Optimizes ranking from click feedback in a bandit setting.
problem Learning to rank from Bernoulli click feedback in a bandit setting.
method Variance-aware confidence sets derived from Bernstein and Chernoff bounds for optimal algorithms.
result Optimal algorithms for the case of small mean rewards, improving on previous suboptimal results.
Algorithm learns diverse recommendations from partial clicks.
problem Learning diverse recommendations from partial user feedback.
method Online learning algorithm for cascade model.
result Gap-free upper bound on n-step regret proved.
TopRank algorithm improves online ranking with better performance and insights.
problem Sequential decision-making in online learning to rank with user feedback.
method Generalized click model and topological sort-based algorithm.
result TopRank outperforms existing algorithms in terms of performance and proof insight.
Proposes a robust learning strategy for RS over implicit feedback.
problem Robustness in training large-scale Recommender Systems over implicit feedback.
method Minimizes pairwise ranking loss over blocks of non-clicked followed by clicked items, discarding updates for abnormal blocks.
result The approach prevents shifts in item distribution due to bots or few user interactions, improving ranking measures and computation time.
A new method optimizes recommender systems by addressing missing-not-at-random implicit feedback.
problem Missing-not-at-random implicit feedback in recommender systems.
method Defining an ideal loss function and proposing an unbiased estimator to optimize relevance.
result The proposed method improves performance, especially for rare items.
New ranking algorithms improve online content delivery by learning from click data.
problem Bias in ranking systems due to production system biases.
method Proposed novel extensions of LinUCB and Linear Thompson Sampling algorithms to handle position-based click model.
result Validated the proposed algorithms through offline and online experiments.
Proposes a new algorithm for click feedback in search results.
problem Learning to predict user clicks based on relevance and position.
method Developed a Bernoulli rank- 1 1 1 bandit learning problem and proposed Rank1ElimKL to improve performance. result Rank1ElimKL outperforms Rank1Elim in various scenarios, including real-world data.
A new algorithm selects web links to maximize revenue while ensuring user attractiveness.
problem Selecting web links to maximize revenue from multi-level feedbacks.
method Constrained Upper Confidence Bound (Con-UCB) algorithm for stochastic multi-armed bandit.
result Proves O ( T ln T ) O(\sqrt{T\ln T}) O ( T ln T ) bounds on regret and attractiveness constraint violation. Paper improves recommendation systems by optimizing sequence of items for clicks.
problem Improving recommendation systems robustness against bots and clicks.
method Minimizing pairwise ranking loss over sequences of items, with thresholds to prevent bot influence.
result The proposed algorithms converge and outperform existing methods in various ranking measures.
Proposes a nonparametric model for predicting conversion rates with delayed feedback.
problem Predicting conversion rates with time delays and unknown distribution.
method Nonparametric delayed feedback model without assuming a specific distribution.
result The proposed model outperforms existing methods in conversion rate prediction.
A method corrects feedback shift in predicting conversion rates with delayed feedback.
problem Delayed feedback leads to mislabeling of positive instances in training data.
method Uses importance weight approach to correct feedback shift.
result Proposed method outperforms existing methods in offline and online experiments.
Proposes a method to train classifiers with delayed feedback using a time window.
problem Training classifiers with delayed feedback that can be biased due to delayed user actions.
method Uses a time window to select samples for training, constructs unbiased empirical risk from all samples.
result Improves classifier performance by using all samples with a time window assumption.
A new dataset tracks user interactions and click responses in online marketplaces.
problem Lack of exposure data in recommender systems datasets.
method Proposes a novel dataset including slates and click responses, allowing more accurate likelihood models.
result Models using exposure data show more natural likelihood, reducing bias towards previously exposed items.
BubbleRank improves online search results using safe exploration.
problem Learning user preferences from scratch in online ranking is costly and risky.
method BubbleRank combines offline and online learning, starting with an initial base list and improving it online by exchanging items.
result BubbleRank achieves a graceful degradation of n-step regret with a good initial base list.
This paper explores LETOR for E-Com search, addressing practical challenges and reporting key findings.
problem Applying LETOR to E-Com search presents unique challenges.
method Investigates practical challenges in LETOR for E-Com search, including feature representation, relevance judgments, and feedback signal exploitation.
result LETOR methods can effectively optimize combinations of popularity-based and relevance-based features, and order rate is the most robust training objective.
Optimizes LLM prompts using logged user feedback.
problem Naive approaches to optimizing LLM prompts suffer from high variance and bias.
method Kernel-based off-policy gradient method leveraging sentence similarity.
result Substantially reduces variance and suppresses bias in optimizing prompts.
Optimizing an interactive system against a predefined online metric is particularly challenging, when the metric is computed from user feedback such as clicks and payments. The key challenge is the counterfactual nature: in the case of Web search, any change to a component of the search engine may result in a different…
Paper proposes a combined model for better recommendation by integrating explicit and implicit feedbacks.
problem Improve recommendation accuracy by considering both explicit and implicit feedbacks.
method Developed three models (RHC-PMF, RV-PMF, RHCV-PMF) that incorporate users' explicit and implicit feedbacks for better rating prediction.
result RHCV-PMF model outperforms other models in cold start scenarios for both users and items.
CausalRM models rewards from user feedback, overcoming noise and bias.
problem Aligning language models with user preferences from noisy, biased feedback.
method Causal-theoretic reward modeling framework addressing noise and bias in observational feedback.
result CausalRM learns accurate reward signals from noisy and biased observational feedback.
Agents learn user preferences with less explicit feedback via spatial interface valuing.
problem Learning user preferences with high cognitive load feedback.
method Spatial Interface Valuing for reduced explicit feedback.
result Agents learn faster with spatial interface valuing compared to explicit feedback.
We develop a learning principle and an efficient algorithm for batch learning from logged bandit feedback. This learning setting is ubiquitous in online systems (e.g., ad placement, web search, recommendation), where an algorithm makes a prediction (e.g., ad ranking) for a given input (e.g., query) and observes bandit …
SetRank tackles collaborative ranking from implicit feedback using setwise Bayesian approach.
problem Challenges in pairwise and listwise approaches for implicit feedback.
method SetRank is a novel setwise Bayesian approach that accommodates implicit feedback characteristics.
result SetRank outperforms state-of-the-art baselines on real-world datasets.
A new model considers fatigue in online content recommendation systems.
problem Fatigue in users due to overexposure and boredom from similar recommendations.
method Proposed a fatigue-aware Dependent Click Model (DCM) and two learning algorithms.
result Developed algorithms with regret bounds for learning content relevance and fatigue effects.
Improves YouTube's recommendation system by correcting biases in logged feedback.
problem Data biases in logged feedback from multiple behavior policies.
method Top-K off-policy correction applied to REINFORCE algorithm.
result Efficacy demonstrated through simulations and live experiments.
New algorithms minimize regret in repeated auctions by estimating values and optimizing bids.
problem Minimizing regret in repeated first-price auctions with limited feedback.
method Incorporates causal inference to estimate private values and optimize bidding strategies under different feedback types.
result Achieves near-optimal regret bounds for both full and binary feedback types.
Detects accidental clicks on mobile ads to reduce advertiser costs and improve revenue.
problem Accidental clicks on mobile ads lead to wasted revenue for advertisers and ad networks.
method Collect and analyze dwell time data to identify accidental clicks and estimate thresholds.
result Our method reduces advertiser costs and improves ad click-through rates and revenue.
Item recommendation is the task of predicting a personalized ranking on a set of items (e.g. websites, movies, products). In this paper, we investigate the most common scenario with implicit feedback (e.g. clicks, purchases). There are many methods for item recommendation from implicit feedback like matrix factorizatio…
Paper addresses bias in search intent affecting click behavior.
problem Bias in user search intent affects click behavior and relevance.
method Proposes a search intent bias hypothesis to improve click models.
result Click models can better interpret user clicks and improve retrieval performance.
Dual learning algorithm addresses delayed conversions in CVR prediction.
problem Challenges in predicting conversion rate due to delayed feedback.
method Proposes two unbiased estimators and a dual learning algorithm.
result Demonstrates practical value of the proposed approach through empirical evaluations.
Algorithm learns fair ranking from biased data.
problem Unfair ranking policies from biased implicit feedback.
method Policy-gradient approach with amortized fairness constraints.
result Efficient algorithm FULTR learns fair policies.
Unweighted matrix factorization can match or outperform weighted methods in recommender systems.
problem Improving recommendation performance with matrix factorization on implicit feedback data.
method Systematic study of various weighting schemes and matrix factorization algorithms.
result Training with unweighted data can perform comparably to, and sometimes outperform, training with weighted data.
New method uses bandit feedback to better evaluate recommender systems.
problem Traditional offline evaluation of recommender systems is inaccurate.
method Exploits bandit feedback to estimate online performance.
result Bandit feedback provides more accurate offline evaluation.
Efficiently evaluates new ranking policies using click models.
problem Evaluate new ranking policies offline and optimize them before deployment.
method Proposes evaluation algorithms using click models to estimate expected clicks from logged data.
result Our estimators are more statistically efficient than those that do not use click models.
We address the problem of maximizing an unknown submodular function that can only be accessed via noisy evaluations. Our work is motivated by the task of summarizing content, e.g., image collections, by leveraging users' feedback in form of clicks or ratings. For summarization tasks with the goal of maximizing coverage…
First online learning to rank algorithm for broad click models.
problem Online learning to rank in stochastic click models.
method BatchRank, an algorithm for a broad class of click models.
result Derives a gap-dependent upper bound on the T T T -step regret of BatchRank. Paper tackles online ranking and diversification in recommender systems.
problem Maximizing relevance and diversity in ranked lists for online recommendation.
method CascadeHybrid approach that combines contextual bandits for relevance and topical diversity.
result CascadeHybrid outperforms baselines in real-world datasets.
Improves online ranker evaluation using multi-dueling bandits.
problem Efficiently evaluating ranking algorithms from limited user feedback.
method Generalized dueling bandits model for simultaneous comparisons of multiple rankers.
result Orders of magnitude improvement in performance compared to state-of-the-art algorithms.
Develops algorithms for personalized ranking in recommender and energy systems.
problem Designing algorithms for personalized ranking in dynamic systems.
method Proposes SAROS for recommender systems and a fault detection algorithm for power systems.
result SAROS outperforms traditional methods in both recommender and energy systems.
Proposes a new IPW-based ranking metric for two-sided markets.
problem Addressing bias in implicit user feedback in two-sided markets.
method Extends IPW estimator to two-sided markets, addressing position bias.
result Proposed estimator is unbiased for ground-truth ranking metric.
DCM bandits optimize search engine recommendations by learning from user clicks.
problem Optimizing search engine recommendations based on user behavior with multiple clicks.
method Online learning algorithm dcmKL-UCB for maximizing satisfactory item recommendation probability.
result Proves dcmKL-UCB's regret bound and matches a lower bound up to logarithmic factors.
Deep neural nets predict click-through rates for sponsored search ads.
problem Predicting click-through rates for sponsored search ads.
method Two novel deep convolutional neural network approaches at character and word levels.
result Deep models significantly outperform baseline models and improve click-through rate prediction accuracy.
A new model predicts conversion rates by analyzing post-click actions.
problem Challenges in predicting conversion rates due to sample selection bias and data sparsity.
method Post-click behavior decomposition and multi-task learning.
result The model effectively addresses sample selection bias and data sparsity issues.
Optimizes clustering from noisy binary feedback in crowdsourcing.
problem Clustering items from binary user feedback with noisy answers.
method Develops algorithms for clustering items using adaptive selection of questions and items.
result Adaptive algorithm achieves performance close to information-theoretical limits.
A new model improves click-through rate prediction for recommendation systems.
problem Improving accuracy of click-through rate prediction in recommendation systems.
method Combines traditional feature engineering with deep neural networks to automate feature combinations.
result The model (FNFM) outperforms current deep learning feature combination models.
This paper improves online ad revenue by optimizing auction performance directly.
problem Disconnection between ad ranking and auction performance in online advertising.
method Proposes new loss functions and ranking functions to maximize revenue.
result Proposed methods outperform state-of-the-art in maximizing platform revenue.
The paper analyzes matrix completion with unlabeled implicit feedback and provides error bounds.
problem Matrix completion with shared low-rank ground truth and sampling distribution.
method Combining subspace recovery theory and matrix completion bounds.
result Error bounds showing contributions from estimating the sampling distribution and ground truth.
Hotel2vec learns hotel embeddings from multiple data sources.
problem Cold-start problem for hotels with insufficient click data.
method Self-supervised neural network architecture combining user clicks, hotel attributes, and geographic info.
result Improved downstream task predictions with structured hotel attributes.
Paper optimizes recommendation systems for long-term business metrics.
problem Short-term reward optimization ignores long-term business metrics.
method Introduced a framework for modeling long-term rewards in RecoGym.
result Proposed a simple extension leading to state-of-the-art results.