The goal of counterfactual learning for statistical machine translation (SMT) is to optimize a target SMT system from logged data that consist of user feedback to translations that were predicted by another, historic SMT system. A challenge arises by the fact that risk-averse commercial SMT systems deterministically lo…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “Logged Bandit Feedback”
Improved PAC learning algorithm for multiclass classification with bandit feedback.
problem Efficiently learning multiclass classification with limited feedback.
method Novel algorithm using stochastic optimization and Frank-Wolfe updates.
result Improved sample complexity bounds for multiclass PAC learning.
Improved algorithm for bandits with delayed feedback, combining adversarial and stochastic performance.
problem Adversarial and stochastic multiarmed bandits with delayed feedback.
method Modified Zimmert and Seldin's algorithm with near-optimal regret guarantees.
result Near-optimal regret guarantees in both adversarial and stochastic settings.
Bayesian CRM improves offline learning from logged bandit data.
problem Offline learning from logged bandit feedback.
method PAC-Bayesian analysis for a new generalization bound, novel regularization technique.
result New technique outperforms standard regularization and is competitive with variance regularization.
Efficiently evaluates and optimizes counterfactual policies from bandit feedback.
problem Statistical inefficiency in evaluating and optimizing policies from bandit feedback.
method Developed estimators with lowest variance for expected reward from counterfactual policies.
result Achieved significant variance reduction in estimators compared to standard methods.
Counterfactual learning from human bandit feedback describes a scenario where user feedback on the quality of outputs of a historic system is logged and used to improve a target system. We show how to apply this learning framework to neural semantic parsing. From a machine learning perspective, the key challenge lies i…
Off-Policy Evaluation and Learning from Logged Bandit Feedback: Error Reduction via Surrogate Policycs.LG
New MLIPS method reduces error in batch bandit learning.
problem Learning from logged bandit feedback with policy discrepancy.
method Estimates a surrogate policy to reduce error in inverse propensity weighting.
result MLIPS has smaller mean squared error than IPS.
Improved Boltzmann exploration reduces regret in stochastic MAB problems.
problem Improving regret in stochastic multi-armed bandit problems.
method A modified Boltzmann exploration with a doubling trick.
result Achieves regret, improving on previous results.
New algorithm improves multiclass classification regret bound.
problem Improving multiclass classification with bandit feedback.
method Developed a new bandit classification algorithm.
result Guaranteed regret of , improving over classical algorithms.
New algorithm for identifying Condorcet team in noisy comparisons.
problem Online learning with noisy comparisons of teams.
method Formalized dueling teams problem, developed algorithms for stochastic and deterministic settings.
result Identifies Condorcet winning team with reduced number of duels.
Study online learning with delays and capacity constraints, achieving optimal regret bounds.
problem Online learning with delays and capacity constraints.
method Novel scheduling and preemptive techniques, matching upper and lower bounds.
result Achieves optimal regret bounds across all capacity levels.
New graph feedback model for bandits with improved regret bounds.
problem Understanding how graph structure affects regret in bandit problems.
method Introduced fractional weak domination number and -packing independence number to capture upper and lower bounds on regret. Used strong duality theorem to derive upper and lower bounds.
result Proved general upper and lower bounds on regret for various graph structures, showing tightness up to a logarithmic factor.
New method reduces bias in learning from large action spaces using selective importance sampling.
problem Learning from large-scale recommendation systems with bandit feedback and supervised labels.
method Selective Importance Sampling (sIS) and Policy Optimization for eXtreme Models (POXM) algorithm.
result POXM method significantly outperforms existing methods in learning from bandit feedback on XMC tasks.
Paper proposes DG-ETC for online submodular maximization with stochastic bandit feedback.
problem Online unconstrained submodular maximization with stochastic bandit feedback.
method Double-Greedy - Explore-then-Commit (DG-ETC) approach.
result DG-ETC achieves logarithmic regret for -approximate pseudo-regret.
Efficient algorithms for online multiclass linear classification with bandit feedback under linear separability conditions.
problem Efficient online multiclass linear classification with bandit feedback for separable data.
method Design of efficient algorithms based on kernel Perceptron for strong and weak linear separability conditions.
result Near-optimal mistake bounds of $O\left( K/γ^2
ight)$ for strong separability and for weak separability.
We develop a learning principle and an efficient algorithm for batch learning from logged bandit feedback. This learning setting is ubiquitous in online systems (e.g., ad placement, web search, recommendation), where an algorithm makes a prediction (e.g., ad ranking) for a given input (e.g., query) and observes bandit …
Algorithm achieves comparable performance to fully dynamic data with only a few batches.
problem High-dimensional multi-armed contextual bandits with batched feedback.
method Provable sample-efficient algorithm using batch allocation method.
result Achieves regret bounds comparable to fully sequential setting with only L = O(log T) batches.
We present the first real-world application of methods for improving neural machine translation (NMT) with human reinforcement, based on explicit and implicit user feedback collected on the eBay e-commerce platform. Previous work has been confined to simulation experiments, whereas in this paper we work with real logge…
Study online learning in MDPs with aggregate bandit feedback, achieving low regret in both stochastic and adversarial settings.
problem Online learning in finite-horizon episodic MDPs with aggregate bandit feedback.
method Best-of-both-worlds (BOBW) algorithms using FTRL over occupancy measures, self-bounding techniques, and new loss estimators.
result First BOBW algorithms for episodic tabular MDPs with aggregate bandit feedback achieving regret in stochastic and regret in adversarial settings.
CMOSS algorithm reduces regret in combinatorial semi-bandits with efficient computation.
problem Efficiently solving combinatorial semi-bandit problems with minimal regret.
method CMOSS algorithm achieves optimal regret bounds with minimal computational overhead.
result CMOSS achieves optimal regret bounds with minimal computational overhead.
Paper reviews methods for learning from bandit feedback in recommender systems.
problem Learning from historical data with unknown rewards.
method Counterfactual Risk Minimisation (CRM) methods using importance sampling and variance reduction.
result Comparison of different off-policy estimators' performance.
New algorithms reduce matching market regret to log(T) with improved stability.
problem Minimizing regret in two-sided matching markets with bandit feedback.
method Phase-based algorithm with local arm deletion to improve stability.
result Achieves Θ(log(T)) regret for markets with uniqueness consistency.
Novel LSE estimator improves off-policy learning and evaluation.
problem High variance and poor performance with low-quality propensity scores and heavy-tailed reward distributions.
method Introduces a novel estimator based on the log-sum-exponential (LSE) operator.
result Achieves convergence rate of for regret bounds.
Incentivized exploration improves MAB performance under reward drift.
problem Improving exploration in multi-armed bandits with biased feedback.
method Analysis of three MAB algorithms (UCB, ε-Greedy, Thompson Sampling) under drifted reward feedback.
result All algorithms achieve regret and compensation under drifted reward.
Three methods for training models on bandit feedback, reviewed and tested.
problem Training machine learning models on recommender system logs.
method Three distinct methods: argmax, distribution adjustment, and inverse propensity score.
result The latter two methods violate principles but perform well in certain settings.
We address the online linear optimization problem with bandit feedback. Our contribution is twofold. First, we provide an algorithm (based on exponential weights) with a regret of order for any finite action set with actions, under the assumption that the instantaneous loss is bounded by 1. This…
Optimizes ranking from click feedback in a bandit setting.
problem Learning to rank from Bernoulli click feedback in a bandit setting.
method Variance-aware confidence sets derived from Bernstein and Chernoff bounds for optimal algorithms.
result Optimal algorithms for the case of small mean rewards, improving on previous suboptimal results.
Algorithm improves query recommendations with immediate user feedback.
problem Lack of adaptability to immediate user feedback in query recommendation algorithms.
method Augmented transformer-based causal language models with multi-armed bandit framework.
result Substantial improvement in per-round regret compared to state-of-the-art models.
DART optimizes subset selection in non-linear bandit problems.
problem Optimizing subset selection in non-linear bandit problems with correlated rewards.
method DART algorithm for combinatorial bandits without individual arm feedback or linearity assumption.
result DART achieves a regret bound of .
New algorithm for minimizing regret in adversarial dueling bandits.
problem Minimizing regret in dueling bandits with adversarial feedback.
method Introduced an algorithm with -round regret of .
result Algorithm achieves regret lower bound.
Study on a new family of problems interpolating expert advice and multi-armed bandits.
problem A new family of problems combining expert advice and multi-armed bandits.
method Proved minimax regret bounds and designed optimal PAC algorithms for pure exploration.
result Tight minimax regret bounds and optimal PAC algorithm for -BAI.
Oracle-efficient algorithms reduce combinatorial semi-bandit regret to logarithmic time.
problem Scalability issue in combinatorial semi-bandit problems due to high combinatorial optimization costs.
method Oracle-efficient frameworks that minimize oracle queries while maintaining tight regret guarantees.
result Achieved regret with oracle queries for worst-case linear rewards.
We investigate contextual online learning with nonparametric (Lipschitz) comparison classes under different assumptions on losses and feedback information. For full information feedback and Lipschitz losses, we design the first explicit algorithm achieving the minimax regret rate (up to log factors). In a partial feedb…
New algorithm achieves near-optimal performance in dueling bandit problem.
problem Optimizing decision-making in dueling bandit problems with limited adaptive rounds.
method Developed a batched algorithm that matches the asymptotic regret bounds of sequential algorithms under the Condorcet condition.
result Asymptotic regret of in rounds.
Capacity-Constrained Online Convex Optimization with Delayed Feedback
problem Online learning with delayed feedback under a hard capacity constraint
method Reduction to a delayed and weighted OCO problem using a scheduler
result First regret guarantees for capacity-constrained OCO under convex and strongly convex losses
Optimizes LLM prompts using logged user feedback.
problem Naive approaches to optimizing LLM prompts suffer from high variance and bias.
method Kernel-based off-policy gradient method leveraging sentence similarity.
result Substantially reduces variance and suppresses bias in optimizing prompts.
New algorithm tackles batched stochastic linear bandits with 1-bit communication constraints.
problem Stochastic linear bandits with 1-bit communication constraints.
method Phased-elimination algorithms based on G-optimal designs and 1-bit mean estimation.
result Achieves near-optimal regret bounds for broad scaling regimes.
Two algorithms for linear contextual bandits with rare updates achieve optimal regret and efficiency.
problem Linear contextual bandits with infrequent parameter updates.
method Two practical algorithms with updates, BLCE-G and BLCE.
result Minimax-optimal regret with low computational complexity.
New algorithm speeds up user preference learning in conversational contexts.
problem Limited performance of existing conversational contextual bandit approaches.
method Proposes ConLinUCB framework and two algorithms, ConLinUCB-BS and ConLinUCB-MCR, with explorative key-term selection.
result Proves tighter regret bounds and achieves significant computational efficiency improvements.
Combines offline causal inference and online bandit learning for better decision-making.
problem Making adaptive decisions using both logged and streaming data to avoid user harm.
method Unified offline causal inference and online learning algorithms, deriving bounds on decision accuracy.
result First upper regret bound for forest-based online bandit algorithms.
Optimistic covariance-adaptive algorithms improve combinatorial semi-bandits regret.
problem Optimal regret in stochastic combinatorial semi-bandits with adaptive covariance estimation.
method Design of OLS-UCB-C and COS-V algorithms leveraging online covariance estimation.
result Improved gap-free regret with T^1/2 complexity for COS-V.
Adapts Exp3 to adversarial bandits with delays and data.
problem Adversarial multi-armed bandits with delayed feedback.
method Tuned Exp3 variants with step-size adaptation and implicit exploration.
result Optimal regret bounds of with high probability.
The dueling bandit is a learning framework wherein the feedback information in the learning process is restricted to a noisy comparison between a pair of actions. In this research, we address a dueling bandit problem based on a cost function over a continuous space. We propose a stochastic mirror descent algorithm and …
Conversational UCB accelerates bandit learning with user feedback.
problem Slow learning speed in traditional contextual bandit algorithms.
method Generalized contextual bandit to conversational contextual bandit, leveraging both behavioral and conversational feedbacks.
result ConUCB achieves a smaller regret upper bound, indicating faster learning speed.
Improved Bandit PCA with optimal regret bound.
problem Minimizing regret in online PCA with bandit feedback.
method Combines online mirror descent and multiscale exploration.
result Minimax optimal regret bound of .
Study optimal arms in combinatorial bandits with semi-bandit feedback and finite budget.
problem Finding optimal arms in combinatorial bandits with semi-bandit feedback and finite budget constraints.
method Proposes a generic algorithm covering various arm elimination strategies and derives lower bounds.
result Demonstrates sufficient and necessary budget requirements for finding the best arm.
Reduces user feedback needed for accurate recommender systems.
problem Limited user feedback in recommender systems.
method Partial Bandit and Semi-Bandit approach for efficient user feedback retrieval.
result Similar global accuracy and learning efficiency with reduced feedback.
New algorithm for linear bandits learns from feature feedback, reducing regret.
problem Linear bandit problem with feature feedback.
method Developed new theory and algorithms for linear bandits with feature feedback.
result Achieves regret scaling like , improving over traditional linear bandits.