Paper solves stochastic contextual linear bandits using linear bandit algorithms.
problem Stochastic contextual linear bandits with unknown context distribution.
method Establishes a reduction framework to convert to linear bandit problems.
result Achieves nearly optimal regret bound of O ( d T log T ) O(d\sqrt{T\log T}) O ( d T log T ) . New algorithm reduces regret from sqrt(T) to polylog(T) in stochastic contextual linear bandits.
problem Achieving logarithmic regret in stochastic contextual linear bandits.
method Low Regret Stochastic Contextual Bandits ( exttt{LR-SCB}) algorithm, exploiting stochastic contexts and parameter estimation.
result Logarithmic regret (polylog(T)) achieved, improving over sqrt(T) lower bound.
First robust bandit algorithm for contextual bandits with sub-linear regret.
problem Vulnerability of linear contextual bandit algorithms to adversarial attacks.
method Proposes a robust bandit algorithm for stochastic linear contextual bandits under fully adaptive and omniscient attacks.
result Sub-linear regret under various attacks without requiring attack information.
New algorithm improves learning efficiency in multi-task contextual bandits.
problem Improving learning efficiency in multi-task contextual bandits.
method Alternating projected gradient descent (GD) and minimization estimator for low-rank feature matrix recovery.
result Proved regret bound for multi-task learning algorithm.
We consider the stochastic linear (multi-armed) contextual bandit problem with the possibility of hidden simple multi-armed bandit structure in which the rewards are independent of the contextual information. Algorithms that are designed solely for one of the regimes are known to be sub-optimal for the alternate regime…
A framework for auto-tuning hyper-parameters in contextual bandit algorithms.
problem Auto-tuning hyper-parameters in real-time for contextual bandit algorithms.
method Proposes a Syndicated Bandits framework to learn multiple hyper-parameters dynamically.
result Achieves optimal regret bounds under certain scenarios and handles multiple contextual bandit algorithms.
This paper improves offline contextual bandits using distributional robustness.
problem Improving offline contextual bandits with robustness.
method Extends Distributionally Robust Optimization (DRO) for offline contextual bandits, introducing a convex reformulation of Counterfactual Risk Minimization.
result Automatic calibration of asymptotic confidence intervals for policy optimization.
A new algorithm tackles graph-based contextual bandits with efficient regret bounds.
problem Predicting labels on graphs with contextual bandit methods.
method Graph-based contextual bandit algorithm using optimal stochastic bandit techniques.
result Regret bounds for line graphs and trees, and improved bounds for general graphs.
Study on selecting between base algorithms in stochastic bandit problems.
problem Model selection in stochastic environments with contextual information.
method Developed a meta-algorithm-base algorithm abstraction with a smoothing transformation for optimal O ( T ) O(\sqrt{T}) O ( T ) guarantees. result Optimal O ( T ) O(\sqrt{T}) O ( T ) model selection guarantees for stochastic contextual bandit problems. New algorithm reduces contextual bandit identification to argmax calls.
problem Best-arm identification in stochastic contextual bandits.
method Instance-optimal PAC algorithm using argmax oracle calls.
result First instance-dependent PAC sample complexity for contextual bandits.
Faster algorithm reduces contextual bandit regret with fewer offline regression calls.
problem Optimizing reward in contextual bandits with unknown functions.
method Designing a simple algorithm with O ( log T ) {O}(\log T) O ( log T ) offline regression calls. result Achieves statistically optimal regret with minimal offline calls.
A new algorithm selects models for contextual bandits, reducing regret.
problem Model selection for stochastic contextual bandits under realizability assumption.
method Adaptive Contextual Bandit (ACB) algorithm, successive refinement phases.
result ACB algorithm achieves similar regret bound to known algorithms, with a model selection cost.
New algorithm reduces regret with diverse contexts in bandits.
problem Impact of context diversity on stochastic linear contextual bandits.
method Design of LinUCB-d algorithm and analysis of its regret performance.
result Cumulative expected regret is bounded by a constant under diverse context assumption.
Optimizes decision-making in dueling bandits with contextual features.
problem Identifying the best arm in dueling bandits with contextual features.
method Develops algorithms for minimizing regret in stochastic contextual dueling bandits.
result Proves optimal regret bounds for contextual dueling bandits.
New method for online statistical inference in contextual bandits using SGD.
problem Online decision-making in contextual bandits with statistical inference.
method Weighted stochastic gradient descent for adaptive data collection.
result Asymptotic normality of the parameter estimator with improved efficiency.
New algorithm for contextual bandits with linear constraints using regression.
problem Contextual bandits with packing and covering constraints.
method Modular Lagrangian approach via regression.
result First vanishing-regret guarantees for CBwLC (or CBwK).
New algorithms protect user data while optimizing personalized decisions.
problem Personalized decision-making with private user data.
method Developed LDP algorithms for stochastic generalized linear bandits using SGD and OLS.
result Achieved the same regret bound as non-privacy settings with LDP.
New algorithm for fair ranking in contextual bandits with concave rewards.
problem Fair ranking in recommendation systems.
method Geometric interpretation of CBCR as optimization, Frank-Wolfe analyses.
result First algorithm with provably vanishing regret for CBCR.
Regularization-induced exploration improves contextual bandit performance.
problem Complex reward models in real-world contextual bandits are hard to explore effectively.
method Regularization-induced exploration using stochasticity in cross-validation.
result Regularization-induced exploration leads to reliable exploration in large-scale business environments.
Designs a single policy for collecting data to train near-optimal policies.
problem Engineering overhead in deploying minimax procedures for stochastic linear contextual bandits.
method Designs a single stochastic policy to collect data from which a near-optimal policy can be extracted.
result The designed policy can collect data from which a near-optimal policy can be extracted.
A new algorithm combines SGD and Thompson Sampling for contextual bandits.
problem Finding efficient algorithms for contextual bandits with low time and memory complexity.
method Online Stochastic Gradient Descent (SGD) combined with Thompson Sampling.
result Achieves i l d e O ( T ) ilde{O}(\sqrt{T}) i l d e O ( T ) regret with linear time complexity in T T T and d d d . Improved Thompson Sampling for high-dimensional sparse bandits.
problem Stochastic linear contextual bandits with high-dimensional features.
method Thompson Sampling with spike-and-slab priors and variational inference.
result Nearly optimal upper bound on expected cumulative regret.
New method tackles high-dimensional contextual bandits with flexible kernel models.
problem Maximizing rewards in decision-making scenarios with many features.
method Introduces stochastic assumptions and no-regret learning for Gaussian kernels.
result Achieves no-regret learning even with feature dimensions growing with samples.
We consider the problem of contextual bandits with stochastic experts, which is a variation of the traditional stochastic contextual bandit with experts problem. In our problem setting, we assume access to a class of stochastic experts, where each expert is a conditional distribution over the arms given a context. We p…
Algorithm minimizes regret in dueling bandits with contextualized utilities.
problem Minimizing regret in dueling bandits with context-dependent utilities.
method Proposes CoLSTIM algorithm based on perturbed utility estimates.
result Achieves regret of order i l d e O ( d T ) ilde O(\sqrt{dT}) i l d e O ( d T ) . Data that is gathered adaptively --- via bandit algorithms, for example --- exhibits bias. This is true both when gathering simple numeric valued data --- the empirical means kept track of by stochastic bandit algorithms are biased downwards --- and when gathering more complicated data --- running hypothesis tests on c…
A collaborative algorithm reduces regret in federated linear contextual bandits.
problem Optimizing decision-making in federated learning with heterogeneous data.
method Fed-PE algorithm, leveraging geometric structure of rewards, multi-client G-optimal design.
result Achieves near-optimal regrets with logarithmic communication costs.
Develops a strategy to minimize loss in both stochastic and adversarial environments for linear contextual bandits.
problem Linear contextual bandits with adversarial corruption.
method Proposes a novel strategy called Best-of-Both-Worlds (BoBW) RealFTRL, extending RealLinExp3 and FTRL.
result Regret upper bound of $O\left(\min\left\{\frac{(\log(T))^3}{Δ_{*}} + \sqrt{\frac{C(\log(T))^3}{Δ_{*}}},\ \ \sqrt{T}(\log(T))^2
ight\}
ight)$ , showing effectiveness in both stochastic and adversarial environments.
Develops a Best-of-Both-Worlds algorithm for linear contextual bandits with Tsallis entropy.
problem Linear contextual bandits with i.i.d. contexts.
method Follow-The-Regularized-Leader (FTRL) with Tsallis entropy.
result Achieves $O\left(\log(T)^{\frac{1+β}{2+β}}T^{\frac{1}{2+β}}
ight)$ regret under margin condition.
Study on adaptivity constraints in linear contextual bandits with optimal design.
problem Impact of adaptivity constraints on linear contextual bandits.
method Two models of limited adaptivity: batch learning and rare policy switches. Proposed distributional optimal design.
result Achieves minimax-optimal regret with optimal number of policy switches and batches.
NeuralRBMLE tackles explore-exploit trade-offs in contextual bandits with neural networks.
problem Stochastic contextual bandit problem with general bounded reward functions.
method Reward-biased maximum likelihood estimation with neural networks to enforce exploration.
result Both NeuralRBMLE variants achieve O ~ ( T ) \widetilde{\mathcal{O}}(\sqrt{T}) O ( T ) regret. Study shows how to improve contextual bandits with loss predictors.
problem Improving minimax regret in contextual bandits with loss predictors.
method Developed novel algorithmic techniques for upper bounds and lower bounds in various settings.
result Optimal regret is O ( min { T , E T 1 4 } ) \mathcal{O}(\min\{\sqrt{T}, \sqrt{\mathcal{E}}T^\frac{1}{4}\}) O ( min { T , E T 4 1 }) when E \mathcal{E} E is known, and O ( E T 1 3 ) \mathcal{O}(\sqrt{\mathcal{E}}T^\frac{1}{3}) O ( E T 3 1 ) if E \mathcal{E} E is unknown. A new framework tunes hyperparameters in real-time for contextual bandits.
problem Optimizing hyperparameters for contextual bandits in real-time.
method CDT (Continuous Dynamic Tuning) framework using Zooming TS algorithm.
result Achieves sublinear regret and performs better than existing methods.
Master algorithm selects best contextual bandit from a collection.
problem Model selection in stochastic contextual bandit setting.
method Random selection with probability adjustment based on comparison of cumulative rewards.
result Achieves the same regret rate as the best candidate in a collection of black-box algorithms.
A new method for sparse linear bandits reduces exploration-exploitation tradeoff.
problem Sparse linear bandits in high-dimensional settings with finite actions.
method Best subset selection for parameter estimation and doubly growing epochs for regret minimization.
result Achieves nearly dimension-independent regret of i l d e O ( s T ) ilde{\mathcal{O}}(s\sqrt{T}) i l d e O ( s T ) with high probability. Paper proposes efficient offline neural bandit method.
problem Offline policy learning with neural networks.
method Provable efficient offline contextual bandit with neural network function approximation.
result Method provably generalizes over unseen contexts.
New algorithms minimize regret in both adversarial and stochastic contexts.
problem Minimizing regret in linear contextual bandits.
method Best-of-both-worlds algorithms using FTRL with Shannon entropy regularizer.
result Achieves near-optimal regret bounds in both adversarial and stochastic regimes.
Adapts two algorithms for online learning with delayed rewards.
problem Online learning with delayed rewards in generalized linear contextual bandits.
method Modifies upper confidence bounds and Thompson sampling algorithms for delayed rewards.
result Both algorithms can be made robust to delays, improving their performance.
A contextual bandit method evaluates and improves inventory control policies.
problem Evaluating and improving periodic review inventory control policies with nonstationary demand.
method Contextual bandit-based algorithm to evaluate and tweak policies.
result The method achieves favorable guarantees in both theory and practice.
New algorithms minimize simple and cumulative regret in contextual bandits.
problem Minimizing simple and cumulative regret in contextual bandit settings.
method Proposed new algorithms using conformal arm sets (CASs).
result Near-optimal minimax guarantees for simple regret and state-of-the-art guarantees for cumulative regret.
Optimal algorithm for maximizing rewards in contextual bandits with resource constraints.
problem Maximizing rewards in contextual bandits with resource constraints.
method Proposed a universal and optimal algorithmic framework for CBwK by reducing it to online regression.
result Established the optimality of the proposed algorithm for various function classes.
Improved algorithm for contextual bandits with reduced regret.
problem Adversarial contextual bandits with i.i.d. contexts.
method Oracle-efficient relaxation with O ( T 2 3 ( K log ( ∣ Π ∣ ) ) 1 3 ) O(T^{\frac{2}{3}}(K\log(|Π|))^{\frac{1}{3}}) O ( T 3 2 ( K log ( ∣Π∣ ) ) 3 1 ) regret bound. result First to improve regret bound and match original bound for stochastic case.
Paper studies CLO with partial feedback, improving decision-making in uncertain contexts.
problem Improving decision-making in contexts with uncertain cost coefficients using partial feedback.
method Unified class of offline learning algorithms for CLO with different types of feedback, using IERM framework.
result Fast-rate regret bound for IERM with partial feedback and misspecified model classes.
New algorithm offers costless model selection in contextual bandits.
problem Minimizing cumulative regret in stochastic contextual bandits.
method Gradually increasing class complexity and adapting to the simplest class with dominant estimation variance.
result Costless model selection is feasible under certain conditions, providing improved regret guarantees.
New algorithms reduce sample complexity for multiclass contextual bandits.
problem Designing efficient algorithms for multiclass contextual bandits with sparse rewards.
method Two complementary approaches: decision-estimation coefficient analysis and low-variance exploration.
result Achieved optimal sample complexity bounds for multiclass contextual bandits.
Proposes a new algorithm for non-stationary bandits.
problem Non-stationary reward distributions in contextual bandits.
method Multiscale changepoint detection for adaptive learning.
result Regret bound analysis and superior performance in experiments.
We study the stochastic contextual bandit problem, where the reward is generated from an unknown function with additive noise. No assumption is made about the reward function other than boundedness. We propose a new algorithm, NeuralUCB, which leverages the representation power of deep neural networks and uses a neural…
We introduce the study of fairness in multi-armed bandit problems. Our fairness definition can be interpreted as demanding that given a pool of applicants (say, for college admission or mortgages), a worse applicant is never favored over a better one, despite a learning algorithm's uncertainty over the true payoffs. We…