Paper solves stochastic contextual linear bandits using linear bandit algorithms.
problem Stochastic contextual linear bandits with unknown context distribution.
method Establishes a reduction framework to convert to linear bandit problems.
result Achieves nearly optimal regret bound of O ( d T log T ) O(d\sqrt{T\log T}) O ( d T log T ) . Optimal algorithm for identifying best arm in stochastic linear bandits with fixed confidence.
problem Identifying the best arm in stochastic linear bandits with fixed confidence.
method Extending an algorithm designed for Best Arm Identification to the ε ε ε -Thresholding Bandit Problem (TBP). result Asymptotically optimal algorithm for TBP.
A new algorithm improves stochastic linear bandit performance using residual bootstrap.
problem Improving performance in stochastic linear bandit problems.
method Residual bootstrap exploration to estimate mean reward and pull the arm with the highest estimate.
result Proposed algorithm exttt{LinReBoot} achieves high-probability sub-linear regret under mild conditions.
New algorithm reduces regret from sqrt(T) to polylog(T) in stochastic contextual linear bandits.
problem Achieving logarithmic regret in stochastic contextual linear bandits.
method Low Regret Stochastic Contextual Bandits ( exttt{LR-SCB}) algorithm, exploiting stochastic contexts and parameter estimation.
result Logarithmic regret (polylog(T)) achieved, improving over sqrt(T) lower bound.
Improved algorithms for stochastic linear bandits using tighter confidence sequences.
problem Stochastic linear bandits with improved worst-case regret guarantees.
method Novel tail bound for adaptive martingale mixtures to construct tighter confidence sequences.
result Linear bandit algorithm achieves competitive worst-case regret.
Bayesian bandit algorithms with approximate inference improve regret bounds in stochastic linear bandits.
problem Theoretical justification for Bayesian bandit algorithms with approximate inference in stochastic linear bandits.
method Proposed a theoretical framework to analyze approximate inference impact and conducted frequentist regret analysis on LinTS and LinBUCB.
result LinTS and LinBUCB preserve their original regret upper bounds with larger constant terms in approximate inference settings.
First robust bandit algorithm for contextual bandits with sub-linear regret.
problem Vulnerability of linear contextual bandit algorithms to adversarial attacks.
method Proposes a robust bandit algorithm for stochastic linear contextual bandits under fully adaptive and omniscient attacks.
result Sub-linear regret under various attacks without requiring attack information.
New algorithms improve linear bandit performance with low computation.
problem Optimizing reward in linear stochastic bandits.
method Reward-biased maximum likelihood method modified for linear and generalized linear bandits.
result New policies achieve order-optimality and competitive empirical performance.
Study shows efficient neural network approach for stochastic bandits.
problem Optimizing decisions in uncertain environments with neural network models.
method OFU-ReLU algorithm that balances exploration and exploitation, using a transformed feature space.
result Achieves i l d e O ( T ) ilde{O}(\sqrt{T}) i l d e O ( T ) regret guarantee for stochastic bandits with ReLU neural networks. Meta-learning improves performance in stochastic linear bandits.
problem Selecting a learning algorithm that performs well across multiple bandit tasks.
method Regularized OFUL algorithm with a bias vector, estimating bias within the learning-to-learn setting.
result Meta-learning strategies improve performance when the number of tasks grows and task variance is small.
RONM method reduces regret in stochastic convex bandits with decreasing noise.
problem Stochastic convex bandit problem with decreasing noise.
method Regularized Online Newton Method (RONM) based on Online Newton Method (ONM).
result RONM achieves polylogarithmic regret in time horizon n.
A meta-UCB method combines stochastic bandit algorithms.
problem Combining multiple stochastic bandit algorithms efficiently.
method Meta-UCB procedure solving an N-armed bandit problem.
result Final regret depends only on the best base algorithm's regret.
Paper addresses privacy and robustness in stochastic linear bandits.
problem Stochastic linear bandits with differential privacy and adversarial robustness.
method Logarithmic batch queries, arm elimination algorithm, two privacy models.
result First algorithms providing differential privacy and adversarial robustness.
New method reduces ensemble size for linear bandits, achieving near optimal regret.
problem Achieving near optimal regret in linear bandits with limited ensemble size.
method Ensemble sampling with a size of order d log T d \log T d log T for a d d d -dimensional stochastic linear bandit. result Regret is at most ( d log T ) 5 / 2 T (d \log T)^{5/2} \sqrt{T} ( d log T ) 5/2 T , improving over linear scaling with T T T . New algorithms tackle RKHS bandits with reduced complexity and improved performance.
problem Adversarial and stochastic RKHS bandit problems with high computational complexity.
method Combining approximation theory with misspecified linear bandit methods.
result First general algorithm for adversarial RKHS bandit problem.
ES reduces high-probability regret in stochastic linear bandits.
problem High-probability regret in stochastic linear bandits.
method Linear ensemble sampling with standard Gaussian perturbations, analyzing m = Θ ( d log n ) m=Θ(d\log n) m = Θ ( d log n ) ensemble size. result ES achieves i l d e O ( d 3 / 2 n ) ilde O(d^{3/2}\sqrt n) i l d e O ( d 3/2 n ) high-probability regret, closing the gap to Thompson sampling. New algorithms protect user data while optimizing personalized decisions.
problem Personalized decision-making with private user data.
method Developed LDP algorithms for stochastic generalized linear bandits using SGD and OLS.
result Achieved the same regret bound as non-privacy settings with LDP.
Algorithm balances online and offline data for linear bandits.
problem Online learning with an offline dataset in linear bandits.
method Proposes a linear bandit algorithm that uses offline data early and increasingly favors exploration as the horizon grows.
result Establishes regret bounds showing competitive performance with both purely online and offline solutions.
Balances and eliminates base algorithms in bandits and RL to bound total regret.
problem Model selection in bandits and reinforcement learning with unknown optimal regret.
method Balances and eliminates base algorithms based on candidate regret bounds.
result Total regret bound is the best valid candidate regret bound times a small multiplicative factor.
New algorithm reduces regret with diverse contexts in bandits.
problem Impact of context diversity on stochastic linear contextual bandits.
method Design of LinUCB-d algorithm and analysis of its regret performance.
result Cumulative expected regret is bounded by a constant under diverse context assumption.
New algorithm improves learning efficiency in multi-task contextual bandits.
problem Improving learning efficiency in multi-task contextual bandits.
method Alternating projected gradient descent (GD) and minimization estimator for low-rank feature matrix recovery.
result Proved regret bound for multi-task learning algorithm.
This paper addresses missing covariates in stochastic linear bandits, providing a high-probability regret bound.
problem Effect of missing covariates on regret in stochastic linear bandit algorithms.
method Proposes an algorithm that provides a high-probability upper bound on regret in terms of covariate sampling probabilities.
result Regret degrades due to missingness by at most ζ m i n 2 ζ_{min}^2 ζ min 2 , where ζ m i n ζ_{min} ζ min is the minimum probability of observing covariates. New algorithm controls linear systems with bandit feedback, achieving optimal regret.
problem Controlling linear systems with bandit feedback under adversarial costs.
method Developed a new algorithm for linear control with memory optimization technique.
result Achieved optimal regret growth proportional to square root of time horizon.
New algorithms combat adversarial attacks in stochastic linear bandits.
problem Adversarial attacks on stochastic linear bandit rewards.
method Two variants of Robust Phased Elimination algorithms, one knowing C C C and one not. result Near-optimal regret in non-corrupted case and additive terms dependent on C C C . We consider the stochastic linear (multi-armed) contextual bandit problem with the possibility of hidden simple multi-armed bandit structure in which the rewards are independent of the contextual information. Algorithms that are designed solely for one of the regimes are known to be sub-optimal for the alternate regime…
New algorithms for stochastic linear bandits with heavy-tailed payoffs achieve nearly optimal regret.
problem Stochastic linear bandits with heavy-tailed payoffs.
method Median of means and dynamic truncation.
result Sublinear regret bound of O ( d 1 2 T 1 1 + ε ) O(d^{\frac{1}{2}}T^{\frac{1}{1+ε}}) O ( d 2 1 T 1 + ε 1 ) for ε ∈ ( 0 , 1 ] ε\in(0,1] ε ∈ ( 0 , 1 ] . This paper explores a new form of the linear bandit problem in which the algorithm receives the usual stochastic rewards as well as stochastic feedback about which features are relevant to the rewards, the latter feedback being the novel aspect. The focus of this paper is the development of new theory and algorithms fo…
Study multi-task learning with low-rank representation in stochastic linear bandits.
problem Transfer learning across multiple linear bandit tasks with shared low-dimensional representation.
method Proposes a greedy policy with trace norm regularization to implicitly learn a low-rank representation without knowing the rank.
result Upper bound on multi-task regret of N d T ( T + d ) r \sqrt{NdT(T+d)r} N d T ( T + d ) r , showing benefit over independent task solving. Study on policy gradient for stochastic bandits using diffusion approximation.
problem Improving policy gradient methods for stochastic bandits with optimal regret bounds.
method Continuous-time diffusion approximation of policy gradient with learning rate analysis.
result Proved optimal regret bound of O ( k log ( k ) log ( n ) / η ) O(k \log(k) \log(n) / η) O ( k log ( k ) log ( n ) / η ) for η = O ( Δ 2 / log ( n ) ) η= O(Δ^2/\log(n)) η = O ( Δ 2 / log ( n )) . Optimal algorithm for high-dimensional stochastic linear bandits with sparse parameters.
problem High-dimensional stochastic linear bandits with sparse parameters.
method Three-stage arm selection algorithm using thresholded Lasso for estimation.
result Achieves exact minimax optimality in cumulative regret.
New method for linear bandits with unknown sparsity, improving sparse regret bounds.
problem Sparse regret bounds for unknown sparsity and adversarial action sets.
method Combines online to confidence set conversions with randomized model selection over nested confidence sets.
result First sparse regret bounds for unknown sparsity and adversarial action sets.
Study on selecting between base algorithms in stochastic bandit problems.
problem Model selection in stochastic environments with contextual information.
method Developed a meta-algorithm-base algorithm abstraction with a smoothing transformation for optimal O ( T ) O(\sqrt{T}) O ( T ) guarantees. result Optimal O ( T ) O(\sqrt{T}) O ( T ) model selection guarantees for stochastic contextual bandit problems. Adaptive algorithm selects models for stochastic linear bandits based on problem complexity.
problem Model selection for stochastic linear bandits with unknown problem complexity.
method Adaptive phase-based algorithm that adapts to true problem complexity.
result Achieves regret scaling of O ( ∥ θ ∗ ∥ T ) O(\|θ^*\|\sqrt{T}) O ( ∥ θ ∗ ∥ T ) for K K K armed mixture bandits and O ( d ∗ T ) O(d^*\sqrt{T}) O ( d ∗ T ) for standard linear bandits. Optimal best-arm identification in linear bandits reduces sampling budget.
problem Identifying the best arm with fixed confidence in stochastic linear bandits.
method A simple algorithm that tracks an optimal proportion of arm draws, updated as rarely as desired.
result The algorithm's sampling complexity matches known lower bounds, asymptotically almost surely and in expectation.
Designs a single policy for collecting data to train near-optimal policies.
problem Engineering overhead in deploying minimax procedures for stochastic linear contextual bandits.
method Designs a single stochastic policy to collect data from which a near-optimal policy can be extracted.
result The designed policy can collect data from which a near-optimal policy can be extracted.
A new method for sparse linear bandits reduces exploration-exploitation tradeoff.
problem Sparse linear bandits in high-dimensional settings with finite actions.
method Best subset selection for parameter estimation and doubly growing epochs for regret minimization.
result Achieves nearly dimension-independent regret of i l d e O ( s T ) ilde{\mathcal{O}}(s\sqrt{T}) i l d e O ( s T ) with high probability. Tr-LinUCB reduces regret in stochastic linear bandits by truncating exploration.
problem Over-exploration in LinUCB leads to suboptimal cumulative regret.
method Proposes Tr-LinUCB, a truncated version of LinUCB.
result Achieves O ( d log ( T ) ) O(d\log(T)) O ( d log ( T )) regret with S = C d log ( T ) S = Cd\log(T) S = C d log ( T ) , matching lower bound. Optimizes stochastic linear bandits with efficient, asymptotically optimal algorithm.
problem Optimizing stochastic linear bandits with multiple actions.
method Frequentist information-directed sampling (IDS) with a surrogate for information gain.
result Asymptotically optimal and nearly worst-case optimal in finite time.
A decentralized algorithm minimizes cumulative regret in stochastic linear bandits with safety constraints.
problem Efficiently solving a linear bandit-optimization problem over a network of agents with safety constraints.
method DLUCB: a fully decentralized algorithm that minimizes cumulative regret through UCB strategy and consensus procedure.
result Near-optimal regret performance of O ( d log N T N T ) \mathcal{O}(d\log{NT}\sqrt{NT}) O ( d log N T N T ) with O ( d N 2 ) \mathcal{O}(dN^2) O ( d N 2 ) communication rate. We introduce the safe linear stochastic bandit framework---a generalization of linear stochastic bandits---where, in each stage, the learner is required to select an arm with an expected reward that is no less than a predetermined (safe) threshold with high probability. We assume that the learner initially has knowledg…
Stochastic multi-armed bandits form a class of online learning problems that have important applications in online recommendation systems, adaptive medical treatment, and many others. Even though potential attacks against these learning algorithms may hijack their behavior, causing catastrophic loss in real-world appli…
New bandit algorithm for non-i.i.d. noise, improving standard rates.
problem Linear stochastic bandit with non-i.i.d. observation noise.
method Developed new confidence sequences and an algorithm based on optimism in uncertainty.
result Regret bounds for the new algorithm, showing recovery of standard rates up to a factor of the mixing time.
Optimizes pure exploration in linear bandits with a new algorithm.
problem Best-arm identification in linear stochastic bandits.
method Developed the first asymptotically optimal algorithm for fixed-confidence pure exploration in linear bandits.
result Avoids the pitfall of a simple but difficult instance and bypasses the need to solve an optimal design problem.
New algorithm reduces bandit regret to log^3(T).
problem Noise model for linear stochastic bandits with vanishing noise.
method Weighted least-squares estimation, leveraging eigenvalue relation.
result Minimax regret scaling as log^3(T) for time horizon T.
A new algorithm identifies one of several nearly optimal arms in linear bandits.
problem Identifying one arm that is close to the best arm in linear bandits.
method Developed a procedure to adapt best-arm identification algorithms for ε \varepsilon ε -best-answer identification in transductive linear stochastic bandits. result Proposed an asymptotically optimal algorithm for ε \varepsilon ε -best-answer identification. Develops model selection for bandits balancing adversarial and stochastic guarantees.
problem Model selection in bandit scenarios with simultaneous adversarial and stochastic high-probability regret.
method Nested policy classes, balanced candidate regret bounds, mis-specification tests.
result Best of both world guarantees in linear bandits with simultaneous adversarial and stochastic environments.
Develops a strategy to minimize loss in both stochastic and adversarial environments for linear contextual bandits.
problem Linear contextual bandits with adversarial corruption.
method Proposes a novel strategy called Best-of-Both-Worlds (BoBW) RealFTRL, extending RealLinExp3 and FTRL.
result Regret upper bound of $O\left(\min\left\{\frac{(\log(T))^3}{Δ_{*}} + \sqrt{\frac{C(\log(T))^3}{Δ_{*}}},\ \ \sqrt{T}(\log(T))^2
ight\}
ight)$ , showing effectiveness in both stochastic and adversarial environments.
CORe uses randomization to explore bandit problems without external noise.
problem Exploration in stochastic bandit problems.
method Randomizes past observations to exploit variance in rewards.
result Achieves i l d e O ( d n log K ) ilde O(d\sqrt{n\log K}) i l d e O ( d n log K ) regret bound in stochastic linear bandits.