New algorithms tackle RKHS bandits with reduced complexity and improved performance.
problem Adversarial and stochastic RKHS bandit problems with high computational complexity.
method Combining approximation theory with misspecified linear bandit methods.
result First general algorithm for adversarial RKHS bandit problem.
Unified theory for UCB policies in total and max bandit problems.
problem Order optimality of UCB policies in max bandit problems.
method Unified definition of UCB policy using oracle quantity and failure count.
result UCB policies are order optimal in both total and max bandit problems.
Study invariant Lipschitz bandits, improving regret bounds.
problem Optimizing decisions under symmetry in online settings.
method Integrates side observations using group orbits into UniformMesh algorithm.
result Improved regret bound for invariant Lipschitz bandit class.
This paper analyzes the multi-armed bandit problem using frequency-domain methods.
problem The exploration-exploitation trade-off in sequential decision-making.
method Proposes a frequency-domain analysis framework, reformulating the bandit process as a signal processing problem.
result Confidence bound term in UCB algorithm is equivalent to a time-varying gain in frequency domain.
The paper extends Thompson Sampling to infinite action spaces using information theory.
problem Addressing the limitation of finite action spaces in Thompson Sampling.
method Information-theoretic analysis, extending rate-distortion theory to infinite action spaces.
result Derives a near-optimal regret bound for bandits with infinite and continuous action spaces.
Survey and compare PAC-Bayes bounds for bandit problems.
problem Designing and evaluating bandit algorithms with strong performance guarantees.
method PAC-Bayes bounds applied to bandit problems.
result PAC-Bayes bounds useful for offline bandit algorithms, but loose for online algorithms.
Optimizes matching in weighted graphs with semi-bandit sampling.
problem Finding optimal pairings in weighted graphs with sequential sampling.
method Leverages rank-1 assumption on adjacency matrix to reduce sample complexity and regret.
result Achieves linear dependency in the number of vertices for sample complexity and regret.
This paper addresses missing covariates in stochastic linear bandits, providing a high-probability regret bound.
problem Effect of missing covariates on regret in stochastic linear bandit algorithms.
method Proposes an algorithm that provides a high-probability upper bound on regret in terms of covariate sampling probabilities.
result Regret degrades due to missingness by at most ζ m i n 2 ζ_{min}^2 ζ min 2 , where ζ m i n ζ_{min} ζ min is the minimum probability of observing covariates. MetaTS learns to explore better by meta-learning prior from bandit instances.
problem Efficient exploration in bandit problems.
method MetaThompson Sampling (MetaTS) that meta-learns the prior from bandit instances.
result MetaTS quickly adapts to unknown prior and achieves better exploration.
SOO uses bandit theory to optimize functions with limited evaluations.
problem Optimizing functions with a limited number of evaluations.
method SOO is a machine learning approach that uses domain partitioning and a tradeoff between exploration and exploitation.
result SOO provides guarantees on the returned solution and is numerically efficient.
Study on the limits of bandit learning, showing hardness and limitations.
problem Understanding the learnability of bandit learning under arbitrary reward functions.
method Investigation into which classes of reward functions are learnable and how they can be learned.
result No combinatorial dimension can characterize bandit learnability, and computational hardness is inherent.
Improved regret bounds for bandits with fixed expert advice using information theory.
problem Optimizing regret in bandit problems with fixed expert distributions.
method Information-theoretic analysis and KL-divergence measures.
result First regret bounds for EXP4 that can get arbitrarily close to zero under certain conditions.
This paper uses bandit theory and Thompson Sampling to optimize protein sequences.
problem Optimizing protein sequences using machine learning and directed evolution.
method Proposes a Thompson Sampling-guided Directed Evolution (TS-DE) framework.
result TS-DE achieves a nearly optimal Bayesian regret of order i l d e O ( d 2 M T ) ilde O(d^{2}\sqrt{MT}) i l d e O ( d 2 M T ) . Data that is gathered adaptively --- via bandit algorithms, for example --- exhibits bias. This is true both when gathering simple numeric valued data --- the empirical means kept track of by stochastic bandit algorithms are biased downwards --- and when gathering more complicated data --- running hypothesis tests on c…
The paper provides robustness guarantees for mode estimation in bandits.
problem Understanding robustness in mode estimation under adversarial data contamination.
method Simple randomization and theoretical analysis of multi-armed bandits.
result Regret guarantees for various modal bandit problems.
EVILL uses randomised perturbations to improve exploration in bandit problems.
problem Improving exploration in structured stochastic bandit problems.
method Solves for the minimiser of a linearly perturbed regularised negative log-likelihood function.
result EVILL matches the performance of Thompson-sampling-style methods in theory and practice.
Improves bandit convex optimization with gradient variations.
problem Bandit Convex Optimization with Gradient Variations.
method Refined analysis of non-consecutive gradient variation.
result Improved dimension dependence for convex and strongly convex functions.
A new method boosts exploration in bandit algorithms, reducing regret.
problem Improving exploration in bandit algorithms with bounded or unbounded rewards.
method Residual Bootstrap Exploration (ReBoot) method that injects data-driven randomness.
result Proves logarithmic regret in Gaussian multi-armed bandits with appropriate variance inflation.
The paper offers efficient algorithms for combinatorial and linear bandits using empirical process theory.
problem Optimal algorithms for combinatorial and linear bandits with practical sample complexity.
method Empirical process theory, Gaussian-width, minimizing experimental design objective.
result Sample complexity matches lower bounds, especially for combinatorial classes.
BLAE solves batched linear bandits with optimal regret and practical performance.
problem Batched linear bandit problem with limited adaptivity.
method Integrates arm elimination with regularized G-optimal design, achieving minimax optimal regret.
result Achieves minimax optimal regret in both large- K K K and small- K K K regimes with O ( log log T ) O(\log\log T) O ( log log T ) batches. New experimental design minimizes regret in bandits.
problem Minimizing regret in online stochastic linear and combinatorial bandits.
method Experimental design-based algorithm balancing information gain and reward.
result State-of-the-art finite time regret guarantees and computational efficiency.
A new framework tunes hyperparameters in real-time for contextual bandits.
problem Optimizing hyperparameters for contextual bandits in real-time.
method CDT (Continuous Dynamic Tuning) framework using Zooming TS algorithm.
result Achieves sublinear regret and performs better than existing methods.
This paper explores a new form of the linear bandit problem in which the algorithm receives the usual stochastic rewards as well as stochastic feedback about which features are relevant to the rewards, the latter feedback being the novel aspect. The focus of this paper is the development of new theory and algorithms fo…
Neural Thompson Sampling uses deep neural networks for contextual bandit problems.
problem Solving contextual multi-armed bandit problems.
method Adapts deep neural networks for exploration and exploitation in a novel posterior distribution.
result Guaranteed cumulative regret of O ( T 1 / 2 ) \mathcal{O}(T^{1/2}) O ( T 1/2 ) for bounded reward functions. Algorithm adapts to non-stationary rewards without prior knowledge.
problem Optimizing decisions in non-stationary environments without prior knowledge of changes.
method Optimization-based algorithm that restarts when non-stationarity is detected.
result Achieves tighter dynamic regret bound and is nearly minimax optimal.
The paper reveals that baselines significantly impact RL algorithms' convergence.
problem Understanding the true impact of baselines on policy optimization.
method Theoretical analysis of bandit and RL problems, focusing on natural policy gradient and EXP3.
result Baselines can determine algorithm convergence, contradicting traditional optimization theory.
A new algorithm reduces inference error in adaptive contextual bandits.
problem Challenges in statistical inference for adaptive contextual bandits.
method Proposes a regularized EXP4 algorithm that satisfies the Lai-Wei stability condition.
result Valid Wald-type confidence intervals for linear functionals can be achieved without the price of adaptivity.
Improves A/B testing power using a two-armed bandit framework.
problem Comparing outcomes under a new policy to a control.
method Doubly robust estimation, two-armed bandit framework, permutation-based method.
result Superior performance in A/B testing compared to existing methods.
New meta-learning framework for minimizing simple regret in bandits.
problem Minimizing simple regret in a sequence of bandit tasks with unknown distributions.
method Developed Bayesian and frequentist meta-learning algorithms for bandits, analyzing their meta simple regret.
result Bayesian algorithm achieves i l d e O ( m / n ) ilde{O}(m / \sqrt{n}) i l d e O ( m / n ) meta simple regret, frequentist algorithm i l d e O ( m n + m / n ) ilde{O}(\sqrt{m} n + m/ \sqrt{n}) i l d e O ( m n + m / n ) . New algorithm controls systems with unknown, changing losses.
problem Control systems with adversarial perturbations and unknown loss function.
method Efficient sublinear regret algorithm for bandit convex optimization with memory.
result Achieves efficient control with sublinear regret in the presence of unknown, changing losses.
New algorithm reduces semi-bandit regret using covariance estimates.
problem Complexity of semi-bandits due to joint distribution of outcomes.
method Develops a new sub-exponential distribution family and an algorithm using covariance estimates.
result Proves a new lower bound on expected regret and constructs an algorithm with asymptotic analysis.
The paper develops a theory for identifying the best arm in non-parametric multi-armed bandits with a fixed budget.
problem Identifying the best arm in non-parametric multi-armed bandits with a limited number of trials.
method The paper proposes upper and lower bounds on the average log-probability of misidentification using information-theoretic quantities and a refined analysis of the successive-rejects strategy.
result The paper provides new upper and lower bounds on the average log-probability of misidentification, which generalize existing bounds.
Study ranking in generalized linear bandits with position and item dependencies.
problem Complex reward function due to position and item dependencies in recommendation systems.
method Model position and item dependencies, design UCB and Thompson Sampling algorithms.
result Generalizes existing studies in position dependencies and graph theory.
New algorithms improve contextual bandit performance by adapting to problem difficulty.
problem Improving contextual bandit performance on problems with varying difficulty.
method Introducing complexity measures and oracle-efficient algorithms.
result Achieves optimal instance-dependent regret bounds for rich policy classes.
Thompson Sampling tackles noisy context in stochastic bandits.
problem Designing an action policy for noisy, corrupted contexts in stochastic bandits.
method Introducing a Thompson Sampling algorithm for Gaussian bandits with Gaussian context noise, adopting an information-theoretic analysis.
result Demonstrates the Bayesian regret of the proposed algorithm concerning the oracle's action policy.
Practical algorithm for contextual bandits with large action spaces.
problem Efficient algorithms for decision making in large, continuous action spaces.
method Uses computational oracles for supervised learning and optimization over the action space.
result Achieves sample complexity, runtime, and memory independent of the size of the action space.
The design of personalized incentives or recommendations to improve user engagement is gaining prominence as digital platform providers continually emerge. We propose a multi-armed bandit framework for matching incentives to users, whose preferences are unknown a priori and evolving dynamically in time, in a resource c…
Improved regret bounds for DP-KLUCB and DP-IMED in Bernoulli bandits.
problem Minimizing regret in stochastic bandits under ε-global Differential Privacy.
method Developed DP versions of KLUCB and IMED, proving tighter lower bounds and matching upper bounds.
result DP-KLUCB and DP-IMED achieve asymptotically optimal regret under ε-global DP.
Improved Thompson Sampling reduces regret in contextual bandits and reinforcement learning.
problem Thompson Sampling's exploration is insufficient in some contexts.
method Developed Feel-Good Thompson Sampling to address exploration issues.
result Feel-Good Thompson Sampling reduces regret compared to standard Thompson Sampling.
New algorithms balance collaboration and adversarial behavior in linear bandits.
problem Minimizing regret in a collaborative linear bandit problem with adversarial agents.
method Robust collaborative phased elimination algorithm with tight analyses.
result Achieves near-optimal regret bounds of $O\left(α+ 1/\sqrt{M}
ight) \sqrt{dT}$ for good agents.
New approach limits regret in non-stationary bandits.
problem Understanding worst case regret in time-varying bandits.
method Belief inertia argument to resist new evidence after changes.
result Linear worst case regret for classical and restarting algorithms.
We investigate the optimality of perturbation based algorithms in the stochastic and adversarial multi-armed bandit problems. For the stochastic case, we provide a unified regret analysis for both sub-Weibull and bounded perturbations when rewards are sub-Gaussian. Our bounds are instance optimal for sub-Weibull pertur…
We study dynamic allocation problems for discrete time multi-armed bandits under uncertainty, based on the the theory of nonlinear expectations. We show that, under strong independence of the bandits and with some relaxation in the definition of optimality, a Gittins allocation index gives optimal choices. This involve…
A UCB algorithm reduces regret in cooperative multi-agent graph bandits.
problem Cooperative multi-agent decision-making on a graph with shared rewards.
method Upper Confidence Bound (UCB) algorithm for minimizing regret.
result The Multi-G-UCB algorithm achieves expected regret O ( γ N log ( T ) [ K T + D K ] ) O(γN\log(T)[\sqrt{KT} + DK]) O ( γ N log ( T ) [ K T + D K ]) . New algorithm uses control variates to improve multi-armed bandit performance.
problem Stochastic multi-armed bandits with auxiliary reward information.
method Developed UCB-CV algorithm using control variates for mean estimation.
result UCB-CV algorithm provides tighter confidence bounds and smaller variance.
New algorithms improve contextual bandits with neural networks and energy models.
problem Inefficient exploration in non-linear models for contextual bandits.
method Maximum entropy exploration using neural networks and energy models.
result Both techniques outperform standard algorithms, with energy models best overall.
New method shows multi-objective bandits are not harder than single-objective ones.
problem Comparing multi-objective bandits to single-objective ones.
method Upper and lower confidence-bound estimators for every arm-objective pair, using top-two races and uncertainty-greedy rule.
result Achieves Pareto regret of \(O(
icefrac{\log T}{g^\dagger})\), matching lower bound of \(Ω(
icefrac{\log T}{g^\dagger})\).
Reciprocal learning unifies various machine learning algorithms.
problem Understanding the convergence of machine learning algorithms.
method Introducing reciprocal learning as a generalization of various algorithms using decision theory.
result Reciprocal learning algorithms converge at linear rates to an approximately optimal model under certain conditions.