Gaussian prior and likelihood improve bandit learning performance.
problem Improving bandit learning with misspecified Gaussian distributions.
method An agent with a bounded information ratio interacts with a Bernoulli bandit based on a Gaussian prior and likelihood.
result The regret increase is at most linear in the square-root of the time horizon for diffuse distributions.
Paper solves stochastic contextual linear bandits using linear bandit algorithms.
problem Stochastic contextual linear bandits with unknown context distribution.
method Establishes a reduction framework to convert to linear bandit problems.
result Achieves nearly optimal regret bound of O ( d T log T ) O(d\sqrt{T\log T}) O ( d T log T ) . New algorithm reduces individual regret and communication costs in cooperative bandits.
problem Optimal individual and group regret in cooperative multi-agent bandits.
method Integrates a new communication policy into a learning algorithm.
result Achieves optimal individual regret and constant communication costs.
Paper designs a bandit algorithm without reward distribution info.
problem Designing bandit algorithms without reward distribution info.
method Alternates between greedy rule and forced exploration.
result Achieves substantial regret upper bounds.
New bandit algorithms improve sparse reward learning.
problem Sparse rewards hinder learning efficiency in real-world bandit applications.
method Developed algorithms based on Upper Confidence Bound and Thompson Sampling for zero-inflated distributions.
result Empirical performance of new algorithms is superior to existing methods.
LMC-TS uses MCMC for efficient posterior sampling in contextual bandits.
problem Efficiency of Thompson sampling for high-dimensional contextual bandits.
method Langevin Monte Carlo for direct posterior sampling.
result LMC-TS achieves sublinear regret bound for linear contextual bandits.
New algorithm reduces worst-case regret for heavy-tailed bandits.
problem Stochastic Multi-Armed Bandit problem with heavy-tailed rewards.
method Modified minimax policy MOSS with saturated empirical mean.
result Worst-case regret matching lower bound for heavy-tailed distributions.
Algorithms for hyperparameter optimization abound, all of which work well under different and often unverifiable assumptions. Motivated by the general challenge of sequentially choosing which algorithm to use, we study the more specific task of choosing among distributions to use for random hyperparameter optimization.…
Paper confirms Feldman's conjecture on two-armed bandit problem.
problem Two-armed bandit problem with general distributions and utility functions.
method Obtained necessary and sufficient condition for myopic strategy optimality.
result Myopic strategy stochastically maximizes wins in Bernoulli bandit problems.
Greedy algorithm achieves sublinear regret for various distributions.
problem Efficient performance of greedy algorithms in linear contextual bandit problems.
method Introduced Local Anti-Concentration (LAC) condition to ensure sublinear regret.
result Greedy algorithm achieves O ( poly log T ) O(\operatorname{poly} \log T) O ( poly log T ) cumulative expected regret. The target of X \mathcal{X} X -armed bandit problem is to find the global maximum of an unknown stochastic function f f f , given a finite budget of n n n evaluations. Recently, X \mathcal{X} X -armed bandits have been widely used in many situations. Many of these applications need to deal with large-scale data sets. To deal with…
A new framework for structured bandits using influence diagrams and variational Thompson sampling.
problem Complex statistical dependencies in structured bandit problems.
method Influence diagram framework, variational Thompson sampling, tracking structured posterior distribution.
result Empirically evaluated algorithms perform as well as or better than existing baselines.
A new policy for contextual bandits adapts to reward vector shifts.
problem Learning under reward vector shifts with ordered rewards.
method Adaptive-discretization and optimistic elimination policy.
result Established upper bounds on preference-based regret.
Efficient bandit exploration for various distributions without distribution-specific tuning.
problem Optimizing exploration in multi-armed bandit models for different distributions.
method Sub-sampling Duelling Algorithms (SDA) with Random Block sampling for efficient exploration.
result Achieves asymptotically optimal regret for Bernoulli, Gaussian, and Poisson distributions.
Bandit algorithms struggle with consistent performance and robustness.
problem Achieving consistent and robust performance in stochastic multi-armed bandit settings.
method Analyzing regret minimization trade-offs and proposing distribution-oblivious algorithms.
result Logarithmic regret is inconsistent and super-logarithmic regret is necessary for consistent learning.
Optimal algorithm for contextual bandits with unknown context distributions.
problem Designing efficient algorithms for contextual bandits with unknown context distributions.
method Cross-learning setting, novel technique for coordinating multiple epochs.
result Nearly tight regret bound of O ~ ( T K ) \widetilde{O}(\sqrt{TK}) O ( T K ) for learning to bid in first-price auctions and sleeping bandits. New trade-off found in bandit problems with unknown range.
problem Stochastic bandit problems with unknown range.
method Exhibit a strategy achieving new trade-off between distribution-dependent and distribution-free regret bounds.
result Achieves the rates for regret indicated by the new trade-off.
New definition resolves ambiguity in non-stationary bandit classification.
problem Ambiguity in classifying non-stationary bandits using existing definitions.
method Introducing a formal definition that resolves ambiguity and provides a unified approach.
result Unified approach applicable to both Bayesian and frequentist formulations, resolves classification issues.
Paper analyzes FTPL's effectiveness in combinatorial semi-bandit problems.
problem Optimizing FTPL policy in combinatorial semi-bandit problems.
method Geometric resampling (GR) and conditional geometric resampling (CGR) for FTPL in semi-bandit setting.
result FTPL achieves optimal regret bounds in both Fréchet and Pareto distributions.
This paper improves offline contextual bandits using distributional robustness.
problem Improving offline contextual bandits with robustness.
method Extends Distributionally Robust Optimization (DRO) for offline contextual bandits, introducing a convex reformulation of Counterfactual Risk Minimization.
result Automatic calibration of asymptotic confidence intervals for policy optimization.
This paper applies Thompson Sampling to asymmetric α \alpha α -stable bandits for financial and wireless data.
problem Optimizing exploration-exploitation in multi-armed bandits with asymmetric α \alpha α -stable distributions. method Thompson Sampling applied to unknown asymmetric α \alpha α -stable reward distributions. result Demonstrates effectiveness of Thompson Sampling for asymmetric α \alpha α -stable bandits. Algorithm optimizes collaborative learning among distributed clients using kernel-based bandits.
problem Optimizing personalized objectives in a distributed system with limited global information.
method Kernel-based bandit framework with surrogate Gaussian process models, sparse approximations.
result Order-optimal regret performance (up to polylogarithmic factors) and reduced communication overhead.
The paper improves bandit algorithms by incorporating random-effect models.
problem Improving statistical efficiency in multi-armed bandit problems with misspecified priors.
method Introduces a random-effect model to bandits, estimating arm means and designing a UCB algorithm ReUCB.
result Derives an upper bound on the Bayes regret of ReUCB, showing improved performance over Thompson sampling.
Unified framework for distributional regret in bandits and reinforcement learning.
problem Characterizing the distribution of regret in multi-armed bandits and reinforcement learning.
method Unified framework with a UCBVI-style algorithm and distributional regret bounds.
result Distributional regret bounds with optimal trade-offs between expected and distributional regret.
New algorithms improve dueling bandit performance in multiplayer settings.
problem Challenges in collaborative exploration of non-informative arm pairs in multiplayer dueling bandits.
method Demonstrated Follow Your Leader approach and message-passing fully distributed protocol.
result Multiplayer algorithms outperform single-player benchmarks.
Proposes a general method to derive regret bounds for multi-armed bandit algorithms.
problem Deriving regret bounds for randomized multi-armed bandit algorithms.
method Checking sufficient conditions on sampling probabilities and distributions.
result Proves logarithmic regret bounds for various bandit algorithms and new models.
In this paper we propose a multi-armed bandit inspired, pool based active learning algorithm for the problem of binary classification. By carefully constructing an analogy between active learning and multi-armed bandits, we utilize ideas such as lower confidence bounds, and self-concordant regularization from the multi…
A distributed algorithm reduces communication cost in linear bandits to near-optimal levels.
problem Cooperative linear bandit optimization with stochastic contexts.
method DisBE-LUCB algorithm, DecBE-LUCB algorithm, sharing information through a central server or immediate neighbors.
result Communication cost of DisBE-LUCB matches information-theoretic lower bound up to logarithmic factors.
Unified formulation bridges adversarial and nonstationary bandits.
problem Handling time-varying reward distributions in multi-armed bandit problems.
method Unified oracle that switches between adversarial and nonstationary bandit oracles based on window size.
result Optimal regret achieved with matching lower bound.
New Thompson sampling algorithm reduces regret for exponential family bandits.
problem Minimizing regret in multi-armed bandit problems with exponential family rewards.
method Proposes ExpTS and ExpTS + ^+ + algorithms using novel sampling distributions. result Minimizes both finite-time and asymptotic regret for exponential family rewards.
DPPS uses DP priors for Bayesian non-parametric multi-arm bandits.
problem Optimizing multi-arm bandit environments with prior beliefs.
method Bayesian non-parametric algorithm based on Dirichlet Process priors.
result DPPS provides principled incorporation of prior beliefs and is optimal in Bayesian regret setup.
This paper improves FTPL algorithm for semi-bandit problems with best-of-both-worlds guarantees.
problem Optimizing regret in adversarial and stochastic m m m -set semi-bandit problems. method Extending FTPL with geometric resampling (GR) to m m m -set semi-bandits and analyzing its performance. result FTPL with Fréchet and Pareto distributions achieves O ( m d T ) O(\sqrt{mdT}) O ( m d T ) regret in adversarial setting and logarithmic regret in stochastic setting. Paper optimizes FTPL for adversarial and stochastic bandits with specific tail distributions.
problem Optimizing Follow-the-Perturbed-Leader (FTPL) policy for bandit problems.
method Analyzes FTPL with Fréchet-type tail distributions in adversarial and stochastic settings.
result FTPL with certain Fréchet-type tail distributions achieves O ( K T ) \mathcal{O}(\sqrt{KT}) O ( K T ) regrets in adversarial bandits. New method reduces regret in nonparametric bandits with unknown covariate shifts.
problem Optimal actions depend on context, but context distributions can change over time.
method Derives new regret bounds for nonparametric bandits under covariate shifts.
result Regret bounds adaptively attainable without knowledge of shift time or magnitude.
Algorithm improves online learning in adversarial bandits.
problem Online learning in adversarial multi-armed bandits with non-uniform best arm distribution.
method Online-within-online setup, inner and outer learners, leveraging non-uniform empirical distribution of best arms.
result Improves regret bounds for non-uniform best arm distributions.
New distributions allow greedy arm selection in sparse bandit problems.
problem Sparse contextual bandit problem with sparse parameters and feature distributions.
method Introduced new distribution classes and demonstrated that mixtures of these distributions are also greedy-applicable.
result Greedy algorithm applicable to a wider range of arm feature distributions, including those with origin-asymmetric support.
In this paper, we propose a Double Thompson Sampling (D-TS) algorithm for dueling bandit problems. As indicated by its name, D-TS selects both the first and the second candidates according to Thompson Sampling. Specifically, D-TS maintains a posterior distribution for the preference matrix, and chooses the pair of arms…
New algorithm optimizes unimodal bandits using empirical divergence.
problem Optimizing decisions in multi-armed bandit problems with unimodal distributions.
method Indexed Minimum Empirical Divergence (IMED) adapted for unimodal structure.
result IMED-UB algorithm optimally exploits unimodal structure.
We extend Bayesian multi-armed bandit (MAB) algorithms beyond their original setting by making use of sequential Monte Carlo (SMC) methods. A MAB is a sequential decision making problem where the goal is to learn a policy that maximizes long term payoff, where only the reward of the executed action is observed. In the …
New meta-learning approach for bandit policies that achieve high average reward.
problem Designing bandit policies that balance between worst-case and Bayesian assumptions.
method Differentiable parameterized policies optimized using policy gradients.
result Proposed algorithm achieves low regret and is practical for various bandit problems.
We study the problem of regret minimization for distributed bandits learning, in which M M M agents work collaboratively to minimize their total regret under the coordination of a central server. Our goal is to design communication protocols with near-optimal regret and little communication cost, which is measured by the…
New algorithm for countable bandits with optimal regret.
problem Stochastic bandit problem with countably many arms.
method Fully adaptive online learning algorithm with O(log n) expected cumulative regret.
result Achieves optimal regret of O(log n) after any number of plays n.
A robust bandit algorithm uses Dirichlet sampling to minimize regret under various distributional assumptions.
problem Robustness of bandit algorithms to model misspecification.
method Dirichlet Sampling (DS) algorithm based on pairwise comparisons and re-sampling of arm observations.
result Different DS variants achieve optimal regret guarantees for bounded distributions and logarithmic regret for semi-bounded distributions.
New meta-learning framework for minimizing simple regret in bandits.
problem Minimizing simple regret in a sequence of bandit tasks with unknown distributions.
method Developed Bayesian and frequentist meta-learning algorithms for bandits, analyzing their meta simple regret.
result Bayesian algorithm achieves i l d e O ( m / n ) ilde{O}(m / \sqrt{n}) i l d e O ( m / n ) meta simple regret, frequentist algorithm i l d e O ( m n + m / n ) ilde{O}(\sqrt{m} n + m/ \sqrt{n}) i l d e O ( m n + m / n ) . New method uses offline data to improve online bandit learning, even when distributions differ.
problem Improving online bandit learning with different offline and online distributions.
method MIN-UCB policy that adapts to offline data when informative, achieving tight regret bounds.
result MIN-UCB policy outperforms UCB policy with offline data and provides tight regret bounds.
CRIMED optimizes regret in bandits with unbounded stochastic corruption.
problem Minimizing regret in bandits with arbitrary unbounded corruptions.
method Introduces CRIMED, an asymptotically-optimal algorithm for Gaussian distributions with known variance.
result Achieves exact lower bound on regret for Gaussian distributions with high corruption probability.
New algorithm reduces semi-bandit regret using covariance estimates.
problem Complexity of semi-bandits due to joint distribution of outcomes.
method Develops a new sub-exponential distribution family and an algorithm using covariance estimates.
result Proves a new lower bound on expected regret and constructs an algorithm with asymptotic analysis.
Thompson sampling used for linear bandits with normal-gamma priors.
problem Optimizing decisions in uncertain environments with linear dependencies and unknown parameters.
method Bayesian Thompson sampling with multivariate normal-gamma priors.
result Derivation of a Bayesian regret bound for the approach.