Bayesian bandit algorithms with approximate inference improve regret bounds in stochastic linear bandits.
problem Theoretical justification for Bayesian bandit algorithms with approximate inference in stochastic linear bandits.
method Proposed a theoretical framework to analyze approximate inference impact and conducted frequentist regret analysis on LinTS and LinBUCB.
result LinTS and LinBUCB preserve their original regret upper bounds with larger constant terms in approximate inference settings.
Paper solves Bayesian bandit problem with continuous-time limit and approximate policy.
problem Finding optimal policy in Bayesian bandit problems with large horizons.
method Reformulates Bayesian bandit problem as continuous Hamilton-Jacobi-Bellman (HJB) equation and proposes approximate Bayes-optimal policy.
result Approximate Bayes-optimal policy for large horizons with constant computational cost.
Bayesian optimization improved for biased data.
problem Adversarial bias in observations, especially hidden confounders.
method Reduction to dueling bandits, information-directed sampling (IDS).
result First efficient kernelized algorithm with regret guarantees.
PAC-Bayesian analysis improves lifelong learning in multi-armed bandits.
problem Improving lifelong learning in multi-armed bandits.
method PAC-Bayesian analysis for deriving lower bounds and proposing lifelong learning algorithms.
result Proposed algorithms outperform baseline methods in lifelong multi-armed bandit problems.
Bayesian method infers contextual bandit policies robustly.
problem Inference of contextual bandit policies in small sample sizes.
method Empirical likelihood for Bayesian inference.
result Accurate uncertainty measurements and policy comparison.
DPPS uses DP priors for Bayesian non-parametric multi-arm bandits.
problem Optimizing multi-arm bandit environments with prior beliefs.
method Bayesian non-parametric algorithm based on Dirichlet Process priors.
result DPPS provides principled incorporation of prior beliefs and is optimal in Bayesian regret setup.
New algorithm minimizes Bayesian regret in offline linear bandits.
problem Minimizing Bayesian regret in offline linear bandits.
method Proposes a new algorithm that directly minimizes upper bounds on Bayesian regret using conic optimization.
result Upper bounds are tight and guarantee superior performance compared to LCB.
New bounds for Bayesian bandits show prior improves performance.
problem Improving regret bounds for Bayesian bandits.
method Upper confidence bound algorithm with finite-time logarithmic regret bounds.
result Derives O ( c Δ log n ) O(c_Δ\log n) O ( c Δ log n ) and O ( c h log 2 n ) O(c_h \log^2 n) O ( c h log 2 n ) upper bounds for Bayesian bandits. Bayesian bandits misspecification affects UX optimization, revealing new models.
problem Misspecification of value models in Bayesian bandits impacts UX optimization.
method Formulated UXO as a restless, sleeping bandit with unobserved confounders and optional stopping. Provided model extensions to address misspecifications.
result Common misspecifications lead to sub-optimal rewards, demonstrating overdispersion's effects on bandit performance.
Bayesian algorithm improves best-arm identification within fixed budget.
problem Maximizing probability of identifying optimal arm within fixed budget.
method Proposes Bayesian elimination algorithm and derives upper bound on misidentification probability.
result Upper bound on misidentification probability reflects prior quality and matches lower bound.
Survey and compare PAC-Bayes bounds for bandit problems.
problem Designing and evaluating bandit algorithms with strong performance guarantees.
method PAC-Bayes bounds applied to bandit problems.
result PAC-Bayes bounds useful for offline bandit algorithms, but loose for online algorithms.
Algorithm identifies best arm with prior info in structured bandits.
problem Bayesian fixed-budget best-arm identification in structured bandits.
method Prior-dependent allocations based on structure and prior information.
result Improved theoretical bounds and robust performance across diverse models.
IDS improves sparse linear bandits by balancing information and regret.
problem Sparse linear bandits in high-dimensional decision-making.
method Information-directed sampling (IDS) with Bayesian regret bounds and empirical Bayesian sparse posterior sampling.
result IDS nearly matches existing lower bounds and significantly reduces regret.
Thompson sampling used for linear bandits with normal-gamma priors.
problem Optimizing decisions in uncertain environments with linear dependencies and unknown parameters.
method Bayesian Thompson sampling with multivariate normal-gamma priors.
result Derivation of a Bayesian regret bound for the approach.
Thompson Sampling is at most twice as bad as any other policy in Bayesian bandit models.
problem Optimizing selection of the best arm in Bayesian bandit models with independent latent processes.
method Thompson Sampling approach applied to models with independent latent arm processes.
result Thompson Sampling makes at most twice the expected number of mistakes compared to any other policy.
Novel Bayesian approach for non-stationary linear contextual bandits.
problem Non-stationary linear contextual bandits.
method Weighted Sequential Bayesian (WSB) inference.
result Established frequentist regret guarantees for new algorithms.
NK bandits improve performance on nonlinear tasks.
problem Improving performance on nonlinear sequential decision tasks.
method Proposed NK bandits using neural kernels to guide policies.
result NK bandits achieve state-of-the-art performance.
Thompson Sampling tackles noisy context in stochastic bandits.
problem Designing an action policy for noisy, corrupted contexts in stochastic bandits.
method Introducing a Thompson Sampling algorithm for Gaussian bandits with Gaussian context noise, adopting an information-theoretic analysis.
result Demonstrates the Bayesian regret of the proposed algorithm concerning the oracle's action policy.
BIS uses bandits to efficiently sample from expensive-to-evaluate densities.
problem Sampling from computationally expensive target densities.
method Sequential selection through multi-armed bandits, optimizing sample set directly.
result BIS achieves accurate sampling with fewer evaluations than adaptive methods.
Bayesian algorithms minimize cumulative regret in decentralized multi-agent bandits.
problem Minimizing cumulative regret in a decentralized multi-agent multi-armed bandit problem.
method Proposed decentralized Bayesian multi-armed bandit framework, including Thompson Sampling and Bayes-UCB algorithms.
result Regret scales logarithmically with constants matching those of an optimal centralized agent.
This review examines bandit problems in AI using statistical methods.
problem Sequential decision-making under uncertainty in AI environments.
method Foundational models, concentration inequalities, minimax regret bounds, frequentist and Bayesian algorithms, K-armed contextual bandits, SCAB, functional data analysis.
result Exploration-exploitation trade-offs and regret analyses in various bandit problems.
ARC algorithm optimizes dynamic pricing with correlated observations.
problem Optimizing dynamic pricing with correlated and generally distributed observations.
method Extends ARC algorithm to batched bandits with generalised linear model.
result ARC algorithm outperforms alternative approaches in dynamic pricing.
EBUCB framework achieves optimal regret with bounded approximate inference error.
problem Theoretical gap between practical performance and theoretical justification of Bayesian bandit algorithms with approximate inference.
method Enhanced Bayesian Upper Confidence Bound (EBUCB) framework that accommodates bandit problems with approximate inference.
result EBUCB achieves optimal regret order O ( log T ) O(\log T) O ( log T ) under certain conditions on inference error. Gaussian prior and likelihood improve bandit learning performance.
problem Improving bandit learning with misspecified Gaussian distributions.
method An agent with a bounded information ratio interacts with a Bernoulli bandit based on a Gaussian prior and likelihood.
result The regret increase is at most linear in the square-root of the time horizon for diffuse distributions.
We extend Bayesian multi-armed bandit (MAB) algorithms beyond their original setting by making use of sequential Monte Carlo (SMC) methods. A MAB is a sequential decision making problem where the goal is to learn a policy that maximizes long term payoff, where only the reward of the executed action is observed. In the …
The paper tightens the regret rate for linear bandit problems.
problem Bayesian regret in linear bandit problems.
method Information-theoretic framework and chaining argument.
result Established a new bound with a tight rate of O ( d T ) O(d\sqrt{T}) O ( d T ) . New meta-learning framework for minimizing simple regret in bandits.
problem Minimizing simple regret in a sequence of bandit tasks with unknown distributions.
method Developed Bayesian and frequentist meta-learning algorithms for bandits, analyzing their meta simple regret.
result Bayesian algorithm achieves i l d e O ( m / n ) ilde{O}(m / \sqrt{n}) i l d e O ( m / n ) meta simple regret, frequentist algorithm i l d e O ( m n + m / n ) ilde{O}(\sqrt{m} n + m/ \sqrt{n}) i l d e O ( m n + m / n ) . New approach for off-policy learning in contextual bandits with performance guarantees.
problem Improving performance of logging policies in contextual bandits.
method PAC-Bayesian analysis of policy mixtures, providing tighter generalization bounds and tractable optimization algorithms.
result Proved tighter generalization bounds and demonstrated effectiveness in practical scenarios.
A simple uncertainty measure improves deep bandit performance.
problem Efficient exploration in complex environments with deep neural networks.
method Sample Average Uncertainty (SAU) for estimating outcome uncertainty directly.
result SAU matches the uncertainty of Thompson Sampling and its regret bounds.
We here adopt Bayesian nonparametric mixture models to extend multi-armed bandits in general, and Thompson sampling in particular, to scenarios where there is reward model uncertainty. In the stochastic multi-armed bandit, the reward for the played arm is generated from an unknown distribution. Reward uncertainty, i.e.…
New algorithm optimizes best arm identification with minimal regret.
problem Best arm identification in multi-armed bandit problems.
method Characterized Bayesian simple regret with continuity conditions of prior, proposed a simple algorithm.
result Proposed algorithm achieves rate-optimal Bayesian simple regret.
BLOB combines organic and bandit signals for better user interest estimation.
problem Combining organic and bandit signals for improved user interest estimation.
method Bayesian Latent Organic Bandit (BLOB) model using variational auto-encoders and local re-parametrization.
result BLOB outperforms organic and bandit-based methods in both organic and bandit-rich environments.
MINTS uses a minimalist Bayesian framework to tackle multi-armed bandits with structural constraints.
problem Sequential decision-making under uncertainty with complex structural constraints.
method Minimalist Bayesian framework with profile likelihood to eliminate nuisance parameters.
result MINTS achieves near-optimal regret guarantees and adapts to unimodal structure.
BFTS uses Bayesian Additive Regression Trees for improved personalized mobile health interventions.
problem Adapting to complex, non-linear user behaviors in personalized mobile health interventions.
method Bayesian Forest Thompson Sampling (BFTS) integrates Bayesian Additive Regression Trees (BART) into the exploration loop of contextual bandits.
result BFTS achieves state-of-the-art regret on tabular benchmarks and improves engagement rates by over 30% in a behavioral intervention study.
We address the problem of finding the maximizer of a nonlinear smooth function, that can only be evaluated point-wise, subject to constraints on the number of permitted function evaluations. This problem is also known as fixed-budget best arm identification in the multi-armed bandit literature. We introduce a Bayesian …
Unified Bayesian framework for efficient off-policy evaluation and learning in large action spaces.
problem Efficient off-policy evaluation and learning in systems with correlated actions.
method Unified Bayesian framework with structured priors and sDM approach.
result sDM leverages action correlations without compromising computational efficiency.
PFN-TS uses Thompson sampling with PFNs to improve contextual bandit performance.
problem Improving contextual bandit performance using Thompson sampling with prior-data fitted networks.
method PFN-TS converts PFN posterior predictives into mean-reward samples using a subsampled predictive central limit theorem.
result PFN-TS achieves the best average rank across nonlinear synthetic and OpenML classification-to-bandit benchmarks.
We consider the multi armed bandit problem in non-stationary environments. Based on the Bayesian method, we propose a variant of Thompson Sampling which can be used in both rested and restless bandit scenarios. Applying discounting to the parameters of prior distribution, we describe a way to systematically reduce the …
New method reduces GP bandit complexity while maintaining good performance.
problem Computational burden in Bayesian optimization with Gaussian processes.
method Information thresholding to compress GP posterior and reduce complexity.
result Sublinear regret bounds with sublinear posterior complexity.
Bayesian metalearning improves performance in linear bandits with misspecified priors.
problem Improper priors lead to suboptimal performance in sequential decision-making.
method Proves performance bounds for metalearning priors in stochastic linear bandits and develops a metalearning algorithm.
result Metalearning can improve performance by learning the prior from multiple tasks.
Deep Bayesian Bandits improve personalized ads by balancing exploration and exploitation.
problem Recommender systems favor items already engaged by users, causing new campaigns to remain unexplored.
method Formulated as a contextual bandit, implemented exploration techniques using posterior distribution and bootstrapped models.
result Demonstrated a positive gain in exploration model in offline simulation and online AB setting.
A new algorithm uses concavity in Gaussian processes to optimize decisions in bandit problems.
problem Optimizing decisions in sequential problems with context-dependent rewards.
method Proposes a UCB algorithm using a shape-constrained reward function estimator based on a Gaussian Process model with concavity constraints.
result Derives regret bounds for the proposed UCB algorithm.
Paper proposes an efficient bandit-based algorithm for hyperparameter optimization.
problem Efficiently evaluating hyperparameters in deep learning models with large search spaces.
method Sub-Sampling (SS) algorithm combined with Bayesian Optimization (BOSS).
result Theoretical proof of optimality and empirical validation of superior performance.
We study contextual linear bandit problems under feature uncertainty, where the features are noisy and have missing entries. To address the challenges posed by this noise, we analyze Bayesian oracles given the observed noisy features. Our Bayesian analysis reveals that the optimal hypothesis can significantly deviate f…
New algorithm improves best arm identification in Bayesian settings.
problem Finding the arm with the highest mean in unknown distributions.
method Developed a variant of successive elimination algorithm.
result Achieved optimal performance in Bayesian setting with logarithmic gap.
A new Bayesian framework simplifies stochastic optimization by focusing on key parameters.
problem Bayesian methods struggle with complex structural constraints.
method Minimalist Bayesian framework that eliminates nuisance parameters via profile likelihood.
result Near-optimal regret guarantees for multi-armed bandits and convex optimization.
In this paper we propose a flexible and efficient framework for handling multi-armed bandits, combining sequential Monte Carlo algorithms with hierarchical Bayesian modeling techniques. The framework naturally encompasses restless bandits, contextual bandits, and other bandit variants under a single inferential model. …
New algorithm reduces regret in sequential decision-making problems.
problem Balancing exploration and exploitation in online sequential decision problems.
method Variational Bayesian optimistic sampling (VBOS) for optimizing policies.
result VBOS achieves i l d e O ( A T ) ilde O(\sqrt{AT}) i l d e O ( A T ) Bayesian regret for stochastic multi-armed bandits.