arXiv research

New algorithm reduces regret in multi-armed bandit problems with Gaussian rewards.

New acquisition function for extreme rewards in bandits.

Gittins indices provide an optimal solution to the classical multi-armed bandit problem. An obstacle to their use has been the common perception that their computation is very difficult. This paper demonstrates an accessible general methodology for the calculating Gittins indices for the multi-armed bandit with a detai…

We present a formal model of human decision-making in explore-exploit tasks using the context of multi-armed bandit problems, where the decision-maker must choose among multiple options with uncertain rewards. We address the standard multi-armed bandit problem, the multi-armed bandit problem with transition costs, and …

Recent work on follow the perturbed leader (FTPL) algorithms for the adversarial multi-armed bandit problem has highlighted the role of the hazard rate of the distribution generating the perturbations. Assuming that the hazard rate is bounded, it is possible to provide regret analyses for a variety of FTPL algorithms f…

The paper honors Lai's contributions to multi-armed bandits and establishes new regret bounds.

Inspired by the Reward-Biased Maximum Likelihood Estimate method of adaptive control, we propose RBMLE -- a novel family of learning algorithms for stochastic multi-armed bandits (SMABs). For a broad range of SMABs including both the parametric Exponential Family as well as the non-parametric sub-Gaussian/Exponential f…

Improved statistical efficiency of Thompson Sampling for combinatorial semi-bandits.

Optimism stabilizes Thompson Sampling for adaptive inference in multi-armed bandits.

I analyse the frequentist regret of the famous Gittins index strategy for multi-armed bandits with Gaussian noise and a finite horizon. Remarkably it turns out that this approach leads to finite-time regret guarantees comparable to those available for the popular UCB algorithm. Along the way I derive finite-time bounds…

Proposes a general method to derive regret bounds for multi-armed bandit algorithms.

Satisficing is a relaxation of maximizing and allows for less risky decision making in the face of uncertainty. We propose two sets of satisficing objectives for the multi-armed bandit problem, where the objective is to achieve reward-based decision-making performance above a given threshold. We show that these new pro…

ERTS uses Thompson sampling for Gaussian entropic risk bandits, achieving regret bounds.

We investigate the optimality of perturbation based algorithms in the stochastic and adversarial multi-armed bandit problems. For the stochastic case, we provide a unified regret analysis for both sub-Weibull and bounded perturbations when rewards are sub-Gaussian. Our bounds are instance optimal for sub-Weibull pertur…

Method predicts future rewards from past actions in a linear Gaussian system.

Upper Confidence Bound (UCB) method is arguably the most celebrated one used in online decision making with partial information feedback. Existing techniques for constructing confidence bounds are typically built upon various concentration inequalities, which thus lead to over-exploration. In this paper, we propose a n…

We here adopt Bayesian nonparametric mixture models to extend multi-armed bandits in general, and Thompson sampling in particular, to scenarios where there is reward model uncertainty. In the stochastic multi-armed bandit, the reward for the played arm is generated from an unknown distribution. Reward uncertainty, i.e.…

New BO method optimizes functions efficiently even with unknown hyperparameters.

New method tightens sub-Gaussian concentration inequalities.

GP-TS optimizes TLM pre-training hyperparameters efficiently.

KL-MS improves regret bounds for multi-armed bandits with bounded rewards.

ARTEO algorithm optimizes safety-critical systems with uncertainty.

New algorithm uses machine learning to predict rewards for decision-making problems.

In this paper, we propose a novel perturbation-based exploration method in bandit algorithms with bounded or unbounded rewards, called residual bootstrap exploration (\texttt{ReBoot}). The \texttt{ReBoot} enforces exploration by injecting data-driven randomness through a residual-based perturbation mechanism. This nove…

Study optimizes identifying the best arm with fixed rounds and Gaussian outcomes.

The stochastic multi-armed bandit problem is well understood when the reward distributions are sub-Gaussian. In this paper we examine the bandit problem under the weaker assumption that the distributions have moments of order 1+ε, for some $ε\in (0,1]$. Surprisingly, moments of order 2 (i.e., finite variance) are suffi…

Paper designs a bandit algorithm without reward distribution info.

Paper presents a new method for optimizing hyperparameters in machine learning models.

New dynamic allocation methods for multi-armed bandit models.

PAC-Bayesian analysis improves lifelong learning in multi-armed bandits.

The Knowledge Gradient (KG) policy was originally proposed for online ranking and selection problems but has recently been adapted for use in online decision making in general and multi-armed bandit problems (MABs) in particular. We study its use in a class of exponential family MABs and identify weaknesses, including …

The paper explores sampling problems and shows minimal exploration is needed.

Improved theoretical guarantees for Top Two algorithms.

Paper uses subjective logic to estimate uncertainty in multi-armed bandit problems.

A new method optimizes in nonstationary environments with many arms efficiently.

New algorithm reduces worst-case regret for heavy-tailed bandits.

Proposes an interpretable machine learning framework for multi-arm HTE estimation.

In this paper we propose a multi-armed bandit inspired, pool based active learning algorithm for the problem of binary classification. By carefully constructing an analogy between active learning and multi-armed bandits, we utilize ideas such as lower confidence bounds, and self-concordant regularization from the multi…

Study finds optimal regret bound for multi-armed bandit problem with expert advice.

A new algorithm balances global reward and group constraints in federated multi-armed bandits.

Quantum algorithms for multi-armed bandits are explored with limited reward access.

Algorithm improves multi-armed bandit performance by transferring reward samples.

New method for identifying best arm in batched multi-armed bandit problems.

A new bandit strategy improves learning speed at the cost of accuracy.

This note gives a short, self-contained, proof of a sharp connection between Gittins indices and Bayesian upper confidence bound algorithms. I consider a Gaussian multi-armed bandit problem with discount factor $γ$. The Gittins index of an arm is shown to equal the $γ$-quantile of the posterior distribution of the arm'…

Study on Pareto optimality in multi-objective bandit problems.

In this paper, we study the multi-armed bandit problem in the batched setting where the employed policy must split data into a small number of batches. While the minimax regret for the two-armed stochastic bandits has been completely characterized in \cite{perchet2016batched}, the effect of the number of arms on the re…

Sequential portfolio selection has attracted increasing interests in the machine learning and quantitative finance communities in recent years. As a mathematical framework for reinforcement learning policies, the stochastic multi-armed bandit problem addresses the primary difficulty in sequential decision making under …