arXiv research

New algorithm tackles adversarial corruption in Lipschitz bandits with sub-linear regret.

Study invariant Lipschitz bandits, improving regret bounds.

New algorithm tackles delayed feedback in Lipschitz bandits with sublinear regret.

Stochastic Lipschitz bandit algorithms balance exploration and exploitation, and have been used for a variety of important task domains. In this paper, we present a framework for Lipschitz bandit methods that adaptively learns partitions of context- and arm-space. Due to this flexibility, the algorithm is able to effic…

Algorithm tackles adaptive discretization in adversarial Lipschitz bandits for dynamic pricing and auctions.

Transfer learning for bandits with latent Lipschitz continuity.

Paper introduces risk assessment for contextual bandits without experiments.

New algorithm tracks changes in infinite action space rewards.

Improved adaptive rates for Lipschitz bandit problem.

The paper tackles multi-player information asymmetry bandits in metric spaces.

Thompson Sampling is a well established approach to bandit and reinforcement learning problems. However its use in continuum armed bandit problems has received relatively little attention. We provide the first bounds on the regret of Thompson Sampling for continuum armed bandits under weak conditions on the function cl…

A new method optimizes in nonstationary environments with many arms efficiently.

Paper tackles robust batched bandits for heavy-tailed rewards.

BLiE optimizes hyperparameters with theoretical guarantees and superior performance.

Unified framework for risk-aware policy learning in contextual bandits.

New algorithm optimizes smooth functions with Hölder exponent > 1.

New algorithms reduce regret for convex bandits with small comparator norms.

ABoB optimizes online configuration tuning by clustering parameters and accelerating learning.

Algorithm improves wildlife protection patrols.

We investigate contextual online learning with nonparametric (Lipschitz) comparison classes under different assumptions on losses and feedback information. For full information feedback and Lipschitz losses, we design the first explicit algorithm achieving the minimax regret rate (up to log factors). In a partial feedb…

The paper tackles minimax optimality in continuum contextual bandits with Hölder continuity.

We consider the Lipschitz bandit optimization problem with an emphasis on practical efficiency. Although there is rich literature on regret analysis of this type of problem, e.g., [Kleinberg et al. 2008, Bubeck et al. 2011, Slivkins 2014], their proposed algorithms suffer from serious practical problems including extre…

We investigate online classification with paid stochastic experts. Here, before making their prediction, each expert must be paid. The amount that we pay each expert directly influences the accuracy of their prediction through some unknown Lipschitz "productivity" function. In each round, the learner must decide how mu…

ECP optimizes expensive functions without knowing Lipschitz constant.

The paper analyzes Random Search and introduces BLiN-MOS for bandit learning.

Efficient algorithms for contextual bandits with smooth regret in continuous action spaces.

We study contextual bandit learning with an abstract policy class and continuous action space. We obtain two qualitatively different regret bounds: one competes with a smoothed version of the policy class under no continuity assumptions, while the other requires standard Lipschitz assumptions. Both bounds exhibit data-…

Nonparametric Thompson Sampling achieves optimal regret for risk-averse bandits with sub-Gaussian rewards.

We analyze the regret of combinatorial Thompson sampling (CTS) for the combinatorial multi-armed bandit with probabilistically triggered arms under the semi-bandit feedback setting. We assume that the learner has access to an exact optimization oracle but does not know the expected base arm outcomes beforehand. When th…

The paper extends Thompson Sampling to infinite action spaces using information theory.

We consider the closely related problems of bandit convex optimization with two-point feedback, and zero-order stochastic convex optimization with two function evaluations per round. We provide a simple algorithm and analysis which is optimal for convex Lipschitz functions. This improves on \cite{dujww13}, which only p…

Multi-armed bandits a simple but very powerful framework for algorithms that make decisions over time under uncertainty. An enormous body of work has accumulated over the years, covered in several books and surveys. This book provides a more introductory, textbook-like treatment of the subject. Each chapter tackles a p…

This paper studies continuum-armed bandits under Besov smoothness conditions and derives minimax rates.

New lower bound shows bandit convex optimization is harder than previously thought.

A bandit problem with filtered Poisson process data.

We consider the problem of maximizing a non-concave Lipschitz multivariate function over a compact domain by sequentially querying its (possibly perturbed) values. We study a natural algorithm designed originally by Piyavskii and Shubert in 1972, for which we prove new bounds on the number of evaluations of the functio…

This paper introduces and addresses a wide class of stochastic bandit problems where the function mapping the arm to the corresponding reward exhibits some known structural properties. Most existing structures (e.g. linear, Lipschitz, unimodal, combinatorial, dueling, ...) are covered by our framework. We derive an asy…

Meta-learning improves performance across similar tasks in adversarial bandit settings.

New algorithm reduces regret for single-index bandits to nearly optimal.

Improved regret for single-index bandits with optimal algorithm.

The paper tackles online learning problems with monotone arm sequences, achieving optimal or near-optimal regret bounds.

Kernelized bandit algorithm tackles adaptive contextual bandits with single-index models.

Stochastic zeroth-order (SZO), or gradient-free, optimization allows to optimize arbitrary functions by relying only on function evaluations under parameter perturbations, however, the iteration complexity of SZO methods suffers a factor proportional to the dimensionality of the perturbed function. We show that in scen…

New method tracks significant shifts in nonparametric bandits.

BaNk-UCB tackles batched nonparametric bandits with k-NN regression and UCB.

The paper tackles causal bandits with unknown SCMs and soft interventions, providing upper and lower bounds on regret.

Certified algorithms optimize functions with varying costs, providing error bounds.

We study online optimization of smoothed piecewise constant functions over the domain [0, 1). This is motivated by the problem of adaptively picking parameters of learning algorithms as in the recently introduced framework by Gupta and Roughgarden (2016). Majority of the machine learning literature has focused on Lipsc…