arXiv research

Optimizes quadratic bandits with tight Hessian-dependent sample complexity bounds.

We consider the exploration-exploitation tradeoff in linear quadratic (LQ) control problems, where the state dynamics is linear and the cost function is quadratic in states and controls. We analyze the regret of Thompson sampling (TS) (a.k.a. posterior-sampling for reinforcement learning) in the frequentist setting, i.…

New algorithm reduces regret in bandit optimization for high-dimensional data.

Quantum algorithms for multi-armed bandits are explored with limited reward access.

In this paper, we propose the first computationally efficient projection-free algorithm for bandit convex optimization (BCO). We show that our algorithm achieves a sublinear regret of $O(nT^{4/5})$ (where $T$ is the horizon and $n$ is the dimension) for any bounded convex functions with uniformly bounded gradients. We …

SCaLE tackles dynamic regret in noisy bandit feedback with switching costs.

The problem of stochastic convex optimization with bandit feedback (in the learning community) or without knowledge of gradients (in the optimization community) has received much attention in recent years, in the form of algorithms and performance upper bounds. However, much less is known about the inherent complexity …

RONM method reduces regret in stochastic convex bandits with decreasing noise.

We consider the quantum version of the bandit problem known as {\em best arm identification} (BAI). We first propose a quantum modeling of the BAI problem, which assumes that both the learning agent and the environment are quantum; we then propose an algorithm based on quantum amplitude amplification to solve BAI. We f…

We study the problem of stochastic combinatorial pure exploration (CPE), where an agent sequentially pulls a set of single arms (a.k.a. a super arm) and tries to find the best super arm. Among a variety of problem settings of the CPE, we focus on the full-bandit setting, where we cannot observe the reward of each singl…

Study on information evolution in interactive decision making using multi-armed bandits.

New algorithms combat adversarial attacks in stochastic linear bandits.

We provide new lower bounds on the regret that must be suffered by adversarial bandit algorithms. The new results show that recent upper bounds that either (a) hold with high-probability or (b) depend on the total lossof the best arm or (c) depend on the quadratic variation of the losses, are close to tight. Besides th…

We propose a contextual-bandit approach for demand side management by offering price incentives. More precisely, a target mean consumption is set at each round and the mean consumption is modeled as a complex function of the distribution of prices sent and of some contextual variables such as the temperature, weather, …

The paper considers the problem of global optimization in the setup of stochastic process bandits. We introduce an UCB algorithm which builds a cascade of discretization trees based on generic chaining in order to render possible his operability over a continuous domain. The theoretical framework applies to functions u…

Ensemble++ uses shared-factor ensembles to scale Thompson Sampling for linear and nonlinear bandits.

Improves reward bounds for prediction with expert advice using abstention.

Quantum algorithms improve regret bounds for bandits with knapsacks.

Optimal mixtures of generative models outperform individual models on image datasets.

New L2 regularization improves softmax MAB performance.

Paper introduces a new framework for optimizing non-convex functions.

Unified analysis of kernel-based and locally adaptive bandit optimization methods.

New algorithm tackles dynamic query routing to multiple embedding models.

Develops new optimization techniques for decision-making under uncertainty.

New algorithm catches moving subspaces in bandit problems.

This work improves online regression and contextual bandits using neural networks.

New algorithm tackles nonstationary linear bandits with latent dynamics.

New method improves neural network robustness by identifying functions rather than parameters.

Algorithm minimizes control regret for non-stationary LQR systems.

We consider online learning for minimizing regret in unknown, episodic Markov decision processes (MDPs) with continuous states and actions. We develop variants of the UCRL and posterior sampling algorithms that employ nonparametric Gaussian process priors to generalize across the state and action spaces. When the trans…

We study how to adapt to smoothly-varying ('easy') environments in well-known online learning problems where acquiring information is expensive. For the problem of label efficient prediction, which is a budgeted version of prediction with expert advice, we present an online algorithm whose regret depends optimally on t…

Optimal algorithm for selecting high-quality arms from infinite bandit arms.

We design and analyze CascadeBAI, an algorithm for finding the best set of $K$ items, also called an arm, within the framework of cascading bandits. An upper bound on the time complexity of CascadeBAI is derived by overcoming a crucial analytical challenge, namely, that of probabilistically estimating the amount of ava…

We consider a partial-feedback variant of the well-studied online PCA problem where a learner attempts to predict a sequence of $d$-dimensional vectors in terms of a quadratic loss, while only having limited feedback about the environment's choices. We focus on a natural notion of bandit feedback where the learner only…

Adaptive linear bandit algorithm with best-of-three-worlds regret bounds.

Gaussian processes (GP) are a well studied Bayesian approach for the optimization of black-box functions. Despite their effectiveness in simple problems, GP-based algorithms hardly scale to high-dimensional functions, as their per-iteration time and space cost is at least quadratic in the number of dimensions $d$ and i…

The trade-off between the cost of acquiring and processing data, and uncertainty due to a lack of data is fundamental in machine learning. A basic instance of this trade-off is the problem of deciding when to make noisy and costly observations of a discrete-time Gaussian random walk, so as to minimise the posterior var…

BanditPAM clusters data faster than traditional methods.

New algorithms tackle RKHS bandits with reduced complexity and improved performance.

Unified formulation bridges adversarial and nonstationary bandits.

BestChanID identifies the channel with maximal capacity using training sequences.

Paper tackles LDP bandits learning with improved results and sub-linear regret.

Paper solves stochastic contextual linear bandits using linear bandit algorithms.

Graph-Triggered Bandits unify rested and restless bandits with graph-defined arm interactions.

New definition resolves ambiguity in non-stationary bandit classification.

Unified approach for non-stationary and clustered bandits.

A framework for auto-tuning hyper-parameters in contextual bandit algorithms.

Stochastic multi-armed bandits form a class of online learning problems that have important applications in online recommendation systems, adaptive medical treatment, and many others. Even though potential attacks against these learning algorithms may hijack their behavior, causing catastrophic loss in real-world appli…