arXiv research

The paper addresses frequentist regret of Linear Thompson Sampling in stochastic linear bandits.

A new algorithm reduces frequentist regret in multi-agent bandit problems with sparse hypergraphs.

Improved Thompson Sampling reduces regret in contextual bandits and reinforcement learning.

New meta-learning framework for minimizing simple regret in bandits.

Bayes optimal algorithm under certain conditions doesn't achieve exponential simple regret.

We consider the exploration-exploitation dilemma in finite-horizon reinforcement learning (RL). When the state space is large or continuous, traditional tabular approaches are unfeasible and some form of function approximation is mandatory. In this paper, we introduce an optimistically-initialized variant of the popula…

We consider the exploration-exploitation tradeoff in linear quadratic (LQ) control problems, where the state dynamics is linear and the cost function is quadratic in states and controls. We analyze the regret of Thompson sampling (TS) (a.k.a. posterior-sampling for reinforcement learning) in the frequentist setting, i.…

Improved regret bound for linear ensemble sampling.

New IDS algorithm refines parameter norm bounds for better bandit performance.

The paper improves bounds on regret in Gaussian process bandits.

In the stochastic bandit problem, the goal is to maximize an unknown function via a sequence of noisy evaluations. Typically, the observation noise is assumed to be independent of the evaluation point and to satisfy a tail bound uniformly on the domain; a restrictive assumption for many applications. In this work, we c…

We present two algorithms for Bayesian optimization in the batch feedback setting, based on Gaussian process upper confidence bound and Thompson sampling approaches, along with frequentist regret guarantees and numerical results.

Improved BO algorithms reduce prediction error under Gaussian noise.

New algorithms reduce regret in reinforcement learning with MNL approximations.

Unified analysis of Gaussian Process Thompson Sampling without discretization.

TSAC achieves optimal frequentist regret in adaptive control of LQRs.

I analyse the frequentist regret of the famous Gittins index strategy for multi-armed bandits with Gaussian noise and a finite horizon. Remarkably it turns out that this approach leads to finite-time regret guarantees comparable to those available for the popular UCB algorithm. Along the way I derive finite-time bounds…

Unified framework for Bayesian and Frequentist statistics.

Stochastic Rank-One Bandits (Katarya et al, (2017a,b)) are a simple framework for regret minimization problems over rank-one matrices of arms. The initially proposed algorithms are proved to have logarithmic regret, but do not match the existing lower bound for this problem. We close this gap by first proving that rank…

Improved Thompson Sampling using fractional posteriors achieves better regret bounds.

New insights into empirical Bayes and compound decision problems with improved regret bounds.

This paper is about index policies for minimizing (frequentist) regret in a stochastic multi-armed bandit model, inspired by a Bayesian view on the problem. Our main contribution is to prove that the Bayes-UCB algorithm, which relies on quantiles of posterior distributions, is asymptotically optimal when the reward dis…

We study a variant of the stochastic multi-armed bandit (MAB) problem in which the rewards are corrupted. In this framework, motivated by privacy preservation in online recommender systems, the goal is to maximize the sum of the (unobserved) rewards, based on the observation of transformation of these rewards through a…

New algorithms reduce reinforcement learning regret in factored MDPs.

In this work, we investigate black-box optimization from the perspective of frequentist kernel methods. We propose a novel batch optimization algorithm, which jointly maximizes the acquisition function and select points from a whole batch in a holistic way. Theoretically, we derive regret bounds for both the noise-free…

Bayesian framework reduces online optimization regret.

New algorithm for competing influence spread in unknown networks.

A new sequential method estimates Poisson means in streaming data, achieving optimality and efficiency.

New algorithm reduces regret for logistic bandits without $κ$ dependency.

Improved exploration in factored average-reward MDPs reduces regret.

No-regret optimization for time-varying functions using uncertainty injection.

We study, to the best of our knowledge, the first Bayesian algorithm for unimodal Multi-Armed Bandit (MAB) problems with graph structure. In this setting, each arm corresponds to a node of a graph and each edge provides a relationship, unknown to the learner, between two nodes in terms of expected reward. Furthermore, …

We tackle the problem of online reward maximisation over a large finite set of actions described by their contexts. We focus on the case when the number of actions is too big to sample all of them even once. However we assume that we have access to the similarities between actions' contexts and that the expected reward…

Logarithmic regret strategies for safe multi-armed bandits with safety risk constraints.

New definition resolves ambiguity in non-stationary bandit classification.

Bayesian bandit algorithms with approximate inference improve regret bounds in stochastic linear bandits.

This review examines bandit problems in AI using statistical methods.

New algorithms for neural bandits learn from context and arm features.

We consider stochastic multi-armed bandit problems with complex actions over a set of basic arms, where the decision maker plays a complex action rather than a basic arm in each round. The reward of the complex action is some function of the basic arms' rewards, and the feedback observed may not necessarily be the rewa…

New technique tracks geometric properties to correct base algorithms' poor performance.

Bayesian UQ matches frequentist UQ for adaptively collected data.

We consider online learning for minimizing regret in unknown, episodic Markov decision processes (MDPs) with continuous states and actions. We develop variants of the UCRL and posterior sampling algorithms that employ nonparametric Gaussian process priors to generalize across the state and action spaces. When the trans…

Improved Thompson sampling for bandits with Langevin algorithms.

Unified framework for ensemble sampling in nonlinear contextual bandits with provable regret bounds.

Novel Bayesian approach for non-stationary linear contextual bandits.

OOD-trained Bayesian neural networks perform similarly to frequentist methods in uncertainty quantification.

Improved POMDP regret to sqrt(T) with known observation model.

New TS algorithms improve performance in non-stationary multi-armed bandit problems.