arXiv research

GP-UCB performs suboptimally under certain conditions, as shown by a new regret lower bound.

A classic setting of the stochastic K-armed bandit problem is considered in this note. In this problem it has been known that KL-UCB policy achieves the asymptotically optimal regret bound and KL-UCB+ policy empirically performs better than the KL-UCB policy although the regret bound for the original form of the KL-UCB…

A new method optimizes robustness measures under input uncertainty using randomized Gaussian process upper confidence bound.

A new UCB algorithm for heavy-tailed bandits with near-optimal regret.

Unified theory for UCB policies in total and max bandit problems.

New findings show many popular bandit algorithms are unstable, contradicting minimax optimality.

Unified proof for various bandit algorithms with logarithmic regret.

We consider the sequential Bayesian optimization problem with bandit feedback, adopting a formulation that allows for the reward function to vary with time. We model the reward function using a Gaussian process whose evolution obeys a simple Markov model. We introduce two natural extensions of the classical Gaussian pr…

Improved UCB method for stochastic bandits using distance tuning.

This paper improves Bayesian optimization methods with tighter regret bounds and practical solutions.

Greedy algorithms outperform UCB in many-armed bandit problems.

New algorithm robust to outliers in Bayesian Optimization.

BaNk-UCB tackles batched nonparametric bandits with k-NN regression and UCB.

UCB algorithm's arm-sampling behavior is revealed, leading to new insights and proofs.

ET-GP-UCB optimizes time-varying functions without knowing change rates.

This paper is about index policies for minimizing (frequentist) regret in a stochastic multi-armed bandit model, inspired by a Bayesian view on the problem. Our main contribution is to prove that the Bayes-UCB algorithm, which relies on quantiles of posterior distributions, is asymptotically optimal when the reward dis…

This paper improves GP-UCB by using a shifted exponential distribution for confidence parameters.

In this work, we address the open problem of finding low-complexity near-optimal multi-armed bandit algorithms for sequential decision making problems. Existing bandit algorithms are either sub-optimal and computationally simple (e.g., UCB1) or optimal and computationally complex (e.g., kl-UCB). We propose a boosting a…

TS-UCB improves Thompson Sampling with minimal extra computation.

First private Bayesian optimization algorithm with provable performance.

New UCB-type algorithms reduce regret bounds for stochastic bandits with heavy and super heavy noise.

Optimal mixtures of generative models outperform individual models on image datasets.

Safe Gaussian Process Bandit Optimization with sub-linear regret bounds.

We consider a novel setting of zeroth order non-convex optimization, where in addition to querying the function value at a given point, we can also duel two points and get the point with the larger function value. We refer to this setting as optimization with dueling-choice bandits since both direct queries and duels a…

UCB algorithm adapted for large-scale, non-sub-Gaussian problems.

UCB algorithms improve on bandit problems with precise regret analysis and adaptive inference.

Multi-armed bandit (MAB) is a class of online learning problems where a learning agent aims to maximize its expected cumulative reward while repeatedly selecting to pull arms with unknown reward distributions. We consider a scenario where the reward distributions may change in a piecewise-stationary fashion at unknown …

Fine-grained gap-dependent regret bounds for reinforcement learning.

UCB exploration improves best arm identification in fixed-budget settings.

Optimized Sharpe Ratio for better risk-adjusted decision-making in multi-armed bandits.

In this paper we propose and explore the k-Nearest Neighbour UCB algorithm for multi-armed bandits with covariates. We focus on a setting where the covariates are supported on a metric space of low intrinsic dimension, such as a manifold embedded within a high dimensional ambient feature space. The algorithm is concept…

The paper analyzes the sliding regret of stochastic bandit algorithms.

A new method optimizes material discovery by balancing exploration and exploitation.

I introduce and analyse an anytime version of the Optimally Confident UCB (OCUCB) algorithm designed for minimising the cumulative regret in finite-armed stochastic bandits with subgaussian noise. The new algorithm is simple, intuitive (in hindsight) and comes with the strongest finite-time regret guarantees for a hori…

New UCB algorithm for learning PSRs with tractable computation and accuracy.

We consider a variant of the classic multi-armed bandit problem where the expected reward of each arm is a function of an unknown parameter. The arms are divided into different groups, each of which has a common parameter. Therefore, when the player selects an arm at each time slot, information of other arms in the sam…

GP-UCB resolves sublinear regret for kernelized bandits.

A new algorithm for competing agents in a two-sided market setting.

Algorithm optimizes two objectives in bandits: minimizing regret and identifying best arm.

A new algorithm for better decision-making in recommendation systems.

UCB-Advantage learns MDPs with $O(\sqrt{H^2SAT})$ regret.

New algorithm outperforms existing ones in multi-player bandit problems without sensing.

New algorithm reduces regret bounds for Bayesian optimization with unknown hyperparameters.

UCB algorithm provides stable sample means for sequential data.

Optimized bandit algorithms have heavy-tailed regret distributions that can grow faster than expected.

New algorithm optimizes Hölder smooth functions in RKHS with tighter regret bounds.

Paper improves regret bounds for Gaussian process upper confidence bound in Bayesian optimization.

AGG-UCB uses neural networks to optimize group behaviors in contextual bandits.