arXiv research

A new algorithm for better decision-making in recommendation systems.

UCB-V algorithm improves on UCB for MAB problems with variance estimates.

A classic setting of the stochastic K-armed bandit problem is considered in this note. In this problem it has been known that KL-UCB policy achieves the asymptotically optimal regret bound and KL-UCB+ policy empirically performs better than the KL-UCB policy although the regret bound for the original form of the KL-UCB…

The multi-armed bandit problem has been extensively studied under the stationary assumption. However in reality, this assumption often does not hold because the distributions of rewards themselves may change over time. In this paper, we propose a change-detection (CD) based framework for multi-armed bandit problems und…

UCB algorithm adapted for large-scale, non-sub-Gaussian problems.

Improved UCB method for stochastic bandits using distance tuning.

We consider the sequential Bayesian optimization problem with bandit feedback, adopting a formulation that allows for the reward function to vary with time. We model the reward function using a Gaussian process whose evolution obeys a simple Markov model. We introduce two natural extensions of the classical Gaussian pr…

Fine-grained gap-dependent regret bounds for reinforcement learning.

A new UCB algorithm for heavy-tailed bandits with near-optimal regret.

This paper improves GP-UCB by using a shifted exponential distribution for confidence parameters.

BaNk-UCB tackles batched nonparametric bandits with k-NN regression and UCB.

GP-UCB performs suboptimally under certain conditions, as shown by a new regret lower bound.

In this paper we propose and explore the k-Nearest Neighbour UCB algorithm for multi-armed bandits with covariates. We focus on a setting where the covariates are supported on a metric space of low intrinsic dimension, such as a manifold embedded within a high dimensional ambient feature space. The algorithm is concept…

A new method optimizes robustness measures under input uncertainty using randomized Gaussian process upper confidence bound.

UCB exploration improves best arm identification in fixed-budget settings.

TS-UCB improves Thompson Sampling with minimal extra computation.

UCB algorithm's arm-sampling behavior is revealed, leading to new insights and proofs.

New method uses offline data to improve online bandit learning, even when distributions differ.

A meta-UCB method combines stochastic bandit algorithms.

GP-UCB resolves sublinear regret for kernelized bandits.

Unified theory for UCB policies in total and max bandit problems.

New UCB algorithm for learning PSRs with tractable computation and accuracy.

This paper improves Bayesian optimization methods with tighter regret bounds and practical solutions.

New algorithm robust to outliers in Bayesian Optimization.

New UCB-type algorithms reduce regret bounds for stochastic bandits with heavy and super heavy noise.

Unified proof for various bandit algorithms with logarithmic regret.

We consider the problem of finitely parameterized multi-armed bandits where the model of the underlying stochastic environment can be characterized based on a common unknown parameter. The true parameter is unknown to the learning agent. However, the set of possible parameters, which is finite, is known a priori. We pr…

New algorithm prevents strategic replication in multi-armed bandit problems.

A new algorithm for competing agents in a two-sided market setting.

This paper is about index policies for minimizing (frequentist) regret in a stochastic multi-armed bandit model, inspired by a Bayesian view on the problem. Our main contribution is to prove that the Bayes-UCB algorithm, which relies on quantiles of posterior distributions, is asymptotically optimal when the reward dis…

We consider a variant of the classic multi-armed bandit problem where the expected reward of each arm is a function of an unknown parameter. The arms are divided into different groups, each of which has a common parameter. Therefore, when the player selects an arm at each time slot, information of other arms in the sam…

New findings show many popular bandit algorithms are unstable, contradicting minimax optimality.

RAVEN-UCB addresses non-stationary MAB problems with tighter regret bounds.

UCB algorithms improve on bandit problems with precise regret analysis and adaptive inference.

New algorithm outperforms existing ones in multi-player bandit problems without sensing.

Characterizes UCB algorithm performance in bandit problems.

UCB algorithm provides stable sample means for sequential data.

ET-GP-UCB optimizes time-varying functions without knowing change rates.

In this work, we address the open problem of finding low-complexity near-optimal multi-armed bandit algorithms for sequential decision making problems. Existing bandit algorithms are either sub-optimal and computationally simple (e.g., UCB1) or optimal and computationally complex (e.g., kl-UCB). We propose a boosting a…

Adaptive KL-UCB algorithm for Markov and i.i.d. rewards.

UCB-Advantage learns MDPs with $O(\sqrt{H^2SAT})$ regret.

A UCB algorithm reduces regret in cooperative multi-agent graph bandits.

First private Bayesian optimization algorithm with provable performance.

Neural-$σ^2$-LinearUCB improves regret in neural contextual bandits.

New algorithm reduces sample complexity for Top Two method.

Algorithm improves multi-armed bandit performance by transferring reward samples.

Multi-armed bandit (MAB) is a class of online learning problems where a learning agent aims to maximize its expected cumulative reward while repeatedly selecting to pull arms with unknown reward distributions. We consider a scenario where the reward distributions may change in a piecewise-stationary fashion at unknown …

A new UCB policy improves reward-cost ratio estimation in budgeted MAB.