arXiv research

UCB algorithm adapted for large-scale, non-sub-Gaussian problems.

Improved UCB method for stochastic bandits using distance tuning.

UCB exploration improves best arm identification in fixed-budget settings.

We show how an ensemble of $Q^*$-functions can be leveraged for more effective exploration in deep reinforcement learning. We build on well established algorithms from the bandit setting, and adapt them to the $Q$-learning setting. We propose an exploration strategy based on upper-confidence bounds (UCB). Our experimen…

Proposes EE-Net for neural exploration in contextual bandits.

A new algorithm learns from raw feature vectors using deep neural networks and UCB for exploration.

Greedy algorithms outperform UCB in many-armed bandit problems.

A fundamental question in reinforcement learning is whether model-free algorithms are sample efficient. Recently, Jin et al. \cite{jin2018q} proposed a Q-learning algorithm with UCB exploration policy, and proved it has nearly optimal regret bound for finite-horizon episodic MDP. In this paper, we adapt Q-learning with…

New algorithm balances exploration cost between groups in multi-armed bandits.

Improved Bayesian optimisation method using randomised Gaussian process UCB.

New algorithm explores reinforcement learning with noisy data.

Real-world networks such as social and communication networks are too large to be observed entirely. Such networks are often partially observed such that network size, network topology, and nodes of the original network are unknown. In this paper we formalize the Adaptive Graph Exploring problem. We assume that we are …

BaNk-UCB tackles batched nonparametric bandits with k-NN regression and UCB.

A bandit algorithm reduces regret in noisy, communication-constrained feedback.

GP-UCB performs suboptimally under certain conditions, as shown by a new regret lower bound.

This paper is about index policies for minimizing (frequentist) regret in a stochastic multi-armed bandit model, inspired by a Bayesian view on the problem. Our main contribution is to prove that the Bayes-UCB algorithm, which relies on quantiles of posterior distributions, is asymptotically optimal when the reward dis…

Proposes a method to avoid excessive exploration in reinforcement learning.

Algorithm optimizes two objectives in bandits: minimizing regret and identifying best arm.

Paper proposes CVaR-TS for risk-constrained MAB problems.

New algorithm reduces regret bounds for Bayesian optimization with unknown hyperparameters.

In this paper we propose and explore the k-Nearest Neighbour UCB algorithm for multi-armed bandits with covariates. We focus on a setting where the covariates are supported on a metric space of low intrinsic dimension, such as a manifold embedded within a high dimensional ambient feature space. The algorithm is concept…

New algorithm prevents strategic replication in multi-armed bandit problems.

In this paper, we consider the challenge of maximizing an unknown function f for which evaluations are noisy and are acquired with high cost. An iterative procedure uses the previous measures to actively select the next estimation of f which is predicted to be the most useful. We focus on the case where the function ca…

This paper improves Bayesian optimization methods with tighter regret bounds and practical solutions.

A new method optimizes material discovery by balancing exploration and exploitation.

RAVEN-UCB addresses non-stationary MAB problems with tighter regret bounds.

Latent variable models improve RL by facilitating efficient learning and exploration.

AGG-UCB uses neural networks to optimize group behaviors in contextual bandits.

Extended analysis of Q-learning's efficiency, matching optimal regret.

New algorithm eliminates arms to minimize regret in complex bandit problems.

We study the explore-exploit tradeoff in distributed cooperative decision-making using the context of the multiarmed bandit (MAB) problem. For the distributed cooperative MAB problem, we design the cooperative UCB algorithm that comprises two interleaved distributed processes: (i) running consensus algorithms for estim…

We study the stochastic contextual bandit problem, where the reward is generated from an unknown function with additive noise. No assumption is made about the reward function other than boundedness. We propose a new algorithm, NeuralUCB, which leverages the representation power of deep neural networks and uses a neural…

Neural-$σ^2$-LinearUCB improves regret in neural contextual bandits.

MADE improves exploration in RL by maximizing deviation from explored regions.

New algorithm reduces unfairness in bandit problems by balancing exploration and exploitation.

Improved control approach for correlated bandits with better performance.

We propose $\tt RandUCB$, a bandit strategy that builds on theoretically derived confidence intervals similar to upper confidence bound (UCB) algorithms, but akin to Thompson sampling (TS), it uses randomization to trade off exploration and exploitation. In the $K$-armed bandit setting, we show that there are infinitel…

In this paper, we propose and study opportunistic bandits - a new variant of bandits where the regret of pulling a suboptimal arm varies under different environmental conditions, such as network load or produce price. When the load/price is low, so is the cost/regret of pulling a suboptimal arm (e.g., trying a suboptim…

The paper analyzes the sliding regret of stochastic bandit algorithms.

Monotonicity helps in safely optimizing unknown functions.

New approach incentivizes strategic agents to explore, making exploration almost free.

SPAQL improves RL by adaptively partitioning state-action space and learning a time-invariant policy.

In this paper, the method UCB-RS, which resorts to recommendation system (RS) for enhancing the upper-confidence bound algorithm UCB, is presented. The proposed method is used for dealing with non-stationary and large-state spaces multi-armed bandit problems. The proposed method has been targeted to the problem of the …

The paper proposes a novel upper confidence bound (UCB) procedure for identifying the arm with the largest mean in a multi-armed bandit game in the fixed confidence setting using a small number of total samples. The procedure cannot be improved in the sense that the number of samples required to identify the best arm i…

We study incentivized exploration for the multi-armed bandit (MAB) problem where the players receive compensation for exploring arms other than the greedy choice and may provide biased feedback on reward. We seek to understand the impact of this drifted reward feedback by analyzing the performance of three instantiatio…

DETC algorithm achieves asymptotic optimality in multi-armed bandit problems.

We study the stochastic multi-armed bandit problem with the graph-based feedback structure introduced by Mannor and Shamir. We analyze the performance of the two most prominent stochastic bandit algorithms, Thompson Sampling and Upper Confidence Bound (UCB), in the graph-based feedback setting. We show that these algor…

A new algorithm for better decision-making in recommendation systems.