arXiv research

NeurWIN learns Whittle indices for restless bandits using deep reinforcement learning.

A simple uncertainty measure improves deep bandit performance.

Contextual multi-armed bandit problems arise frequently in important industrial applications. Existing solutions model the context either linearly, which enables uncertainty driven (principled) exploration, or non-linearly, by using epsilon-greedy exploration policies. Here we present a deep learning framework for cont…

A new algorithm learns from raw feature vectors using deep neural networks and UCB for exploration.

We study the neural-linear bandit model for solving sequential decision-making problems with high dimensional side information. Neural-linear bandits leverage the representation power of deep neural networks and combine it with efficient exploration mechanisms, designed for linear contextual bandits, on top of the last…

Neural-$σ^2$-LinearUCB improves regret in neural contextual bandits.

Deep Bayesian Bandits improve personalized ads by balancing exploration and exploitation.

We introduce a new and completely online contextual bandit algorithm called Gated Linear Contextual Bandits (GLCB). This algorithm is based on Gated Linear Networks (GLNs), a recently introduced deep learning architecture with properties well-suited to the online setting. Leveraging data-dependent gating properties of …

Neural Thompson Sampling uses deep neural networks for contextual bandit problems.

DFPV improves PCL for confounded bandit policy evaluation.

Paper proposes an efficient bandit-based algorithm for hyperparameter optimization.

Adambs adapts Adam to prioritize important training examples.

A new algorithm for bandits with hierarchical rewards.

Improves bandit convex optimization with gradient variations.

Study identifies methods for handling sparse data in web settings with contextual bandits.

Master-slave architecture tackles combinatorial multi-armed bandits with diversity constraints.

We study the stochastic contextual bandit problem, where the reward is generated from an unknown function with additive noise. No assumption is made about the reward function other than boundedness. We propose a new algorithm, NeuralUCB, which leverages the representation power of deep neural networks and uses a neural…

Study Whittle index learning algorithms for restless bandits with constant stepsizes.

Efficient exploration remains a major challenge for reinforcement learning. One reason is that the variability of the returns often depends on the current state and action, and is therefore heteroscedastic. Classical exploration strategies such as upper confidence bound algorithms and Thompson sampling fail to appropri…

New RL algorithms learn Gittins indices for unknown Markovian states.

New algorithms for neural bandits learn from context and arm features.

This paper proposes GProp, a deep reinforcement learning algorithm for continuous policies with compatible function approximation. The algorithm is based on two innovations. Firstly, we present a temporal-difference based method for learning the gradient of the value-function. Secondly, we present the deviator-actor-cr…

A new adversarial attack method using structured search and contextual bandits.

New algorithms reduce dueling bandits' regret with neural networks and efficient exploration.

CRB tackles rising rewards in combinatorial online learning.

Recent advances in deep reinforcement learning have made significant strides in performance on applications such as Go and Atari games. However, developing practical methods to balance exploration and exploitation in complex domains remains largely unsolved. Thompson Sampling and its extension to reinforcement learning…

CAE repurposes critics for efficient exploration in RL.

Modern deep learning methods are very sensitive to many hyperparameters, and, due to the long training times of state-of-the-art models, vanilla Bayesian hyperparameter optimization is typically computationally infeasible. On the other hand, bandit-based configuration evaluation approaches based on random search lack g…

TS-UCB improves Thompson Sampling with minimal extra computation.

Deep Reinforcement Learning has been shown to be very successful in complex games, e.g. Atari or Go. These games have clearly defined rules, and hence allow simulation. In many practical applications, however, interactions with the environment are costly and a good simulator of the environment is not available. Further…

We show how an ensemble of $Q^*$-functions can be leveraged for more effective exploration in deep reinforcement learning. We build on well established algorithms from the bandit setting, and adapt them to the $Q$-learning setting. We propose an exploration strategy based on upper-confidence bounds (UCB). Our experimen…

The paper analyzes Random Search and introduces BLiN-MOS for bandit learning.

Adaptive anomaly detection for IoT data reduces delay without sacrificing accuracy.

Unified framework for scalable black-box optimization.

New algorithms tackle RKHS bandits with reduced complexity and improved performance.

Unified formulation bridges adversarial and nonstationary bandits.

Performance of machine learning algorithms depends critically on identifying a good set of hyperparameters. While recent approaches use Bayesian optimization to adaptively select configurations, we focus on speeding up random search through adaptive resource allocation and early-stopping. We formulate hyperparameter op…

Paper tackles LDP bandits learning with improved results and sub-linear regret.

Paper solves stochastic contextual linear bandits using linear bandit algorithms.

SGD shows distinct phases in learning single-index models, achieving optimal sample complexity and regret.

Graph-Triggered Bandits unify rested and restless bandits with graph-defined arm interactions.

New definition resolves ambiguity in non-stationary bandit classification.

Recommendations are broadly used in marketplaces to match users with items relevant to their interests and needs. To understand user intent and tailor recommendations to their needs, we use deep learning to explore various heterogeneous data available in marketplaces. This paper focuses on the challenge of measuring re…

Unified approach for non-stationary and clustered bandits.

A framework for auto-tuning hyper-parameters in contextual bandit algorithms.

Stochastic multi-armed bandits form a class of online learning problems that have important applications in online recommendation systems, adaptive medical treatment, and many others. Even though potential attacks against these learning algorithms may hijack their behavior, causing catastrophic loss in real-world appli…

New algorithm learns optimal exploration parameters for contextual bandits.

Investigates sequential problems on graph structures and large action spaces.