arXiv research

Proposes a new algorithm for graph-based semi-parametric contextual bandits.

DPPS uses DP priors for Bayesian non-parametric multi-arm bandits.

This paper refines the weighted strategy for non-stationary parametric bandits, improving regret bounds.

We study a nonparametric contextual bandit problem where the expected reward functions belong to a Hölder class with smoothness parameter $β$. We show how this interpolates between two extremes that were previously studied in isolation: non-differentiable bandits ($β\leq1$), where rate-optimal regret is achieved by run…

This paper refines the weighted strategy for non-stationary parametric bandits and MDPs, improving regret bounds.

The paper develops a theory for identifying the best arm in non-parametric multi-armed bandits with a fixed budget.

A new exploration strategy for contextual bandits reduces regret and is computationally efficient.

Efficient bandit exploration for various distributions without distribution-specific tuning.

Contextual bandit algorithms are sensitive to the estimation method of the outcome model as well as the exploration method used, particularly in the presence of rich heterogeneity or complex outcome models, which can lead to difficult estimation problems along the path of learning. We study a consideration for the expl…

We address challenges in estimating parameters from adaptively collected data.

Bayesian optimization improved for biased data.

New bandit algorithms improve sparse reward learning.

Inspired by the Reward-Biased Maximum Likelihood Estimate method of adaptive control, we propose RBMLE -- a novel family of learning algorithms for stochastic multi-armed bandits (SMABs). For a broad range of SMABs including both the parametric Exponential Family as well as the non-parametric sub-Gaussian/Exponential f…

We study a policy gradient method with L2 regularization for MAB problems.

Proposes a general method to derive regret bounds for multi-armed bandit algorithms.

Proposes a new semi-parametric framework for batched bandits with covariates.

New algorithms for best arm identification in bandits robust to misspecified parameters.

Develops confidence bounds for off-policy evaluation in contextual bandits.

Simple greedy algorithms can excel in multi-objective bandits with multiple good arms.

New method clusters items with bandit feedback without parametric assumptions.

Thompson Sampling has been demonstrated in many complex bandit models, however the theoretical guarantees available for the parametric multi-armed bandit are still limited to the Bernoulli case. Here we extend them by proving asymptotic optimality of the algorithm using the Jeffreys prior for 1-dimensional exponential …

Existing strategies for finite-armed stochastic bandits mostly depend on a parameter of scale that must be known in advance. Sometimes this is in the form of a bound on the payoffs, or the knowledge of a variance or subgaussian parameter. The notable exceptions are the analysis of Gaussian bandits with unknown mean and…

We present a new strategy for gap estimation in randomized algorithms for multiarmed bandits and combine it with the EXP3++ algorithm of Seldin and Slivkins (2014). In the stochastic regime the strategy reduces dependence of regret on a time horizon from $(\ln t)^3$ to $(\ln t)^2$ and eliminates an additive factor of o…

Nonparametric Thompson Sampling achieves optimal regret for risk-averse bandits with sub-Gaussian rewards.

New algorithms use neural networks to minimize regret in contextual bandits.

We investigate the use of bootstrapping in the bandit setting. We first show that the commonly used non-parametric bootstrapping (NPB) procedure can be provably inefficient and establish a near-linear lower bound on the regret incurred by it under the bandit model with Bernoulli rewards. We show that NPB with an approp…

In stochastic multi-armed bandits, the reward distribution of each arm is assumed to be stationary. This assumption is often violated in practice (e.g., in recommendation systems), where the reward of an arm may change whenever is selected, i.e., rested bandit setting. In this paper, we consider the non-parametric rott…

Efficiently clusters data sequences with varying distributions.

Many efficient algorithms with strong theoretical guarantees have been proposed for the contextual multi-armed bandit problem. However, applying these algorithms in practice can be difficult because they require domain expertise to build appropriate features and to tune their parameters. We propose a new method for the…

Unified framework for human-like decision making in various sequential tasks.

We consider $K$-armed stochastic bandits and consider cumulative regret bounds up to time $T$. We are interested in strategies achieving simultaneously a distribution-free regret bound of optimal order $\sqrt{KT}$ and a distribution-dependent regret that is asymptotically optimal, that is, matching the $κ\ln T$ lower b…

Improved theoretical guarantees for Top Two algorithms.

New findings show optimization is crucial for OPL in large action spaces.

New algorithm reduces best-in-class regret in contextual bandits.

A federated learning algorithm tackles unknown contexts in multi-arm bandits.

BaNk-UCB tackles batched nonparametric bandits with k-NN regression and UCB.

New algorithm for linear bandits tackles Optimal Transport problems.

Motivated by the application of real-time pricing in e-commerce platforms, we consider the problem of revenue-maximization in a setting where the seller can leverage contextual information describing the customer's history and the product's type to predict her valuation of the product. However, her true valuation is un…

New algorithm tackles non-linear utility in MNL bandits with $ilde{O}(\sqrt{T})$ regret.

Improved algorithm for logistic bandits with better regret bounds.

We propose a bandit algorithm that explores by randomizing its history of rewards. Specifically, it pulls the arm with the highest mean reward in a non-parametric bootstrap sample of its history with pseudo rewards. We design the pseudo rewards such that the bootstrap mean is optimistic with a sufficiently high probabi…

Upper Confidence Bound (UCB) method is arguably the most celebrated one used in online decision making with partial information feedback. Existing techniques for constructing confidence bounds are typically built upon various concentration inequalities, which thus lead to over-exploration. In this paper, we propose a n…

This study presents two new algorithms for solving linear stochastic bandit problems. The proposed methods use an approach from non-parametric statistics called bootstrapping to create confidence bounds. This is achieved without making any assumptions about the distribution of noise in the underlying system. We present…

Algorithm BGLM-OFU minimizes regret in combinatorial causal bandits with binary models.

Efficient exploration remains a major challenge for reinforcement learning. One reason is that the variability of the returns often depends on the current state and action, and is therefore heteroscedastic. Classical exploration strategies such as upper confidence bound algorithms and Thompson sampling fail to appropri…

Method learns evolving policies in healthcare contexts.

New GLB algorithm reduces regret in non-stationary settings.

The computational costs of inference and planning have confined Bayesian model-based reinforcement learning to one of two dismal fates: powerful Bayes-adaptive planning but only for simplistic models, or powerful, Bayesian non-parametric models but using simple, myopic planning strategies such as Thompson sampling. We …