arXiv research

Study nonparametric contextual bandits with batched updates, achieving optimal regret.

GPE algorithm optimizes nonparametric contextual bandits with efficient regret bounds.

BaNk-UCB tackles batched nonparametric bandits with k-NN regression and UCB.

Develops locally private methods for nonparametric contextual bandits.

SPARKLE handles high-dimensional covariates for online decision-making.

We here adopt Bayesian nonparametric mixture models to extend multi-armed bandits in general, and Thompson sampling in particular, to scenarios where there is reward model uncertainty. In the stochastic multi-armed bandit, the reward for the played arm is generated from an unknown distribution. Reward uncertainty, i.e.…

This work optimizes identifying good arms in nonparametric multi-armed bandits.

New method tracks significant shifts in nonparametric bandits.

New method reduces regret in nonparametric bandits with unknown covariate shifts.

Paper tackles transfer learning for contextual multi-armed bandits under covariate shift.

Study sparsity benefits in infinite feature contextual bandits.

We analyze the $K$-armed bandit problem where the reward for each arm is a noisy realization based on an observed context under mild nonparametric assumptions. We attain tight results for top-arm identification and a sublinear regret of $\widetilde{O}\Big(T^{\frac{1+D}{2+D}}\Big)$, where $D$ is the context dimension, f…

Paper solves nonparametric contextual bandit with unbounded contexts.

New method clusters items with bandit feedback without parametric assumptions.

New study reveals a polynomial penalty for adapting to unknown margin parameters in batched nonparametric bandits.

Develops new methods for risk-aware decision-making in medical bandits.

Consider a nonparametric contextual multi-arm bandit problem where each arm $a \in [K]$ is associated to a nonparametric reward function $f_a: [0,1] \to \mathbb{R}$ mapping from contexts to the expected reward. Suppose that there is a large set of arms, yet there is a simple but unknown structure amongst the arm reward…

New algorithm reduces regret for single-index bandits to nearly optimal.

Improved regret for single-index bandits with optimal algorithm.

We study a multi-armed bandit problem with covariates in a setting where there is a possible delay in observing the rewards. Under some mild assumptions on the probability distributions for the delays and using an appropriate randomization to select the arms, the proposed strategy is shown to be strongly consistent.

Optimizes reward learning design for complex tasks using nonparametric methods.

Proposes a new semi-parametric framework for batched bandits with covariates.

Optimal strategy proposed for maximizing cumulative reward in continuum-armed bandits.

We study a nonparametric contextual bandit problem where the expected reward functions belong to a Hölder class with smoothness parameter $β$. We show how this interpolates between two extremes that were previously studied in isolation: non-differentiable bandits ($β\leq1$), where rate-optimal regret is achieved by run…

New algorithm reduces contextual bandits to efficient regression.

We consider the stochastic contextual bandit problem with additional regularization. The motivation comes from problems where the policy of the agent must be close to some baseline policy which is known to perform well on the task. To tackle this problem we use a nonparametric model and propose an algorithm splitting t…

A new algorithm reduces regret in high-dimensional online learning problems.

We consider a multi-armed bandit problem in a setting where each arm produces a noisy reward realization which depends on an observable random covariate. As opposed to the traditional static multi-armed bandit problem, this setting allows for dynamically changing rewards that better describe applications where side inf…

Nonparametric Thompson Sampling achieves optimal regret for risk-averse bandits with sub-Gaussian rewards.

Real-world networks such as social and communication networks are too large to be observed entirely. Such networks are often partially observed such that network size, network topology, and nodes of the original network are unknown. In this paper we formalize the Adaptive Graph Exploring problem. We assume that we are …

New method tackles high-dimensional contextual bandits with flexible kernel models.

PEAK tests means of multiple data streams with sequential betting.

Eluder dimension and information gain are equivalent for reproducing kernel Hilbert spaces.

We investigate contextual online learning with nonparametric (Lipschitz) comparison classes under different assumptions on losses and feedback information. For full information feedback and Lipschitz losses, we design the first explicit algorithm achieving the minimax regret rate (up to log factors). In a partial feedb…

Study explores strategies for randomized allocation in delayed rewards bandits.

Kernelized bandit algorithm tackles adaptive contextual bandits with single-index models.

We propose nonparametric methods for individual calibration in regression models.

This paper proposes GProp, a deep reinforcement learning algorithm for continuous policies with compatible function approximation. The algorithm is based on two innovations. Firstly, we present a temporal-difference based method for learning the gradient of the value-function. Secondly, we present the deviator-actor-cr…

Method learns evolving policies in healthcare contexts.

We consider online learning for minimizing regret in unknown, episodic Markov decision processes (MDPs) with continuous states and actions. We develop variants of the UCRL and posterior sampling algorithms that employ nonparametric Gaussian process priors to generalize across the state and action spaces. When the trans…

Develops algorithms to optimize machine replacement schedules using operational data.

A new method learns the optimal pricing map for semiparametric dynamic pricing problems.

New methods for anytime-valid off-policy inference in contextual bandits.

We consider the problem of adaptively placing sensors along an interval to detect stochastically-generated events. We present a new formulation of the problem as a continuum-armed bandit problem with feedback in the form of partial observations of realisations of an inhomogeneous Poisson process. We design a solution m…

The sample mean is among the most well studied estimators in statistics, having many desirable properties such as unbiasedness and consistency. However, when analyzing data collected using a multi-armed bandit (MAB) experiment, the sample mean is biased and much remains to be understood about its properties. For exampl…

New algorithm tackles smooth online learning with optimal regret.

Bayesian nonparametrics adapt model complexity to diverse datasets.

NP-iMCMC algorithm for nonparametric models in universal PPLs.