arXiv research

Paper introduces risk-sensitive bandits with optimal arm mixtures.

New framework shifts bandit algorithms from expected reward to preference metrics, optimizing mixtures of arms.

New distributions allow greedy arm selection in sparse bandit problems.

This paper presents new deviation inequalities that are valid uniformly in time under adaptive sampling in a multi-armed bandit model. The deviations are measured using the Kullback-Leibler divergence in a given one-dimensional exponential family, and may take into account several arms at a time. They are obtained by c…

We here adopt Bayesian nonparametric mixture models to extend multi-armed bandits in general, and Thompson sampling in particular, to scenarios where there is reward model uncertainty. In the stochastic multi-armed bandit, the reward for the played arm is generated from an unknown distribution. Reward uncertainty, i.e.…

Optimal mixtures of generative models outperform individual models on image datasets.

We consider a scenario where an agent has multiple available strategies to explore an unknown environment. For each new interaction with the environment, the agent must select which exploration strategy to use. We provide a new strategy-agnostic method that treat the situation as a Multi-Armed Bandits problem where the…

Paper tackles best mixed arm identification with cost constraints in bandit models.

ContextWIN uses neural networks and reinforcement learning to optimize decisions in dynamic environments.

This paper presents a novel approach for incremental semiparametric inverse dynamics learning. In particular, we consider the mixture of two approaches: Parametric modeling based on rigid body dynamics equations and nonparametric modeling based on incremental kernel methods, with no prior information on the mechanical …

New method identifies latent treatment effects from proxy models.

Modeling glucose distribution changes over time using neural ODEs.

The Hierarchical Mixture of Experts (HME) is a well-known tree-based model for regression and classification, based on soft probabilistic splits. In its original formulation it was trained by maximum likelihood, and is therefore prone to over-fitting. Furthermore the maximum likelihood framework offers no natural metri…

In many biomedical, science, and engineering problems, one must sequentially decide which action to take next so as to maximize rewards. One general class of algorithms for optimizing interactions with the world, while simultaneously learning how the world operates, is the multi-armed bandit setting and, in particular,…

Deleting refusal directions from models leads to systematically more optimistic decisions.

We study a bad arm existing checking problem in which a player's task is to judge whether a positive arm exists or not among given K arms by drawing as small number of arms as possible. Here, an arm is positive if its expected loss suffered by drawing the arm is at least a given threshold. This problem is a formalizati…

New algorithms for streaming bandits with limited memory.

Study best arm identification in restless Markov multi-armed bandits with state-dependent transitions.

Adaptive algorithm selects models for stochastic linear bandits based on problem complexity.

Paper tackles identifying an odd arm in a multi-armed bandit with restless Markov processes and trembling hand.

Improved theoretical guarantees for Top Two algorithms.

We address the M-best-arm identification problem in multi-armed bandits. A player has a limited budget to explore K arms (M<K), and once pulled, each arm yields a reward drawn (independently) from a fixed, unknown distribution. The goal is to find the top M arms in the sense of expected reward. We develop an algorithm …

Sequential screening and dynamic regret in multi-armed bandits with arriving arms

Paper tackles good arm identification in stochastic bandits.

We consider the best-arm identification problem in multi-armed bandits, which focuses purely on exploration. A player is given a fixed budget to explore a finite set of arms, and the rewards of each arm are drawn independently from a fixed, unknown distribution. The player aims to identify the arm with the largest expe…

A novel algorithm reduces communication costs in federated best arm identification.

New algorithms identify best arm with less pulls, adapting to arm covariances.

Study best arm identification in restless bandits with unknown TPMs.

New method for identifying best arm in batched multi-armed bandit problems.

Optimal best-arm identification with known number of optimal arms.

We consider a novel multi-armed bandit framework where the rewards obtained by pulling the arms are functions of a common latent random variable. The correlation between arms due to the common random source can be used to design a generalized upper-confidence-bound (UCB) algorithm that identifies certain arms as $non-c…

Paper tackles outlier detection in multi-armed bandits, achieving high accuracy with reduced exploration costs.

We study a strategic version of the multi-armed bandit problem, where each arm is an individual strategic agent and we, the principal, pull one arm each round. When pulled, the arm receives some private reward $v_a$ and can choose an amount $x_a$ to pass on to the principal (keeping $v_a-x_a$ for itself). All non-pulle…

Improved best-arm identification in correlated multi-armed bandits.

Optimal best-arm identification in linear bandits reduces sampling budget.

Study optimizes identifying the best arm with fixed rounds and Gaussian outcomes.

This work optimizes identifying good arms in nonparametric multi-armed bandits.

We analyze the regret of combinatorial Thompson sampling (CTS) for the combinatorial multi-armed bandit with probabilistically triggered arms under the semi-bandit feedback setting. We assume that the learner has access to an exact optimization oracle but does not know the expected base arm outcomes beforehand. When th…

Develops a framework for clustering and distribution matching with bandit feedback.

New algorithm reduces super-arm selection complexity exponentially.

We propose a new online algorithm for cumulative regret minimization in a stochastic linear bandit. The algorithm pulls the arm with the highest estimated reward in a linear model trained on its perturbed history. Therefore, we call it perturbed-history exploration in a linear bandit (LinPHE). The perturbed history is …

Domain adaptation performance of a learning algorithm on a target domain is a function of its source domain error and a divergence measure between the data distribution of these two domains. We present a study of various distance-based measures in the context of NLP tasks, that characterize the dissimilarity between do…

New algorithm identifies best target arm with known additive relationship between source and target MAB instances.

This paper studies Thompson sampling's arm-pull dynamics and inference, revealing key differences from UCB algorithms.

Study best arm identification with limited precision sampling in bandits.

The paper tackles a bandit problem with infinitely many arms per group, aiming to identify the group with the highest quantile reward.

New algorithm reduces worst-case sample complexity for learning best arm.

Greedy algorithms outperform UCB in many-armed bandit problems.