arXiv research

New model for multi-armed bandits with growing arms.

Optimized bandit algorithms have heavy-tailed regret distributions that can grow faster than expected.

New algorithm reduces super-arm selection complexity exponentially.

AR design reduces multi-armed bandit experiment cost.

In recent years, multi-armed bandit (MAB) framework has attracted a lot of attention in various applications, from recommender systems and information retrieval to healthcare and finance, due to its stellar performance combined with certain attractive properties, such as learning from less feedback. The multi-armed ban…

New algorithms improve performance guarantees for multi-armed bandits problems.

Graph-Triggered Bandits unify rested and restless bandits with graph-defined arm interactions.

UCB algorithm provides stable sample means for sequential data.

New framework improves restless bandit policies for large numbers of arms.

New algorithm for combinatorial bandit problems reduces regret.

Proposes Genetic Thompson Sampling for multi-armed bandits, improving performance in nonstationary settings.

TRIPLE efficiently optimizes prompts with a budget constraint.

Tiny Eats GRU detects eating episodes on a microcontroller.

We consider the classical stochastic multi-armed bandit problem with a constraint that limits the total cost incurred by switching between actions to be no larger than a given switching budget. For this problem, we prove matching upper and lower bounds on the optimal (i.e., minimax) regret, and provide efficient rate-o…

New policy for non-stationary multi-armed bandits with recovering rewards.

Greedy policies perform poorly in imperfectly observed contextual bandits.

Thompson Sampling improves decision-making in partially observed contexts.

Optimal strategy for A/B/n testing with control in stratified populations.

A new algorithm reduces regret in multi-player bandits without collision info.

Proposes auditing for envy-freeness in recommender systems to assess individual preferences.

PROBE optimizes best-arm identification with cheap proxies, improving sample complexity.

DETC algorithm achieves asymptotic optimality in multi-armed bandit problems.

We consider the problem of learning in single-player and multiplayer multiarmed bandit models. Bandit problems are classes of online learning problems that capture exploration versus exploitation tradeoffs. In a multiarmed bandit model, players can pick among many arms, and each play of an arm generates an i.i.d. rewar…

There is growing interest in being able to run neural networks on sensors, wearables and internet-of-things (IoT) devices. However, the computational demands of neural networks make them difficult to deploy on resource-constrained edge devices. To meet this need, our work introduces a new recurrent unit architecture th…

An online reinforcement learning algorithm is anytime if it does not need to know in advance the horizon T of the experiment. A well-known technique to obtain an anytime algorithm from any non-anytime algorithm is the "Doubling Trick". In the context of adversarial or stochastic multi-armed bandits, the performance of …

Motivated by posted price auctions where buyers are grouped in an unknown number of latent types characterized by their private values for the good on sale, we investigate revenue maximization in stochastic dynamic pricing when the distribution of buyers' private values is supported on an unknown set of points in [0,1]…

The growing number of low-power smart devices in the Internet of Things is coupled with the concept of "Edge Computing", that is moving some of the intelligence, especially machine learning, towards the edge of the network. Enabling machine learning algorithms to run on resource-constrained hardware, typically on low-p…

Generalized Linear Bandits (GLBs), a natural extension of the stochastic linear bandits, has been popular and successful in recent years. However, existing GLBs scale poorly with the number of rounds and the number of arms, limiting their utility in practice. This paper proposes new, scalable solutions to the GLB probl…

New algorithm minimizes regret in multi-armed bandits with network interference.

New algorithm CROP achieves asymptotic optimality with bounded regret.

We study a bad arm existing checking problem in which a player's task is to judge whether a positive arm exists or not among given K arms by drawing as small number of arms as possible. Here, an arm is positive if its expected loss suffered by drawing the arm is at least a given threshold. This problem is a formalizati…

New algorithms for streaming bandits with limited memory.

Study best arm identification in restless Markov multi-armed bandits with state-dependent transitions.

Paper tackles identifying an odd arm in a multi-armed bandit with restless Markov processes and trembling hand.

Improved theoretical guarantees for Top Two algorithms.

We address the M-best-arm identification problem in multi-armed bandits. A player has a limited budget to explore K arms (M<K), and once pulled, each arm yields a reward drawn (independently) from a fixed, unknown distribution. The goal is to find the top M arms in the sense of expected reward. We develop an algorithm …

Sequential screening and dynamic regret in multi-armed bandits with arriving arms

Paper tackles good arm identification in stochastic bandits.

We consider the best-arm identification problem in multi-armed bandits, which focuses purely on exploration. A player is given a fixed budget to explore a finite set of arms, and the rewards of each arm are drawn independently from a fixed, unknown distribution. The player aims to identify the arm with the largest expe…

A novel algorithm reduces communication costs in federated best arm identification.

New algorithms identify best arm with less pulls, adapting to arm covariances.

Study best arm identification in restless bandits with unknown TPMs.

New method for identifying best arm in batched multi-armed bandit problems.

Optimal best-arm identification with known number of optimal arms.

We consider a novel multi-armed bandit framework where the rewards obtained by pulling the arms are functions of a common latent random variable. The correlation between arms due to the common random source can be used to design a generalized upper-confidence-bound (UCB) algorithm that identifies certain arms as $non-c…

Paper tackles outlier detection in multi-armed bandits, achieving high accuracy with reduced exploration costs.

We study a strategic version of the multi-armed bandit problem, where each arm is an individual strategic agent and we, the principal, pull one arm each round. When pulled, the arm receives some private reward $v_a$ and can choose an amount $x_a$ to pass on to the principal (keeping $v_a-x_a$ for itself). All non-pulle…

Improved best-arm identification in correlated multi-armed bandits.