arXiv research

Greedy algorithms outperform UCB in many-armed bandit problems.

We consider a stochastic bandit problem with infinitely many arms. In this setting, the learner has no chance of trying all the arms even once and has to dedicate its limited number of samples only to a certain number of arms. All previous algorithms for this setting were designed for minimizing the cumulative regret o…

Study tackles infinitely many-armed bandits with rotting rewards, achieving tight regret bounds.

New algorithm reduces regret in infinitely many-armed bandits with decreasing rewards.

We study the problem of finding the most mutually correlated arms among many arms. We show that adaptive arms sampling strategies can have significant advantages over the non-adaptive uniform sampling strategy. Our proposed algorithms rely on a novel correlation estimator. The use of this accurate estimator allows us t…

A new method optimizes in nonstationary environments with many arms efficiently.

The paper tackles a bandit problem with infinitely many arms per group, aiming to identify the group with the highest quantile reward.

Study on policy gradient for stochastic bandits using diffusion approximation.

Proposes a max-utility arm selection strategy for reducing cumulative regret in sequential query recommendations.

We give examples of data-generating models under which Breiman's random forest may be extremely slow to converge to the optimal predictor or even fail to be consistent. The evidence provided for these properties is based on mostly intuitive arguments, similar to those used earlier with simpler examples, and on numerica…

Investigates sequential problems on graph structures and large action spaces.

Batch Thompson Sampling reduces exploration-exploitation trade-off in online decision making.

New algorithm for countable bandits with optimal regret.

We propose a generic, Bayesian, information geometric approach to the exploration--exploitation trade-off in multi-armed bandit problems. Our approach, BelMan, uniformly supports pure exploration, exploration--exploitation, and two-phase bandit problems. The knowledge on bandit arms and their reward distributions is su…

Certified algorithms optimize functions with varying costs, providing error bounds.

New methods optimize personalized treatment assignment in trials with many arms.

We propose $\tt RandUCB$, a bandit strategy that builds on theoretically derived confidence intervals similar to upper confidence bound (UCB) algorithms, but akin to Thompson sampling (TS), it uses randomization to trade off exploration and exploitation. In the $K$-armed bandit setting, we show that there are infinitel…

In a linear stochastic bandit model, each arm is a vector in an Euclidean space and the observed return at each time step is an unknown linear function of the chosen arm at that time step. In this paper, we investigate the problem of learning the best arm in a linear stochastic bandit model, where each arm's expected r…

We study the problem of dynamic assortment personalization with large, heterogeneous populations and wide arrays of products, and demonstrate the importance of structural priors for effective, efficient large-scale personalization. Assortment personalization is the problem of choosing, for each individual (type), a bes…

We consider the problem of learning in single-player and multiplayer multiarmed bandit models. Bandit problems are classes of online learning problems that capture exploration versus exploitation tradeoffs. In a multiarmed bandit model, players can pick among many arms, and each play of an arm generates an i.i.d. rewar…

This paper introduces a new metric, ULI, for RL that ensures both cumulative and instantaneous performance.

New bandit model for healthcare intervention planning.

Optimal best arm identification for multi-objective bandits with fixed error probability.

New algorithm reduces cold-start costs in multi-armed bandits for many products.

Adaptive algorithm selects models for stochastic linear bandits based on problem complexity.

Study best arm identification in restless bandits with unknown TPMs.