arXiv research

Study on indexability of restless multi-armed bandits and rollout policy performance.

Proposes a fair RMAB framework ensuring equal exposure to arms.

The paper tackles restless bandits with limited observation, proposing a method to analyze and approximate their optimal strategies.

Study best arm identification in restless Markov multi-armed bandits with state-dependent transitions.

Restless bandit problems are instances of non-stationary multi-armed bandits. These problems have been studied well from the optimization perspective, where the goal is to efficiently find a near-optimal policy when system parameters are known. However, very few papers adopt a learning perspective, where the parameters…

Study Whittle index learning algorithms for restless bandits with constant stepsizes.

We obtain the conditions for the emergence of the swarm intelligence effect in an interactive game of restless multi-armed bandit (rMAB). A player competes with multiple agents. Each bandit has a payoff that changes with a probability $p_{c}$ per round. The agents and player choose one of three options: (1) Exploit (a …

New RL algorithm tackles adversarial RMAB with unknown transitions and bandit feedback.

Paper tackles identifying an odd arm in a multi-armed bandit with restless Markov processes and trembling hand.

Study best arm identification in restless bandits with unknown TPMs.

Optimal policy for multi-armed multi-action bandits with unknown parameters.

Paper tackles anomaly detection in restless Markov arms with unknown TPMs.

Improved control approach for correlated bandits with better performance.

DOPL learns from preference feedback to solve RMAB problems.

NeurWIN learns Whittle indices for restless bandits using deep reinforcement learning.

New bandit model for healthcare intervention planning.

New algorithm optimizes resource allocation in non-stationary networks.

New algorithm optimizes decision-making for complex systems with varying parameters.

ContextWIN uses neural networks and reinforcement learning to optimize decisions in dynamic environments.

Neural Index Policy for multi-action bandits with heterogeneous budgets.

New algorithms reduce regret in non-stationary bandits with increasing payoffs.

In this paper we propose a flexible and efficient framework for handling multi-armed bandits, combining sequential Monte Carlo algorithms with hierarchical Bayesian modeling techniques. The framework naturally encompasses restless bandits, contextual bandits, and other bandit variants under a single inferential model. …

New algorithm uses Whittle index to improve Q-learning for restless bandits.

Graph-Triggered Bandits unify rested and restless bandits with graph-defined arm interactions.

New algorithm handles both decaying and non-decaying bandit problems.

We consider the multi armed bandit problem in non-stationary environments. Based on the Bayesian method, we propose a variant of Thompson Sampling which can be used in both rested and restless bandit scenarios. Applying discounting to the parameters of prior distribution, we describe a way to systematically reduce the …

The multi-armed restless bandit problem is studied in the case where the pay-off distributions are stationary $\varphi$-mixing. This version of the problem provides a more realistic model for most real-world applications, but cannot be optimally solved in practice, since it is known to be PSPACE-hard. The objective of …

A new algorithm for restless bandits handles long-range dependencies.

We consider a class of restless multi-armed bandit (RMAB) problems with unknown arm dynamics. At each time, a player chooses an arm out of N arms to play, referred to as an active arm, and receives a random reward from a finite set of reward states. The reward state of the active arm transits according to an unknown Ma…

New model predicts optimal control for restless bandit problems.

New approach tackles non-Markovian behavior in maternal health programs.

We study the stochastic multi-armed bandit problem in the case when the arm samples are dependent over time and generated from so-called weak $\cC$-mixing processes. We establish a $\cC-$Mix Improved UCB agorithm and provide both problem-dependent and independent regret analysis in two different scenarios. In the first…

We study the restless bandit associated with an extremely simple scalar Kalman filter model in discrete time. Under certain assumptions, we prove that the problem is indexable in the sense that the Whittle index is a non-decreasing function of the relevant belief state. In spite of the long history of this problem, thi…

New framework improves restless bandit policies for large numbers of arms.

We consider the restless Markov bandit problem, in which the state of each arm evolves according to a Markov process independently of the learner's actions. We suggest an algorithm that after $T$ steps achieves $\tilde{O}(\sqrt{T})$ regret with respect to the best policy that knows the distributions of all arms. No ass…

Restless bandit problems assume time-varying reward distributions of the arms, which adds flexibility to the model but makes the analysis more challenging. We study learning algorithms over the unknown reward distributions and prove a sub-linear, $O(\sqrt{T}\log T)$, regret bound for a variant of Thompson sampling. Our…

Method predicts future rewards from past actions in a linear Gaussian system.

Bayesian bandits misspecification affects UX optimization, revealing new models.

New dynamic allocation methods for multi-armed bandit models.

PAC-Bayesian analysis improves lifelong learning in multi-armed bandits.

Paper uses subjective logic to estimate uncertainty in multi-armed bandit problems.

Algorithm improves multi-armed bandit performance by transferring reward samples.

Quantum algorithms for multi-armed bandits are explored with limited reward access.

Study on Pareto optimality in multi-objective bandit problems.

Unified formulation bridges adversarial and nonstationary bandits.

In this paper we propose a multi-armed bandit inspired, pool based active learning algorithm for the problem of binary classification. By carefully constructing an analogy between active learning and multi-armed bandits, we utilize ideas such as lower confidence bounds, and self-concordant regularization from the multi…

Study finds optimal regret bound for multi-armed bandit problem with expert advice.

The paper tackles lifelong learning in multi-armed bandits, aiming to minimize average regret over multiple tasks.