arXiv research

New algorithms detect changes in non-stationary MABs for better performance.

Paper designs a bandit algorithm without reward distribution info.

Multi-armed bandit (MAB) is a class of online learning problems where a learning agent aims to maximize its expected cumulative reward while repeatedly selecting to pull arms with unknown reward distributions. We consider a scenario where the reward distributions may change in a piecewise-stationary fashion at unknown …

A survey is performed of various Multi-Armed Bandit (MAB) strategies in order to examine their performance in circumstances exhibiting non-stationary stochastic reward functions in conjunction with delayed feedback. We run several MAB simulations to simulate an online eCommerce platform for grocery pick up, optimizing …

We extend Bayesian multi-armed bandit (MAB) algorithms beyond their original setting by making use of sequential Monte Carlo (SMC) methods. A MAB is a sequential decision making problem where the goal is to learn a policy that maximizes long term payoff, where only the reward of the executed action is observed. In the …

Study incentivizes exploration in non-stationary MAB with compensation.

New TS algorithms improve performance in non-stationary multi-armed bandit problems.

The Multi-Armed Bandits (MAB) framework highlights the tension between acquiring new knowledge (Exploration) and leveraging available knowledge (Exploitation). In the classical MAB problem, a decision maker must choose an arm at each time step, upon which she receives a reward. The decision maker's objective is to maxi…

Algorithm reduces decision-making errors in multi-agent bandit problems.

The problem of time-series clustering is considered in the case where each data-point is a sample generated by a piecewise stationary ergodic process. Stationary processes are perhaps the most general class of processes considered in non-parametric statistics and allow for arbitrary long-range dependence between variab…

We investigate the piecewise-stationary combinatorial semi-bandit problem. Compared to the original combinatorial semi-bandit problem, our setting assumes the reward distributions of base arms may change in a piecewise-stationary manner at unknown time steps. We propose an algorithm, \texttt{GLR-CUCB}, which incorporat…

A decentralized policy achieves logarithmic regret for multi-agent MAB problems with communication constraints.

DAL enhances black-box bandit algorithms for non-stationary environments.

RAVEN-UCB addresses non-stationary MAB problems with tighter regret bounds.

New algorithms reduce regret in non-stationary bandits with increasing payoffs.

In this work, we consider the popular tree-based search strategy within the framework of reinforcement learning, the Monte Carlo Tree Search (MCTS), in the context of infinite-horizon discounted cost Markov Decision Process (MDP). While MCTS is believed to provide an approximate value function for a given state with en…

Algorithm identifies best arm in piecewise stationary linear bandits with minimal samples.

Cascading bandit (CB) is a popular model for web search and online advertising, where an agent aims to learn the $K$ most attractive items out of a ground set of size $L$ during the interaction with a user. However, the stationary CB model may be too simple to apply to real-world problems, where user preferences may ch…

DS-TS adapts to abrupt and smooth changes in bandit problems.

Paper introduces novel Bandit algorithms for non-stationary environments in finance.

New method tracks shifts in infinite-armed bandits without prior knowledge.

The paper tackles non-stationary MAB with periodic rewards.

For a bounded domain equipped with a piecewise Lipschitz continuous Riemannian metric g, we consider harmonic map from $(Ω, g)$ to a compact Riemannian manifold $(N,h)\subset\mathbb R^k$ without boundary. We generalize the notion of stationary harmonic map and prove the partial regularity. We also discuss the global Li…

In a multi-armed bandit (MAB) problem a gambler needs to choose at each round of play one of K arms, each characterized by an unknown reward distribution. Reward realizations are only observed when an arm is selected, and the gambler's objective is to maximize his cumulative expected earnings over some given horizon of…

Study shows nonstationary bandits require T-dependent regret even with minimal nonstationarity.

Study adapts combinatorial semi-bandit for piecewise stationary, causally related rewards.

The multi-armed bandit problem has been extensively studied under the stationary assumption. However in reality, this assumption often does not hold because the distributions of rewards themselves may change over time. In this paper, we propose a change-detection (CD) based framework for multi-armed bandit problems und…

New algorithms control loss and constraints in uncertain, changing environments.

We introduce GLR-klUCB, a novel algorithm for the piecewise iid non-stationary bandit problem with bounded rewards. This algorithm combines an efficient bandit algorithm, kl-UCB, with an efficient, parameter-free, changepoint detector, the Bernoulli Generalized Likelihood Ratio Test, for which we provide new theoretica…

Master algorithm fails to detect non-stationarity in practical settings.

The paper tackles rested bandits with non-decreasing and concave rewards, deriving lower bounds and an efficient algorithm.

New algorithms minimize dynamic regret in non-stationary online learning.

Proposes a new TS algorithm for non-stationary bandits using KS tests.

The Bivariate Dynamic Contagion Processes (BDCP) are a broad class of bivariate point processes characterized by the intensities as a general class of piecewise deterministic Markov processes. The BDCP describes a rich dynamic structure where the system is under the influence of both external and internal factors model…

We study the non-stationary stochastic multiarmed bandit (MAB) problem and propose two generic algorithms, namely, the limited memory deterministic sequencing of exploration and exploitation (LM-DSEE) and the Sliding-Window Upper Confidence Bound# (SW-UCB#). We rigorously analyze these algorithms in abruptly-changing a…

Paper closes the gap in MP-MAB problems with novel adaptive communication and exploration.

We formulate the problem of sampling and recovering clustered graph signal as a multi-armed bandit (MAB) problem. This formulation lends naturally to learning sampling strategies using the well-known gradient MAB algorithm. In particular, the sampling strategy is represented as a probability distribution over the indiv…

Study non-stationary bandits with resource constraints.

A new bandit problem where experiments can be interrupted if results are not promising.

In this work, we study recommendation systems modelled as contextual multi-armed bandit (MAB) problems. We propose a graph-based recommendation system that learns and exploits the geometry of the user space to create meaningful clusters in the user domain. This reduces the dimensionality of the recommendation problem w…

Memory-based models can learn to approximate Bayes-optimal predictors for non-stationary data.

New method optimizes policies in non-stationary environments.

Improved algorithm detects changes in RL environments with non-stationary MDPs.

Trust-aware MAB improves learning performance by accounting for human deviation.

New algorithm for multi-fidelity bandits reduces costs and improves regret.

New algorithm ensures consistent results in constrained MAB problems.

DARLING tackles non-stationary RL with guarantees, improving dynamic regret.

New algorithm reduces dynamic regret for noisy gradient feedback with piecewise polynomial comparators.