arXiv research

New algorithm tackles delayed feedback in Lipschitz bandits with sublinear regret.

Banker-OMD improves online learning with delayed feedback.

New algorithm tackles stochastic bandits with varying arm-dependent delays.

New algorithm reduces regret in delayed feedback generalised linear bandits.

Paper tackles dueling bandits with delayed feedback, revealing preference bias.

New algorithms ensure fair selection in combinatorial semi-bandit with unrestricted delays.

Study non-oblivious adversarial bandits with delayed feedback and propose algorithms with improved regret bounds.

This paper deals with bandit online learning problems involving feedback of unknown delay that can emerge in multi-armed bandit (MAB) and bandit convex optimization (BCO) settings. MAB and BCO require only values of the objective function involved that become available through feedback, and are used to estimate the gra…

Adapts two algorithms for online learning with delayed rewards.

New algorithm handles delayed feedback robustly, reducing regret without knowing delay bounds.

We study a variant of the stochastic $K$-armed bandit problem, which we call "bandits with delayed, aggregated anonymous feedback". In this problem, when the player pulls an arm, a reward is generated, however it is not immediately observed. Instead, at the end of each round the player observes only the sum of a number…

Study explores strategies for randomized allocation in delayed rewards bandits.

Proposes DEXP3.M for unknown delay in multi-arm bandit with multiple play.

A new algorithm tackles delayed combinatorial semi-bandit with causal relations.

We study a multi-armed bandit problem with covariates in a setting where there is a possible delay in observing the rewards. Under some mild assumptions on the probability distributions for the delays and using an appropriate randomization to select the arms, the proposed strategy is shown to be strongly consistent.

Improved algorithm for bandits with delayed feedback, combining adversarial and stochastic performance.

Adapts Exp3 to adversarial bandits with delays and data.

New algorithms handle online prediction with bandit and delayed feedback, improving regret bounds.

Stochastic linear bandits are a natural and well-studied model for structured exploration/exploitation problems and are widely used in applications such as online marketing and recommendation. One of the main challenges faced by practitioners hoping to apply existing algorithms is that usually the feedback is randomly …

Capacity-Constrained Online Convex Optimization with Delayed Feedback

New algorithm tackles non-stationary delayed feedback in recommender systems.

We consider a stochastic multi-armed bandit (MAB) problem with delayed impact of actions. In our setting, actions taken in the past impact the arm rewards in the subsequent future. This delayed impact of actions is prevalent in the real world. For example, the capability to pay back a loan for people in a certain socia…

Study cooperative bandit learning with imperfect communication, achieving near-optimal performance.

New algorithm for multi-armed bandits with delayed, partially observed rewards.

New algorithm tackles multi-armed bandit with arbitrary delays and general bounded losses.

Study online learning with delays and capacity constraints, achieving optimal regret bounds.

New algorithm reduces regret in collaborative multi-agent bandit problems.

New algorithm optimizes noisy function evaluations with delayed feedback.

New algorithm optimizes for long-term user satisfaction in delayed reward settings.

We investigate multiarmed bandits with delayed feedback, where the delays need neither be identical nor bounded. We first prove that "delayed" Exp3 achieves the $O(\sqrt{(KT + D)\ln K} )$ regret bound conjectured by Cesa-Bianchi et al. [2019] in the case of variable, but bounded delays. Here, $K$ is the number of actio…

We propose a new algorithm for adversarial multi-armed bandits with unrestricted delays. The algorithm is based on a novel hybrid regularizer applied in the Follow the Regularized Leader (FTRL) framework. It achieves $\mathcal{O}(\sqrt{kn}+\sqrt{D\log(k)})$ regret guarantee, where $k$ is the number of arms, $n$ is the …

We propose a generalization of the best arm identification problem in stochastic multi-armed bandits (MAB) to the setting where every pull of an arm is associated with delayed feedback. The delay in feedback increases the effective sample complexity of standard algorithms, but can be offset if we have access to partial…

A survey is performed of various Multi-Armed Bandit (MAB) strategies in order to examine their performance in circumstances exhibiting non-stationary stochastic reward functions in conjunction with delayed feedback. We run several MAB simulations to simulate an online eCommerce platform for grocery pick up, optimizing …

New algorithm optimizes long-term user satisfaction in recommendation systems.

Online learning with delayed feedback has received increasing attention recently due to its several applications in distributed, web-based learning problems. In this paper we provide a systematic study of the topic, and analyze the effect of delay on the regret of online learning algorithms. Somewhat surprisingly, it t…

Thompson Sampling tackles noisy context in stochastic bandits.

We explore a novel setting of the Multi-Armed Bandit (MAB) problem inspired from real world applications which we call bandits with "stochastic delayed composite anonymous feedback (SDCAF)". In SDCAF, the rewards on pulling arms are stochastic with respect to time but spread over a fixed number of time steps in the fut…

A new algorithm for cryo-EM data collection that balances reward and latency.

Adaptive anomaly detection for IoT data reduces delay without sacrificing accuracy.

Delaytron learns multi-class classifiers with delayed feedbacks, achieving efficient regret bounds.

PCTS optimizes noisy, delayed, multi-fidelity feedbacks in black-box optimization.

New algorithm improves cache management with delayed feedback and decaying costs.

We consider a novel stochastic multi-armed bandit setting, where playing an arm makes it unavailable for a fixed number of time slots thereafter. This models situations where reusing an arm too often is undesirable (e.g. making the same product recommendation repeatedly) or infeasible (e.g. compute job scheduling on ma…

A new algorithm reduces regret in cooperative multi-agent bandits with heavy-tailed data.

Optimizes sensor usage for detecting abrupt changes in sensor data.

The paper tackles carousel personalization in music streaming apps using contextual bandits.

We study a decentralized cooperative stochastic multi-armed bandit problem with $K$ arms on a network of $N$ agents. In our model, the reward distribution of each arm is the same for each agent and rewards are drawn independently across agents and time steps. In each round, each agent chooses an arm to play and subsequ…

Adapts bandit algorithms for online survival analysis under Cox PH model.