arXiv research

Study gap-dependent regret bounds for risk-sensitive RL.

Fine-grained gap-dependent regret bounds for reinforcement learning.

This paper improves Q-learning bounds using reference-advantage decomposition.

New method reduces regret and communication costs in federated Q-learning.

This paper establishes that optimistic algorithms attain gap-dependent and non-asymptotic logarithmic regret for episodic MDPs. In contrast to prior work, our bounds do not suffer a dependence on diameter-like quantities or ergodicity, and smoothly interpolate between the gap dependent logarithmic-regret, and the $\wid…

Improved gap-dependent bounds for reinforcement learning with linear approximations.

The paper analyzes Q-learning in 2-player Markov games and provides gap-dependent logarithmic regret bounds.

Unified framework for corruption-robust linear bandits with optimal gap-dependent misspecification bounds.

New algorithm reduces regret in private online learning with optimal gap-dependent rate.

New algorithm reduces MDP regret by accounting for state suboptimality gaps and variance.

Adaptive smooth non-stationary bandits achieve optimal regret rates without knowing parameters.

New algorithms minimize regret in streaming MAB with memory constraints.

New RL algorithms reduce costs for single-agent and federated learning.

Algorithm reduces regret in misspecified linear contextual bandits.

New insights into balancing reward and fairness in stochastic MAB.

Improves policy optimization with polylog(T) regret bounds for stochastic losses.

RAVEN-UCB addresses non-stationary MAB problems with tighter regret bounds.

Unified framework for distributional regret in bandits and reinforcement learning.

KL-MS improves regret bounds for multi-armed bandits with bounded rewards.

A matroid is a notion of independence in combinatorial optimization which is closely related to computational efficiency. In particular, it is well known that the maximum of a constrained modular function can be found greedily if and only if the constraints are associated with a matroid. In this paper, we bring togethe…

We propose an online algorithm for cumulative regret minimization in a stochastic multi-armed bandit. The algorithm adds $O(t)$ i.i.d. pseudo-rewards to its history in round $t$ and then pulls the arm with the highest average reward in its perturbed history. Therefore, we call it perturbed-history exploration (PHE). Th…

Logarithmic regret achieved in RL with linear function approximation.

Paper develops robust estimators and strategies for stochastic MABs with heavy-tailed rewards.

Logarithmic regret strategies for safe multi-armed bandits with safety risk constraints.

New algorithms achieve no-regret learning even with adversarial transitions and losses.

Improved reinforcement learning for episodes with varying action sets.

In this paper, we study the combinatorial multi-armed bandit problem (CMAB) with probabilistically triggered arms (PTAs). Under the assumption that the arm triggering probabilities (ATPs) are positive for all arms, we prove that a class of upper confidence bound (UCB) policies, named Combinatorial UCB with exploration …

A search engine usually outputs a list of $K$ web pages. The user examines this list, from the first web page to the last, and chooses the first attractive page. This model of user behavior is known as the cascade model. In this paper, we propose cascading bandits, a learning variant of the cascade model where the obje…

New algorithm achieves data-dependent regret bounds in MDPs with unknown transitions.

New algorithms reduce regret in online MDPs by adapting to data and variance.

This work improves regret minimization for logistic bandits by reducing dependence on a large constant.

The paper tackles fair sequential decision making with biased linear bandit feedback.

A stochastic combinatorial semi-bandit is an online learning problem where at each step a learning agent chooses a subset of ground items subject to constraints, and then observes stochastic weights of these items and receives their sum as a payoff. In this paper, we close the problem of computationally and sample effi…

Non-stationarity appears in many online applications such as web search and advertising. In this paper, we study the online learning to rank problem in a non-stationary environment where user preferences change abruptly at an unknown moment in time. We consider the problem of identifying the K most attractive items and…

Study online learning in MDPs with aggregate bandit feedback, achieving low regret in both stochastic and adversarial settings.

New algorithms improve contextual bandit performance by adapting to problem difficulty.

A new reinforcement learning framework separates users into risk-tolerant and risk-averse groups for better performance.

New algorithm optimizes multi-armed bandit performance in stochastic and adversarial settings.

Algorithm optimizes multi-player learning with noisy rewards without direct communication.

We make three contributions to the theory of k-armed adversarial bandits. First, we prove a first-order bound for a modified variant of the INF strategy by Audibert and Bubeck [2009], without sacrificing worst case optimality or modifying the loss estimators. Second, we provide a variance analysis for algorithms based …

A search engine recommends to the user a list of web pages. The user examines this list, from the first page to the last, and clicks on all attractive pages until the user is satisfied. This behavior of the user can be described by the dependent click model (DCM). We propose DCM bandits, an online learning variant of t…

Many important optimization problems, such as the minimum spanning tree and minimum-cost flow, can be solved optimally by a greedy method. In this work, we study a learning variant of these problems, where the model of the problem is unknown and has to be learned by interacting repeatedly with the environment in the ba…

The paper tackles resource allocation for arms with unknown and random rewards, achieving optimal regret bounds.

New algorithm optimizes dueling bandits for both stochastic and adversarial preferences.

New algorithm resists corruption in linear contextual bandits.

Online learning to rank is a core problem in information retrieval and machine learning. Many provably efficient algorithms have been recently proposed for this problem in specific click models. The click model is a model of how the user interacts with a list of documents. Though these results are significant, their im…

Motivated by applications in computational advertising and systems biology, we consider the problem of identifying the best out of several possible soft interventions at a source node $V$ in an acyclic causal directed graph, to maximize the expected value of a target node $Y$ (located downstream of $V$). Our setting im…

We consider the dynamic assortment optimization problem under the multinomial logit model (MNL) with unknown utility parameters. The main question investigated in this paper is model mis-specification under the $\varepsilon$-contamination model, which is a fundamental model in robust statistics and machine learning. In…