arXiv research

Optimistic algorithms achieve logarithmic regret bounds for MDPs without diameter dependence.

This paper improves Q-learning bounds using reference-advantage decomposition.

New algorithm improves bandit with graph feedback by decomposing regret.

Paper proposes FedQ-Advantage for federated Q-learning with near-optimal regret and low communication cost.

This paper begins with a study on the dual representations of risk and regret measures and their impact on modeling multistage decision making under uncertainty. A relationship between risk envelopes and regret envelopes is established by using the Lagrangian duality theory. Such a relationship opens a door to a decomp…

The paper introduces new measures to quantify variability in decision tree models due to observational multiplicity.

The paper tackles ride-hailing fleet repositioning with a calibrated demand approach.

UCB-Advantage learns MDPs with $O(\sqrt{H^2SAT})$ regret.

A new linear contextual bandit algorithm with improved regret bound.

Paper analyzes adaptive optimization algorithms and provides new insights.

New method reduces multi-armed bandit regret to near-optimal levels.

New method identifies algo trading strategies as liquidity consumers or providers.

Paper analyzes and improves KL-regularized RL for LLMs with logarithmic regret.

Evaluating AI investment strategies

A new algorithm improves regret bounds for contextual bandits.

The paper studies reward concentration in MDPs, covering asymptotic and non-asymptotic settings.

Algorithm POLO learns low-rank MDPs with adversarial changes in full-info feedback.

New algorithm reduces decision switching in dynamic environments.

New RL algorithm reduces sample complexity for optimal learning.

Reinforcement learning (RL) in Markov decision processes (MDPs) with large state spaces is a challenging problem. The performance of standard RL algorithms degrades drastically with the dimensionality of state space. However, in practice, these large MDPs typically incorporate a latent or hidden low-dimensional structu…

ACP-UCB1 ranks arms based on upper-tail performance, improving stochastic bandit algorithms.

The paper decomposes probabilistic scores into reliability, uncertainty, and information loss.

Most of machine learning deals with vector parameters. Ideally we would like to take higher order information into account and make use of matrix or even tensor parameters. However the resulting algorithms are usually inefficient. Here we address on-line learning with matrix parameters. It is often easy to obtain onlin…

Interactive learning with hindsight instruction feedback achieves better performance than traditional methods.

Paper proposes an online learning method with multi-level adaptivity for diverse loss functions.

Algorithm improves wildlife protection patrols.

Paper proposes algorithms to minimize both dynamic and adaptive regret simultaneously.

This paper addresses the problem of online learning in a dynamic setting. We consider a social network in which each individual observes a private signal about the underlying state of the world and communicates with her neighbors at each time period. Unlike many existing approaches, the underlying state is dynamic, and…

New measure of policy regret shows compatibility with traditional external regret in adversarial games.

The paper analyzes the sliding regret of stochastic bandit algorithms.

This paper analyzes regret bounds for Gaussian process Thompson sampling.

Efficiently minimizes regret in non-convex games with gradient-based methods.

New definition of regret for nonconvex online learning models.

Online learning algorithms are designed to learn even when their input is generated by an adversary. The widely-accepted formal definition of an online algorithm's ability to learn is the game-theoretic notion of regret. We argue that the standard definition of regret becomes inadequate if the adversary is allowed to a…

We propose a new reinforcement learning algorithm for partially observable Markov decision processes (POMDP) based on spectral decomposition methods. While spectral methods have been previously employed for consistent learning of (passive) latent variable models such as hidden Markov models, POMDPs are more challenging…

Balances the regret of different algorithms in bandit and RL problems.

Paper studies how to combine regret minimizers for solving complex games.

Study Thompson Sampling in adversarial bit prediction, finding regret bounds and optimal sequences.

Data-driven model selection reduces regret in sequential decisions.

Optimistic Hedge achieves optimal regret bounds in two-player zero-sum games.

New algorithm reduces regret in contextual bandits.

We propose a new reinforcement learning algorithm for partially observable Markov decision processes (POMDP) based on spectral decomposition methods. While spectral methods have been previously employed for consistent learning of (passive) latent variable models such as hidden Markov models, POMDPs are more challenging…

Optimal switching regret for all segmentations in online convex optimisation.

New approach for distributed online optimization of non-convex losses with sublinear regret.

New algorithm SELECT minimizes satisficing regret in bandits.

The paper tackles efficient online learning by achieving minimal regret with respect to the best expert.

Proximal online gradient minimizes dynamic regret in evolving environments.

This paper considers the stability of online learning algorithms and its implications for learnability (bounded regret). We introduce a novel quantity called {\em forward regret} that intuitively measures how good an online learning algorithm is if it is allowed a one-step look-ahead into the future. We show that given…