arXiv research

This paper improves Q-learning bounds using reference-advantage decomposition.

New algorithm improves bandit with graph feedback by decomposing regret.

Paper proposes FedQ-Advantage for federated Q-learning with near-optimal regret and low communication cost.

This paper establishes that optimistic algorithms attain gap-dependent and non-asymptotic logarithmic regret for episodic MDPs. In contrast to prior work, our bounds do not suffer a dependence on diameter-like quantities or ergodicity, and smoothly interpolate between the gap dependent logarithmic-regret, and the $\wid…

This paper begins with a study on the dual representations of risk and regret measures and their impact on modeling multistage decision making under uncertainty. A relationship between risk envelopes and regret envelopes is established by using the Lagrangian duality theory. Such a relationship opens a door to a decomp…

The paper introduces new measures to quantify variability in decision tree models due to observational multiplicity.

The paper tackles ride-hailing fleet repositioning with a calibrated demand approach.

UCB-Advantage learns MDPs with $O(\sqrt{H^2SAT})$ regret.

A new linear contextual bandit algorithm with improved regret bound.

New method reduces multi-armed bandit regret to near-optimal levels.

New method identifies algo trading strategies as liquidity consumers or providers.

Paper analyzes and improves KL-regularized RL for LLMs with logarithmic regret.

A new algorithm improves regret bounds for contextual bandits.

Evaluating AI investment strategies

The paper studies reward concentration in MDPs, covering asymptotic and non-asymptotic settings.

Algorithm POLO learns low-rank MDPs with adversarial changes in full-info feedback.

New algorithm reduces decision switching in dynamic environments.

New RL algorithm reduces sample complexity for optimal learning.

Reinforcement learning (RL) in Markov decision processes (MDPs) with large state spaces is a challenging problem. The performance of standard RL algorithms degrades drastically with the dimensionality of state space. However, in practice, these large MDPs typically incorporate a latent or hidden low-dimensional structu…

ACP-UCB1 ranks arms based on upper-tail performance, improving stochastic bandit algorithms.

The paper decomposes probabilistic scores into reliability, uncertainty, and information loss.

Most of machine learning deals with vector parameters. Ideally we would like to take higher order information into account and make use of matrix or even tensor parameters. However the resulting algorithms are usually inefficient. Here we address on-line learning with matrix parameters. It is often easy to obtain onlin…

Interactive learning with hindsight instruction feedback achieves better performance than traditional methods.

Paper proposes an online learning method with multi-level adaptivity for diverse loss functions.

Recently, much work has been done on extending the scope of online learning and incremental stochastic optimization algorithms. In this paper we contribute to this effort in two ways: First, based on a new regret decomposition and a generalization of Bregman divergences, we provide a self-contained, modular analysis of…

Algorithm improves wildlife protection patrols.

This paper addresses the problem of online learning in a dynamic setting. We consider a social network in which each individual observes a private signal about the underlying state of the world and communicates with her neighbors at each time period. Unlike many existing approaches, the underlying state is dynamic, and…

Regret minimization is treated as the golden rule in the traditional study of online learning. However, regret minimization algorithms tend to converge to the static optimum, thus being suboptimal for changing environments. To address this limitation, new performance measures, including dynamic regret and adaptive regr…

The paper analyzes the sliding regret of stochastic bandit algorithms.

This paper analyzes regret bounds for Gaussian process Thompson sampling.

The notion of \emph{policy regret} in online learning is a well defined? performance measure for the common scenario of adaptive adversaries, which more traditional quantities such as external regret do not take into account. We revisit the notion of policy regret and first show that there are online learning settings …

Online learning algorithms are designed to learn even when their input is generated by an adversary. The widely-accepted formal definition of an online algorithm's ability to learn is the game-theoretic notion of regret. We argue that the standard definition of regret becomes inadequate if the adversary is allowed to a…

Balances the regret of different algorithms in bandit and RL problems.

Data-driven model selection reduces regret in sequential decisions.

Optimistic Hedge achieves optimal regret bounds in two-player zero-sum games.

We consider an online learning process to forecast a sequence of outcomes for nonconvex models. A typical measure to evaluate online learning algorithms is regret but such standard definition of regret is intractable for nonconvex models even in offline settings. Hence, gradient based definition of regrets are common f…

Optimal switching regret for all segmentations in online convex optimisation.

New approach for distributed online optimization of non-convex losses with sublinear regret.

We propose a new reinforcement learning algorithm for partially observable Markov decision processes (POMDP) based on spectral decomposition methods. While spectral methods have been previously employed for consistent learning of (passive) latent variable models such as hidden Markov models, POMDPs are more challenging…

Regret minimization is a powerful tool for solving large-scale problems; it was recently used in breakthrough results for large-scale extensive-form game solving. This was achieved by composing simplex regret minimizers into an overall regret-minimization framework for extensive-form game strategy spaces. In this paper…

New algorithm SELECT minimizes satisficing regret in bandits.

We study the Thompson sampling algorithm in an adversarial setting, specifically, for adversarial bit prediction. We characterize the bit sequences with the smallest and largest expected regret. Among sequences of length $T$ with $k < \frac{T}{2}$ zeros, the sequences of largest regret consist of alternating zeros and …

This paper considers the stability of online learning algorithms and its implications for learnability (bounded regret). We introduce a novel quantity called {\em forward regret} that intuitively measures how good an online learning algorithm is if it is allowed a one-step look-ahead into the future. We show that given…

Paper introduces a new $G^\star$ regret measure for online convex optimization with smooth losses.

We propose a new reinforcement learning algorithm for partially observable Markov decision processes (POMDP) based on spectral decomposition methods. While spectral methods have been previously employed for consistent learning of (passive) latent variable models such as hidden Markov models, POMDPs are more challenging…

Bandit algorithms struggle with consistent performance and robustness.

Paper improves worst-case regret bounds for RLSVI in reinforcement learning.

Paper solves no-swap regret minimization for combinatorial bandits with polylogarithmic dependence on N.