arXiv research

New RL algorithm handles delayed feedback with posterior sampling.

While policy-based reinforcement learning (RL) achieves tremendous successes in practice, it is significantly less understood in theory, especially compared with value-based RL. In particular, it remains elusive how to design a provably efficient policy optimization algorithm that incorporates exploration. To bridge su…

We analyze reinforcement learning algorithms using a distributional approach.

New issue found in value-based reinforcement learning for stochastic environments.

Overparameterized models generalize well in offline contextual bandits, but policy-based algorithms struggle.

Bayesian classifier improves robustness with optimistic score ratio.

We show that the optimistic limits of the colored Jones polynomials of the hyperbolic knots coincide with the optimistic limits of the Kashaev invariants modulo $4π^2$.

We discuss the relative merits of optimistic and randomized approaches to exploration in reinforcement learning. Optimistic approaches presented in the literature apply an optimistic boost to the value estimate at each state-action pair and select actions that are greedy with respect to the resulting optimistic value f…

This work explores representation complexity in RL paradigms, revealing model-based RL as the easiest task.

The optimistic limit is the mathematical formulation of the classical limit which is a physical method to expect the actual limit by using saddle point method of certain potential function. The original optimistic limit of the Kashaev invariant was formulated by Yokota, and a modified formulation was suggested by the a…

Optimistic estimate predicts best fitting performance of nonlinear models.

Optimistic initialisation is an effective strategy for efficient exploration in reinforcement learning (RL). In the tabular case, all provably efficient model-free algorithms rely on it. However, model-free deep RL algorithms do not use optimistic initialisation despite taking inspiration from these provably efficient …

New algorithm reduces regret in sequential decision-making problems.

Modern deep learning methods provide effective means to learn good representations. However, is a good representation itself sufficient for sample efficient reinforcement learning? This question has largely been studied only with respect to (worst-case) approximation error, in the more classical approximate dynamic pro…

Optimistic NPG improves policy optimization in online RL with efficient sample complexity.

The basic financial purpose of a firm is to maximize its value. An inventory management system should also contribute to realization of this basic aim. Many current asset management models currently found in financial management literature were constructed with the assumption of book profit maximization as basic aim. H…

Characterizes concept classes for optimistic online learning.

We show how to take any two parameter-free online learning algorithms with different regret guarantees and obtain a single algorithm whose regret is the minimum of the two base algorithms. Our method is embarrassingly simple: just add the iterates. This trick can generate efficient algorithms that adapt to many norms s…

This paper improves reinforcement learning policies in a scalable way.

New algorithms reduce forecasting errors by leveraging optimistic learning and hinting.

Omega method mitigates noise in stochastic game optimization.

New method minimizes regret in AMDP with high probability.

Unified framework for model- and value-optimistic reinforcement learning.

Proposes a value-based method for continuous control without an actor.

Value functions struggle to represent transition dynamics, impacting statistical efficiency.

It is well known that in stochastic multi-armed bandits (MAB), the sample mean of an arm is typically not an unbiased estimator of its true mean. In this paper, we decouple three different sources of this selection bias: adaptive \emph{sampling} of arms, adaptive \emph{stopping} of the experiment, and adaptively \emph{…

Efficiently solves exploration-exploitation in LQR using Lagrangian relaxation.

Proposes H-UCRL for efficient model-based RL with sublinear regret.

A fundamental problem arising in many areas of machine learning is the evaluation of the likelihood of a given observation under different nominal distributions. Frequently, these nominal distributions are themselves estimated from data, which makes them susceptible to estimation errors. We thus propose to replace each…

DR-RPO optimizes robust policies in RL with limited interaction, achieving sublinear regret.

New method for optimistic planning in MDPs using regularization.

TF-Coder simplifies tensor manipulation programming in TensorFlow.

Optimistic Hedge achieves optimal regret bounds in two-player zero-sum games.

Optimistic method adapted for faster convex-concave min-max problems.

Value-based methods constitute a fundamental methodology in planning and deep reinforcement learning (RL). In this paper, we propose to exploit the underlying structures of the state-action value function, i.e., Q function, for both planning and deep RL. In particular, if the underlying system dynamics lead to some glo…

This paper develops a model of reference-dependent assessment of subjective beliefs in which loss-averse people optimally choose the expectation as the reference point to balance the current felicity from the optimistic anticipation and the future disappointment from the realisation. The choice of over-optimism or over…

Optimistic PPO variant solves linear MDPs with improved regret bound.

Unified framework for CO problems using RL, providing optimal solutions and convergence guarantees.

Study evaluates valuation models for UK companies using case studies.

Study optimizes linear regression analysis for high-dimensional settings.

Novel approach to universal online learning for bounded losses, closing open problems.

We address the issue of limit cycling behavior in training Generative Adversarial Networks and propose the use of Optimistic Mirror Decent (OMD) for training Wasserstein GANs. Recent theoretical results have shown that optimistic mirror decent (OMD) can enjoy faster regret rates in the context of zero-sum games. WGANs …

Solves open problem on universally consistent online learning with unbounded losses.

In this paper, we prove some convergence results of a special case of optimistic policy iteration algorithm for stochastic shortest path problem. We consider both Monte Carlo and $TD(λ)$ methods for the policy evaluation step under the condition that the termination state will eventually be reached almost surely.

Stochastic Q-learning tackles large action spaces with reduced computation.

Accelerates policy optimization in RL with optimistic and adaptive updates.

Unified view on selective credit assignment for reinforcement learning.

Unified framework for model-based RL with sample complexity guarantees.