arXiv research

Proposes Optimistic Pessimistically Initialised Q-Learning (OPIQ) for better exploration in RL.

Proposes H-UCRL for efficient model-based RL with sublinear regret.

We discuss the relative merits of optimistic and randomized approaches to exploration in reinforcement learning. Optimistic approaches presented in the literature apply an optimistic boost to the value estimate at each state-action pair and select actions that are greedy with respect to the resulting optimistic value f…

Efficiently solves exploration-exploitation in LQR using Lagrangian relaxation.

New algorithm reduces regret in sequential decision-making problems.

Optimistic NPG improves policy optimization in online RL with efficient sample complexity.

Omega method mitigates noise in stochastic game optimization.

The paper analyzes CMDPs, balancing exploration and exploitation to avoid constraint violations.

While policy-based reinforcement learning (RL) achieves tremendous successes in practice, it is significantly less understood in theory, especially compared with value-based RL. In particular, it remains elusive how to design a provably efficient policy optimization algorithm that incorporates exploration. To bridge su…

Free lunch from noise reveals linear spectral features for RL.

New algorithm explores reinforcement learning with noisy data.

New approach uses unlabeled prior data to accelerate exploration in sparse reward tasks.

Optimistic RL algorithms are simplified for deep RL with competitive performance.

Model-based Bayesian Reinforcement Learning (BRL) allows a found formalization of the problem of acting optimally while facing an unknown environment, i.e., avoiding the exploration-exploitation dilemma. However, algorithms explicitly addressing BRL suffer from such a combinatorial explosion that a large body of work r…

Optimistic actor-critic tackles linear MDPs with parametric policies.

New method boosts skill learning by encouraging optimistic exploration.

New rule reduces exploration regret to logarithmic, improving bad episode handling.

This paper analyzes how randomizing rewards in MBRL can improve performance without being overly optimistic.

RandQL is a new model-free algorithm for MDPs with a novel learning rate randomization approach.

Robust Optimization has traditionally taken a pessimistic, or worst-case viewpoint of uncertainty which is motivated by a desire to find sets of optimal policies that maintain feasibility under a variety of operating conditions. In this paper, we explore an optimistic, or best-case view of uncertainty and show that it …

New RL algorithm optimizes policies with bandit feedback, matching previous bounds.

New batch RL method avoids overly optimistic policies.

New RL algorithm tackles online robust MDPs with uncertainty.

Actor-critic methods, a type of model-free Reinforcement Learning, have been successfully applied to challenging tasks in continuous control, often achieving state-of-the art performance. However, wide-scale adoption of these methods in real-world domains is made difficult by their poor sample efficiency. We address th…

New framework improves learning across multiple distributions.

Bayesian classifier improves robustness with optimistic score ratio.

Study designs steering rewards for MFGs with unknown dynamics and model uncertainty.

We show that the optimistic limits of the colored Jones polynomials of the hyperbolic knots coincide with the optimistic limits of the Kashaev invariants modulo $4π^2$.

The thesis optimizes quantum state exploration using bandit algorithms.

The optimistic limit is the mathematical formulation of the classical limit which is a physical method to expect the actual limit by using saddle point method of certain potential function. The original optimistic limit of the Kashaev invariant was formulated by Yokota, and a modified formulation was suggested by the a…

Optimistic estimate predicts best fitting performance of nonlinear models.

This paper studies systematic exploration for reinforcement learning with rich observations and function approximation. We introduce a new model called contextual decision processes, that unifies and generalizes most prior settings. Our first contribution is a complexity measure, the Bellman rank, that we show enables …

Characterizes concept classes for optimistic online learning.

Bayesian optimisation (BO) is a well-known efficient algorithm for finding the global optimum of expensive, black-box functions. The current practical BO algorithms have regret bounds ranging from $\mathcal{O}(\frac{logN}{\sqrt{N}})$ to $\mathcal O(e^{-\sqrt{N}})$, where $N$ is the number of evaluations. This paper exp…

We show how to take any two parameter-free online learning algorithms with different regret guarantees and obtain a single algorithm whose regret is the minimum of the two base algorithms. Our method is embarrassingly simple: just add the iterates. This trick can generate efficient algorithms that adapt to many norms s…

While designing the state space of an MDP, it is common to include states that are transient or not reachable by any policy (e.g., in mountain car, the product space of speed and position contains configurations that are not physically reachable). This leads to defining weakly-communicating or multi-chain MDPs. In this…

EQO uses a simple bonus term for efficient exploration in tabular RL.

New algorithms reduce forecasting errors by leveraging optimistic learning and hinting.

In many cases an intelligent agent may want to learn how to mimic a single observed demonstrated trajectory. In this work we consider how to perform such procedural learning from observation, which could help to enable agents to better use the enormous set of video data on observation sequences. Our approach exploits t…

New method minimizes regret in AMDP with high probability.

We consider the exploration-exploitation dilemma in finite-horizon reinforcement learning (RL). When the state space is large or continuous, traditional tabular approaches are unfeasible and some form of function approximation is mandatory. In this paper, we introduce an optimistically-initialized variant of the popula…

Unified framework for model- and value-optimistic reinforcement learning.

New algorithms ensure policies perform at least as good as a baseline in reinforcement learning.

Kernel-UCBVI algorithm balances exploration and exploitation in metric state-action spaces.

New algorithm fills gaps in offline data for hybrid RL, achieving similar gains without coverage assumptions.

Optimal algorithm for linear bandits on ellipsoids with minimax regret bound.

Online Apprenticeship Learning aims to match expert performance without access to cost functions.

Improved control approach for correlated bandits with better performance.