arXiv research

We discuss the relative merits of optimistic and randomized approaches to exploration in reinforcement learning. Optimistic approaches presented in the literature apply an optimistic boost to the value estimate at each state-action pair and select actions that are greedy with respect to the resulting optimistic value f…

Proposes Optimistic Pessimistically Initialised Q-Learning (OPIQ) for better exploration in RL.

New algorithm reduces regret in sequential decision-making problems.

Optimistic Hedge achieves optimal regret bounds in two-player zero-sum games.

Safe linear bandits over unknown polytopes avoid safety violations and suboptimal actions.

New algorithm optimizes online decision-making with dynamically generated actions.

In many cases an intelligent agent may want to learn how to mimic a single observed demonstrated trajectory. In this work we consider how to perform such procedural learning from observation, which could help to enable agents to better use the enormous set of video data on observation sequences. Our approach exploits t…

New framework for reinforcement learning with sporadic state observations.

Proposes H-UCRL for efficient model-based RL with sublinear regret.

Accelerates policy optimization in RL with optimistic and adaptive updates.

A new framework for structured bandits using influence diagrams and variational Thompson sampling.

We design a new provably efficient algorithm for episodic reinforcement learning with generalized linear function approximation. We analyze the algorithm under a new expressivity assumption that we call "optimistic closure," which is strictly weaker than assumptions from prior analyses for the linear setting. With opti…

We consider the problem of online planning in a Markov Decision Process when given only access to a generative model, restricted to open-loop policies - i.e. sequences of actions - and under budget constraint. In this setting, the Open-Loop Optimistic Planning (OLOP) algorithm enjoys good theoretical guarantees but is …

New batch RL method avoids overly optimistic policies.

Optimistic Thompson Sampling reduces regret in unknown multi-player games.

Algorithm converges to Nash equilibria in competitive games.

Actor-critic methods, a type of model-free Reinforcement Learning, have been successfully applied to challenging tasks in continuous control, often achieving state-of-the art performance. However, wide-scale adoption of these methods in real-world domains is made difficult by their poor sample efficiency. We address th…

Adaptive discretization improves model-based RL in large spaces.

The thesis optimizes quantum state exploration using bandit algorithms.

New principle optimizes bandit decisions with context.

Optimistic Q-learning reduces sample complexity for systems with known dynamics.

New findings on universal learning in contextual bandits with adversarial rewards.

New algorithm achieves consistent learning from context in bandit problems.

RandQL is a new model-free algorithm for MDPs with a novel learning rate randomization approach.

Improved POMDP regret to sqrt(T) with known observation model.

Bayesian classifier improves robustness with optimistic score ratio.

We show that the optimistic limits of the colored Jones polynomials of the hyperbolic knots coincide with the optimistic limits of the Kashaev invariants modulo $4π^2$.

OPSRL algorithm reduces regret with few samples in reinforcement learning.

The optimistic limit is the mathematical formulation of the classical limit which is a physical method to expect the actual limit by using saddle point method of certain potential function. The original optimistic limit of the Kashaev invariant was formulated by Yokota, and a modified formulation was suggested by the a…

Optimistic estimate predicts best fitting performance of nonlinear models.

Optimistic NPG improves policy optimization in online RL with efficient sample complexity.

We give a simple optimistic algorithm for which it is easy to derive regret bounds of $\tilde{O}(\sqrt{t_{\rm mix} SAT})$ after $T$ steps in uniformly ergodic Markov decision processes with $S$ states, $A$ actions, and mixing time parameter $t_{\rm mix}$. These bounds are the first regret bounds in the general, non-epi…

Characterizes concept classes for optimistic online learning.

We show how to take any two parameter-free online learning algorithms with different regret guarantees and obtain a single algorithm whose regret is the minimum of the two base algorithms. Our method is embarrassingly simple: just add the iterates. This trick can generate efficient algorithms that adapt to many norms s…

Paper optimizes reinforcement learning in self-play games with reduced steps.

Real-world applications require RL algorithms to act safely. During learning process, it is likely that the agent executes sub-optimal actions that may lead to unsafe/poor states of the system. Exploration is particularly brittle in high-dimensional state/action space due to increased number of low-performing actions. …

We study online reinforcement learning for finite-horizon deterministic control systems with {\it arbitrary} state and action spaces. Suppose that the transition dynamics and reward function is unknown, but the state and action space is endowed with a metric that characterizes the proximity between different states and…

New algorithms reduce forecasting errors by leveraging optimistic learning and hinting.

Kernel-UCBVI algorithm balances exploration and exploitation in metric state-action spaces.

Omega method mitigates noise in stochastic game optimization.

New method minimizes regret in AMDP with high probability.

Optimal algorithm for linear bandits on ellipsoids with minimax regret bound.

We establish that an optimistic variant of Q-learning applied to a fixed-horizon episodic Markov decision process with an aggregated state representation incurs regret $\tilde{\mathcal{O}}(\sqrt{H^5 M K} + εHK)$, where $H$ is the horizon, $M$ is the number of aggregate states, $K$ is the number of episodes, and $ε$ is …

Unified framework for model- and value-optimistic reinforcement learning.

New RL algorithm learns good actions from offline data, reducing uncertainty and divergence.

It is well known that in stochastic multi-armed bandits (MAB), the sample mean of an arm is typically not an unbiased estimator of its true mean. In this paper, we decouple three different sources of this selection bias: adaptive \emph{sampling} of arms, adaptive \emph{stopping} of the experiment, and adaptively \emph{…

Efficiently solves exploration-exploitation in LQR using Lagrangian relaxation.

New RL method handles large state-action spaces with complex models.