arXiv research

New exploration bonuses improve reinforcement learning efficiency.

This paper provides an empirical evaluation of recently developed exploration algorithms within the Arcade Learning Environment (ALE). We study the use of different reward bonuses that incentives exploration in reinforcement learning. We do so by fixing the learning algorithm used and focusing only on the impact of the…

HyperX uses reward bonuses to enable efficient exploration in meta-learning.

Improved exploration in adversarial MDPs via dilated bonuses.

Efficient exploration is necessary to achieve good sample efficiency for reinforcement learning in general. From small, tabular settings such as gridworlds to large, continuous and sparse reward settings such as robotic object manipulation tasks, exploration through adding an uncertainty bonus to the reward function ha…

Achieving efficient and scalable exploration in complex domains poses a major challenge in reinforcement learning. While Bayesian and PAC-MDP approaches to the exploration problem offer strong formal guarantees, they are often impractical in higher dimensions due to their reliance on enumerating the state-action space.…

Proposes Optimistic Pessimistically Initialised Q-Learning (OPIQ) for better exploration in RL.

Langevin DQN achieves deep exploration using Gaussian noise.

The paper analyzes the intrinsic exploration terms in policy-gradient algorithms.

All reinforcement learning algorithms must handle the trade-off between exploration and exploitation. Many state-of-the-art deep reinforcement learning methods use noise in the action selection, such as Gaussian noise in policy gradient methods or $ε$-greedy in Q-learning. While these methods are appealing due to their…

Sample-efficient exploration is crucial not only for discovering rewarding experiences but also for adapting to environment changes in a task-agnostic fashion. A principled treatment of the problem of optimal input synthesis for system identification is provided within the framework of sequential Bayesian experimental …

Although exploration in reinforcement learning is well understood from a theoretical point of view, provably correct methods remain impractical. In this paper we study the interplay between exploration and approximation, what we call approximate exploration. Our main goal is to further our theoretical understanding of …

Model-Based Reinforcement Learning (MBRL) offers a promising direction for sample efficient learning, often achieving state of the art results for continuous control tasks. However, many existing MBRL methods rely on combining greedy policies with exploration heuristics, and even those which utilize principled explorat…

We consider the exploration/exploitation problem in reinforcement learning. For exploitation, it is well known that the Bellman equation connects the value at any time-step to the expected value at subsequent time-steps. In this paper we consider a similar \textit{uncertainty} Bellman equation (UBE), which connects the…

We introduce and analyse two algorithms for exploration-exploitation in discrete and continuous Markov Decision Processes (MDPs) based on exploration bonuses. SCAL$^+$ is a variant of SCAL (Fruit et al., 2018) that performs efficient exploration-exploitation in any unknown weakly-communicating MDP for which an upper bo…

MADE improves exploration in RL by maximizing deviation from explored regions.

In this paper, we focus on a prediction-based novelty estimation strategy upon the deep reinforcement learning (DRL) framework, and present a flow-based intrinsic curiosity module (FICM) to exploit the prediction errors from optical flow estimation as exploration bonuses. We propose the concept of leveraging motion fea…

New algorithms achieve logarithmic regret in KL-regularized Markov games.

New approach incentivizes strategic agents to explore, making exploration almost free.

Paper simplifies complex AI exploration by predicting future rewards.

RandQL is a new model-free algorithm for MDPs with a novel learning rate randomization approach.

This paper improves Q-learning bounds using reference-advantage decomposition.

We consider the problem of provably optimal exploration in reinforcement learning for finite horizon MDPs. We show that an optimistic modification to value iteration achieves a regret bound of $\tilde{O}( \sqrt{HSAT} + H^2S^2A+H\sqrt{T})$ where $H$ is the time horizon, $S$ the number of states, $A$ the number of action…

Curriculum learning speeds up agent learning in Minecraft, a complex visual domain.

This paper proposes a formal approach to online learning and planning for agents operating in a priori unknown, time-varying environments. The proposed method computes the maximally likely model of the environment, given the observations about the environment made by an agent earlier in the system run and assuming know…

New algorithm for personalized healthcare with privacy guarantees.

Improved reinforcement learning algorithm with linear approximation for unknown dynamics.

New method improves RL in continuous spaces with kernel smoothing.

New algorithm tackles multi-agent reinforcement learning with optimal convergence rate.

New methods tackle robust reinforcement learning in sparse, corrupted data.

Improved sample complexity for identifying best policies in risk-sensitive reinforcement learning.

Bayes-UCBVI tackles reinforcement learning with a new upper confidence bound method.

Study differential privacy in multi-agent RL, achieving efficient and private learning.

Proposes a new risk model using stable laws to manage company-wide losses.

Improved privacy in RL with near-optimal regret bounds.

Most of the prior work on multi-agent reinforcement learning (MARL) achieves optimal collaboration by directly controlling the agents to maximize a common reward. In this paper, we aim to address this from a different angle. In particular, we consider scenarios where there are self-interested agents (i.e., worker agent…

BYOL-Explore learns to explore visually-rich environments by predicting world dynamics.

A new multi-objective RL framework improves intrinsic exploration performance.

Study incentivizes exploration in non-stationary MAB with compensation.

Effective and intelligent exploration has been an unresolved problem for reinforcement learning. Most contemporary reinforcement learning relies on simple heuristic strategies such as $ε$-greedy exploration or adding Gaussian noise to actions. These heuristics, however, are unable to intelligently distinguish the well …

New findings on when to use action space exploration in reinforcement learning.

Proposes a method to avoid excessive exploration in reinforcement learning.

Balancing exploration and exploitation remains a key challenge in reinforcement learning (RL). State-of-the-art RL algorithms suffer from high sample complexity, particularly in the sparse reward case, where they can do no better than to explore in all directions until the first positive rewards are found. To mitigate …

A grand challenge in reinforcement learning is intelligent exploration, especially when rewards are sparse or deceptive. Two Atari games serve as benchmarks for such hard-exploration domains: Montezuma's Revenge and Pitfall. On both games, current RL algorithms perform poorly, even those with intrinsic motivation, whic…

New algorithm reduces regret by allowing free exploration in multi-armed bandits.

New meta-RL method avoids exploration-exploitation trade-off.

The paper tackles pure exploration in multi-armed bandits with low rank structure using oblivious sampling.

Regularization-induced exploration improves contextual bandit performance.