arXiv research

RECODE uses clustering and embedding to track state visitation counts in RL.

A new neural network model using weighted Lehmer means and multiplets.

BYOL-Explore learns to explore visually-rich environments by predicting world dynamics.

A new multi-objective RL framework improves intrinsic exploration performance.

Study incentivizes exploration in non-stationary MAB with compensation.

Effective and intelligent exploration has been an unresolved problem for reinforcement learning. Most contemporary reinforcement learning relies on simple heuristic strategies such as $ε$-greedy exploration or adding Gaussian noise to actions. These heuristics, however, are unable to intelligently distinguish the well …

New exploration bonuses improve reinforcement learning efficiency.

New findings on when to use action space exploration in reinforcement learning.

Proposes a method to avoid excessive exploration in reinforcement learning.

Balancing exploration and exploitation remains a key challenge in reinforcement learning (RL). State-of-the-art RL algorithms suffer from high sample complexity, particularly in the sparse reward case, where they can do no better than to explore in all directions until the first positive rewards are found. To mitigate …

A grand challenge in reinforcement learning is intelligent exploration, especially when rewards are sparse or deceptive. Two Atari games serve as benchmarks for such hard-exploration domains: Montezuma's Revenge and Pitfall. On both games, current RL algorithms perform poorly, even those with intrinsic motivation, whic…

New algorithm reduces regret by allowing free exploration in multi-armed bandits.

New meta-RL method avoids exploration-exploitation trade-off.

The paper tackles pure exploration in multi-armed bandits with low rank structure using oblivious sampling.

Regularization-induced exploration improves contextual bandit performance.

The Exploration-Exploitation tradeoff arises in Reinforcement Learning when one cannot tell if a policy is optimal. Then, there is a constant need to explore new actions instead of exploiting past experience. In practice, it is common to resolve the tradeoff by using a fixed exploration mechanism, such as $ε$-greedy ex…

We introduce the community exploration problem that has many real-world applications such as online advertising. In the problem, an explorer allocates limited budget to explore communities so as to maximize the number of members he could meet. We provide a systematic study of the community exploration problem, from off…

MADE improves exploration in RL by maximizing deviation from explored regions.

Efficient exploration is one of the key challenges for reinforcement learning (RL) algorithms. Most traditional sample efficiency bounds require strategic exploration. Recently many deep RL algorithms with simple heuristic exploration strategies that have few formal guarantees, achieve surprising success in many domain…

Maximizes Rényi entropy for efficient exploration in reward-free RL.

An agent explores indefinitely in an environment with unlimited rewards.

Proposes EVE for efficient exploration in reinforcement learning.

Exploration is an extremely challenging problem in reinforcement learning, especially in high dimensional state and action spaces and when only sparse rewards are available. Effective representations can indicate which components of the state are task relevant and thus reduce the dimensionality of the space to explore.…

This paper provides an empirical evaluation of recently developed exploration algorithms within the Arcade Learning Environment (ALE). We study the use of different reward bonuses that incentives exploration in reinforcement learning. We do so by fixing the learning algorithm used and focusing only on the impact of the…

In this article we explore an alternative approach to address deep exploration and we introduce the ISL algorithm, which is efficient at performing deep exploration. Similarly to maximum entropy RL, we derive the algorithm by augmenting the traditional RL objective with a novel regularization term. A distinctive featur…

ASE safely explores unknown MDPs with unknown dynamics, improving sample efficiency.

Achieving efficient and scalable exploration in complex domains poses a major challenge in reinforcement learning. While Bayesian and PAC-MDP approaches to the exploration problem offer strong formal guarantees, they are often impractical in higher dimensions due to their reliance on enumerating the state-action space.…

Black-box optimizers that explore in parameter space have often been shown to outperform more sophisticated action space exploration methods developed specifically for the reinforcement learning problem. We examine these black-box methods closely to identify situations in which they are worse than action space explorat…

Proposes a method to enhance exploration in RL using temporal difference uncertainties.

Task-agnostic RL tackles exploration in MDPs with multiple tasks.

Proposes EE-Net for neural exploration in contextual bandits.

A fundamental issue in reinforcement learning algorithms is the balance between exploration of the environment and exploitation of information already obtained by the agent. Especially, exploration has played a critical role for both efficiency and efficacy of the learning process. However, Existing works for explorati…

We address the challenge of effective exploration while maintaining good performance in policy gradient methods. As a solution, we propose diverse exploration (DE) via conjugate policies. DE learns and deploys a set of conjugate policies which can be conveniently generated as a byproduct of conjugate gradient descent. …

Hyper addresses the hyperparameter tuning challenge in RL, improving exploration efficiency and robustness.

Recently deep reinforcement learning (DRL) has achieved outstanding success on solving many difficult and large-scale RL problems. However the high sample cost required for effective learning often makes DRL unaffordable in resource-limited applications. With the aim of improving sample efficiency and learning performa…

Algorithm achieves optimal pricing with minimal exploration for dynamic markets.

Langevin DQN achieves deep exploration using Gaussian noise.

New $L_1$-Coverage objective simplifies exploration in reinforcement learning.

This work introduces CAET, an algorithm for cost-aware pairwise pure exploration.

BeBold improves exploration in sparse-reward tasks by regulating visitation counts.

Efficient exploration in complex environments remains a major challenge for reinforcement learning. We propose bootstrapped DQN, a simple algorithm that explores in a computationally and statistically efficient manner through use of randomized value functions. Unlike dithering strategies such as epsilon-greedy explorat…

A single policy suffices for near-optimal parallel exploration in RL.

Study how untrained policies explore in RL environments.

New RL method explores environments without rewards, achieving efficient policy generation.

The paper analyzes the intrinsic exploration terms in policy-gradient algorithms.

Efficient exploration remains a challenging problem in reinforcement learning, especially for those tasks where rewards from environments are sparse. A commonly used approach for exploring such environments is to introduce some "intrinsic" reward. In this work, we focus on model uncertainty estimation as an intrinsic r…

Simpler ε-greedy with longer action durations improves exploration.

We study an exploration method for model-free RL that generalizes the counter-based exploration bonus methods and takes into account long term exploratory value of actions rather than a single step look-ahead. We propose a model-free RL method that modifies Delayed Q-learning and utilizes the long-term exploration bonu…