arXiv research

Develops HMRL for sparse reward RL problems, improving meta policy efficiency and transferability.

Dealing with sparse rewards is a longstanding challenge in reinforcement learning. The recent use of hindsight methods have achieved success on a variety of sparse-reward tasks, but they fail on complex tasks such as stacking multiple blocks with a robot arm in simulation. Curiosity-driven exploration using the predict…

New RL method uses distance between states instead of rewards for sparse reward environments.

Action guidance helps agents learn true objectives in games with sparse rewards.

Proposes a method to boost deep reinforcement learning with sparse rewards.

New approach uses unlabeled prior data to accelerate exploration in sparse reward tasks.

PlanGAN uses GANs to plan efficient trajectories for multi-goal tasks in sparse reward environments.

ETGL-DDPG improves DDPG for sparse reward control with new exploration and replay techniques.

Imitation learning is an effective alternative approach to learn a policy when the reward function is sparse. In this paper, we consider a challenging setting where an agent and an expert use different actions from each other. We assume that the agent has access to a sparse reward function and state-only expert observa…

SAIL learns from sub-optimal demonstrations to improve sample efficiency in sparse reward tasks.

We propose Scheduled Auxiliary Control (SAC-X), a new learning paradigm in the context of Reinforcement Learning (RL). SAC-X enables learning of complex behaviors - from scratch - in the presence of multiple sparse reward signals. To this end, the agent is equipped with a set of general auxiliary tasks, that it attempt…

Reinforcement Learning(RL) with sparse rewards is a major challenge. We propose \emph{Hindsight Trust Region Policy Optimization}(HTRPO), a new RL algorithm that extends the highly successful TRPO algorithm with \emph{hindsight} to tackle the challenge of sparse rewards. Hindsight refers to the algorithm's ability to l…

New bandit algorithms improve sparse reward learning.

New RL agent learns sparse rewards efficiently.

A framework helps reinforcement learning agents understand and decompose tasks from human demonstrations.

A new EM framework for goal-conditioned RL improves performance on sparse reward tasks.

A new approach for deep exploration in sparse reward reinforcement learning.

Sparse rewards with different magnitudes speed up learning in model-based reinforcement learning.

Deep learning has achieved remarkable successes in solving challenging reinforcement learning (RL) problems when dense reward function is provided. However, in sparse reward environment it still often suffers from the need to carefully shape reward function to guide policy optimization. This limits the applicability of…

This paper explores how environmental properties can simplify reinforcement learning in non-episodic settings.

New RL method MAC improves performance in sparse reward settings.

VO$Q$L optimizes RL with sparse rewards using weighted bounds.

Study intrinsic motivation for synergistic tasks in reinforcement learning.

Successfully navigating a complex environment to obtain a desired outcome is a difficult task, that up to recently was believed to be capable only by humans. This perception has been broken down over time, especially with the introduction of deep reinforcement learning, which has greatly increased the difficulty of tas…

R3L uses planning algorithms to efficiently explore sparse reward environments.

BeBold improves exploration in sparse-reward tasks by regulating visitation counts.

Exploration in environments with continuous control and sparse rewards remains a key challenge in reinforcement learning (RL). Recently, surprise has been used as an intrinsic reward that encourages systematic and efficient exploration. We introduce a new definition of surprise and its RL implementation named Variation…

A new reward shaping method balances learning efficiency and effectiveness for robot manipulation.

We propose Deep Q-Networks (DQN) with model-based exploration, an algorithm combining both model-free and model-based approaches that explores better and learns environments with sparse rewards more efficiently. DQN is a general-purpose, model-free algorithm and has been proven to perform well in a variety of tasks inc…

The most data-efficient algorithms for reinforcement learning in robotics are model-based policy search algorithms, which alternate between learning a dynamical model of the robot and optimizing a policy to maximize the expected return given the model and its uncertainties. However, the current algorithms lack an effec…

New AI learns like neurons, generalizing from sparse rewards.

Sparse reward is one of the biggest challenges in reinforcement learning (RL). In this paper, we propose a novel method called Generative Exploration and Exploitation (GENE) to overcome sparse reward. GENE automatically generates start states to encourage the agent to explore the environment and to exploit received rew…

Learning in sparse reward settings remains a challenge in Reinforcement Learning, which is often addressed by using intrinsic rewards. One promising strategy is inspired by human curiosity, requiring the agent to learn to predict the future. In this paper a curiosity-driven agent is extended to use these predictions di…

In environments with continuous state and action spaces, state-of-the-art actor-critic reinforcement learning algorithms can solve very complex problems, yet can also fail in environments that seem trivial, but the reason for such failures is still poorly understood. In this paper, we contribute a formal explanation of…

PixL2R maps natural language to pixel-based rewards for RL, improving sample efficiency.

This thesis proposes a derivatives hedging framework using deep learning and reinforcement learning.

This study benchmarks AI agents for personalized retail promotions using simulations.

The estimation of advantage is crucial for a number of reinforcement learning algorithms, as it directly influences the choices of future paths. In this work, we propose a family of estimates based on the order statistics over the path ensemble, which allows one to flexibly drive the learning process, towards or agains…

A method for self-supervised representation learning in partially observable environments.

We propose an algorithm for deterministic continuous Markov Decision Processes with sparse rewards that computes the optimal policy exactly with no dependency on the size of the state space. The algorithm has time complexity of $O( |R|^3 \times |A|^2 )$ and memory complexity of $O( |R| \times |A| )$, where $|R|$ is the…

Reinforcement learning agents need exploratory behaviors to escape from local optima. These behaviors may include both immediate dithering perturbation and temporally consistent exploration. To achieve these, a stochastic policy model that is inherently consistent through a period of time is in desire, especially for t…

New algorithm reduces semi-bandit regret using covariance estimates.

This paper proposes an exploration method for deep reinforcement learning based on parameter space noise. Recent studies have experimentally shown that parameter space noise results in better exploration than the commonly used action space noise. Previous methods devised a way to update the diagonal covariance matrix o…

A new method for robot manipulation tasks using imagined object goals.

Hindsight Experience Replay (HER) is a multi-goal reinforcement learning algorithm for sparse reward functions. The algorithm treats every failure as a success for an alternative (virtual) goal that has been achieved in the episode. Virtual goals are randomly selected, irrespective of which are most instructive for the…

ESPD improves learning efficiency in sparse reward reinforcement learning.

Self-supervised reward prediction improves RL in sparse reward settings.

SPReD uses uncertainty to decide imitation from demonstrations.