arXiv research

New reward function improves GAIL performance in task-based environments.

Many reinforcement-learning researchers treat the reward function as a part of the environment, meaning that the agent can only know the reward of a state if it encounters that state in a trial run. However, we argue that this is an unnecessary limitation and instead, the reward function should be provided to the learn…

We present a novel method for learning a set of disentangled reward functions that sum to the original environment reward and are constrained to be independently obtainable. We define independent obtainability in terms of value functions with respect to obtaining one learned reward while pursuing another learned reward…

This work characterizes reward function partial identifiability and its impact on policy optimization.

EPIC quantifies reward differences without policy optimization.

Paper proposes operator deep Q-learning for quick reward adaptation.

AIRL learns robust, generalizable reward functions from demonstrations.

In many sequential decision making tasks, it is challenging to design reward functions that help an RL agent efficiently learn behavior that is considered good by the agent designer. A number of different formulations of the reward-design problem, or close variants thereof, have been proposed in the literature. In this…

Paper proposes RRD to learn proxy rewards for sparse delayed rewards in episodic reinforcement learning.

Enhances reward specification in RL with a novel language-based approach.

PQR estimates reward functions from actions and states without assuming state-only rewards.

This paper introduces a new reward shaping method for average-reward reinforcement learning.

Learning reward functions can lead to poor policy performance despite low error.

One obstacle to applying reinforcement learning algorithms to real-world problems is the lack of suitable reward functions. Designing such reward functions is difficult in part because the user only has an implicit understanding of the task objective. This gives rise to the agent alignment problem: how do we create age…

Improves imitation learning in RL by learning reward function efficiently.

Bayesian REX learns Atari games from demonstrations efficiently.

Paper introduces a novel reward function for noisy financial markets using imitation learning.

New algorithm for reward-free RL with linear function approximation, reducing sample complexity.

New algorithm finds optimal policies without knowing reward functions.

Reinforcement learning (RL) has achieved tremendous success as a general framework for learning how to make decisions. However, this success relies on the interactive hand-tuning of a reward function by RL experts. On the other hand, inverse reinforcement learning (IRL) seeks to learn a reward function from readily-obt…

This thesis tackles learning reward functions from human comparative feedback.

To solve complex real-world problems with reinforcement learning, we cannot rely on manually specified reward functions. Instead, we can have humans communicate an objective to the agent directly. In this work, we combine two approaches to learning from human feedback: expert demonstrations and trajectory preferences. …

Bayesian inverse reinforcement learning (IRL) methods are ideal for safe imitation learning, as they allow a learning agent to reason about reward uncertainty and the safety of a learned policy. However, Bayesian IRL is computationally intractable for high-dimensional problems because each sample from the posterior req…

Reinforcement learning is a promising framework for solving control problems, but its use in practical situations is hampered by the fact that reward functions are often difficult to engineer. Specifying goals and tasks for autonomous machines, such as robots, is a significant challenge: conventionally, reward function…

Imitation Learning describes the problem of recovering an expert policy from demonstrations. While inverse reinforcement learning approaches are known to be very sample-efficient in terms of expert demonstrations, they usually require problem-dependent reward functions or a (task-)specific reward-function regularizatio…

Reward tweaking optimizes behavior for long-term goals by adjusting the reward function.

A significant challenge for the practical application of reinforcement learning in the real world is the need to specify an oracle reward function that correctly defines a task. Inverse reinforcement learning (IRL) seeks to avoid this challenge by instead inferring a reward function from expert behavior. While appealin…

Reinforcement learning with sparse rewards is still an open challenge. Classic methods rely on getting feedback via extrinsic rewards to train the agent, and in situations where this occurs very rarely the agent learns slowly or cannot learn at all. Similarly, if the agent receives also rewards that create suboptimal m…

Reinforcement learning in complex environments is a challenging problem. In particular, the success of reinforcement learning algorithms depends on a well-designed reward function. Inverse reinforcement learning (IRL) solves the problem of recovering reward functions from expert demonstrations. In this paper, we solve …

Unified framework for estimating reward functions in competitive games.

Providing a suitable reward function to reinforcement learning can be difficult in many real world applications. While inverse reinforcement learning (IRL) holds promise for automatically learning reward functions from demonstrations, several major challenges remain. First, existing IRL methods learn reward functions f…

Develops statistical framework for resolving reward function ambiguity in inverse reinforcement learning.

Reinforcement Learning (RL) agents require the specification of a reward signal for learning behaviours. However, introduction of corrupt or stochastic rewards can yield high variance in learning. Such corruption may be a direct result of goal misspecification, randomness in the reward signal, or correlation of the rew…

Unified LP framework for offline reward learning from human demonstrations and feedback.

ORIL learns a reward function from unlabeled data to improve robot learning.

Learning reward functions from data is a promising path towards achieving scalable Reinforcement Learning (RL) for robotics. However, a major challenge in training agents from learned reward models is that the agent can learn to exploit errors in the reward model to achieve high reward behaviors that do not correspond …

The paper proves the convergence of Q-value for Gaussian rewards.

Learning robust value functions given raw observations and rewards is now possible with model-free and model-based deep reinforcement learning algorithms. There is a third alternative, called Successor Representations (SR), which decomposes the value function into two components -- a reward predictor and a successor ma…

Paper proposes a method to learn and exceed expert demonstrations in unknown reward environments.

A new framework isolates exploration challenges in RL without explicit rewards.

New RL method explores environments without rewards, achieving efficient policy generation.

Text generation is a crucial task in NLP. Recently, several adversarial generative models have been proposed to improve the exposure bias problem in text generation. Though these models gain great success, they still suffer from the problems of reward sparsity and mode collapse. In order to address these two problems, …

Improved RCPs for MABs using normalized weight functions.

New method learns time-invariant rewards from demonstrations.

A language for specifying complex reinforcement learning tasks.

Abstract MDPs enable strategic exploration and fast reward transfer in complex environments.

New IRL algorithm identifies optimal reward and policy from expert demonstrations.

New algorithm learns optimal policies without explicit rewards.