arXiv research

A new learning method for prosthetic arms without explicit rewards.

Paper develops methods to optimize policies directly from human feedback without reward inference.

Bayesian REX learns Atari games from demonstrations efficiently.

EPIC quantifies reward differences without policy optimization.

ORIL learns a reward function from unlabeled data to improve robot learning.

Learning reward functions from data is a promising path towards achieving scalable Reinforcement Learning (RL) for robotics. However, a major challenge in training agents from learned reward models is that the agent can learn to exploit errors in the reward model to achieve high reward behaviors that do not correspond …

New algorithm finds optimal policies without knowing reward functions.

There exist a number of reinforcement learning algorithms which learnby climbing the gradient of expected reward. Their long-runconvergence has been proved, even in partially observableenvironments with non-deterministic actions, and without the need fora system model. However, the variance of the gradient estimator ha…

To solve complex real-world problems with reinforcement learning, we cannot rely on manually specified reward functions. Instead, we can have humans communicate an objective to the agent directly. In this work, we combine two approaches to learning from human feedback: expert demonstrations and trajectory preferences. …

Paper improves sample complexity for reward-free RL in low-rank MDPs.

HyperX uses reward bonuses to enable efficient exploration in meta-learning.

Dealing with sparse rewards is a longstanding challenge in reinforcement learning. The recent use of hindsight methods have achieved success on a variety of sparse-reward tasks, but they fail on complex tasks such as stacking multiple blocks with a robot arm in simulation. Curiosity-driven exploration using the predict…

This paper introduces a new reward shaping method for average-reward reinforcement learning.

New algorithm reduces reinforcement learning regret to sqrt(T) without strong dynamics assumptions.

PQR estimates reward functions from actions and states without assuming state-only rewards.

The combination of deep neural network models and reinforcement learning algorithms can make it possible to learn policies for robotic behaviors that directly read in raw sensory inputs, such as camera images, effectively subsuming both estimation and control into one model. However, real-world applications of reinforc…

A method to train multi-agent reinforcement learning models without intrinsic rewards.

Recent reinforcement learning (RL) approaches have shown strong performance in complex domains such as Atari games, but are often highly sample inefficient. A common approach to reduce interaction time with the environment is to use reward shaping, which involves carefully designing reward functions that provide the ag…

Unified LP framework for offline reward learning from human demonstrations and feedback.

New RLHF algorithm identifies optimal policies from human feedback without explicit reward inference.

New RL method explores environments without rewards, achieving efficient policy generation.

New method uses simple sensor intentions to learn complex tasks.

Observer learns optimal policy from learner's actions without rewards.

The AlphaZero algorithm for the learning of strategy games via self-play, which has produced superhuman ability in the games of Go, chess, and shogi, uses a quantitative reward function for game outcomes, requiring the users of the algorithm to explicitly balance different components of the reward against each other, s…

New AI learns like neurons, generalizing from sparse rewards.

GE finds failures in autonomous systems without domain heuristics.

This paper proposes a new algorithm for learning guidance rewards in RL.

Improves imitation learning in RL by learning reward function efficiently.

CoCoRL learns safe constraints from demonstrations with unknown rewards.

New findings on universal learning in contextual bandits with adversarial rewards.

Transforming sparse outcomes into dense process rewards for efficient reinforcement learning.

New algorithm learns policies without explicit rewards for MDPs.

Reinforcement learning algorithms rely on carefully engineering environment rewards that are extrinsic to the agent. However, annotating each environment with hand-designed, dense rewards is not scalable, motivating the need for developing reward functions that are intrinsic to the agent. Curiosity is a type of intrins…

This study proposes hidden state curiosity to enhance RL models' resilience against noise.

New approach uses unlabeled prior data to accelerate exploration in sparse reward tasks.

Learning to control an environment without hand-crafted rewards or expert data remains challenging and is at the frontier of reinforcement learning research. We present an unsupervised learning algorithm to train agents to achieve perceptually-specified goals using only a stream of observations and actions. Our agent s…

Reinforcement learning requires manual specification of a reward function to learn a task. While in principle this reward function only needs to specify the task goal, in practice reinforcement learning can be very time-consuming or even infeasible unless the reward function is shaped so as to provide a smooth gradient…

Solving tasks with sparse rewards is one of the most important challenges in reinforcement learning. In the single-agent setting, this challenge is addressed by introducing intrinsic rewards that motivate agents to explore unseen regions of their state spaces; however, applying these techniques naively to the multi-age…

Recent studies have shown that reinforcement learning (RL) models are vulnerable in various noisy scenarios. For instance, the observed reward channel is often subject to noise in practice (e.g., when rewards are collected through sensors), and is therefore not credible. In addition, for applications such as robotics, …

We seek to align agent behavior with a user's objectives in a reinforcement learning setting with unknown dynamics, an unknown reward function, and unknown unsafe states. The user knows the rewards and unsafe states, but querying the user is expensive. To address this challenge, we propose an algorithm that safely and …

This paper studies poisoning attacks in episodic RL and discovers their effectiveness depends on reward bounds.

It is difficult to be able to imitate well in unknown states from a small amount of expert data and sampling data. Supervised learning methods such as Behavioral Cloning do not require sampling data, but usually suffer from distribution shift. The methods based on reinforcement learning, such as inverse reinforcement l…

OSIL learns safe policies from unsafe demonstrations.

Reinforcement learning methods require careful design involving a reward function to obtain the desired action policy for a given task. In the absence of hand-crafted reward functions, prior work on the topic has proposed several methods for reward estimation by using expert state trajectories and action pairs. However…

New algorithm learns optimal policies without explicit rewards.

Bayesian inverse reinforcement learning (IRL) methods are ideal for safe imitation learning, as they allow a learning agent to reason about reward uncertainty and the safety of a learned policy. However, Bayesian IRL is computationally intractable for high-dimensional problems because each sample from the posterior req…

It is a long-standing challenge to enable an intelligent agent to learn in one environment and generalize to an unseen environment without further data collection and finetuning. In this paper, we consider a zero shot generalization problem setup that complies with biological intelligent agents' learning and generaliza…

Algorithm optimizes multi-player learning with noisy rewards without direct communication.