arXiv research

This paper studies GAIL's global convergence for general MDP and nonlinear rewards.

This paper improves understanding of GAIL's generalization and computational efficiency.

New algorithm improves GAIL for image sequences with global encoder and reward penalization.

GAIL with neural networks converges to global optima and has a known rate.

Triple-GAIL learns from multiple sources to improve imitation learning for complex behaviors.

We study risk-sensitive imitation learning where the agent's goal is to perform at least as well as the expert in terms of a risk profile. We first formulate our risk-sensitive imitation learning setting. We consider the generative adversarial approach to imitation learning (GAIL) and derive an optimization problem for…

Combines BC and GAIL for efficient imitation learning.

The paper analyzes the value discrepancies of imitation learning methods.

New model learns better policies from expert demonstrations with higher efficiency.

New reward function improves GAIL performance in task-based environments.

Paper optimizes GAIL for online and offline learning with linear approximations.

We address the problem of imitation learning with multi-modal demonstrations. Instead of attempting to learn all modes, we argue that in many tasks it is sufficient to imitate any one of them. We show that the state-of-the-art methods such as GAIL and behavior cloning, due to their choice of loss function, often incorr…

The paper explores how duality applies to reinforcement learning.

TRAIL improves robot imitation learning by focusing on task-relevant features.

GAIL is a recent successful imitation learning architecture that exploits the adversarial training procedure introduced in GANs. Albeit successful at generating behaviours similar to those demonstrated to the agent, GAIL suffers from a high sample complexity in the number of interactions it has to carry out in the envi…

This study compares 6 imitation learning algorithms using a common dataset and hyperparameter budget.

WDAIL uses Wasserstein distance for more effective reward shaping in IL.

Imitation learning (IL) aims to learn an optimal policy from demonstrations. However, such demonstrations are often imperfect since collecting optimal ones is costly. To effectively learn from imperfect demonstrations, we propose a novel approach that utilizes confidence scores, which describe the quality of demonstrat…

Learning to imitate expert behavior from demonstrations can be challenging, especially in environments with high-dimensional, continuous observations and unknown dynamics. Supervised learning methods based on behavioral cloning (BC) suffer from distribution shift: because the agent greedily imitates demonstrated action…

The use of imitation learning to learn a single policy for a complex task that has multiple modes or hierarchical structure can be challenging. In fact, previous work has shown that when the modes are known, learning separate policies for each mode or sub-task can greatly improve the performance of imitation learning. …

Compared to reinforcement learning, imitation learning (IL) is a powerful paradigm for training agents to learn control policies efficiently from expert demonstrations. However, in most cases, obtaining demonstration data is costly and laborious, which poses a significant challenge in some scenarios. A promising altern…

DSAC improves robustness of imitation learning.

We consider the problem of imitation learning from expert demonstrations in partially observable Markov decision processes (POMDPs). Belief representations, which characterize the distribution over the latent states in a POMDP, have been modeled using recurrent neural networks and probabilistic latent variable models, …

Paper addresses reward learning issues in RL, improving both under- and over-estimation.

Paper improves imitation learning from observations by minimizing inverse dynamics disagreement.

An accurate model of a patient's individual survival distribution can help determine the appropriate treatment for terminal patients. Unfortunately, risk scores (e.g., from Cox Proportional Hazard models) do not provide survival probabilities, single-time probability models (e.g., the Gail model, predicting 5 year prob…

New method simplifies and improves imitation learning without adversarial techniques.

BCO* improves BCO by concurrently training inverse dynamics and expert policy.

Online Apprenticeship Learning aims to match expert performance without access to cost functions.

Integration of reinforcement learning and imitation learning is an important problem that has been studied for a long time in the field of intelligent robotics. Reinforcement learning optimizes policies to maximize the cumulative reward, whereas imitation learning attempts to extract general knowledge about the traject…

Unified probabilistic perspective on imitation learning methods using divergence minimization.