New method simplifies and improves imitation learning without adversarial techniques.
problem Stable optimization and convergence issues in adversarial imitation learning methods.
method Proposes a non-adversarial framework for imitation learning, providing stronger convergence guarantees.
result Shows AIRL as a special case and derives new algorithms for offline imitation learning.
TRAIL improves robot imitation learning by focusing on task-relevant features.
problem Discriminator networks learn spurious associations, providing poor reward signals.
method Constrained discriminator optimization to learn task-relevant rewards.
result TRAIL outperforms GAIL and behaviour cloning in robotic manipulation tasks.
HGAIL learns policies without expert demonstrations.
problem Lack of expert demonstrations in imitation learning.
method Combines hindsight and GAIL to learn policies.
result Comparable performance to current methods, with curriculum learning.
New algorithm reduces sample complexity for imitation learning.
problem High sample complexity limits deployment of adversarial imitation algorithms.
method Trajectory-centric reinforcement learning ideas.
result Improved learning rate and efficiency in imitation tasks.
GASIL encourages agents to imitate past good trajectories in reinforcement learning.
problem Long-term credit assignment in sparse and delayed reward environments.
method Generative Adversarial Imitation Learning (GASIL) framework.
result GASIL improves performance in reinforcement learning tasks with delayed rewards.
SAIL improves AIL by weighting adversarial rewards with support estimation.
problem Training instability and reward bias in AIL.
method Support-weighted Adversarial Imitation Learning (SAIL) extends AIL with support estimation to improve reinforcement signals.
result SAIL achieves better performance and stability on benchmark tasks.
A new method removes policy optimization in adversarial imitation learning.
problem Adversarial imitation learning's delicate alternated optimization.
method Explicitly condition discriminator on two policies, solving generator's optimization problem directly.
result Simpler approach competitive to prevalent methods.
Relational Mimic improves visual imitation learning from video demonstrations.
problem Improving robustness and sample efficiency in visual imitation learning.
method Combines generative adversarial networks and relational learning.
result Improves agent performance in challenging locomotion tasks.
WDAIL uses Wasserstein distance for more effective reward shaping in IL.
problem Fixed reward functions in GAIL limit performance on complex tasks.
method Introduces Wasserstein distance and PPO for improved reward shaping and stability.
result Significant performance improvement in complex MuJoCo tasks.
Improves imitation learning in RL by learning reward function efficiently.
problem Lack of effective reward function approximation in AIRL for imitation tasks.
method Proposes Off-Policy AIRL that combines adversarial learning with efficient reward function approximation.
result Shows superior imitation performance and efficiency compared to state-of-the-art AIL algorithms.
We study risk-sensitive imitation learning where the agent's goal is to perform at least as well as the expert in terms of a risk profile. We first formulate our risk-sensitive imitation learning setting. We consider the generative adversarial approach to imitation learning (GAIL) and derive an optimization problem for…
End-to-end Dyna-AIL combines planning and learning for faster convergence.
problem Efficient imitation learning with minimal interactions.
method Dyna-like framework integrating model-based planning and model-free learning.
result Our approach converges to optimal policies with fewer interactions.
Safe imitation learning with a safety layer for flexible training.
problem Flexible yet safe imitation learning for complex tasks.
method Theory and modular method with a safety layer for continuous policy, adversarial training, and worst-case safety guarantees.
result Robustness advantage of safety layer during training compared to test time.
False negatives hinder adversarial imitation learning, leading to worse performance.
problem False negatives impede the performance of adversarial imitation learning.
method Proposed a method to alleviate the impact of false negatives.
result The method consistently improves sample efficiency by at least an order of magnitude.
Imitation from observation (IfO) is the problem of learning directly from state-only demonstrations without having access to the demonstrator's actions. The lack of action information both distinguishes IfO from most of the literature in imitation learning, and also sets it apart as a method that may enable agents to l…
New method learns agent actions from raw video demonstrations.
problem Imitation learning for natural-looking movements.
method Generative Adversarial Networks (GANs) for joint state and reward learning.
result Adversarial imitation learning on raw videos produces similar performance to state-of-the-art methods.
Algorithm improves imitation learning from visual data in partially observable environments.
problem Imitation learning from visual observations with missing expert actions and partial observability.
method Theoretical analysis and Latent Adversarial Imitation from Observations algorithm combining adversarial and latent representations.
result Latent Adversarial Imitation from Observations achieves state-of-the-art performance in high-dimensional robotic tasks.
Paper improves AIRL by enhancing policy imitation and addressing reward recovery issues.
problem Inadequate policy imitation and limited transferable reward recovery in AIRL.
method Substituted built-in algorithm with SAC for policy updating and proposed PPO-AIRL + SAC hybrid framework.
result SAC improves policy imitation but hinders reward recovery; PPO-AIRL + SAC achieves satisfactory transfer effect.
New reward function improves GAIL performance in task-based environments.
problem Reward bias in adversarial imitation learning.
method Proposed a new reward function to overcome existing biases.
result New reward function outperforms existing methods in task-based environments.
New model learns better policies from expert demonstrations with higher efficiency.
problem Learning accurate policies from expert demonstrations with high efficiency.
method Generative adversarial imitation learning (GAIL) model that learns f-divergence automatically. result Learns better policies with higher data efficiency in physics-based control tasks.
Triple-GAIL learns from multiple sources to improve imitation learning for complex behaviors.
problem Limited scalability of GAIL in real-world scenarios like autonomous vehicles.
method Integrates expert demonstrations and generated experiences with an auxiliary skill selector.
result Triple-GAIL outperforms state-of-the-art methods in learning complex behaviors.
A new method uses optimal transport for imitation learning, improving efficiency and performance.
problem Recovering expert policies from demonstrations with efficient and general reward functions.
method Proposes Wasserstein Adversarial Imitation Learning, using Kantorovich potentials as reward functions and regularized optimal transport.
result Significantly improves sample-efficiency and average cumulative rewards in robotic experiments.
New framework integrates imitation and reinforcement learning for better robot performance.
problem Combining reinforcement and imitation learning for intelligent robotics.
method Extends probabilistic generative model framework for reinforcement learning and develops pMDP-MO for Markov decision processes.
result Significantly better performance than reinforcement or imitation learning alone.
DAC-SSM learns domain-agnostic states for better imitation learning.
problem Domain shifts hinder imitation learning in partially observable tasks.
method DAC-SSM uses adversarial training to remove domain-dependent information from states.
result DAC-SSM achieves comparable performance to experts in sparse reward tasks.
Paper uses Sinkhorn distances to improve imitation learning effectiveness.
problem Improving imitation learning algorithms by comparing occupancy measures.
method Formulates imitation learning as Sinkhorn distance minimization, combining optimal transport and cosine distances.
result Proposes a new critic network and transport plan that guide imitation learning.
DSAC improves robustness of imitation learning.
problem Learning from few expert data and distribution shift.
method DSAC uses adversarial reward function.
result DSAC performs well on PyBullet environments.
New method improves imitation learning from expert observations.
problem Challenges in imitation learning from observation setting.
method Reparameterized Variational Divergence Minimization.
result Our method outperforms baseline approaches in low-dimensional tasks.
VDB constrains discriminator to improve imitation learning and GANs.
problem Stability and performance issues in adversarial learning methods.
method Information bottleneck technique to modulate discriminator's accuracy.
result Significant improvements across imitation learning, inverse RL, and GANs.
Study shows a linear quadratic regulator's imitation learning converges globally.
problem Global convergence of imitation learning for linear quadratic regulators.
method Analyzed alternating gradient algorithm and established Q-linear rate of convergence.
result Established a unique saddle point for globally optimal policy and reward function.
AGAIL learns policies from incomplete demonstrations by separating state and action trajectories.
problem Learning policies from incomplete demonstrations where actions are partially available.
method Action-Guided Adversarial Imitation Learning (AGAIL) separates state and action trajectories, using actions as auxiliary information to guide policy training.
result AGAIL consistently delivers comparable performance to state-of-the-art methods even with partially available action sequences.
Reinforcement learning mimics expert behavior.
problem Learning from expert demonstrations in reinforcement learning.
method Reduction to reinforcement learning with a stationary reward.
result Expert reward can be recovered and imitation learning is bounded.
A new framework estimates expert policy support to create a reward function for imitation learning.
problem Imitation learning from expert trajectories without reinforcement signals.
method Estimating the support of the expert policy to compute a fixed reward function.
result Comparable or better performance than state-of-the-art methods on discrete and continuous domains.
NDI learns from expert demonstrations by estimating occupancy measures.
problem Imitation Learning (IL) for complex systems.
method Density estimation of expert's occupancy measure followed by RL.
result NDI achieves state-of-the-art performance on control tasks.
A new approach for learning from expert demonstrations using multiple perspectives.
problem Learning from expert demonstrations with limited information from different perspectives.
method Generative adversarial network-based active learning approach.
result Our approach effectively learns from expert demonstrations and highlights the importance of architectural choices.
Generative model learns vehicle trajectory distributions for better data generalization.
problem Data sparsity and privacy issues in urban vehicle trajectory analysis.
method Generative adversarial imitation learning framework for urban vehicle trajectory generation.
result TrajGAIL model produces synthetic trajectories similar to real ones, achieving significant performance gains.
Unified framework for imitation learning via moment matching.
problem Closing the gap between imitation and real-world performance.
method Classifying imitation learning algorithms based on reward or action-value moment matching, considering adversarial divergences.
result Derivation of bounds on policy performance for all algorithms in each class, and introduction of moment recoverability.
Generative adversarial learning is a popular new approach to training generative models which has been proven successful for other related problems as well. The general idea is to maintain an oracle D that discriminates between the expert's data distribution and that of the generative model G. The generative model …
Protects proprietary policies from imitation learning by training adversarial policy ensembles.
problem Protecting policies from external observers cloning them.
method Introduces a reinforcement learning framework that trains an ensemble of near-optimal policies, making demonstrations useless for external observers.
result Demonstrates the existence of 'non-clonable' ensembles and provides a solution to the optimization problem.
Paper uses deep imitation learning to predict aircraft trajectories accurately.
problem Inefficient and costly Air Traffic Management system limits predictability.
method Generative Adversarial Imitation Learning framework with trajectory clustering and classification.
result Accurate predictions for entire trajectory stages, pre- and tactical.
New method improves IL from imperfect demos using confidence scores.
problem Learning optimal policies from imperfect demonstrations is challenging.
method Proposes two confidence-based IL methods: 2IWIL and IC-GAIL.
result Confidence scores from sub-optimal demos significantly improve IL performance.
New method learns belief representations for GAIL in POMDPs.
problem Imitation learning in partially observable Markov decision processes (POMDPs).
method Joint learning of belief module and policy with task-aware imitation loss and belief regularization.
result Our BMIL approach outperforms GAIL and task-agnostic belief learning.
The paper shows how imitation learning can be exploited to extract DRL policies.
problem The confidentiality and integrity of DRL policies are vulnerable to model extraction attacks.
method Developed proof-of-concept attacks using imitation learning techniques.
result Demonstrated the feasibility of launching black-box attacks against DRL policies.
GAIL is a recent successful imitation learning architecture that exploits the adversarial training procedure introduced in GANs. Albeit successful at generating behaviours similar to those demonstrated to the agent, GAIL suffers from a high sample complexity in the number of interactions it has to carry out in the envi…
Imitation learning algorithms can be used to learn a policy from expert demonstrations without access to a reward signal. However, most existing approaches are not applicable in multi-agent settings due to the existence of multiple (Nash) equilibria and non-stationary environments. We propose a new framework for multi-…
New algorithm learns sub-task policies from unsegmented demonstrations.
problem Challenges in learning hierarchical policies from unsegmented demonstrations.
method Generative adversarial imitation learning framework with directed information maximization.
result Automatic learning of sub-task policies from unsegmented demonstrations.
This paper improves understanding of GAIL's generalization and computational efficiency.
problem Understanding the theoretical properties of GAIL, especially its generalization and computational aspects.
method Investigates GAIL's theoretical properties, showing guarantees for generalization and computational efficiency.
result GAIL can be efficiently solved by stochastic first order optimization algorithms with sublinear convergence.
Bayesian Robust Optimization for Imitation Learning (BROIL) balances risk and reward.
problem Learning robust policies for new states in imitation learning.
method Bayesian reward function inference and user-specific risk tolerance.
result BROIL outperforms risk-sensitive and risk-neutral algorithms.
The paper analyzes the value discrepancies of imitation learning methods.
problem Analyzing the value discrepancies of imitation learning methods.
method Discrepancy propagation analysis for infinite-horizon settings.
result GAIL has less compounding errors than behavioral cloning.