A new method removes policy optimization in adversarial imitation learning.
problem Adversarial imitation learning's delicate alternated optimization.
method Explicitly condition discriminator on two policies, solving generator's optimization problem directly.
result Simpler approach competitive to prevalent methods.
HGAIL learns policies without expert demonstrations.
problem Lack of expert demonstrations in imitation learning.
method Combines hindsight and GAIL to learn policies.
result Comparable performance to current methods, with curriculum learning.
A new algorithm trains experts to safely guide agents in partially observed environments.
problem Existing imitation learning methods for POMDPs can lead to sub-optimal or unsafe policies.
method Derive an objective to encourage the expert to maximize the agent's reward, then use it to train both expert and agent.
result The algorithm produces an expert policy that the agent can safely imitate, outperforming fixed expert policies.
Paper improves AIRL by enhancing policy imitation and addressing reward recovery issues.
problem Inadequate policy imitation and limited transferable reward recovery in AIRL.
method Substituted built-in algorithm with SAC for policy updating and proposed PPO-AIRL + SAC hybrid framework.
result SAC improves policy imitation but hinders reward recovery; PPO-AIRL + SAC achieves satisfactory transfer effect.
New method uses variational inference to handle confounding in imitation learning.
problem Confounding due to different sensory inputs between expert and imitating agent.
method Train variational inference model to infer expert's latent information and use for latent-conditional policy training.
result Algorithm converges to correct interventional policy and achieves asymptotically optimal performance.
ICIL learns policies invariant to multiple environments, improving generalization.
problem Learning policies from multiple environments leads to spurious correlations.
method ICIL learns invariant feature representations and a matching imitation policy.
result ICIL policies generalize better to unseen environments.
A new imitation learning method uses random search for simple policies, outperforming complex models.
problem Complexity and reward dependency issues in imitation learning.
method Derivative-free optimization with simple linear policies and random search.
result The proposed method achieves competitive performance on MuJoCo locomotion tasks without a direct reward signal.
A new framework estimates expert policy support to create a reward function for imitation learning.
problem Imitation learning from expert trajectories without reinforcement signals.
method Estimating the support of the expert policy to compute a fixed reward function.
result Comparable or better performance than state-of-the-art methods on discrete and continuous domains.
Continuous control imitation learning fails if expert actions are smooth.
problem Continuous control imitation learning fails if expert actions are smooth.
method Study of imitation learning in discrete-time, continuous state-and-action control systems.
result Any smooth, deterministic imitator policy suffers exponentially larger error than the expert.
Improves imitation learning in RL by learning reward function efficiently.
problem Lack of effective reward function approximation in AIRL for imitation tasks.
method Proposes Off-Policy AIRL that combines adversarial learning with efficient reward function approximation.
result Shows superior imitation performance and efficiency compared to state-of-the-art AIL algorithms.
Recent work on imitation learning has generated policies that reproduce expert behavior from multi-modal data. However, past approaches have focused only on recreating a small number of distinct, expert maneuvers, or have relied on supervised learning techniques that produce unstable policies. This work extends InfoGAI…
New model learns better policies from expert demonstrations with higher efficiency.
problem Learning accurate policies from expert demonstrations with high efficiency.
method Generative adversarial imitation learning (GAIL) model that learns f-divergence automatically. result Learns better policies with higher data efficiency in physics-based control tasks.
Method solves long-horizon robotic tasks via imitation and reinforcement learning.
problem Long-horizon robotic tasks with complex sequences of actions.
method Two-phase approach: imitation learning followed by reinforcement learning.
result Method can scale to challenging long-horizon tasks.
Proposes a method to imitate active learning heuristics for better performance.
problem The performance of active learning heuristics depends on the classifier model and data structure.
method Imitates the selection of the best active learning heuristic using DAGGER.
result Outperforms state-of-the-art imitation learners and heuristics on well-known datasets.
Active-GRPO improves molecular optimization by actively deciding when to imitate or self-improve.
problem Training robust and efficient molecular optimization models with large language models.
method Active-GRPO combines imitation and reinforcement learning, upgrading references and policies dynamically.
result Improves molecular optimization performance, achieving statistically significant gains.
Dual-structured method improves cross-domain imitation learning.
problem Learning policies in a target domain from source domain expert demonstrations.
method Extracts domain and policy expertness features, synthesizes new features.
result Superior performance compared to other algorithms in MuJoCo tasks.
New method simplifies and improves imitation learning without adversarial techniques.
problem Stable optimization and convergence issues in adversarial imitation learning methods.
method Proposes a non-adversarial framework for imitation learning, providing stronger convergence guarantees.
result Shows AIRL as a special case and derives new algorithms for offline imitation learning.
Protects proprietary policies from imitation learning by training adversarial policy ensembles.
problem Protecting policies from external observers cloning them.
method Introduces a reinforcement learning framework that trains an ensemble of near-optimal policies, making demonstrations useless for external observers.
result Demonstrates the existence of 'non-clonable' ensembles and provides a solution to the optimization problem.
MAPS and MAPS-SE learn from multiple suboptimal experts to improve policies efficiently.
problem Learning from multiple suboptimal experts in reinforcement learning.
method Active policy improvement from multiple black-box oracles.
result MAPS and MAPS-SE achieve sample efficiency and state-wise imitation learning.
This work explains why online imitation learning improves faster than theory predicts.
problem Online imitation learning's empirical policy improvement speed exceeds theoretical predictions.
method The authors analyze online imitation learning with a convex, smooth, and non-negative loss function, proving policy improvement in expectation and high probability.
result Adopting a sufficiently expressive policy class in online IL increases both policy improvement speed and performance bias.
Relational Mimic improves visual imitation learning from video demonstrations.
problem Improving robustness and sample efficiency in visual imitation learning.
method Combines generative adversarial networks and relational learning.
result Improves agent performance in challenging locomotion tasks.
AGAIL learns policies from incomplete demonstrations by separating state and action trajectories.
problem Learning policies from incomplete demonstrations where actions are partially available.
method Action-Guided Adversarial Imitation Learning (AGAIL) separates state and action trajectories, using actions as auxiliary information to guide policy training.
result AGAIL consistently delivers comparable performance to state-of-the-art methods even with partially available action sequences.
Compressed imitation learning uses simplicity priors for efficient expert behavior copying.
problem Efficiently learn expert behaviors with minimal data.
method Utilizes policy simplicity as a prior for sample-efficient imitation learning.
result Significantly higher scores achieved with limited expert demonstrations.
PROPEL learns interpretable programmatic policies using imitation and projection.
problem Learning interpretable programmatic policies in reinforcement learning.
method PROPEL is a meta-algorithm based on three insights: optimization in policy space, neural-program mixing, and imitation synthesis.
result PROPEL significantly outperforms state-of-the-art approaches in learning programmatic policies.
SPReD uses uncertainty to decide imitation from demonstrations.
problem Learning from sparse rewards with few demonstrations.
method Ensemble methods to model Q-value distributions, probabilistic and advantage-based uncertainty quantification.
result Significant gains in reinforcement learning across multiple tasks.
Combines BC and GAIL for efficient imitation learning.
problem Efficient imitation learning without reward signals.
method Integrates Behavior Cloning and Generative Adversarial Imitation Learning.
result Combination leads to stable and sample-efficient learning.
RPI combines imitation and reinforcement learning to improve policies efficiently.
problem High sample complexity in reinforcement learning.
method Active interleaving between imitation and reinforcement learning, using oracle queries for exploration.
result RPI outperforms existing methods across various domains.
The paper shows how imitation learning can be exploited to extract DRL policies.
problem The confidentiality and integrity of DRL policies are vulnerable to model extraction attacks.
method Developed proof-of-concept attacks using imitation learning techniques.
result Demonstrated the feasibility of launching black-box attacks against DRL policies.
New algorithm learns from expert demonstrations more efficiently.
problem Efficient imitation learning from expert demonstrations.
method Transformed off-policy distribution matching objective.
result Achieves state-of-the-art sample efficiency and performance.
Bayesian Robust Optimization for Imitation Learning (BROIL) balances risk and reward.
problem Learning robust policies for new states in imitation learning.
method Bayesian reward function inference and user-specific risk tolerance.
result BROIL outperforms risk-sensitive and risk-neutral algorithms.
Paper tackles imitation learning with sparse rewards and heterogeneous actions.
problem Challenges of imitation learning with sparse rewards and different actions.
method Proposes a method that balances imitation and reinforcement learning objectives.
result Agent efficiently leverages sparse rewards and learns from different actions.
Aims to learn from multiple unpredictable teachers with minimal interaction.
problem Learning from multiple non-deterministic teachers with low interaction cost.
method Develops a framework and an active learning algorithm to estimate a distribution over policy space.
result Significantly reduces interaction with teachers without compromising performance.
A new method for learning policies from demonstrations without reinforcement.
problem Learning policies from demonstrations without access to reinforcement signals.
method Energy-based distribution matching (EDM) to learn policy parameters and state marginals.
result EDM yields consistent performance gains over existing algorithms for strictly batch imitation learning.
QFIL improves offline RL by filtering data to reduce bias and variance.
problem Improving offline reinforcement learning policies with limited data.
method QFIL uses a filtered dataset to improve policies, trading off bias and variance through quantile selection.
result QFIL provides a safe policy improvement step with function approximation and effectively balances bias and variance.
The current dominant paradigm for imitation learning relies on strong supervision of expert actions to learn both 'what' and 'how' to imitate. We pursue an alternative paradigm wherein an agent first explores the world without any expert supervision and then distills its experience into a goal-conditioned skill policy …
Paper uses imitation learning to create efficient insulin policies from MPC demonstrations.
problem Resource-constrained medical devices struggle with complex MPC optimizations and state estimation errors.
method Imitation learning of neural network policies from MPC-computed demonstrations, using Bayesian inference with Monte Carlo Dropout.
result Trained policies generalize well to different patient cohorts, outperforming traditional MPC with state estimation.
CFIL uses coupled flows to model state distributions for imitation learning.
problem Lack of explicit modeling of state distributions in reinforcement and imitation learning.
method Coupled normalizing flows for state and state-action distributions.
result CFIL achieves state-of-the-art performance on benchmark tasks.
New method aligns states for better imitation learning.
problem Different dynamics models between imitator and expert.
method State alignment-based reinforcement learning.
result Superior performance on various imitation learning settings.
This paper explores a simple regularizer for reinforcement learning by proposing Generative Adversarial Self-Imitation Learning (GASIL), which encourages the agent to imitate past good trajectories via generative adversarial imitation learning framework. Instead of directly maximizing rewards, GASIL focuses on reproduc…
BAIL learns high-performing policies from batch data without additional interactions.
problem Learning high-performing policies from batch data without additional interactions.
method BAIL learns a V function, selects high-performing actions, and trains a policy network using those actions.
result BAIL outperforms other batch Q-learning and imitation-learning schemes in MuJoCo benchmark.
Variational inference improves hierarchical imitation learning of control programs.
problem Learning structured control policies from demonstrations.
method Variational inference for discovering hierarchical structure in observation-action traces.
result Variational inference leads to more efficient and generalized control policies.
H-ReIL learns to drive safely in near-accident scenarios.
problem Driving safely in high-risk near-accident situations.
method Hierarchical RL and IL approach.
result High-level policy switches between low-level policies for safe driving.
GPRIL learns efficient policies from few expert demonstrations.
problem Sample-efficient imitation learning for robots.
method Generative predecessor models to match expert demonstrations.
result GPRIL achieves higher sample efficiency on real robots.
The use of imitation learning to learn a single policy for a complex task that has multiple modes or hierarchical structure can be challenging. In fact, previous work has shown that when the modes are known, learning separate policies for each mode or sub-task can greatly improve the performance of imitation learning. …
New algorithm learns tasks from video demonstrations using proprioceptive information.
problem Lack of idealized conditions in imitation learning from video demonstrations.
method Proposes an algorithm that leverages proprioceptive state representations for policy learning.
result Outperforms other IfO algorithms by a large margin in MuJoCo domains.
New framework for learning policies that converge in out-of-sample regions.
problem Reliable out-of-sample recovery in imitation learning.
method Contractive dynamical systems and recurrent equilibrium networks.
result Policy rollouts converge regardless of perturbations, enabling efficient OOS recovery.
New method learns belief representations for GAIL in POMDPs.
problem Imitation learning in partially observable Markov decision processes (POMDPs).
method Joint learning of belief module and policy with task-aware imitation loss and belief regularization.
result Our BMIL approach outperforms GAIL and task-agnostic belief learning.
Sample efficiency is critical in solving real-world reinforcement learning problems, where agent-environment interactions can be costly. Imitation learning from expert advice has proved to be an effective strategy for reducing the number of interactions required to train a policy. Online imitation learning, which inter…