Proves EM algorithm guarantees for hierarchical imitation learning.
problem Learning hierarchical policies from expert demonstrations.
method Converts hierarchical imitation learning to latent variable model inference, analyzes EM algorithm.
result Proves EM algorithm converges to true parameter under regular conditions.
Novel online algorithm for hierarchical imitation learning.
problem Scalability issue in reinforcement learning and options discovery.
method Online Baum-Welch algorithm for hierarchical imitation learning.
result The online algorithm outperforms the batch version in both discrete and continuous environments.
Variational inference improves hierarchical imitation learning of control programs.
problem Learning structured control policies from demonstrations.
method Variational inference for discovering hierarchical structure in observation-action traces.
result Variational inference leads to more efficient and generalized control policies.
The use of imitation learning to learn a single policy for a complex task that has multiple modes or hierarchical structure can be challenging. In fact, previous work has shown that when the modes are known, learning separate policies for each mode or sub-task can greatly improve the performance of imitation learning. …
CompILE learns reusable segments from demonstrations for hierarchical task execution.
problem Learning reusable, variable-length segments of hierarchical behavior from demonstrations.
method Unsupervised, fully-differentiable sequence segmentation module for latent encoding and re-composition.
result Model generalizes to longer sequences and unseen environments, learns task boundaries and event encodings.
We study how to effectively leverage expert feedback to learn sequential decision-making policies. We focus on problems with sparse rewards and long time horizons, which typically pose significant challenges in reinforcement learning. We propose an algorithmic framework, called hierarchical guidance, that leverages the…
Hierarchical decoupling improves sample efficiency for complex robots.
problem Learning long-range behaviors on complex robots.
method Two-part policy: low-level imitation and high-level transfer, with KL regularization.
result Hierarchical transfer significantly improves zero-shot high-level transfer and stabilizes learning.
Survey of methods for learning from observation without requiring expert actions.
problem Lack of access to expert actions in imitation learning.
method Survey and classification of state-only imitation learning (SOIL) methods.
result Identification of open problems and future research directions.
New algorithm trains neural nets on simple skills to learn complex tasks faster.
problem Learning complex tasks through simple imitation.
method Train neural networks on simple, easy-to-learn skills to accelerate learning of complex, hard-to-learn tasks.
result Consistently outperforms state-of-the-art baseline in training speed and performance.
Agent learns third-person manipulation tasks from a single video.
problem Learning from third-person videos to perform novel tasks.
method Decoupling high-level task generation from low-level action prediction.
result Agent successfully learns and performs tasks in unseen scenarios.
Method solves long-horizon robotic tasks via imitation and reinforcement learning.
problem Long-horizon robotic tasks with complex sequences of actions.
method Two-phase approach: imitation learning followed by reinforcement learning.
result Method can scale to challenging long-horizon tasks.
H-ReIL learns to drive safely in near-accident scenarios.
problem Driving safely in high-risk near-accident situations.
method Hierarchical RL and IL approach.
result High-level policy switches between low-level policies for safe driving.
New model shows hierarchical proof structure helps theorem provers.
problem How to efficiently learn proofs from teacher traces.
method Model proof search as MDP, analyze imitation learning from traces.
result Hierarchical proof structure leads to more efficient learning.
Robot learns multiple tasks hierarchically by transferring knowledge.
problem Learning multiple complex tasks in open-ended environments.
method Task-oriented procedures, goal-babbling, imitation learning, active learning, intrinsic motivation.
result Robots can learn complex tasks more efficiently by transferring knowledge from simpler ones.
Method learns multi-stage tasks from single video, overcoming challenges of raw pixel learning and insufficient demonstrations.
problem Learning multi-stage vision-based tasks from a single video of a human performing the task.
method Learn primitive behaviors from video demonstrations and dynamically compose them to perform multi-stage tasks.
result Demonstrated learning of various tasks on real robots using raw pixel inputs and minimal demonstrations.
We present an active learning architecture that allows a robot to actively learn which data collection strategy is most efficient for acquiring motor skills to achieve multiple outcomes, and generalise over its experience to achieve new outcomes. The robot explores its environment both via interactive learning and goal…
MineRL Competition reduced reinforcement learning sample needs.
problem Sample inefficiency in reinforcement learning.
method Human demonstrations and imitation learning integrated into reinforcement learning algorithms.
result Top solutions used deep reinforcement learning and imitation learning.
New method aligns states for better imitation learning.
problem Different dynamics models between imitator and expert.
method State alignment-based reinforcement learning.
result Superior performance on various imitation learning settings.
HGAIL learns policies without expert demonstrations.
problem Lack of expert demonstrations in imitation learning.
method Combines hindsight and GAIL to learn policies.
result Comparable performance to current methods, with curriculum learning.
ADVISOR dynamically balances imitation and reinforcement learning to overcome the imitation gap.
problem The gap between imitation learning and reinforcement learning when teaching agents have privileged information.
method Adaptive Insubordination (ADVISOR) dynamically weights imitation and reward-based reinforcement learning losses.
result On-the-fly switching with ADVISOR outperforms pure imitation, pure reinforcement learning, and their combinations.
A learning-based approach optimizes automated lane changes with mixed-integer optimization and machine learning.
problem Traditional motion planning methods are inefficient and lack generalization capability.
method Mixed-Integer Quadratic Problem (MIQP) for optimal trajectories, supervised learning for fast decision-making.
result The proposed model outperforms existing motion planning methods in optimality, efficiency, and generalization.
Reinforcement learning mimics expert behavior.
problem Learning from expert demonstrations in reinforcement learning.
method Reduction to reinforcement learning with a stationary reward.
result Expert reward can be recovered and imitation learning is bounded.
A new benchmark task for evaluating policy learning in complex, high-dimensional action spaces.
problem Lack of a commonly accepted benchmark for evaluating policy learning in hierarchical tasks with high-dimensional action spaces.
method Proposed DinerDash Gym benchmark and Decomposed Policy Graph Modelling (DPGM) algorithm.
result DPGM achieves significant improvement over baselines and effectively injects domain knowledge.
New algorithm reduces sample complexity for imitation learning.
problem High sample complexity limits deployment of adversarial imitation algorithms.
method Trajectory-centric reinforcement learning ideas.
result Improved learning rate and efficiency in imitation tasks.
TRAIL improves robot imitation learning by focusing on task-relevant features.
problem Discriminator networks learn spurious associations, providing poor reward signals.
method Constrained discriminator optimization to learn task-relevant rewards.
result TRAIL outperforms GAIL and behaviour cloning in robotic manipulation tasks.
New framework integrates imitation and reinforcement learning for better robot performance.
problem Combining reinforcement and imitation learning for intelligent robotics.
method Extends probabilistic generative model framework for reinforcement learning and develops pMDP-MO for Markov decision processes.
result Significantly better performance than reinforcement or imitation learning alone.
Guarantees for third-person imitation learning from offline data.
problem Improving generalizability in imitation learning.
method Problem-dependent statistical learning guarantees for third-person imitation from offline observation.
result Strong performance guarantees for transferred policies in the offline setting.
New metric solves correspondence problem for robotic arm imitation learning.
problem Establishing corresponding states and actions between different robotic arms.
method Introducing a distance measure between dissimilar robotic arms and using it as a loss function.
result The distance measure effectively learns imitation policies by minimizing distance between robotic arms.
New method uses bi-level optimization to learn useful representations for imitation learning.
problem Learning useful representations for multiple tasks in imitation learning settings.
method Formulates representation learning as a bi-level optimization problem.
result Bi-level optimization framework provides sample complexity benefits for imitation learning.
Researchers improved Minecraft game performance using imitation learning.
problem Achieving state-of-the-art performance in immersive environments like Minecraft.
method Applied imitation learning to Minecraft, optimizing network architecture, loss function, and data augmentation.
result Reported stronger results than previous experiments, reaching second place in a competition.
New self-imitation learning method improves performance in continuous control tasks.
problem Improving off-policy learning in continuous control tasks.
method Proposes a n-step lower bound to generalize lower-bound Q-learning and introduces a new family of self-imitation learning algorithms.
result n-step lower bound Q-learning achieves a better trade-off between bias and contraction rate, leading to improved performance.
Paper tackles robust imitation learning from noisy demonstrations.
problem Learning from noisy demonstrations is challenging.
method Optimizes a classification risk with a symmetric loss, combining pseudo-labeling and co-training.
result Our method is more robust than state-of-the-art methods.
Paper uses Sinkhorn distances to improve imitation learning effectiveness.
problem Improving imitation learning algorithms by comparing occupancy measures.
method Formulates imitation learning as Sinkhorn distance minimization, combining optimal transport and cosine distances.
result Proposes a new critic network and transport plan that guide imitation learning.
Proof shows imitation of expert's reward and solutions in multi-objective optimization.
problem Multi-objective optimization with reward and solution imitation.
method Wasserstein inverse reinforcement learning.
result Wasserstein inverse reinforcement learning enables imitation of expert's reward and solutions in multi-objective optimization.
The paper analyzes the value discrepancies of imitation learning methods.
problem Analyzing the value discrepancies of imitation learning methods.
method Discrepancy propagation analysis for infinite-horizon settings.
result GAIL has less compounding errors than behavioral cloning.
GASIL encourages agents to imitate past good trajectories in reinforcement learning.
problem Long-term credit assignment in sparse and delayed reward environments.
method Generative Adversarial Imitation Learning (GASIL) framework.
result GASIL improves performance in reinforcement learning tasks with delayed rewards.
GWIL uses Gromov-Wasserstein distance to align expert and imitation agent states.
problem Cross-domain imitation learning challenges due to different system dimensions and stationary distributions.
method Gromov-Wasserstein Imitation Learning (GWIL) using Gromov-Wasserstein distance.
result GWIL effectively aligns expert and imitation agent states in various continuous control domains.
New method improves imitation learning from expert observations.
problem Challenges in imitation learning from observation setting.
method Reparameterized Variational Divergence Minimization.
result Our method outperforms baseline approaches in low-dimensional tasks.
NDI learns from expert demonstrations by estimating occupancy measures.
problem Imitation Learning (IL) for complex systems.
method Density estimation of expert's occupancy measure followed by RL.
result NDI achieves state-of-the-art performance on control tasks.
New method simplifies and improves imitation learning without adversarial techniques.
problem Stable optimization and convergence issues in adversarial imitation learning methods.
method Proposes a non-adversarial framework for imitation learning, providing stronger convergence guarantees.
result Shows AIRL as a special case and derives new algorithms for offline imitation learning.
D-REX learns reward functions from ranked demonstrations to beat the demonstrator's performance.
problem Limited performance of imitation learning compared to the demonstrator.
method Disturbance-based Reward Extrapolation (D-REX) that generates ranked demonstrations from behavioral cloning.
result D-REX significantly outperforms standard imitation learning approaches and beats the demonstrator's performance.
Relational Mimic improves visual imitation learning from video demonstrations.
problem Improving robustness and sample efficiency in visual imitation learning.
method Combines generative adversarial networks and relational learning.
result Improves agent performance in challenging locomotion tasks.
We study risk-sensitive imitation learning where the agent's goal is to perform at least as well as the expert in terms of a risk profile. We first formulate our risk-sensitive imitation learning setting. We consider the generative adversarial approach to imitation learning (GAIL) and derive an optimization problem for…
New method minimizes f-divergence for better imitation learning.
problem Learning from multi-modal demonstrations, especially interpolating between modes.
method Minimizing reverse KL divergence or I-projection for any f-divergence.
result Our method reliably imitates multi-modal behaviors better than existing methods.
Proposes a method to imitate active learning heuristics for better performance.
problem The performance of active learning heuristics depends on the classifier model and data structure.
method Imitates the selection of the best active learning heuristic using DAGGER.
result Outperforms state-of-the-art imitation learners and heuristics on well-known datasets.
A new method removes policy optimization in adversarial imitation learning.
problem Adversarial imitation learning's delicate alternated optimization.
method Explicitly condition discriminator on two policies, solving generator's optimization problem directly.
result Simpler approach competitive to prevalent methods.
A new framework estimates expert policy support to create a reward function for imitation learning.
problem Imitation learning from expert trajectories without reinforcement signals.
method Estimating the support of the expert policy to compute a fixed reward function.
result Comparable or better performance than state-of-the-art methods on discrete and continuous domains.
A new algorithm trains experts to safely guide agents in partially observed environments.
problem Existing imitation learning methods for POMDPs can lead to sub-optimal or unsafe policies.
method Derive an objective to encourage the expert to maximize the agent's reward, then use it to train both expert and agent.
result The algorithm produces an expert policy that the agent can safely imitate, outperforming fixed expert policies.