Method learns reusable actions from human demonstrations.
problem Learning reusable actions from human demonstrations.
method Skill segmentation, inverse reinforcement learning, one-class support vector machine.
result Inferred options improve learning and planning.
Robust real-world learning should benefit from both demonstrations and interactions with the environment. Current approaches to learning from demonstration and reward perform supervised learning on expert demonstration data and use reinforcement learning to further improve performance based on the reward received from …
This work proposes a method to learn from demonstrations and trial-and-error, improving complex tasks.
problem Learning complex tasks from limited demonstrations.
method Meta-imitation learning with trial-and-error and sparse reward feedback.
result Significantly outperforms prior approaches on vision-based control tasks.
LEARN-SAM improves RL from sub-optimal demonstrations by localizing expert policies and selectively using demonstrations.
problem Improving RL from sub-optimal or sparse demonstrations.
method Local Ensemble and Reparameterization with Split and Merge of expert policies (LEARN-SAM).
result LEARN-SAM boosts learning speed and accuracy by selectively using demonstrations.
A framework helps reinforcement learning agents understand and decompose tasks from human demonstrations.
problem Sparse-reward tasks where demonstrations are used as sources of causal knowledge.
method Develops causal models through observation and reasons from this knowledge to decompose tasks.
result A basic implementation of Reasoning from Demonstration (RfD) is effective in sparse-reward tasks.
New method improves IL from imperfect demos using confidence scores.
problem Learning optimal policies from imperfect demonstrations is challenging.
method Proposes two confidence-based IL methods: 2IWIL and IC-GAIL.
result Confidence scores from sub-optimal demos significantly improve IL performance.
A model learns rewards from diverse demonstrations for structurally similar tasks.
problem Difficulty in learning reward functions from demonstrations in real-world applications.
method Deep latent variable model that infers rewards from heterogeneous demonstrations of related tasks.
result Model can infer rewards for new tasks from a single demonstration.
Bayesian Experience Reuse improves learning from multiple experts.
problem Learning from multiple experts with conflicting goals.
method Bayesian neural networks with shared features to model uncertainty and derive a probability distribution over expert models.
result BERS method effectively samples demonstrations from the derived distribution to reuse them in new tasks.
CoCoRL learns safe constraints from demonstrations with unknown rewards.
problem Learning safe constraints from demonstrations with different unknown rewards.
method Convex Constraint Learning for Reinforcement Learning (CoCoRL) constructs a convex safe set based on demonstrations.
result CoCoRL learns constraints that lead to safe driving behavior and can safely transfer to different tasks and environments.
T-REX improves reinforcement learning from suboptimal demonstrations.
problem Inability of existing IRL methods to significantly outperform the demonstrator.
method Trajectory-ranked Reward EXtrapolation (T-REX) algorithm.
result T-REX outperforms state-of-the-art methods on Atari and MuJoCo tasks.
Paper proposes a pretraining method for soft Q-learning from imperfect demonstrations.
problem Challenges in exploiting expert demonstrations while maintaining exploration potentials.
method γ-discounted biased policy evaluation with entropy regularization.
result Our method effectively learns from imperfect demonstrations and outperforms other methods.
Teaches manifolds from teacher's structured data.
problem Learning a manifold from teacher's structured data.
method Extends existing approaches to learning from randomly sampled data points, considering structured data provided by a teacher.
result Demonstrations can significantly reduce data points needed for manifold learning.
AGAIL learns policies from incomplete demonstrations by separating state and action trajectories.
problem Learning policies from incomplete demonstrations where actions are partially available.
method Action-Guided Adversarial Imitation Learning (AGAIL) separates state and action trajectories, using actions as auxiliary information to guide policy training.
result AGAIL consistently delivers comparable performance to state-of-the-art methods even with partially available action sequences.
Novel framework learns policies from noisy expert demonstrations.
problem Learning effective policies with noisy expert demonstrations.
method Adaptive learning framework that jointly interacts with the environment and expert demonstrations, assigning weights to filter out noisy demonstrations.
result The proposed approach learns robustly with noisy demonstrations and achieves higher performance in fewer iterations.
VILD learns policies from mixed-quality demonstrations by modeling expert levels.
problem Challenges in learning from diverse-quality demonstrations.
method Explicitly models expert levels with a probabilistic graphical model and variational approach.
result VILD outperforms state-of-the-art methods in continuous-control benchmarks.
New algorithm learns tasks from video demonstrations using proprioceptive information.
problem Lack of idealized conditions in imitation learning from video demonstrations.
method Proposes an algorithm that leverages proprioceptive state representations for policy learning.
result Outperforms other IfO algorithms by a large margin in MuJoCo domains.
New algorithm robustly learns from corrupted demonstrations, even with constant fraction of noise.
problem Learning from corrupted demonstrations where a fraction of data is noise or outliers.
method Proposes a novel robust algorithm using a Median-of-Means (MOM) objective.
result Guarantees accurate policy estimation even with constant fraction of outliers, similar to classical methods in expert demonstration settings.
PODNet discovers plannable options from unstructured demonstrations.
problem Learning from unstructured, multi-objective demonstrations.
method Custom categorical variational autoencoder, recurrent option inference network, option-conditioned policy network, and option dynamics model.
result PODNet enables learning from demonstration for multiple tasks and planning.
New method learns Atari game Montezuma's Revenge from a single demonstration.
problem Learning from sparse rewards in complex exploration tasks.
method Maximizing rewards directly from a single demonstration state, combined with off-the-shelf reinforcement learning.
result Trained agent achieves high-score of 74,500 in Montezuma's Revenge.
The paper learns robot skills from demonstrations without supervision.
problem Discovering robotic options from unlabelled demonstrations.
method Temporal variational inference for latent variable learning.
result The framework can learn options across multiple datasets.
Estimates rewards from a demonstrator's learning process.
problem Estimating rewards from a demonstrator's behavior.
method Leveraging the demonstrator's exploration phase for reward estimation.
result Consistent reward estimation possible without identifiability issues.
Imitation from observation (IfO) is the problem of learning directly from state-only demonstrations without having access to the demonstrator's actions. The lack of action information both distinguishes IfO from most of the literature in imitation learning, and also sets it apart as a method that may enable agents to l…
Method distills reward and strategies from diverse demonstrators.
problem Reward ambiguity and heterogeneity in human demonstrations.
method Reward network distillation to infer task goal and strategies.
result Better recovery of task and strategy rewards.
New approach learns traffic behavior from unlabelled video.
problem Learning from demonstration in settings with unlabelled raw video data.
method Calibrates camera, detects objects, tracks them, and uses trajectories for LfD.
result Can learn traffic behavior from raw video without additional expert knowledge.
Method learns from video demonstrations with human feedback.
problem Teaching autonomous agents using video demonstrations and human feedback.
method Constructs a mapping between standard and visual representations using a neural network.
result Effective in teaching a hopper agent to perform a backflip with minimal human feedback.
SAIL learns from sub-optimal demonstrations to improve sample efficiency in sparse reward tasks.
problem Reducing sample complexity in sparse-rewarded tasks.
method Self-Adaptive Imitation Learning (SAIL) that exploits sub-optimal demonstrations and efficient exploration.
result Significantly improved sample efficiency and better final performance across various tasks.
Paper tackles robust imitation learning from noisy demonstrations.
problem Learning from noisy demonstrations is challenging.
method Optimizes a classification risk with a symmetric loss, combining pseudo-labeling and co-training.
result Our method is more robust than state-of-the-art methods.
Method learns multi-stage tasks from single video, overcoming challenges of raw pixel learning and insufficient demonstrations.
problem Learning multi-stage vision-based tasks from a single video of a human performing the task.
method Learn primitive behaviors from video demonstrations and dynamically compose them to perform multi-stage tasks.
result Demonstrated learning of various tasks on real robots using raw pixel inputs and minimal demonstrations.
New method learns to answer questions from correct demonstrations without assuming bounded complexity of the demonstrator.
problem Learning to generate answers from correct demonstrations with multiple correct answers.
method Formalizes imitation learning in contextual bandits, relying on reward model complexity, not policy complexity.
result Achieves nearly optimal performance with sample complexity logarithmic in reward class cardinality.
SQIL uses a simple reward strategy to encourage long-horizon imitation of expert demonstrations.
problem Challenges in imitation learning with high-dimensional, continuous observations and unknown dynamics.
method Imitates expert demonstrations by providing a constant reward of +1 for matching actions in demonstrated states, and 0 for all others.
result Empirically outperforms behavioral cloning and achieves competitive results compared to GAIL.
Due to burdensome data requirements, learning from demonstration often falls short of its promise to allow users to quickly and naturally program robots. Demonstrations are inherently ambiguous and incomplete, making correct generalization to unseen situations difficult without a large number of demonstrations in varyi…
D-REX learns reward functions from ranked demonstrations to beat the demonstrator's performance.
problem Limited performance of imitation learning compared to the demonstrator.
method Disturbance-based Reward Extrapolation (D-REX) that generates ranked demonstrations from behavioral cloning.
result D-REX significantly outperforms standard imitation learning approaches and beats the demonstrator's performance.
A method learns to imitate from a single video demonstration using contrastive training and Siamese networks.
problem Learning to imitate from video demonstrations without direct access to state or action information.
method Contrastive training with Siamese recurrent neural networks to learn rewards and an RL policy to minimize distance.
result The method significantly improves policy learning and outperforms current techniques in various environments.
GPRIL learns efficient policies from few expert demonstrations.
problem Sample-efficient imitation learning for robots.
method Generative predecessor models to match expert demonstrations.
result GPRIL achieves higher sample efficiency on real robots.
Paper improves RL from imperfect demonstrations with soft expert guidance.
problem Improper and insufficient expert demonstrations in RLfD.
method Formalizes imperfect expert setting, tackles optimality and convergence issues with soft constraints, and uses local linear search on dual form.
result Method achieves consistent improvement over other RLfD methods.
Robots learn state representation from demonstrations.
problem Robots need a compact state representation for efficient interaction.
method Imitation learning using a multi-head neural network.
result Trained representation improves performance and efficiency in reinforcement learning.
Improved deep RL by pre-training on human demonstrations.
problem Data inefficiency in deep RL learning from raw images.
method Pre-training on non-expert human demonstrations using supervised learning.
result Significant improvements in learning speed with noisy and low-quality demonstrations.
Paper proposes a method to learn goal-reaching behaviors from scratch using imitation learning.
problem Current reinforcement learning algorithms are brittle and require expert demonstrations.
method Iterated supervised learning where agents relabel and imitate generated trajectories.
result Improved goal-reaching performance and robustness over current RL algorithms.
KL-regularized RL from expert demos can lead to slow, unstable learning.
problem Pathological training dynamics in KL-regularized RL from expert demonstrations.
method Empirical analysis and non-parametric behavioral reference policies.
result KL-regularized RL can be significantly improved by using non-parametric behavioral policies.
State-only imitation learning improves dexterous manipulation learning from videos.
problem High sample complexity in complex domains like dexterous manipulation.
method Train an inverse dynamics model to predict actions from states and train the policy jointly.
result Performs on par with state-action approaches and outperforms RL alone.
OSIL learns safe policies from unsafe demonstrations.
problem Offline safe imitation learning with implicit safety.
method Formulates CMDP, infers safety from non-preferred trajectories, learns cost model.
result OSIL learns safer policies without degrading reward performance.
A significant challenge for the practical application of reinforcement learning in the real world is the need to specify an oracle reward function that correctly defines a task. Inverse reinforcement learning (IRL) seeks to avoid this challenge by instead inferring a reward function from expert behavior. While appealin…
Paper proposes a curriculum for RL agents to learn from expert demonstrations.
problem Training RL agents efficiently from expert demonstrations.
method Develops a training curriculum that starts from the goal and works backwards.
result Improves sample efficiency for some tasks compared to PPO.
HGAIL learns policies without expert demonstrations.
problem Lack of expert demonstrations in imitation learning.
method Combines hindsight and GAIL to learn policies.
result Comparable performance to current methods, with curriculum learning.
B-REX efficiently learns Atari game policies from pixel inputs using Bayesian methods.
problem Learning reward functions from visual inputs with uncertainty and safety considerations.
method Bayesian Reward Extrapolation (B-REX) using successor features and preferences.
result B-REX generates posterior samples efficiently, enabling high-confidence performance bounds.
Paper proposes a method to learn and exceed expert demonstrations in unknown reward environments.
problem Learning to outperform expert demonstrations in unknown reward environments.
method A novel concurrent reward and action policy learning approach with a stereo utility definition.
result The proposed method can outperform expert demonstrations in various environments.
SPReD uses uncertainty to decide imitation from demonstrations.
problem Learning from sparse rewards with few demonstrations.
method Ensemble methods to model Q-value distributions, probabilistic and advantage-based uncertainty quantification.
result Significant gains in reinforcement learning across multiple tasks.
RIDM combines imitation and RL with a single demo, no action info needed.
problem Learning from a single observed demonstration without action information.
method Reinforced Inverse Dynamics Modeling (RIDM) that operates on raw state features.
result RIDM performs favorably compared to baseline on simulated and real tasks.