New metric solves correspondence problem for robotic arm imitation learning.
problem Establishing corresponding states and actions between different robotic arms.
method Introducing a distance measure between dissimilar robotic arms and using it as a loss function.
result The distance measure effectively learns imitation policies by minimizing distance between robotic arms.
Robot learns to imitate human interactions through deep learning.
problem Teaching robots to coordinate actions with human partners.
method Deep learning framework for motion embedding, prediction, and trajectory generation.
result Importance of predictive and adaptive components for successful imitation.
BAIL learns high-performing policies from batch data without additional interactions.
problem Learning high-performing policies from batch data without additional interactions.
method BAIL learns a V function, selects high-performing actions, and trains a policy network using those actions.
result BAIL outperforms other batch Q-learning and imitation-learning schemes in MuJoCo benchmark.
Paper uses deep imitation learning to predict aircraft trajectories accurately.
problem Inefficient and costly Air Traffic Management system limits predictability.
method Generative Adversarial Imitation Learning framework with trajectory clustering and classification.
result Accurate predictions for entire trajectory stages, pre- and tactical.
A new imitation learning method uses random search for simple policies, outperforming complex models.
problem Complexity and reward dependency issues in imitation learning.
method Derivative-free optimization with simple linear policies and random search.
result The proposed method achieves competitive performance on MuJoCo locomotion tasks without a direct reward signal.
This study compares 6 imitation learning algorithms using a common dataset and hyperparameter budget.
problem Difficulty in comparing different imitation learning algorithms due to varying datasets, base RL algorithms, and evaluation settings.
method Reimplemented and updated 6 different imitation learning algorithms, using a common off-policy algorithm (SAC) and a widely-used dataset (D4RL). Evaluated on a range of expert trajectories.
result GAIL consistently performs well across different sample sizes, while AdRIL performs well with one important hyperparameter to tune and behavioral cloning remains a strong baseline when data is plentiful.
The paper shows how imitation learning can be exploited to extract DRL policies.
problem The confidentiality and integrity of DRL policies are vulnerable to model extraction attacks.
method Developed proof-of-concept attacks using imitation learning techniques.
result Demonstrated the feasibility of launching black-box attacks against DRL policies.
B-REX efficiently learns Atari game policies from pixel inputs using Bayesian methods.
problem Learning reward functions from visual inputs with uncertainty and safety considerations.
method Bayesian Reward Extrapolation (B-REX) using successor features and preferences.
result B-REX generates posterior samples efficiently, enabling high-confidence performance bounds.
AutoOD automates outlier detection using curiosity-guided search and self-imitation learning.
problem Automated outlier detection for complex tasks with big data.
method Curiosity-guided search strategy and self-imitation learning.
result AutoOD identifies optimal neural network models with superior performance.
The growing use of virtual autonomous agents in applications like games and entertainment demands better control policies for natural-looking movements and actions. Unlike the conventional approach of hard-coding motion routines, we propose a deep learning method for obtaining control policies by directly mimicking raw…
Automated theorem proving without imitation, using tf-idf for premise selection.
problem Automated theorem proving in large knowledge bases without human guidance.
method Exploration mechanism mixing tf-idf selected premises in a deep reinforcement learning context.
result The prover trained with this approach outperforms provers trained only on human proofs.
PROPEL learns interpretable programmatic policies using imitation and projection.
problem Learning interpretable programmatic policies in reinforcement learning.
method PROPEL is a meta-algorithm based on three insights: optimization in policy space, neural-program mixing, and imitation synthesis.
result PROPEL significantly outperforms state-of-the-art approaches in learning programmatic policies.
New method aligns states for better imitation learning.
problem Different dynamics models between imitator and expert.
method State alignment-based reinforcement learning.
result Superior performance on various imitation learning settings.
Proposes efficient training method for deep thin networks.
problem Deploying deep learning models with accuracy and compactness.
method Three-stage method: widen, warm up, fine tune.
result Deep thin networks trained with method outperform standard deep networks.
The paper uses a novel framework to learn option prices by imitating principal investor behavior.
problem Challenges in modeling stock price changes and decision making in equity markets.
method Non-deterministic Markov decision process, Bayesian deep neural network, reinforcement learning.
result Optimal option prices learned through imitation of principal investor behavior.
This paper proposes Self-Imitation Learning (SIL), a simple off-policy actor-critic algorithm that learns to reproduce the agent's past good decisions. This algorithm is designed to verify our hypothesis that exploiting past good experiences can indirectly drive deep exploration. Our empirical results show that SIL sig…
NDPs embed dynamical systems into neural networks for efficient sensorimotor learning.
problem Training policies directly in raw action spaces limits scalability for continuous tasks.
method Embed dynamical systems into neural networks to learn robot behaviors via demonstrations.
result NDPs outperform prior methods in both imitation and reinforcement learning setups.
Imitation Learning (IL) is an appealing approach to learn desirable autonomous behavior. However, directing IL to achieve arbitrary goals is difficult. In contrast, planning-based algorithms use dynamics models and reward functions to achieve goals. Yet, reward functions that evoke desirable behavior are often difficul…
Paper presents first model extraction attack against DRL models.
problem Stealing DRL models from interaction with environment.
method Imitation learning approach to recover DRL models from predicted actions.
result Successfully recovers DRL models with high fidelity and accuracy.
QTNet uses deep reinforcement learning to automate trading strategies.
problem Handling noisy and high-frequency financial data, balancing exploration and exploitation.
method QTNet employs deep reinforcement learning (DRL) with imitative learning to autonomously formulate trading strategies.
result QTNet demonstrates proficiency in extracting robust market features and adaptability to diverse conditions.
HGAIL learns policies without expert demonstrations.
problem Lack of expert demonstrations in imitation learning.
method Combines hindsight and GAIL to learn policies.
result Comparable performance to current methods, with curriculum learning.
ADVISOR dynamically balances imitation and reinforcement learning to overcome the imitation gap.
problem The gap between imitation learning and reinforcement learning when teaching agents have privileged information.
method Adaptive Insubordination (ADVISOR) dynamically weights imitation and reward-based reinforcement learning losses.
result On-the-fly switching with ADVISOR outperforms pure imitation, pure reinforcement learning, and their combinations.
UAIL uses uncertainty estimation to improve control systems in safety-critical tasks.
problem Improving control systems in safety-critical domains like autonomous driving.
method UAIL applies Monte Carlo Dropout to estimate uncertainty in control output and selectively acquire new training data.
result UAIL can reliably predict infractions and outperforms existing algorithms.
Reinforcement learning mimics expert behavior.
problem Learning from expert demonstrations in reinforcement learning.
method Reduction to reinforcement learning with a stationary reward.
result Expert reward can be recovered and imitation learning is bounded.
New algorithm reduces sample complexity for imitation learning.
problem High sample complexity limits deployment of adversarial imitation algorithms.
method Trajectory-centric reinforcement learning ideas.
result Improved learning rate and efficiency in imitation tasks.
MineRL Competition reduced reinforcement learning sample needs.
problem Sample inefficiency in reinforcement learning.
method Human demonstrations and imitation learning integrated into reinforcement learning algorithms.
result Top solutions used deep reinforcement learning and imitation learning.
TRAIL improves robot imitation learning by focusing on task-relevant features.
problem Discriminator networks learn spurious associations, providing poor reward signals.
method Constrained discriminator optimization to learn task-relevant rewards.
result TRAIL outperforms GAIL and behaviour cloning in robotic manipulation tasks.
New algorithm learns expert reward structures from batch data.
problem Learning expert reward structures from batch data without dynamics models.
method Deep Successor Feature Networks (DSFN) and transition-regularized imitation network.
result Superior performance on control benchmarks and sepsis management.
New framework integrates imitation and reinforcement learning for better robot performance.
problem Combining reinforcement and imitation learning for intelligent robotics.
method Extends probabilistic generative model framework for reinforcement learning and develops pMDP-MO for Markov decision processes.
result Significantly better performance than reinforcement or imitation learning alone.
Guarantees for third-person imitation learning from offline data.
problem Improving generalizability in imitation learning.
method Problem-dependent statistical learning guarantees for third-person imitation from offline observation.
result Strong performance guarantees for transferred policies in the offline setting.
New method uses bi-level optimization to learn useful representations for imitation learning.
problem Learning useful representations for multiple tasks in imitation learning settings.
method Formulates representation learning as a bi-level optimization problem.
result Bi-level optimization framework provides sample complexity benefits for imitation learning.
Researchers improved Minecraft game performance using imitation learning.
problem Achieving state-of-the-art performance in immersive environments like Minecraft.
method Applied imitation learning to Minecraft, optimizing network architecture, loss function, and data augmentation.
result Reported stronger results than previous experiments, reaching second place in a competition.
New self-imitation learning method improves performance in continuous control tasks.
problem Improving off-policy learning in continuous control tasks.
method Proposes a n-step lower bound to generalize lower-bound Q-learning and introduces a new family of self-imitation learning algorithms.
result n-step lower bound Q-learning achieves a better trade-off between bias and contraction rate, leading to improved performance.
Paper tackles robust imitation learning from noisy demonstrations.
problem Learning from noisy demonstrations is challenging.
method Optimizes a classification risk with a symmetric loss, combining pseudo-labeling and co-training.
result Our method is more robust than state-of-the-art methods.
Hybrid RL learns from expert state sequences without full action data.
problem Learning from expert state sequences without full action data.
method Tensor-based model to infer unobserved actions; hybrid RL objective.
result Hybrid RL outperforms pure RL and tensor-based action inference.
Paper uses Sinkhorn distances to improve imitation learning effectiveness.
problem Improving imitation learning algorithms by comparing occupancy measures.
method Formulates imitation learning as Sinkhorn distance minimization, combining optimal transport and cosine distances.
result Proposes a new critic network and transport plan that guide imitation learning.
Proof shows imitation of expert's reward and solutions in multi-objective optimization.
problem Multi-objective optimization with reward and solution imitation.
method Wasserstein inverse reinforcement learning.
result Wasserstein inverse reinforcement learning enables imitation of expert's reward and solutions in multi-objective optimization.
The paper analyzes the value discrepancies of imitation learning methods.
problem Analyzing the value discrepancies of imitation learning methods.
method Discrepancy propagation analysis for infinite-horizon settings.
result GAIL has less compounding errors than behavioral cloning.
Behavior cloning can achieve horizon-independent sample complexity in offline imitation learning.
problem Sample complexity in imitation learning increases with problem horizon.
method New analysis of behavior cloning with logarithmic loss.
result Behavior cloning can achieve linear dependence on horizon in offline IL under dense rewards.
GWIL uses Gromov-Wasserstein distance to align expert and imitation agent states.
problem Cross-domain imitation learning challenges due to different system dimensions and stationary distributions.
method Gromov-Wasserstein Imitation Learning (GWIL) using Gromov-Wasserstein distance.
result GWIL effectively aligns expert and imitation agent states in various continuous control domains.
NDI learns from expert demonstrations by estimating occupancy measures.
problem Imitation Learning (IL) for complex systems.
method Density estimation of expert's occupancy measure followed by RL.
result NDI achieves state-of-the-art performance on control tasks.
New method improves imitation learning from expert observations.
problem Challenges in imitation learning from observation setting.
method Reparameterized Variational Divergence Minimization.
result Our method outperforms baseline approaches in low-dimensional tasks.
New method simplifies and improves imitation learning without adversarial techniques.
problem Stable optimization and convergence issues in adversarial imitation learning methods.
method Proposes a non-adversarial framework for imitation learning, providing stronger convergence guarantees.
result Shows AIRL as a special case and derives new algorithms for offline imitation learning.
This paper tackles efficient cooperative control for large-scale traffic signals using tensor-based deep learning.
problem Efficient training and control for large-scale multi-intersection traffic signals.
method Tensor representation, multi-task learning, imitation learning, proximal policy optimization.
result The proposed model achieves better performance compared to existing methods.
MetaTrader combines diverse expert strategies to optimize portfolio performance.
problem Optimizing portfolio performance in changing financial markets.
method Two-stage RL approach: imitation learning followed by a meta-policy.
result MetaTrader significantly outperforms state-of-the-art baselines in balancing profits and risks.
D-REX learns reward functions from ranked demonstrations to beat the demonstrator's performance.
problem Limited performance of imitation learning compared to the demonstrator.
method Disturbance-based Reward Extrapolation (D-REX) that generates ranked demonstrations from behavioral cloning.
result D-REX significantly outperforms standard imitation learning approaches and beats the demonstrator's performance.
Relational Mimic improves visual imitation learning from video demonstrations.
problem Improving robustness and sample efficiency in visual imitation learning.
method Combines generative adversarial networks and relational learning.
result Improves agent performance in challenging locomotion tasks.
We study risk-sensitive imitation learning where the agent's goal is to perform at least as well as the expert in terms of a risk profile. We first formulate our risk-sensitive imitation learning setting. We consider the generative adversarial approach to imitation learning (GAIL) and derive an optimization problem for…