Consider an imitation learning problem that the imitator and the expert have different dynamics models. Most of the current imitation learning methods fail because they focus on imitating actions. We propose a novel state alignment-based imitation learning method to train the imitator to follow the state sequences in e…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “imitation learning”
ADVISOR dynamically balances imitation and reinforcement learning to overcome the imitation gap.
problem The gap between imitation learning and reinforcement learning when teaching agents have privileged information.
method Adaptive Insubordination (ADVISOR) dynamically weights imitation and reward-based reinforcement learning losses.
result On-the-fly switching with ADVISOR outperforms pure imitation, pure reinforcement learning, and their combinations.
Compared to reinforcement learning, imitation learning (IL) is a powerful paradigm for training agents to learn control policies efficiently from expert demonstrations. However, in most cases, obtaining demonstration data is costly and laborious, which poses a significant challenge in some scenarios. A promising altern…
Reinforcement learning mimics expert behavior.
problem Learning from expert demonstrations in reinforcement learning.
method Reduction to reinforcement learning with a stationary reward.
result Expert reward can be recovered and imitation learning is bounded.
Guarantees for third-person imitation learning from offline data.
problem Improving generalizability in imitation learning.
method Problem-dependent statistical learning guarantees for third-person imitation from offline observation.
result Strong performance guarantees for transferred policies in the offline setting.
New metric solves correspondence problem for robotic arm imitation learning.
problem Establishing corresponding states and actions between different robotic arms.
method Introducing a distance measure between dissimilar robotic arms and using it as a loss function.
result The distance measure effectively learns imitation policies by minimizing distance between robotic arms.
New method uses bi-level optimization to learn useful representations for imitation learning.
problem Learning useful representations for multiple tasks in imitation learning settings.
method Formulates representation learning as a bi-level optimization problem.
result Bi-level optimization framework provides sample complexity benefits for imitation learning.
Researchers improved Minecraft game performance using imitation learning.
problem Achieving state-of-the-art performance in immersive environments like Minecraft.
method Applied imitation learning to Minecraft, optimizing network architecture, loss function, and data augmentation.
result Reported stronger results than previous experiments, reaching second place in a competition.
New self-imitation learning method improves performance in continuous control tasks.
problem Improving off-policy learning in continuous control tasks.
method Proposes a n-step lower bound to generalize lower-bound Q-learning and introduces a new family of self-imitation learning algorithms.
result n-step lower bound Q-learning achieves a better trade-off between bias and contraction rate, leading to improved performance.
Paper tackles robust imitation learning from noisy demonstrations.
problem Learning from noisy demonstrations is challenging.
method Optimizes a classification risk with a symmetric loss, combining pseudo-labeling and co-training.
result Our method is more robust than state-of-the-art methods.
Paper uses Sinkhorn distances to improve imitation learning effectiveness.
problem Improving imitation learning algorithms by comparing occupancy measures.
method Formulates imitation learning as Sinkhorn distance minimization, combining optimal transport and cosine distances.
result Proposes a new critic network and transport plan that guide imitation learning.
A proof of imitation of Wasserstein inverse reinforcement learning for multi-objective optimizationcs.LG
Proof shows imitation of expert's reward and solutions in multi-objective optimization.
problem Multi-objective optimization with reward and solution imitation.
method Wasserstein inverse reinforcement learning.
result Wasserstein inverse reinforcement learning enables imitation of expert's reward and solutions in multi-objective optimization.
We show that a critical vulnerability in adversarial imitation is the tendency of discriminator networks to learn spurious associations between visual features and expert labels. When the discriminator focuses on task-irrelevant features, it does not provide an informative reward signal, leading to poor task performanc…
NDI learns from expert demonstrations by estimating occupancy measures.
problem Imitation Learning (IL) for complex systems.
method Density estimation of expert's occupancy measure followed by RL.
result NDI achieves state-of-the-art performance on control tasks.
GWIL uses Gromov-Wasserstein distance to align expert and imitation agent states.
problem Cross-domain imitation learning challenges due to different system dimensions and stationary distributions.
method Gromov-Wasserstein Imitation Learning (GWIL) using Gromov-Wasserstein distance.
result GWIL effectively aligns expert and imitation agent states in various continuous control domains.
New method improves imitation learning from expert observations.
problem Challenges in imitation learning from observation setting.
method Reparameterized Variational Divergence Minimization.
result Our method outperforms baseline approaches in low-dimensional tasks.
Imitation from observation is the framework of learning tasks by observing demonstrated state-only trajectories. Recently, adversarial approaches have achieved significant performance improvements over other methods for imitating complex behaviors. However, these adversarial imitation algorithms often require many demo…
New method simplifies and improves imitation learning without adversarial techniques.
problem Stable optimization and convergence issues in adversarial imitation learning methods.
method Proposes a non-adversarial framework for imitation learning, providing stronger convergence guarantees.
result Shows AIRL as a special case and derives new algorithms for offline imitation learning.
Integration of reinforcement learning and imitation learning is an important problem that has been studied for a long time in the field of intelligent robotics. Reinforcement learning optimizes policies to maximize the cumulative reward, whereas imitation learning attempts to extract general knowledge about the traject…
We study risk-sensitive imitation learning where the agent's goal is to perform at least as well as the expert in terms of a risk profile. We first formulate our risk-sensitive imitation learning setting. We consider the generative adversarial approach to imitation learning (GAIL) and derive an optimization problem for…
Proposes a method to imitate active learning heuristics for better performance.
problem The performance of active learning heuristics depends on the classifier model and data structure.
method Imitates the selection of the best active learning heuristic using DAGGER.
result Outperforms state-of-the-art imitation learners and heuristics on well-known datasets.
A new method removes policy optimization in adversarial imitation learning.
problem Adversarial imitation learning's delicate alternated optimization.
method Explicitly condition discriminator on two policies, solving generator's optimization problem directly.
result Simpler approach competitive to prevalent methods.
A new algorithm trains experts to safely guide agents in partially observed environments.
problem Existing imitation learning methods for POMDPs can lead to sub-optimal or unsafe policies.
method Derive an objective to encourage the expert to maximize the agent's reward, then use it to train both expert and agent.
result The algorithm produces an expert policy that the agent can safely imitate, outperforming fixed expert policies.
Recent work has demonstrated that problems-- particularly imitation learning and structured prediction-- where a learner's predictions influence the input-distribution it is tested on can be naturally addressed by an interactive approach and analyzed using no-regret online learning. These approaches to imitation learni…
The performance of imitation learning is typically upper-bounded by the performance of the demonstrator. While recent empirical results demonstrate that ranked demonstrations allow for better-than-demonstrator performance, preferences over demonstrations may be difficult to obtain, and little is known theoretically abo…
Imitation learning trains a policy from expert demonstrations. Imitation learning approaches have been designed from various principles, such as behavioral cloning via supervised learning, apprenticeship learning via inverse reinforcement learning, and GAIL via generative adversarial learning. In this paper, we propose…
Curriculum learning and imitation learning improve control over financial time-series data.
problem Improving control performance over complex financial time-series data.
method Data augmentation for curriculum learning and policy distillation for imitation learning.
result Curriculum learning shows significant improvement over time-series control tasks.
New method uses variational inference to handle confounding in imitation learning.
problem Confounding due to different sensory inputs between expert and imitating agent.
method Train variational inference model to infer expert's latent information and use for latent-conditional policy training.
result Algorithm converges to correct interventional policy and achieves asymptotically optimal performance.
Algorithm improves imitation learning from visual data in partially observable environments.
problem Imitation learning from visual observations with missing expert actions and partial observability.
method Theoretical analysis and Latent Adversarial Imitation from Observations algorithm combining adversarial and latent representations.
result Latent Adversarial Imitation from Observations achieves state-of-the-art performance in high-dimensional robotic tasks.
Dual-structured method improves cross-domain imitation learning.
problem Learning policies in a target domain from source domain expert demonstrations.
method Extracts domain and policy expertness features, synthesizes new features.
result Superior performance compared to other algorithms in MuJoCo tasks.
Recent work on imitation learning has generated policies that reproduce expert behavior from multi-modal data. However, past approaches have focused only on recreating a small number of distinct, expert maneuvers, or have relied on supervised learning techniques that produce unstable policies. This work extends InfoGAI…
SAIL improves AIL by weighting adversarial rewards with support estimation.
problem Training instability and reward bias in AIL.
method Support-weighted Adversarial Imitation Learning (SAIL) extends AIL with support estimation to improve reinforcement signals.
result SAIL achieves better performance and stability on benchmark tasks.
Imitation from observation (IfO) is the problem of learning directly from state-only demonstrations without having access to the demonstrator's actions. The lack of action information both distinguishes IfO from most of the literature in imitation learning, and also sets it apart as a method that may enable agents to l…
Paper improves AIRL by enhancing policy imitation and addressing reward recovery issues.
problem Inadequate policy imitation and limited transferable reward recovery in AIRL.
method Substituted built-in algorithm with SAC for policy updating and proposed PPO-AIRL + SAC hybrid framework.
result SAC improves policy imitation but hinders reward recovery; PPO-AIRL + SAC achieves satisfactory transfer effect.
Paper introduces a novel reward function for noisy financial markets using imitation learning.
problem Noisy reward function in financial markets hinders RL agent performance.
method Integrates imitation learning feedback with reinforcement learning to improve reward function design.
result Improves financial performance metrics compared to traditional benchmarks and RL agents.
A new imitation learning method uses random search for simple policies, outperforming complex models.
problem Complexity and reward dependency issues in imitation learning.
method Derivative-free optimization with simple linear policies and random search.
result The proposed method achieves competitive performance on MuJoCo locomotion tasks without a direct reward signal.
Imitation learning algorithms can be used to learn a policy from expert demonstrations without access to a reward signal. However, most existing approaches are not applicable in multi-agent settings due to the existence of multiple (Nash) equilibria and non-stationary environments. We propose a new framework for multi-…
CFIL uses coupled flows to model state distributions for imitation learning.
problem Lack of explicit modeling of state distributions in reinforcement and imitation learning.
method Coupled normalizing flows for state and state-action distributions.
result CFIL achieves state-of-the-art performance on benchmark tasks.
Improves imitation learning in RL by learning reward function efficiently.
problem Lack of effective reward function approximation in AIRL for imitation tasks.
method Proposes Off-Policy AIRL that combines adversarial learning with efficient reward function approximation.
result Shows superior imitation performance and efficiency compared to state-of-the-art AIL algorithms.
New model learns better policies from expert demonstrations with higher efficiency.
problem Learning accurate policies from expert demonstrations with high efficiency.
method Generative adversarial imitation learning (GAIL) model that learns -divergence automatically.
result Learns better policies with higher data efficiency in physics-based control tasks.
This paper explores a simple regularizer for reinforcement learning by proposing Generative Adversarial Self-Imitation Learning (GASIL), which encourages the agent to imitate past good trajectories via generative adversarial imitation learning framework. Instead of directly maximizing rewards, GASIL focuses on reproduc…
Continuous control imitation learning fails if expert actions are smooth.
problem Continuous control imitation learning fails if expert actions are smooth.
method Study of imitation learning in discrete-time, continuous state-and-action control systems.
result Any smooth, deterministic imitator policy suffers exponentially larger error than the expert.
The current dominant paradigm for imitation learning relies on strong supervision of expert actions to learn both 'what' and 'how' to imitate. We pursue an alternative paradigm wherein an agent first explores the world without any expert supervision and then distills its experience into a goal-conditioned skill policy …
This work improves imitation learning and goal-conditioned RL by estimating value densities.
problem Effective solutions for imitation and goal-conditioned reinforcement learning require reliably reaching specified states or demonstrations.
method The approach uses recent advances in density estimation to learn value functions efficiently and without hindsight bias.
result The method achieves state-of-the-art demonstration sample-efficiency in imitation learning and is both efficient and bias-free in goal-conditioned reinforcement learning.
ICIL learns policies invariant to multiple environments, improving generalization.
problem Learning policies from multiple environments leads to spurious correlations.
method ICIL learns invariant feature representations and a matching imitation policy.
result ICIL policies generalize better to unseen environments.
A new approach for learning from expert demonstrations using multiple perspectives.
problem Learning from expert demonstrations with limited information from different perspectives.
method Generative adversarial network-based active learning approach.
result Our approach effectively learns from expert demonstrations and highlights the importance of architectural choices.
Safe imitation learning with a safety layer for flexible training.
problem Flexible yet safe imitation learning for complex tasks.
method Theory and modular method with a safety layer for continuous policy, adversarial training, and worst-case safety guarantees.
result Robustness advantage of safety layer during training compared to test time.
We propose Generative Predecessor Models for Imitation Learning (GPRIL), a novel imitation learning algorithm that matches the state-action distribution to the distribution observed in expert demonstrations, using generative models to reason probabilistically about alternative histories of demonstrated states. We show …