ORRB enables fast, customizable rendering of robotics environments.
problem Fast and customizable rendering of robotics environments.
method Based on Unity3d and MuJoCo, optimized for cloud deployment.
result Visual domain randomization for improved simulation.
MVPI framework optimizes risk in reinforcement learning, improving performance in robot simulations.
problem Optimizing risk in reinforcement learning control problems.
method Mean-Variance Policy Iteration (MVPI) framework for risk-averse control in MDPs.
result Risk-averse TD3 outperforms previous methods in robot simulation tasks.
Algorithm learns actions from past states in complex tasks.
problem Learning policies from human feedback is expensive.
method Combining learned feature encoder with inverse models to simulate past actions.
result Algorithm can infer specific skills from single state.
A novel meta-learning method using ES for efficient reinforcement learning.
problem Sample inefficiency in reinforcement learning.
method Evolution strategies (ES) for exploration in parameter space, deterministic policy gradients for adaptation.
result Demonstrates improved performance in high-dimensional control tasks compared to gradient-based methods.
Meta-gradient D4PG optimizes performance and constraint adherence in RL.
problem Balancing performance and adherence to complex constraints in RL.
method Uses meta-gradients to find a balance between expected return and minimizing constraint violations.
result Meta-gradient D4PG consistently outperforms baselines across MuJoCo domains.
The paper proposes an algorithm to learn efficient and effective exploration policies in reinforcement learning.
problem Balancing exploration and exploitation in reinforcement learning.
method Formalized a counterfactual metric for exploration utility and used meta-learning to learn an end-to-end exploration policy.
result Demonstrated improved performance in high-dimensional control tasks in MuJoCo simulator compared to previous methods.
Novel PO algorithms improve LLM alignment tasks.
problem Evaluating PO algorithms on LLM alignment is costly and noisy.
method Designed a diagnostic suite of MuJoCo tasks and datasets, used evolutionary strategies to discover specialized PO algorithms.
result Proposed MPO algorithms significantly outperform existing PO algorithms in LLM alignment.
Unified framework for maximum entropy RL using Tsallis entropy.
problem Generalizing maximum entropy reinforcement learning with various entropies.
method Tsallis MDPs with Tsallis entropy maximization, controlling entropic index.
result Different entropic indices lead to different optimal policies and exploration tendencies.
Unified reinforcement learning objective improves policy performance prediction.
problem Improving policy performance prediction in reinforcement learning.
method Proposed a counterfactual objective and an emphatic approach to compute unbiased policy gradients.
result Geoff-PAC algorithm demonstrates empirical success in deep RL benchmarks.
Recent advances in policy gradient methods and deep learning have demonstrated their applicability for complex reinforcement learning problems. However, the variance of the performance gradient estimates obtained from the simulation is often excessive, leading to poor sample efficiency. In this paper, we apply the stoc…
Method learns from video demonstrations with human feedback.
problem Teaching autonomous agents using video demonstrations and human feedback.
method Constructs a mapping between standard and visual representations using a neural network.
result Effective in teaching a hopper agent to perform a backflip with minimal human feedback.
New method learns policies without limiting to Gaussian distributions.
problem Limitations of Gaussian parameterization in policy learning.
method Advantage weighted quantile regression for implicit policy modeling.
result Comparable or superior performance on MuJoCo benchmarks.
New RL approach speeds up training across tasks.
problem Training reinforcement learning models quickly and efficiently.
method Combines planning quasi-metric and task-specific aimers.
result Achieves multiple-fold speed-up on bit-flip and robotic arm tasks.
Bayesian approach uses Gaussian process for reinforcement learning.
problem Robotic locomotion environments
method Bayesian actor-critic, model-free reinforcement learning with Gaussian process for exploration and policy optimization.
result Gaussian process method outperforms current algorithms in robotic locomotion environments.
WALL-E accelerates reinforcement learning by parallelizing experience collection.
problem Slow experience collection time in reinforcement learning.
method Parallel rollout samplers to speed up experience generation.
result Achieved higher average rewards with parallel processes.
Mutual-information regularization improves RL algorithms, especially in high-dimensional domains.
problem Value overestimation and exploration in reinforcement learning.
method Developed a novel mutual-information regularized actor-critic learning (MIRACLE) algorithm.
result MIRACLE outperforms state-of-the-art algorithms in continuous action spaces.
Algorithm mitigates performance loss in constrained reinforcement learning with model misspecification.
problem Performance loss in reinforcement learning policies due to model misspecification in constrained control systems.
method Proposes an algorithm to handle constrained model misspecification in continuous control systems.
result Algorithm successfully mitigates performance loss in real-world reinforcement learning tasks.
New loss function equivalence reveals PER's uniform sampling can be improved.
problem Improving Prioritized Experience Replay (PER) for better learning efficiency.
method Transforming non-uniformly sampled data loss functions into uniformly sampled ones.
result Some environments can replace PER with a new loss function without performance loss.
We propose a new sample-efficient methodology, called Supervised Policy Update (SPU), for deep reinforcement learning. Starting with data generated by the current policy, SPU formulates and solves a constrained optimization problem in the non-parameterized proximal policy space. Using supervised regression, it then con…
Deep RL approach improves MIS for complex environments.
problem Improving off-policy evaluation for complex environments.
method Uses successor representation from deep RL to decouple reward and dynamics.
result Empirically stable and applicable to high-dimensional domains.
Policy Prediction Network improves continuous control problems with model-free and model-based learning.
problem Improving sample complexity and performance in continuous control problems.
method Integrates model-free and model-based reinforcement learning, introduces implicit model-based learning for continuous action space.
result First to introduce implicit model-based learning to Policy Gradient algorithms for continuous action space.
CVRL tackles complex visual observations in reinforcement learning.
problem Complex visual observations in natural environments.
method Contrastive Variational Reinforcement Learning (CVRL) learns a contrastive variational model by maximizing mutual information between latent states and observations.
result CVRL achieves comparable performance with state-of-the-art model-based DRL methods and significantly outperforms them on tasks with complex observations.
Classifier-based AI safety gates fail in self-improvement, even with advanced verification methods.
problem Maintaining reliable oversight of AI systems as they improve over iterations.
method Comprehensive empirical testing on neural controllers and MuJoCo benchmarks, using various classifiers and verification methods.
result Classifier-based safety gates fail in maintaining reliable oversight, even with advanced verification methods.
Revisits PPO design choices, exposing failure modes and proposing alternatives.
problem Failure modes of standard PPO in new environments.
method Revisits standard PPO design choices, exposes failure modes, and proposes alternative approaches.
result Alternative design choices prevent failure modes in new environments.
BAIL learns high-performing policies from batch data without additional interactions.
problem Learning high-performing policies from batch data without additional interactions.
method BAIL learns a V function, selects high-performing actions, and trains a policy network using those actions.
result BAIL outperforms other batch Q-learning and imitation-learning schemes in MuJoCo benchmark.
Simplifies RL training with fewer techniques, reducing bias and instability.
problem Training instabilities and high sample complexity in RL.
method Introduced a simple deterministic policy gradient, used propensity estimation, and delayed policy updates.
result Improved performance and reduced sample complexity through these techniques.
DiCE uses diverse agents to explore and learn, avoiding local minima.
problem Local minima in RL due to limited exploration and correlated behavior.
method DiCE employs a group of heterogeneous agents to explore simultaneously and share experiences, with a diversity regularization mechanism.
result DiCE achieves substantial improvement over baselines in MuJoCo locomotion tasks.
PWIL learns agent behavior from expert using Wasserstein distance.
problem Learning agent behavior from expert demonstrations efficiently.
method Primal Wasserstein Imitation Learning (PWIL) with offline reward function.
result PWIL achieves expert behavior on MuJoCo tasks with minimal interactions.
Dual-structured method improves cross-domain imitation learning.
problem Learning policies in a target domain from source domain expert demonstrations.
method Extracts domain and policy expertness features, synthesizes new features.
result Superior performance compared to other algorithms in MuJoCo tasks.
New algorithm learns tasks from video demonstrations using proprioceptive information.
problem Lack of idealized conditions in imitation learning from video demonstrations.
method Proposes an algorithm that leverages proprioceptive state representations for policy learning.
result Outperforms other IfO algorithms by a large margin in MuJoCo domains.
MAGE optimizes policies using action gradients from model-based learning.
problem Lack of direct gradient information from critics in actor-critic methods.
method Model-based actor-critic algorithm that learns action-value gradient.
result MAGE outperforms model-free and model-based baselines on continuous control tasks.
ESPD improves learning efficiency in sparse reward reinforcement learning.
problem Sparse reward reinforcement learning challenges.
method Evolutionary Stochastic Policy Distillation (ESPD) based on drifted random walk insight.
result High learning efficiency demonstrated in MuJoCo robotics control suite experiments.
The paper proposes a framework to reason about object dynamics for faster reinforcement learning.
problem Current reinforcement learning approaches lack prior knowledge about the environment, limiting efficiency.
method Integrates object dynamics and behavior into reinforcement learning to improve efficiency.
result Demonstrates the need for reasoning about object behavior and dynamics, leading to faster learning.
The paper tackles calibrating long-term behaviors with multiple styles using programmatic style-consistency.
problem Generating long-term sequential behaviors with multiple styles simultaneously.
method Leverage programmatic labeling functions to specify controllable styles and derive style-consistency as a learning objective.
result Learned policies can be calibrated for up to 1024 distinct style combinations.
T-REX improves reinforcement learning from suboptimal demonstrations.
problem Inability of existing IRL methods to significantly outperform the demonstrator.
method Trajectory-ranked Reward EXtrapolation (T-REX) algorithm.
result T-REX outperforms state-of-the-art methods on Atari and MuJoCo tasks.
A scalable online method for Gaussian processes that improves decision-making in various applications.
problem Scalability issues with Gaussian processes for online decision-making.
method Online variational conditioning (OVC) for SVGPs.
result OVC enables efficient online learning and decision-making with SVGPs.
A new method for robot manipulation tasks using imagined object goals.
problem Learning robot manipulation in sparse reward environments.
method Train objects to reach target positions, then use predictions to create a curriculum of tasks.
result Higher success rates in challenging learning scenarios compared to alternatives.
New approach relaxes inductive biases of physics-inspired NNs for better performance.
problem Challenges in applying physics-inspired NNs to real-world systems.
method Examined and relaxed inductive biases of Hamiltonian NNs, improving performance on non-conservative systems.
result Improved performance on practical, non-conservative systems by relaxing inductive biases.
New algorithm infers reward function from agent's learning trajectories.
problem Inferring reward function from agent's learning data.
method Gradient-based approach to recover reward function.
result Improved performance compared to state-of-the-art methods.
Paper introduces DTAE to optimize RL algorithms, balancing exploration and exploitation.
problem Balancing exploration and exploitation in reinforcement learning.
method Soft policy optimization with entropy and dual-track advantage estimator (DTAE).
result DTAE accelerates RL algorithm convergence and improves performance.
A common belief in model-free reinforcement learning is that methods based on random search in the parameter space of policies exhibit significantly worse sample complexity than those that explore the space of actions. We dispel such beliefs by introducing a random search method for training static, linear policies for…
Unified approach to neural network learning with PAC-Bayes bounds.
problem Overfitting and extrapolation issues in neural networks.
method Combines variational inference and PAC-Bayes for scalable learning.
result Validates theory and shows better generalization on high-dimensional tasks.
This work formalizes robustness criteria for reinforcement learning actions and improves performance in perturbed environments.
problem Improving reinforcement learning policies to perform well in uncertain or adversarial action scenarios.
method Formalized two robustness criteria for reinforcement learning actions, considering adversarial actions and action perturbations. Developed algorithms for tabular and deep reinforcement learning settings.
result Action-robust reinforcement learning policies improve performance in perturbed environments and are a form of implicit regularization.
New methods improve deep reinforcement learning by accelerating credit assignment.
problem Challenges in achieving fast and stable off-policy learning in deep reinforcement learning.
method Extends the generalized PBE objective to support multistep credit assignment and derives three gradient-based methods.
result Proposed methods outperform PPO and StreamQ in MuJoCo and MinAtar environments.
AdaRL improves robust RL by adaptively adjusting policy complexity.
problem Handling epistemic uncertainty in environment dynamics.
method Bi-level optimization framework with adaptive rank adjustment.
result AdaRL outperforms existing methods on MuJoCo benchmarks.
A new method for ILO with transition model disparity using an intermediary policy.
problem Learning tasks from expert observations with different transition dynamics.
method Training an intermediary policy to match the state transitions of the expert dataset.
result Our method outperforms existing ILO approaches with transition model mismatch.
This study compares 6 imitation learning algorithms using a common dataset and hyperparameter budget.
problem Difficulty in comparing different imitation learning algorithms due to varying datasets, base RL algorithms, and evaluation settings.
method Reimplemented and updated 6 different imitation learning algorithms, using a common off-policy algorithm (SAC) and a widely-used dataset (D4RL). Evaluated on a range of expert trajectories.
result GAIL consistently performs well across different sample sizes, while AdRIL performs well with one important hyperparameter to tune and behavioral cloning remains a strong baseline when data is plentiful.
Rewards are sparse in the real world and most of today's reinforcement learning algorithms struggle with such sparsity. One solution to this problem is to allow the agent to create rewards for itself - thus making rewards dense and more suitable for learning. In particular, inspired by curious behaviour in animals, obs…