A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Novel PO algorithms improve LLM alignment tasks.
MVPI framework optimizes risk in reinforcement learning, improving performance in robot simulations.
New loss function equivalence reveals PER's uniform sampling can be improved.
Meta-gradient D4PG optimizes performance and constraint adherence in RL.
Deep RL approach improves MIS for complex environments.
CVRL tackles complex visual observations in reinforcement learning.
Classifier-based AI safety gates fail in self-improvement, even with advanced verification methods.
In this paper, we present a method for learning from video demonstrations by using human feedback to construct a mapping between the standard representation of the agent and the visual representation of the demonstration. In this way, we leverage the advantages of both these representations, i.e., we learn the policy u…
Simplifies RL training with fewer techniques, reducing bias and instability.
DiCE uses diverse agents to explore and learn, avoiding local minima.
PWIL learns agent behavior from expert using Wasserstein distance.
Dual-structured method improves cross-domain imitation learning.
MAGE optimizes policies using action gradients from model-based learning.
ESPD improves learning efficiency in sparse reward reinforcement learning.
The paper proposes a framework to reason about object dynamics for faster reinforcement learning.
New approach relaxes inductive biases of physics-inspired NNs for better performance.
Algorithm learns actions from past states in complex tasks.
Paper introduces DTAE to optimize RL algorithms, balancing exploration and exploitation.
We present the OpenAI Remote Rendering Backend (ORRB), a system that allows fast and customizable rendering of robotics environments. It is based on the Unity3d game engine and interfaces with the MuJoCo physics simulation library. ORRB was designed with visual domain randomization in mind. It is optimized for cloud de…
New methods improve deep reinforcement learning by accelerating credit assignment.
AdaRL improves robust RL by adaptively adjusting policy complexity.
A new method for ILO with transition model disparity using an intermediary policy.
This study compares 6 imitation learning algorithms using a common dataset and hyperparameter budget.
Rewards are sparse in the real world and most of today's reinforcement learning algorithms struggle with such sparsity. One solution to this problem is to allow the agent to create rewards for itself - thus making rewards dense and more suitable for learning. In particular, inspired by curious behaviour in animals, obs…
This paper proposes Self-Imitation Learning (SIL), a simple off-policy actor-critic algorithm that learns to reproduce the agent's past good decisions. This algorithm is designed to verify our hypothesis that exploiting past good experiences can indirectly drive deep exploration. Our empirical results show that SIL sig…
Soft Actor-Critic (SAC) is an off-policy actor-critic deep reinforcement learning (DRL) algorithm based on maximum entropy reinforcement learning. By combining off-policy updates with an actor-critic formulation, SAC achieves state-of-the-art performance on a range of continuous-action benchmark tasks, outperforming pr…
PPO's gradients are heavy-tailed, affecting learning; a robust estimator improves performance.
Paper tackles covariate shift in offline IL using less proficient behavior data.
New state-only IL algorithm tackles MDP transition mismatch.
Novel algorithm reduces computational burden in IRL with finite-time guarantees.
This paper explores a simple regularizer for reinforcement learning by proposing Generative Adversarial Self-Imitation Learning (GASIL), which encourages the agent to imitate past good trajectories via generative adversarial imitation learning framework. Instead of directly maximizing rewards, GASIL focuses on reproduc…
FORK improves model-free reinforcement learning performance.
New RL method improves robustness across various conditions.
RL agents learn from a few tasks to generalize to new ones.
Many continuous control tasks have easily formulated objectives, yet using them directly as a reward in reinforcement learning (RL) leads to suboptimal policies. Therefore, many classical control tasks guide RL training using complex rewards, which require tedious hand-tuning. We automate the reward search with AutoRL,…
Policy optimization on high-dimensional continuous control tasks exhibits its difficulty caused by the large variance of the policy gradient estimators. We present the action subspace dependent gradient (ASDG) estimator which incorporates the Rao-Blackwell theorem (RB) and Control Variates (CV) into a unified framework…
Recent advances in policy gradient methods and deep learning have demonstrated their applicability for complex reinforcement learning problems. However, the variance of the performance gradient estimates obtained from the simulation is often excessive, leading to poor sample efficiency. In this paper, we apply the stoc…
There are two halves to RL systems: experience collection time and policy learning time. For a large number of samples in rollouts, experience collection time is the major bottleneck. Thus, it is necessary to speed up the rollout generation time with multi-process architecture support. Our work, dubbed WALL-E, utilizes…
In hierarchical reinforcement learning a major challenge is determining appropriate low-level policies. We propose an unsupervised learning scheme, based on asymmetric self-play from Sukhbaatar et al. (2018), that automatically learns a good representation of sub-goals in the environment and a low-level policy that can…
In this technical report, we consider an approach that combines the PPO objective and K-FAC natural gradient optimization, for which we call PPOKFAC. We perform a range of empirical analysis on various aspects of the algorithm, such as sample complexity, training speed, and sensitivity to batch size and training epochs…
We propose a new objective, the counterfactual objective, unifying existing objectives for off-policy policy gradient algorithms in the continuing reinforcement learning (RL) setting. Compared to the commonly used excursion objective, which can be misleading about the performance of the target policy when deployed, our…
Genetic algorithms have been widely used in many practical optimization problems. Inspired by natural selection, operators, including mutation, crossover and selection, provide effective heuristics for search and black-box optimization. However, they have not been shown useful for deep reinforcement learning, possibly …
New RL approach speeds up training across tasks.
CoNES optimizes blackbox functions using convex optimization and information geometry.
Bayesian approach uses Gaussian process for reinforcement learning.
Continuous control tasks in reinforcement learning are important because they provide an important framework for learning in high-dimensional state spaces with deceptive rewards, where the agent can easily become trapped into suboptimal solutions. One way to avoid local optima is to use a population of agents to ensure…
This paper proposes a novel deep reinforcement learning architecture that was inspired by previous tree structured architectures which were only useable in discrete action spaces. Policy Prediction Network offers a way to improve sample complexity and performance on continuous control problems in exchange for extra com…
A fundamental issue in reinforcement learning algorithms is the balance between exploration of the environment and exploitation of information already obtained by the agent. Especially, exploration has played a critical role for both efficiency and efficacy of the learning process. However, Existing works for explorati…