A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Paper tackles overestimation bias in continuous control, improving performance by 25%.
Tabular Q-Learning with learned state abstractions solves continuous control tasks.
Deep RBVFs improve continuous control in RL.
New model learning objective improves continuous control tasks.
Current Deep Reinforcement Learning algorithms still heavily rely on handcrafted neural network architectures. We propose a novel approach to automatically find strong topologies for continuous control tasks while only adding a minor overhead in terms of interactions in the environment. To achieve this, we combine Neur…
We provide a framework for incorporating robustness -- to perturbations in the transition dynamics which we refer to as model misspecification -- into continuous control Reinforcement Learning (RL) algorithms. We specifically focus on incorporating robustness into a state-of-the-art continuous control RL algorithm call…
Continuous control tasks in reinforcement learning are important because they provide an important framework for learning in high-dimensional state spaces with deceptive rewards, where the agent can easily become trapped into suboptimal solutions. One way to avoid local optima is to use a population of agents to ensure…
Reinforcement learning algorithms rely on exploration to discover new behaviors, which is typically achieved by following a stochastic policy. In continuous control tasks, policies with a Gaussian distribution have been widely adopted. Gaussian exploration however does not result in smooth trajectories that generally c…
New self-imitation learning method improves performance in continuous control tasks.
GP-PSRL achieves sublinear regret for continuous control with unbounded state space.
Paper establishes baselines for offline RL from visual observations.
Proposes a value-based method for continuous control without an actor.
In this paper, we present a novel penalty approach for the numerical solution of continuously controlled HJB equations and HJB obstacle problems. Our results include estimates of the penalisation error for a class of penalty terms, and we show that variations of Newton's method can be used to obtain globally convergent…
HiDe learns hierarchical control for complex tasks by separating planning and control.
Study investigates key design choices in on-policy RL algorithms.
This work evaluates task-agnostic exploration methods for fixed-batch learning.
DyNODE uses neural ODEs to model system dynamics in continuous control tasks.
Exploration in environments with continuous control and sparse rewards remains a key challenge in reinforcement learning (RL). Recently, surprise has been used as an intrinsic reward that encourages systematic and efficient exploration. We introduce a new definition of surprise and its RL implementation named Variation…
RP1 uses active learning to improve world model in fewest samples.
RichID learns optimal control policies from nonlinear observations.
Unified framework for policy improvement in RL with benefits in data efficiency and computation.
A fundamental challenge in artificial intelligence is to build an agent that generalizes and adapts to unseen environments. A common strategy is to build a decoder that takes the context of the unseen new environment as input and generates a policy accordingly. The current paper studies how to build a decoder for the f…
Improves RL algorithms with two techniques.
Action chunking and data exploration improve behavior cloning in robotics.
Many continuous control tasks have easily formulated objectives, yet using them directly as a reward in reinforcement learning (RL) leads to suboptimal policies. Therefore, many classical control tasks guide RL training using complex rewards, which require tedious hand-tuning. We automate the reward search with AutoRL,…
Policy optimization on high-dimensional continuous control tasks exhibits its difficulty caused by the large variance of the policy gradient estimators. We present the action subspace dependent gradient (ASDG) estimator which incorporates the Rao-Blackwell theorem (RB) and Control Variates (CV) into a unified framework…
Many continuous control tasks have bounded action spaces. When policy gradient methods are applied to such tasks, out-of-bound actions need to be clipped before execution, while policies are usually optimized as if the actions are not clipped. We propose a policy gradient estimator that exploits the knowledge of action…
Paper introduces a meta-critic for accelerating off-policy actor-critic learning.
Reliable and effective multi-task learning is a prerequisite for the development of robotic agents that can quickly learn to accomplish related, everyday tasks. However, in the reinforcement learning domain, multi-task learning has not exhibited the same level of success as in other domains, such as computer vision. In…
USAC balances pessimism and optimism in actor-critic training for better exploration and performance.
In recent years significant progress has been made in dealing with challenging problems using reinforcement learning.Despite its great success, reinforcement learning still faces challenge in continuous control tasks. Conventional methods always compute the derivatives of the optimal goal with a costly computation reso…
PFPN uses particle filtering to improve character control in physics-based simulations.
Improved reinforcement learning with deep learning.
We observe that several existing policy gradient methods (such as vanilla policy gradient, PPO, A2C) may suffer from overly large gradients when the current policy is close to deterministic (even in some very simple environments), leading to an unstable training process. To address this issue, we propose a new method, …
Policy optimization is an effective reinforcement learning approach to solve continuous control tasks. Recent achievements have shown that alternating online and offline optimization is a successful choice for efficient trajectory reuse. However, deciding when to stop optimizing and collect new trajectories is non-triv…
A hybrid model combines Q-learning and PID controller for continuous vehicle control.
We identify a fundamental problem in policy gradient-based methods in continuous control. As policy gradient methods require the agent's underlying probability distribution, they limit policy representation to parametric distribution classes. We show that optimizing over such sets results in local movement in the actio…
Despite recent successes in Reinforcement Learning, value-based methods often suffer from high variance hindering performance. In this paper, we illustrate this in a continuous control setting where state of the art methods perform poorly whenever sensor noise is introduced. To overcome this issue, we introduce Recurre…
Traditional model-based RL relies on hand-specified or learned models of transition dynamics of the environment. These methods are sample efficient and facilitate learning in the real world but fail to generalize to subtle variations in the underlying dynamics, e.g., due to differences in mass, friction, or actuators a…
Algorithm mitigates performance loss in constrained reinforcement learning with model misspecification.
Evolutionary Strategies optimize hyper-parameters for off-policy learning.
GWIL uses Gromov-Wasserstein distance to align expert and imitation agent states.
Weakly-supervised RL identifies meaningful tasks, improving performance in complex environments.
Proposes a Quasi-Newton trust region method for policy optimization in reinforcement learning.
Safe reinforcement learning framework using optimal transport for robustness.
New framework improves offline RL performance without complex methods.
Following Roe and others (see, e.g., [MR1451755]), we (re)develop coarse geometry from the foundations, taking a categorical point of view. In this paper, we concentrate on the discrete case in which topology plays no role. Our theory is particularly suited to the development of the_Roe (C*-)algebras_ C*(X) and their K…
AQL uses amortized inference to handle high-dimensional action spaces in Q-learning.