A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Meta-SAC automatically tunes SAC's entropy temperature for better exploration.
Soft Actor-Critic (SAC) is an off-policy actor-critic deep reinforcement learning (DRL) algorithm based on maximum entropy reinforcement learning. By combining off-policy updates with an actor-critic formulation, SAC achieves state-of-the-art performance on a range of continuous-action benchmark tasks, outperforming pr…
PI-SAC agents learn predictive information to improve RL efficiency.
Band-limited SAC improves learning efficiency and stability in simulated environments.
The ability to discover approximately optimal policies in domains with sparse rewards is crucial to applying reinforcement learning (RL) in many real-world scenarios. Approaches such as neural density models and continuous exploration (e.g., Go-Explore) have been proposed to maintain the high exploration rate necessary…
LC-SAC tackles non-stationary dynamics in reinforcement learning.
Paper uses SAC and DDPG to optimize cryptocurrency portfolios.
Paper improves AIRL by enhancing policy imitation and addressing reward recovery issues.
Self attention mechanisms have become a key building block in many state-of-the-art language understanding models. In this paper, we show that the self attention operator can be formulated in terms of 1x1 convolution operations. Following this observation, we propose several novel operators: First, we introduce a 2D ve…
Model-free deep reinforcement learning (RL) algorithms have been successfully applied to a range of challenging sequential decision making and control tasks. However, these methods typically suffer from two major challenges: high sample complexity and brittleness to hyperparameters. Both of these challenges limit the a…
Paper uses SAC RL to optimize market-making strategies.
TOPPO improves PPO for MTRL by balancing critic gradients, outperforming SAC.
We introduce higher order mean curvatures of screen almost conformal (SAC) half-lightlike submanifolds of indefinite contact manifolds, admitting a semi-symmetric non-metric connection, and use them to generalize some known results of [6]. Also, we derive a new set of integration formulae via the divergence of some spe…
While most current research in Reinforcement Learning (RL) focuses on improving the performance of the algorithms in controlled environments, the use of RL under constraints like those met in the video game industry is rarely studied. Operating under such constraints, we propose Hybrid SAC, an extension of the Soft Act…
Paper uses RL to optimize multi-asset portfolios in fluctuating markets.
Paper improves SVaR estimation for stress testing under macro scenarios using a hybrid GPR-HS framework.
We aim to develop off-policy DRL algorithms that not only exceed state-of-the-art performance but are also simple and minimalistic. For standard continuous control benchmarks, Soft Actor-Critic (SAC), which employs entropy maximization, currently provides state-of-the-art performance. We first demonstrate that the entr…
The optimal policy of a reinforcement learning problem is often discontinuous and non-smooth. I.e., for two states with similar representations, their optimal policies can be significantly different. In this case, representing the entire policy with a function approximator (FA) with shared parameters for all states may…
We propose Scheduled Auxiliary Control (SAC-X), a new learning paradigm in the context of Reinforcement Learning (RL). SAC-X enables learning of complex behaviors - from scratch - in the presence of multiple sparse reward signals. To this end, the agent is equipped with a set of general auxiliary tasks, that it attempt…
We propose a new policy iteration theory as an important extension of soft policy iteration and Soft Actor-Critic (SAC), one of the most efficient model free algorithms for deep reinforcement learning. Supported by the new theory, arbitrary entropy measures that generalize Shannon entropy, such as Tsallis entropy and R…
Deep Reinforcement Learning (DRL) algorithms for continuous action spaces are known to be brittle toward hyperparameters as well as \cut{being}sample inefficient. Soft Actor Critic (SAC) proposes an off-policy deep actor critic algorithm within the maximum entropy RL framework which offers greater stability and empiric…
New PAC-Bayesian approach stabilizes actor-critic learning.
Researchers analyze how algorithmic and implementation choices affect RL performance.
Paper uses DRL to improve volatility fitting in equity derivatives.
Simple image augmentation boosts deep RL from pixels.
This paper analyzes DRL strategies in finance, revealing unique trading patterns and performance differences.
CausalCOMRL improves RL task representations by integrating causal relationships, enhancing generalizability.
ESAC improves reinforcement learning by lookahead and intuition.
Safe-FinRL uses DRL for high-frequency stock trading, reducing bias and variance.
From a young age humans learn to use grammatical principles to hierarchically combine words into sentences. Action grammars is the parallel idea, that there is an underlying set of rules (a "grammar") that govern how we hierarchically combine actions to form new, more complex actions. We introduce the Action Grammar Re…
We present a method for fast training of vision based control policies on real robots. The key idea behind our method is to perform multi-task Reinforcement Learning with auxiliary tasks that differ not only in the reward to be optimized but also in the state-space in which they operate. In particular, we allow auxilia…
TT-DAC-PS: A deterministic actor-critic approach for optimal trade execution
ESAC combines genetic methods with RL to improve scalability and efficiency.
Develops RL for optimal market-making in non-Markov processes.
Cumulative entropy regularization introduces a regulatory signal to the reinforcement learning (RL) problem that encourages policies with high-entropy actions, which is equivalent to enforcing small deviations from a uniform reference marginal policy. This has been shown to improve exploration and robustness, and it ta…
The paper combines several fortunate mini miracles to achieve its two objectives. These were woven together in a several year's effort to answer a question raised by Iz Singer a decade ago. Our answer is accessible to the topologist, to the differential geometer and to the analyst who appreciates the statement of the I…
Deep RL algorithms struggle with noisy rewards in portfolio optimisation.
ApolloRL offers a platform for RL research in autonomous driving.
Mirror descent method improved RL algorithms.
Continuous control tasks in reinforcement learning are important because they provide an important framework for learning in high-dimensional state spaces with deceptive rewards, where the agent can easily become trapped into suboptimal solutions. One way to avoid local optima is to use a population of agents to ensure…
New reinforcement learning bound improves generalization for sequential data.
Recent successful deep reinforcement learning algorithms, such as Trust Region Policy Optimization (TRPO) or Proximal Policy Optimization (PPO), are fundamentally variations of conservative policy iteration (CPI). These algorithms iterate policy evaluation followed by a softened policy improvement step. As so, they are…
A new RL framework handles autocorrelated actions for better learning and stability.
Variational inference with α-divergences has been widely used in modern probabilistic machine learning. Compared to Kullback-Leibler (KL) divergence, a major advantage of using α-divergences (with positive α values) is their mass-covering property. However, estimating and optimizing α-divergences require to use importa…
SMEs provide a transparent testbed for RL evaluation.
Training an agent to solve control tasks directly from high-dimensional images with model-free reinforcement learning (RL) has proven difficult. A promising approach is to learn a latent representation together with the control policy. However, fitting a high-capacity encoder using a scarce reward signal is sample inef…
Sample efficiency is a crucial problem in deep reinforcement learning. Recent algorithms, such as REDQ and DroQ, found a way to improve the sample efficiency by increasing the update-to-data (UTD) ratio to 20 gradient update steps on the critic per environment sample. However, this comes at the expense of a greatly inc…
The paper improves model-based reinforcement learning by using multi-timestep objectives.