arXiv research

Meta-SAC automatically tunes SAC's entropy temperature for better exploration.

Paper uses SAC and DDPG to optimize cryptocurrency portfolios.

Band-limited SAC improves learning efficiency and stability in simulated environments.

The ability to discover approximately optimal policies in domains with sparse rewards is crucial to applying reinforcement learning (RL) in many real-world scenarios. Approaches such as neural density models and continuous exploration (e.g., Go-Explore) have been proposed to maintain the high exploration rate necessary…

LC-SAC tackles non-stationary dynamics in reinforcement learning.

TOPPO improves PPO for MTRL by balancing critic gradients, outperforming SAC.

Paper uses SAC RL to optimize market-making strategies.

Soft Actor-Critic (SAC) is an off-policy actor-critic deep reinforcement learning (DRL) algorithm based on maximum entropy reinforcement learning. By combining off-policy updates with an actor-critic formulation, SAC achieves state-of-the-art performance on a range of continuous-action benchmark tasks, outperforming pr…

PI-SAC agents learn predictive information to improve RL efficiency.

Paper uses RL to optimize multi-asset portfolios in fluctuating markets.

While most current research in Reinforcement Learning (RL) focuses on improving the performance of the algorithms in controlled environments, the use of RL under constraints like those met in the video game industry is rarely studied. Operating under such constraints, we propose Hybrid SAC, an extension of the Soft Act…

The optimal policy of a reinforcement learning problem is often discontinuous and non-smooth. I.e., for two states with similar representations, their optimal policies can be significantly different. In this case, representing the entire policy with a function approximator (FA) with shared parameters for all states may…

Paper improves AIRL by enhancing policy imitation and addressing reward recovery issues.

Self attention mechanisms have become a key building block in many state-of-the-art language understanding models. In this paper, we show that the self attention operator can be formulated in terms of 1x1 convolution operations. Following this observation, we propose several novel operators: First, we introduce a 2D ve…

Model-free deep reinforcement learning (RL) algorithms have been successfully applied to a range of challenging sequential decision making and control tasks. However, these methods typically suffer from two major challenges: high sample complexity and brittleness to hyperparameters. Both of these challenges limit the a…

CausalCOMRL improves RL task representations by integrating causal relationships, enhancing generalizability.

We introduce higher order mean curvatures of screen almost conformal (SAC) half-lightlike submanifolds of indefinite contact manifolds, admitting a semi-symmetric non-metric connection, and use them to generalize some known results of [6]. Also, we derive a new set of integration formulae via the divergence of some spe…

Paper improves SVaR estimation for stress testing under macro scenarios using a hybrid GPR-HS framework.

Paper uses DRL to improve volatility fitting in equity derivatives.

We aim to develop off-policy DRL algorithms that not only exceed state-of-the-art performance but are also simple and minimalistic. For standard continuous control benchmarks, Soft Actor-Critic (SAC), which employs entropy maximization, currently provides state-of-the-art performance. We first demonstrate that the entr…

ESAC improves reinforcement learning by lookahead and intuition.

Develops RL for optimal market-making in non-Markov processes.

TT-DAC-PS: A deterministic actor-critic approach for optimal trade execution

We propose Scheduled Auxiliary Control (SAC-X), a new learning paradigm in the context of Reinforcement Learning (RL). SAC-X enables learning of complex behaviors - from scratch - in the presence of multiple sparse reward signals. To this end, the agent is equipped with a set of general auxiliary tasks, that it attempt…

We propose a new policy iteration theory as an important extension of soft policy iteration and Soft Actor-Critic (SAC), one of the most efficient model free algorithms for deep reinforcement learning. Supported by the new theory, arbitrary entropy measures that generalize Shannon entropy, such as Tsallis entropy and R…

Mirror descent method improved RL algorithms.

We present a method for fast training of vision based control policies on real robots. The key idea behind our method is to perform multi-task Reinforcement Learning with auxiliary tasks that differ not only in the reward to be optimized but also in the state-space in which they operate. In particular, we allow auxilia…

Deep RL algorithms struggle with noisy rewards in portfolio optimisation.

Deep Reinforcement Learning (DRL) algorithms for continuous action spaces are known to be brittle toward hyperparameters as well as \cut{being}sample inefficient. Soft Actor Critic (SAC) proposes an off-policy deep actor critic algorithm within the maximum entropy RL framework which offers greater stability and empiric…

Cumulative entropy regularization introduces a regulatory signal to the reinforcement learning (RL) problem that encourages policies with high-entropy actions, which is equivalent to enforcing small deviations from a uniform reference marginal policy. This has been shown to improve exploration and robustness, and it ta…

New PAC-Bayesian approach stabilizes actor-critic learning.

Researchers analyze how algorithmic and implementation choices affect RL performance.

ESAC combines genetic methods with RL to improve scalability and efficiency.

ApolloRL offers a platform for RL research in autonomous driving.

Simple image augmentation boosts deep RL from pixels.

This paper analyzes DRL strategies in finance, revealing unique trading patterns and performance differences.

Recent successful deep reinforcement learning algorithms, such as Trust Region Policy Optimization (TRPO) or Proximal Policy Optimization (PPO), are fundamentally variations of conservative policy iteration (CPI). These algorithms iterate policy evaluation followed by a softened policy improvement step. As so, they are…

SMEs provide a transparent testbed for RL evaluation.

Safe-FinRL uses DRL for high-frequency stock trading, reducing bias and variance.

A new RL framework handles autocorrelated actions for better learning and stability.

From a young age humans learn to use grammatical principles to hierarchically combine words into sentences. Action grammars is the parallel idea, that there is an underlying set of rules (a "grammar") that govern how we hierarchically combine actions to form new, more complex actions. We introduce the Action Grammar Re…

New reinforcement learning bound improves generalization for sequential data.

Study optimizes GCS operations with deep learning and reinforcement learning.

Study uses RL to optimize crypto portfolios with two-sided transactions and lending.

Variational inference with α-divergences has been widely used in modern probabilistic machine learning. Compared to Kullback-Leibler (KL) divergence, a major advantage of using α-divergences (with positive α values) is their mass-covering property. However, estimating and optimizing α-divergences require to use importa…

AdaRL improves robust RL by adaptively adjusting policy complexity.

The paper combines several fortunate mini miracles to achieve its two objectives. These were woven together in a several year's effort to answer a question raised by Iz Singer a decade ago. Our answer is accessible to the topologist, to the differential geometer and to the analyst who appreciates the statement of the I…

Model-based reinforcement learning (MBRL) with model-predictive control or online planning has shown great potential for locomotion control tasks in terms of both sample efficiency and asymptotic performance. Despite their initial successes, the existing planning methods search from candidate sequences randomly generat…