A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
AR-A3C improves A3C's robustness to noisy environments.
Policy gradient is an efficient technique for improving a policy in a reinforcement learning setting. However, vanilla online variants are on-policy only and not able to take advantage of off-policy data. In this paper we describe a new technique that combines policy gradient with off-policy Q-learning, drawing experie…
We present an approach for reconfiguration of dynamic visual sensor networks with deep reinforcement learning (RL). Our RL agent uses a modified asynchronous advantage actor-critic framework and the recently proposed Relational Network module at the foundation of its network architecture. To address the issue of sample…
After presenting Actor Critic Methods (ACM), we show ACM are control variate estimators. Using the projection theorem, we prove that the Q and Advantage Actor Critic (A2C) methods are optimal in the sense of the norm for the control variate estimators spanned by functions conditioned by the current state and acti…
Multi-simulator training has contributed to the recent success of Deep Reinforcement Learning by stabilizing learning and allowing for higher training throughputs. We propose Gossip-based Actor-Learner Architectures (GALA) where several actor-learners (such as A2C agents) are organized in a peer-to-peer communication t…
Motivated by recent advance of machine learning using Deep Reinforcement Learning this paper proposes a modified architecture that produces more robust agents and speeds up the training process. Our architecture is based on Asynchronous Advantage Actor-Critic (A3C) algorithm where the total input dimensionality is halv…
Smaller actor-critic models lead to performance degradation and overfitting, highlighting the critic's role in value underestimation.
SLM Lab is a framework for reproducible RL research with modular algorithms.
In traditional reinforcement learning, an agent maximizes the reward collected during its interaction with the environment by approximating the optimal policy through the estimation of value functions. Typically, given a state s and action a, the corresponding value is the expected discounted sum of rewards. The optima…
Deep reinforcement learning is poised to revolutionise the field of AI and represents a step towards building autonomous systems with a higher level understanding of the visual world. Currently, deep learning is enabling reinforcement learning to scale to problems that were previously intractable, such as learning to p…
TT-DAC-PS: A deterministic actor-critic approach for optimal trade execution
Deep reinforcement learning has shown its success in game playing. However, 2.5D fighting games would be a challenging task to handle due to ambiguity in visual appearances like height or depth of the characters. Moreover, actions in such games typically involve particular sequential action orders, which also makes the…
PEARL uses reinforcement learning to improve matrix preconditioners.
Deep reinforcement learning has achieved great successes in recent years, but there are still open challenges, such as convergence to locally optimal policies and sample inefficiency. In this paper, we contribute a novel self-supervised auxiliary task, i.e., Terminal Prediction (TP), estimating temporal closeness to te…
Reinforcement learning improves trading performance on stock exchanges.
In 2015, Google's DeepMind announced an advancement in creating an autonomous agent based on deep reinforcement learning (DRL) that could beat a professional player in a series of 49 Atari games. However, the current manifestation of DRL is still immature, and has significant drawbacks. One of DRL's imperfections is it…
Based on graphic lambda calculus, we propose a program for a new model of asynchronous distributed computing, inspired from Hewitt Actor Model, as well as several investigation paths, concerning how one may graft lambda calculus and knot diagrammatics.
This paper proposes Self-Imitation Learning (SIL), a simple off-policy actor-critic algorithm that learns to reproduce the agent's past good decisions. This algorithm is designed to verify our hypothesis that exploiting past good experiences can indirectly drive deep exploration. Our empirical results show that SIL sig…
The paper tackles scalarization issues in A2C RL algorithms, proposing methods to avoid gradient overlap and noise.
Deep reinforcement learning (deep RL) has achieved superior performance in complex sequential tasks by using deep neural networks as function approximators to learn directly from raw input images. However, learning directly from raw images is data inefficient. The agent must learn feature representation of complex stat…
Improved SAC with AWMP for better control tasks.
D2D-SPL uses discrete states and a classifier to train RL faster.
New method combines value function decomposition and policy gradients for cooperative multi-agent reinforcement learning.
Paper provides convergence guarantees for off-policy NAC with finite sample complexity.
The paper analyzes an actor-critic algorithm with target networks for deep reinforcement learning.
Actor-critic methods solve reinforcement learning problems by updating a parameterized policy known as an actor in a direction that increases an estimate of the expected return known as a critic. However, existing actor-critic methods only use values or gradients of the critic to update the policy parameter. In this pa…
New PAC-Bayesian approach stabilizes actor-critic learning.
Single-timescale actor-critic finds globally optimal policy.
Actor-critic methods can achieve incredible performance on difficult reinforcement learning problems, but they are also prone to instability. This is partly due to the interaction between the actor and critic during learning, e.g., an inaccurate step taken by one of them might adversely affect the other and destabilize…
New analysis shows actor-critic method converges efficiently in practical settings.
This work analyzes actor-critic methods for faster convergence.
New algorithm reduces bias in off-policy reinforcement learning.
Despite the empirical success of the actor-critic algorithm, its theoretical understanding lags behind. In a broader context, actor-critic can be viewed as an online alternating update algorithm for bilevel optimization, whose convergence is known to be fragile. To understand the instability of actor-critic, we focus o…
Paper introduces a meta-critic for accelerating off-policy actor-critic learning.
Efficient learning-based MPC for unknown nonlinear systems with state constraints.
WRAAC uses Wasserstein distance for robust reinforcement learning.
Deep Reinforcement Learning (DRL) algorithms are known to be data inefficient. One reason is that a DRL agent learns both the feature and the policy tabula rasa. Integrating prior knowledge into DRL algorithms is one way to improve learning efficiency since it helps to build helpful representations. In this work, we co…
Actor-critic algorithms converge to an ODE as data samples change dynamically.
Current imitation learning techniques are too restrictive because they require the agent and expert to share the same action space. However, oftentimes agents that act differently from the expert can solve the task just as good. For example, a person lifting a box can be imitated by a ceiling mounted robot or a desktop…
COF-PAC converges with novel critic and learning method.
WAVE improves stability in reinforcement learning by adaptively weighting critic's loss.
Capsule Networks improve autonomous navigation in sparse environments.
Neural networks trained with actor-critic algorithms converge to ODEs under weak convergence analysis.
Study uses reinforcement learning to optimize portfolios under recursive utility.
Deep reinforcement learning boosts commodities trading performance.
Actor-Critic method achieves optimal regret for unichain MDPs.
We present an actor-critic framework for MDPs where the objective is the variance-adjusted expected return. Our critic uses linear function approximation, and we extend the concept of compatible features to the variance-adjusted setting. We present an episodic actor-critic algorithm and show that it converges almost su…
In this paper we propose a hybrid architecture of actor-critic algorithms for reinforcement learning in parameterized action space, which consists of multiple parallel sub-actor networks to decompose the structured action space into simpler action spaces along with a critic network to guide the training of all sub-acto…