A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Model visualizes and analyzes multilayer networks in a latent space.
Proposes a new model for predicting future motion of road actors in autonomous vehicles.
This paper considers a distributed reinforcement learning problem in which a network of multiple agents aim to cooperatively maximize the globally averaged return through communication with only local neighbors. A randomized communication-efficient multi-agent actor-critic algorithm is proposed for possibly unidirectio…
We establish a new connection between value and policy based reinforcement learning (RL) based on a relationship between softmax temporal value consistency and policy optimality under entropy regularization. Specifically, we show that softmax consistent action values correspond to optimal entropy regularized policy pro…
Modern control theories such as systems engineering approaches try to solve nonlinear system problems by revelation of causal relationship or co-relationship among the components; most of those approaches focus on control of sophisticatedly modeled white-boxed systems. We suggest an application of actor-critic reinforc…
Study of public and private VC relationships in France using qualitative methods.
Actor-critic methods solve reinforcement learning problems by updating a parameterized policy known as an actor in a direction that increases an estimate of the expected return known as a critic. However, existing actor-critic methods only use values or gradients of the critic to update the policy parameter. In this pa…
The paper analyzes an actor-critic algorithm with target networks for deep reinforcement learning.
Single-timescale actor-critic finds globally optimal policy.
New PAC-Bayesian approach stabilizes actor-critic learning.
Actor-critic methods can achieve incredible performance on difficult reinforcement learning problems, but they are also prone to instability. This is partly due to the interaction between the actor and critic during learning, e.g., an inaccurate step taken by one of them might adversely affect the other and destabilize…
FORK improves model-free reinforcement learning performance.
Smaller actor-critic models lead to performance degradation and overfitting, highlighting the critic's role in value underestimation.
New analysis shows actor-critic method converges efficiently in practical settings.
This work analyzes actor-critic methods for faster convergence.
Most previous studies on multi-agent reinforcement learning focus on deriving decentralized and cooperative policies to maximize a common reward and rarely consider the transferability of trained policies to new tasks. This prevents such policies from being applied to more complex multi-agent tasks. To resolve these li…
Despite the empirical success of the actor-critic algorithm, its theoretical understanding lags behind. In a broader context, actor-critic can be viewed as an online alternating update algorithm for bilevel optimization, whose convergence is known to be fragile. To understand the instability of actor-critic, we focus o…
Actor-critic algorithms converge to an ODE as data samples change dynamically.
AGAC uses an adversary to enhance exploration in complex tasks.
Many policy gradient methods are variants of Actor-Critic (AC), where a value function (critic) is learned to facilitate updating the parameterized policy (actor). The update to the actor involves a log-likelihood update weighted by the action-values, with the addition of entropy regularization for soft variants. In th…
Neural networks trained with actor-critic algorithms converge to ODEs under weak convergence analysis.
SAVO actor improves reinforcement learning by avoiding local optima in complex Q-functions.
Traffic actors' future motion predicted using a hybrid graph model.
In this paper we propose a hybrid architecture of actor-critic algorithms for reinforcement learning in parameterized action space, which consists of multiple parallel sub-actor networks to decompose the structured action space into simpler action spaces along with a critic network to guide the training of all sub-acto…
Actor-Critic method achieves optimal regret for unichain MDPs.
TT-DAC-PS: A deterministic actor-critic approach for optimal trade execution
Proposes a value-based method for continuous control without an actor.
Optimistic actor-critic tackles linear MDPs with parametric policies.
Paper introduces a meta-critic for accelerating off-policy actor-critic learning.
National statistical systems are the enterprises tasked with collecting, validating and reporting societal attributes. These data serve many purposes - they allow governments to improve services, economic actors to traverse markets, and academics to assess social theories. National statistical systems vary in quality, …
ESAC improves reinforcement learning by lookahead and intuition.
We present an actor-critic framework for MDPs where the objective is the variance-adjusted expected return. Our critic uses linear function approximation, and we extend the concept of compatible features to the variance-adjusted setting. We present an episodic actor-critic algorithm and show that it converges almost su…
We report a data mining pipeline and subsequent analysis to understand the core periphery power structure created in three national newspapers in Bangladesh, as depicted by statements made by people appearing in news. Statements made by one actor about another actor can be considered a form of public conversation. Name…
WAVE improves stability in reinforcement learning by adaptively weighting critic's loss.
Study uses actor-critic method for continuous-time mean-field control with entropy regularisation.
A simple guide to understanding hierarchical causality in complex systems.
We model messaging activities as a hierarchical doubly stochastic point process with three main levels, and develop an iterative algorithm for inferring actors' relative latent positions from a stream of messaging activity data. Each of the message-exchanging actors is modeled as a process in a latent space. The actors…
A new method improves actor-critic RL by integrating HMC, enhancing policy distribution and exploration.
After presenting Actor Critic Methods (ACM), we show ACM are control variate estimators. Using the projection theorem, we prove that the Q and Advantage Actor Critic (A2C) methods are optimal in the sense of the norm for the control variate estimators spanned by functions conditioned by the current state and acti…
Model-free deep reinforcement learning (RL) algorithms have been demonstrated on a range of challenging decision making and control tasks. However, these methods typically suffer from two major challenges: very high sample complexity and brittle convergence properties, which necessitate meticulous hyperparameter tuning…
Study visualizes actor-critic loss landscapes for inventory optimization.
Reinforcement learning for continuous-time risk-sensitive asset allocation
In a dynamic social or biological environment, the interactions between the actors can undergo large and systematic changes. In this paper we propose a model-based approach to analyze what we will refer to as the dynamic tomography of such time-evolving networks. Our approach offers an intuitive but powerful tool to in…
New testing method for robust actor-critic bandit algorithms.
Paper analyzes convergence rates of two time-scale AC and NAC algorithms.
We address one of the crucial aspects necessary for safe and efficient operations of autonomous vehicles, namely predicting future state of traffic actors in the autonomous vehicle's surroundings. We introduce a deep learning-based approach that takes into account a current world state and produces raster images of eac…
Proposes online learning for Hawkes processes with network structure and event interaction.
New algorithm solves mean-field control problems using actor-critic learning with moment neural networks.