arXiv research

Improved forecasting of investment dynamics across heterogeneous panels using a two-stage model.

Deep RL solves complex economic models with heterogeneous agents.

Model shows PoS networks can be captured by external finance, leading to centralization.

The statistical description and modeling of volatility plays a prominent role in econometrics, risk management and finance. GARCH and stochastic volatility models have been extensively studied and are routinely fitted to market data, albeit providing a phenomenological description only. In contrast, the field of econop…

Deep RL agent secures 2nd place in CityLearn Challenge for district demand management.

Our focus is on realistically modeling and forecasting dynamic networks of face-to-face contacts among individuals. Important aspects of such data that lead to problems with current methods include the tendency of the contacts to move between periods of slow and rapid changes, and the dynamic heterogeneity in the actor…

Actor-critic methods solve reinforcement learning problems by updating a parameterized policy known as an actor in a direction that increases an estimate of the expected return known as a critic. However, existing actor-critic methods only use values or gradients of the critic to update the policy parameter. In this pa…

The paper analyzes an actor-critic algorithm with target networks for deep reinforcement learning.

Single-timescale actor-critic finds globally optimal policy.

New PAC-Bayesian approach stabilizes actor-critic learning.

Actor-critic methods can achieve incredible performance on difficult reinforcement learning problems, but they are also prone to instability. This is partly due to the interaction between the actor and critic during learning, e.g., an inaccurate step taken by one of them might adversely affect the other and destabilize…

FORK improves model-free reinforcement learning performance.

Smaller actor-critic models lead to performance degradation and overfitting, highlighting the critic's role in value underestimation.

New analysis shows actor-critic method converges efficiently in practical settings.

This work analyzes actor-critic methods for faster convergence.

Despite the empirical success of the actor-critic algorithm, its theoretical understanding lags behind. In a broader context, actor-critic can be viewed as an online alternating update algorithm for bilevel optimization, whose convergence is known to be fragile. To understand the instability of actor-critic, we focus o…

Actor-critic algorithms converge to an ODE as data samples change dynamically.

AGAC uses an adversary to enhance exploration in complex tasks.

Many policy gradient methods are variants of Actor-Critic (AC), where a value function (critic) is learned to facilitate updating the parameterized policy (actor). The update to the actor involves a log-likelihood update weighted by the action-values, with the addition of entropy regularization for soft variants. In th…

Neural networks trained with actor-critic algorithms converge to ODEs under weak convergence analysis.

SAVO actor improves reinforcement learning by avoiding local optima in complex Q-functions.

The notion that economies should normally be in equilibrium is by now well-established; equally well-established is that economies are almost never precisely in equilibrium. Using a very general formulation, we show that under dynamics that are second-order in time a price system can remain away from equilibrium with p…

Traffic actors' future motion predicted using a hybrid graph model.

In this paper we propose a hybrid architecture of actor-critic algorithms for reinforcement learning in parameterized action space, which consists of multiple parallel sub-actor networks to decompose the structured action space into simpler action spaces along with a critic network to guide the training of all sub-acto…

Proposes a new model for predicting future motion of road actors in autonomous vehicles.

Actor-Critic method achieves optimal regret for unichain MDPs.

TT-DAC-PS: A deterministic actor-critic approach for optimal trade execution

Proposes a value-based method for continuous control without an actor.

Optimistic actor-critic tackles linear MDPs with parametric policies.

We study the emergence of instabilities in a stylized model of a financial market, when different market actors calculate prices according to different (local) market measures. We derive typical properties for ensembles of large random markets using techniques borrowed from statistical mechanics of disordered systems. …

Paper introduces a meta-critic for accelerating off-policy actor-critic learning.

ESAC improves reinforcement learning by lookahead and intuition.

We present an actor-critic framework for MDPs where the objective is the variance-adjusted expected return. Our critic uses linear function approximation, and we extend the concept of compatible features to the variance-adjusted setting. We present an episodic actor-critic algorithm and show that it converges almost su…

We report a data mining pipeline and subsequent analysis to understand the core periphery power structure created in three national newspapers in Bangladesh, as depicted by statements made by people appearing in news. Statements made by one actor about another actor can be considered a form of public conversation. Name…

WAVE improves stability in reinforcement learning by adaptively weighting critic's loss.

Study uses actor-critic method for continuous-time mean-field control with entropy regularisation.

A simple guide to understanding hierarchical causality in complex systems.

We model messaging activities as a hierarchical doubly stochastic point process with three main levels, and develop an iterative algorithm for inferring actors' relative latent positions from a stream of messaging activity data. Each of the message-exchanging actors is modeled as a process in a latent space. The actors…

A new method improves actor-critic RL by integrating HMC, enhancing policy distribution and exploration.

After presenting Actor Critic Methods (ACM), we show ACM are control variate estimators. Using the projection theorem, we prove that the Q and Advantage Actor Critic (A2C) methods are optimal in the sense of the $L^2$ norm for the control variate estimators spanned by functions conditioned by the current state and acti…

Model-free deep reinforcement learning (RL) algorithms have been demonstrated on a range of challenging decision making and control tasks. However, these methods typically suffer from two major challenges: very high sample complexity and brittle convergence properties, which necessitate meticulous hyperparameter tuning…

Study visualizes actor-critic loss landscapes for inventory optimization.

Reinforcement learning for continuous-time risk-sensitive asset allocation

Model visualizes and analyzes multilayer networks in a latent space.

In a dynamic social or biological environment, the interactions between the actors can undergo large and systematic changes. In this paper we propose a model-based approach to analyze what we will refer to as the dynamic tomography of such time-evolving networks. Our approach offers an intuitive but powerful tool to in…

New testing method for robust actor-critic bandit algorithms.

Paper analyzes convergence rates of two time-scale AC and NAC algorithms.

We address one of the crucial aspects necessary for safe and efficient operations of autonomous vehicles, namely predicting future state of traffic actors in the autonomous vehicle's surroundings. We introduce a deep learning-based approach that takes into account a current world state and produces raster images of eac…