arXiv research

Study designs incentives for adapting multi-agent systems without knowing their learning dynamics.

Paper introduces PRMs to learn non-Markovian stochastic rewards for reinforcement learning.

Describes state variables in sequential decision problems, linking them to Markovian and non-Markovian models.

A new metric detects non-Markovian states in partially observable environments.

This paper studies the equilibrium pricing of asset shares in the presence of dynamic private information. The market consists of a risk-neutral informed agent who observes the firm value, noise traders, and competitive market makers who set share prices using the total order flow as a noisy signal of the insider's inf…

ARL bridges non-Markovian decision processes with reinforcement learning, improving foresight and stability.

This paper develops a new methodology for studying continuous-time Nash equilibrium in a financial market with asymmetrically informed agents. This approach allows us to lift the restriction of risk neutrality imposed on market makers by the current literature. It turns out that, when the market makers are risk averse,…

Most real-world problems have huge state and/or action spaces. Therefore, a naive application of existing tabular solution methods is not tractable on such problems. Nonetheless, these solution methods are quite useful if an agent has access to a relatively small state-action space homomorphism of the true environment …

In a Markovian stochastic volatility model, we consider financial agents whose investment criteria are modelled by forward exponential performance processes. The problem of contingent claim indifference valuation is first addressed and a number of properties are proved and discussed. Special attention is given to the c…

We propose a deep neural network-based algorithm to identify the Markovian Nash equilibrium of general large $N$-player stochastic differential games. Following the idea of fictitious play, we recast the $N$-player game into $N$ decoupled decision problems (one for each player) and solve them iteratively. The individua…

FedSARSA converges with heterogeneous agents, achieving linear speed-up.

We prove the global existence of an incomplete, continuous-time finite-agent Radner equilibrium in which exponential agents optimize their expected utility over both running consumption and terminal wealth. The market consists of a traded annuity, and, along with unspanned income, the market is incomplete. Set in a Bro…

Paper analyzes convergence of decentralized algorithms with noise and bias.

DASA speeds up SA with delayed agents, achieving N-fold speedup.

Optimal strategies are found for a repeated betting game using diffusion approximation.

We propose a general non-linear order book model that is built from the individual behaviours of the agents. Our framework encompasses Markovian and Hawkes based models. Under mild assumptions, we prove original results on the ergodicity and diffusivity of such system. Then we provide closed form formulas for various q…

UD-SGD analysis shows efficient sampling by a few agents can outperform others.

Motivated by the emerging use of multi-agent reinforcement learning (MARL) in engineering applications such as networked robotics, swarming drones, and sensor networks, we investigate the policy evaluation problem in a fully decentralized setting, using temporal-difference (TD) learning with linear function approximati…

It is shown how the generating functional method of De Dominicis can be used to solve the dynamics of the original version of the minority game (MG), in which agents observe real as opposed to fake market histories. Here one again finds exact closed equations for correlation and response functions, but now these are de…

Quasi-equilibrium models for aggregate variables are widely-used throughout finance and economics. The validity of such models depends crucially upon assuming that the systems' participants behave both independently and in a Markovian fashion. We present a simplified market model to demonstrate that herding effects bet…

PFedRL-Rep learns shared and personalized policies for heterogeneous environments.

Non-Markovian point process shows power-law scaling, similar to nonlinear Markovian process.

Develops variational framework for LQG risk-sensitive MFGs with major-minor interactions.

Projects Markovian processes from Itô semimartingales with jumps.

Study on price formation in financial markets with a single default event.

This paper proposes DeepSynth, a method for effective training of deep Reinforcement Learning (RL) agents when the reward is sparse and non-Markovian, but at the same time progress towards the reward requires achieving an unknown sequence of high-level objectives. Our method employs a novel algorithm for synthesis of c…

This paper extends Markovian projections to semimartingales with jumps.

In this paper we investigate the local risk-minimization approach for a semimartingale financial market where there are restrictions on the available information to agents who can observe at least the asset prices. We characterize the optimal strategy in terms of suitable decompositions of a given contingent claim, wit…

This paper solves the inversion problem for jump processes using Markovian projections.

Analyzes non-Markovian environments in stochastic approximation.

We propose a class of Markovian agent based models for the time evolution of a share price in an interactive market. The models rely on a microscopic description of a market of buyers and sellers who change their opinion about the stock value in a stochastic way. The actual price is determined in realistic way by match…

In this paper, we present a family of a control-stopping games which arise naturally in equilibrium-based models of market microstructure, as well as in other models with strategic buyers and sellers. A distinctive feature of this family of games is the fact that the agents do not have any exogenously given fundamental…

Study adds investment gains and losses to recursive utility model, proving existence and uniqueness of utility process.

Paper proposes MAMRL for efficient energy dispatch in self-powered edge computing systems.

Breaks down complex nonlinear dynamics into simpler components.

Stochastic differential equation approximation for linear TD(0) under Markovian noise

Paper establishes convergence rates and concentration bounds for stochastic approximation and reinforcement learning with Markovian noise.

Unified analytical tool for non-Markovian jump processes.

Study improves covariance estimation for SGD under Markovian data, matching best rates.

We develop a Markovian approximation for SVV models to compute hedging strategies.

A new HOM model improves forecasting of Indian base metal prices.

We prove existence and uniqueness of stochastic equilibria in a class of incomplete continuous-time financial environments where the market participants are exponential utility maximizers with heterogeneous risk-aversion coefficients and general Markovian random endowments. The incompleteness featured in our setting - …

Deep learning solves non-Markovian FBSDEs for utility maximization.

Investigates optimal consumption and investment strategies in non-Markovian markets with unbounded parameters.

We simplify a complex volatility model to make it easier to price options.

We consider an agent who is involved in a Markov decision process and receives a vector of outcomes every round. Her objective is to maximize a global concave reward function on the average vectorial outcome. The problem models applications such as multi-objective optimization, maximum entropy exploration, and constrai…

This paper first describes a class of uncertain stochastic control systems with Markovian switching, and derives an Itô-Liu formula for Markov-modulated processes. And we characterize an optimal control law, which satisfies the generalized Hamilton-Jacobi-Bellman (HJB) equation with Markovian switching. Then, by using …

Modeling high-frequency order book data with Hawkes-Markovian process.