arXiv research

Analyzes non-Markovian environments in stochastic approximation.

Paper introduces MVS to detect non-Markovian observations in reinforcement learning.

Investigates optimal consumption and investment strategies in non-Markovian markets with unbounded parameters.

Paper tackles robust offline RL for non-Markovian processes, improving efficiency and applicability.

This paper studies a class of non$-$Markovian singular stochastic control problems, for which we provide a novel probabilistic representation. The solution of such control problem is proved to identify with the solution of a $Z-$constrained BSDE, with dynamics associated to a non singular underlying forward process. Du…

ARL bridges non-Markovian decision processes with reinforcement learning, improving foresight and stability.

Study shows how certain stochastic models reach a steady state over time.

Paper solves Merton's portfolio problem in a non-Markovian, non-semimartingale model.

This paper proposes DeepSynth, a method for effective training of deep Reinforcement Learning (RL) agents when the reward is sparse and non-Markovian, but at the same time progress towards the reward requires achieving an unknown sequence of high-level objectives. Our method employs a novel algorithm for synthesis of c…

Efficient RL in PRMs with improved regret bound.

FinFlowRL learns from experts to optimize financial control in changing markets.

Partially observable environments present an important open challenge in the domain of sequential control learning with delayed rewards. Despite numerous attempts during the two last decades, the majority of reinforcement learning algorithms and associated approximate models, applied to this context, still assume Marko…

This paper investigates Merton's portfolio problem in a rough stochastic environment described by Volterra Heston model. The model has a non-Markovian and non-semimartingale structure. By considering an auxiliary random process, we solve the portfolio optimization problem with the martingale optimality principle. Optim…

Non-Markovian point process shows power-law scaling, similar to nonlinear Markovian process.

A new metric detects non-Markovian states in partially observable environments.

Paper introduces PRMs to learn non-Markovian stochastic rewards for reinforcement learning.

Deep learning solves non-Markovian FBSDEs for utility maximization.

Rough stochastic volatility models have attracted a lot of attentions recently, in particular for the linear option pricing problem. In this paper, starting with power utilities, we propose to use a martingale distortion representation of the optimal value function for the nonlinear asset allocation problem in a (non-M…

Most real-world problems have huge state and/or action spaces. Therefore, a naive application of existing tabular solution methods is not tractable on such problems. Nonetheless, these solution methods are quite useful if an agent has access to a relatively small state-action space homomorphism of the true environment …

Unified analytical tool for non-Markovian jump processes.

We present a novel Monte Carlo based LSV calibration algorithm that applies to all stochastic volatility models, including the non-Markovian rough volatility family. Our framework overcomes the limitations of the particle method proposed by Guyon and Henry-Labordère (2012) and theoretically guarantees a variance reduct…

Two signature-based methods solve optimal stopping in non-Markovian frameworks.

The paper develops a deep signature approach for option pricing under non-Markovian stochastic volatility models.

Study optimal portfolios in a non-Markovian regime-switching model with random time horizon.

Study develops numerical schemes for non-Markovian volatility models with memory.

In this paper, we present a discrete-type approximation scheme to solve continuous-time optimal stopping problems based on fully non-Markovian continuous processes adapted to the Brownian motion filtration. The approximations satisfy suitable variational inequalities which allow us to construct $ε$-optimal stopping tim…

Study analyzes non-Markovian effects in financial markets over multiple years.

Paper improves neural ODEs for forecasting non-Markovian processes.

Paper tackles non-Markovian control problems with new learning methods.

We establish existence, uniqueness and regularity of solution results for a class of backward stochastic partial differential equations with singular terminal condition. The equation describes the value function of non-Markovian stochastic optimal control problem in which the terminal state of the controlled process is…

Extends pricing methods for index options under rough volatility.

Path signatures improve hedging of exotic derivatives in non-Markovian models.

The paper develops methods to price options under rough volatility models using BSPDEs.

Fractional stochastic volatility models have been widely used to capture the non-Markovian structure revealed from financial time series of realized volatility. On the other hand, empirical studies have identified scales in stock price volatility: both fast-time scale on the order of days and slow-scale on the order of…

A new method predicts non-Markovian closure terms for complex systems.

HS-FNO models non-Markovian PDEs by learning history and future states.

New model controls memory in seq2seq tasks, revealing learning regimes.

Develops RL algorithm for non-Markovian, non-stationary reward streams.

Describes state variables in sequential decision problems, linking them to Markovian and non-Markovian models.

Representation learning on networks offers a powerful alternative to the oft painstaking process of manual feature engineering, and as a result, has enjoyed considerable success in recent years. However, all the existing representation learning methods are based on the first-order network (FON), that is, the network th…

New method optimizes decision-making in uncertain environments.

By appealing to renewal theory we determine the equations that the mean exit time of a continuous-time random walk with drift satisfies both when the present coincides with a jump instant or when it does not. Particular attention is paid to the corrections ensuing from the non-Markovian nature of the process. We show t…

We study stochastic differential equations (SDEs) whose drift and diffusion coefficients are path-dependent and controlled. We construct a value process on the canonical path space, considered simultaneously under a family of singular measures, rather than the usual family of processes indexed by the controls. This val…

Study on kinetic Langevin diffusions and their couplings, showing subtle TV bounds and new non-Markovian couplings.

New approach tackles non-Markovian behavior in maternal health programs.

FLDD improves discrete diffusion models by learning a non-Markovian noising process.

In this paper, we present a Longstaff-Schwartz-type algorithm for optimal stopping time problems based on the Brownian motion filtration. The algorithm is based on Leão, Ohashi and Russo and, in contrast to previous works, our methodology applies to optimal stopping problems for fully non-Markovian and non-semimartinga…

Develops non-Markovian couplings for sub-Riemannian Brownian motions.