arXiv research

Paper introduces IO-NPF for efficient Bayesian experimental design.

Paper introduces PRMs to learn non-Markovian stochastic rewards for reinforcement learning.

A new method predicts non-Markovian closure terms for complex systems.

This work tackles large action spaces in RL by binarizing actions.

New approach tackles non-Markovian behavior in maternal health programs.

SRMC framework reduces Monte Carlo variance by history-based sampling in high-dimensional spaces.

ARL bridges non-Markovian decision processes with reinforcement learning, improving foresight and stability.

We present a novel method in the family of particle MCMC methods that we refer to as particle Gibbs with ancestor sampling (PG-AS). Similarly to the existing PG with backward simulation (PG-BS) procedure, we use backward sampling to (considerably) improve the mixing of the PG kernel. Instead of using separate forward a…

This paper proposes DeepSynth, a method for effective training of deep Reinforcement Learning (RL) agents when the reward is sparse and non-Markovian, but at the same time progress towards the reward requires achieving an unknown sequence of high-level objectives. Our method employs a novel algorithm for synthesis of c…

Particle Markov chain Monte Carlo (PMCMC) is a systematic way of combining the two main tools used for Monte Carlo statistical inference: sequential Monte Carlo (SMC) and Markov chain Monte Carlo (MCMC). We present a novel PMCMC algorithm that we refer to as particle Gibbs with ancestor sampling (PGAS). PGAS provides t…

Analyzes non-Markovian environments in stochastic approximation.

Paper tackles robust offline RL for non-Markovian processes, improving efficiency and applicability.

Non-Markovian point process shows power-law scaling, similar to nonlinear Markovian process.

The paper develops a deep signature approach for option pricing under non-Markovian stochastic volatility models.

Study optimal portfolios in a non-Markovian regime-switching model with random time horizon.

Investigates optimal consumption and investment strategies in non-Markovian markets with unbounded parameters.

Study develops numerical schemes for non-Markovian volatility models with memory.

Deep learning solves non-Markovian FBSDEs for utility maximization.

Unified analytical tool for non-Markovian jump processes.

Bayesian inference in state-space models is challenging due to high-dimensional state trajectories. A viable approach is particle Markov chain Monte Carlo, combining MCMC and sequential Monte Carlo to form "exact approximations" to otherwise intractable MCMC methods. The performance of the approximation is limited to t…

Paper tackles non-Markovian control problems with new learning methods.

Path signatures improve hedging of exotic derivatives in non-Markovian models.

We establish existence, uniqueness and regularity of solution results for a class of backward stochastic partial differential equations with singular terminal condition. The equation describes the value function of non-Markovian stochastic optimal control problem in which the terminal state of the controlled process is…

Two signature-based methods solve optimal stopping in non-Markovian frameworks.

New model controls memory in seq2seq tasks, revealing learning regimes.

The paper tackles robust control for insurance contracts under uncertain transition rates.

The paper develops methods to price options under rough volatility models using BSPDEs.

A new metric detects non-Markovian states in partially observable environments.

Describes state variables in sequential decision problems, linking them to Markovian and non-Markovian models.

HS-FNO models non-Markovian PDEs by learning history and future states.

Paper solves Merton's portfolio problem in a non-Markovian, non-semimartingale model.

In this paper, we present a discrete-type approximation scheme to solve continuous-time optimal stopping problems based on fully non-Markovian continuous processes adapted to the Brownian motion filtration. The approximations satisfy suitable variational inequalities which allow us to construct $ε$-optimal stopping tim…

Study analyzes non-Markovian effects in financial markets over multiple years.

Paper improves neural ODEs for forecasting non-Markovian processes.

FLDD improves discrete diffusion models by learning a non-Markovian noising process.

New method for pricing European options in rough LSV models.

A new model predicts price concavity and reversion after metaorder execution.

The paper values variable annuities using complex stochastic models and deep learning.

This paper studies a class of non$-$Markovian singular stochastic control problems, for which we provide a novel probabilistic representation. The solution of such control problem is proved to identify with the solution of a $Z-$constrained BSDE, with dynamics associated to a non singular underlying forward process. Du…

Paper introduces MVS to detect non-Markovian observations in reinforcement learning.

Extends pricing methods for index options under rough volatility.

Paper solves complex game theory problems with new equations.

Efficient RL in PRMs with improved regret bound.

Develops RL algorithm for non-Markovian, non-stationary reward streams.

Representation learning on networks offers a powerful alternative to the oft painstaking process of manual feature engineering, and as a result, has enjoyed considerable success in recent years. However, all the existing representation learning methods are based on the first-order network (FON), that is, the network th…

Empirical mode modeling improves state-space analysis of noisy data.

By appealing to renewal theory we determine the equations that the mean exit time of a continuous-time random walk with drift satisfies both when the present coincides with a jump instant or when it does not. Particular attention is paid to the corrections ensuing from the non-Markovian nature of the process. We show t…

We study stochastic differential equations (SDEs) whose drift and diffusion coefficients are path-dependent and controlled. We construct a value process on the canonical path space, considered simultaneously under a family of singular measures, rather than the usual family of processes indexed by the controls. This val…