arXiv research

Paper introduces PRMs to learn non-Markovian stochastic rewards for reinforcement learning.

Paper tackles robust offline RL for non-Markovian processes, improving efficiency and applicability.

This paper investigates multi-task reinforcement learning in non-Markovian decision making, showing benefits in sample efficiency.

Describes state variables in sequential decision problems, linking them to Markovian and non-Markovian models.

New method optimizes decision-making in uncertain environments.

Develops RL algorithm for non-Markovian, non-stationary reward streams.

ARL bridges non-Markovian decision processes with reinforcement learning, improving foresight and stability.

A new macroscopic market making model connects market making and optimal execution.

A new metric detects non-Markovian states in partially observable environments.

We solve non-Markovian optimal switching problems in discrete time on an infinite horizon, when the decision maker is risk aware and the filtration is general, and establish existence and uniqueness of solutions for the associated reflected backward stochastic difference equations. An example application to hydropower …

Efficient RL in PRMs with improved regret bound.

Analyzes non-Markovian environments in stochastic approximation.

FinFlowRL learns from experts to optimize financial control in changing markets.

Non-Markovian point process shows power-law scaling, similar to nonlinear Markovian process.

Study on kinetic Langevin diffusions and their couplings, showing subtle TV bounds and new non-Markovian couplings.

Deep learning solves non-Markovian FBSDEs for utility maximization.

FLDD improves discrete diffusion models by learning a non-Markovian noising process.

Expands Thiele equation for non-Markovian life insurance cash flows.

New method for pricing European options in rough LSV models.

Develops non-Markovian couplings for sub-Riemannian Brownian motions.

Unified analytical tool for non-Markovian jump processes.

Investigates optimal consumption and investment strategies in non-Markovian markets with unbounded parameters.

Two signature-based methods solve optimal stopping in non-Markovian frameworks.

The paper develops a deep signature approach for option pricing under non-Markovian stochastic volatility models.

Signature portfolios approximate optimal wealth in non-Markovian markets.

Novel signature approach for pricing and hedging path-dependent options with market frictions.

Market making is one of the most important aspects of algorithmic trading, and it has been studied quite extensively from a theoretical point of view. The practical implementation of so-called "optimal strategies" however suffers from the failure of most order book models to faithfully reproduce the behaviour of real m…

This work analyzes $Q$-learning with adaptive stepsizes for finite-time convergence.

Study optimal portfolios in a non-Markovian regime-switching model with random time horizon.

Study develops numerical schemes for non-Markovian volatility models with memory.

In this paper, we present a discrete-type approximation scheme to solve continuous-time optimal stopping problems based on fully non-Markovian continuous processes adapted to the Brownian motion filtration. The approximations satisfy suitable variational inequalities which allow us to construct $ε$-optimal stopping tim…

Partially observable environments present an important open challenge in the domain of sequential control learning with delayed rewards. Despite numerous attempts during the two last decades, the majority of reinforcement learning algorithms and associated approximate models, applied to this context, still assume Marko…

Study analyzes non-Markovian effects in financial markets over multiple years.

Paper improves neural ODEs for forecasting non-Markovian processes.

In many settings (e.g., robotics) demonstrations provide a natural way to specify tasks; however, most methods for learning from demonstrations either do not provide guarantees that the artifacts learned for the tasks, such as rewards or policies, can be safely composed and/or do not explicitly capture history dependen…

Paper tackles non-Markovian control problems with new learning methods.

We establish existence, uniqueness and regularity of solution results for a class of backward stochastic partial differential equations with singular terminal condition. The equation describes the value function of non-Markovian stochastic optimal control problem in which the terminal state of the controlled process is…

A new sampling method using log-concave Markov chains.

DeepSynth synthesizes automata to guide deep RL agents through sparse, non-Markovian rewards.

This paper studies a class of non$-$Markovian singular stochastic control problems, for which we provide a novel probabilistic representation. The solution of such control problem is proved to identify with the solution of a $Z-$constrained BSDE, with dynamics associated to a non singular underlying forward process. Du…

Paper introduces MVS to detect non-Markovian observations in reinforcement learning.

Path signatures improve hedging of exotic derivatives in non-Markovian models.

Extends pricing methods for index options under rough volatility.

The paper develops methods to price options under rough volatility models using BSPDEs.

The problem of high-dimensional path-dependent optimal stopping (OS) is important to multiple academic communities and applications. Modern OS tasks often have a large number of decision epochs, and complicated non-Markovian dynamics, making them especially challenging. Standard approaches, often relying on ADP, dualit…

A new method predicts non-Markovian closure terms for complex systems.

This review explores causal decision-making to improve decision quality.

HS-FNO models non-Markovian PDEs by learning history and future states.