This paper first describes a class of uncertain stochastic control systems with Markovian switching, and derives an Itô-Liu formula for Markov-modulated processes. And we characterize an optimal control law, which satisfies the generalized Hamilton-Jacobi-Bellman (HJB) equation with Markovian switching. Then, by using …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Markovian controllers”
New framework for policy gradient methods in continuous time reinforcement learning.
problem Addressing policy gradient methods for continuous time reinforcement learning.
method Control randomisation technique to derive policy gradient representation for various Markovian control problems.
result Demonstrated application to optimal switching problems in the energy sector.
We establish existence, uniqueness and regularity of solution results for a class of backward stochastic partial differential equations with singular terminal condition. The equation describes the value function of non-Markovian stochastic optimal control problem in which the terminal state of the controlled process is…
Paper tackles non-Markovian control problems with new learning methods.
problem Non-Markovian stochastic control problems with unknown parameters.
method Off-model training and importance sampling for deep neural network approximation.
result Quantitative error bounds for adaptive learning under model uncertainty.
This paper studies a class of nonMarkovian singular stochastic control problems, for which we provide a novel probabilistic representation. The solution of such control problem is proved to identify with the solution of a constrained BSDE, with dynamics associated to a non singular underlying forward process. Du…
In this paper we study a class of time-inconsistent terminal Markovian control problems in discrete time subject to model uncertainty. We combine the concept of the sub-game perfect strategies with the adaptive robust stochastic to tackle the theoretical aspects of the considered stochastic control problem. Consequentl…
Paper analyzes Greedy-GQ for reinforcement learning with Markovian noise.
problem Analyzing Greedy-GQ for reinforcement learning with Markovian noise.
method Develops finite-sample analysis for Greedy-GQ with linear function approximation under Markovian noise.
result Provides theoretical justification for choosing stepsizes for faster convergence.
Two deep learning algorithms solve utility maximisation problems in finance.
problem Solving utility maximisation problems in finance with deep learning.
method Two algorithms: one for Markovian problems via HJB equation and 2BSDE, the other for non-Markovian problems via adjoint BSDE.
result Highly accurate results with low computational cost, solving problems with power, log, and non-HARA utilities in various models.
Study optimal portfolios in a non-Markovian regime-switching model with random time horizon.
problem Optimal portfolio selection in a market with non-Markovian regime-switching and random time horizon.
method Formulated as a constrained stochastic linear-quadratic optimal control problem, derived closed-form expressions for optimal portfolios and efficient frontier.
result Closed-form expressions for optimal portfolios and efficient frontier derived under non-Markovian regime-switching and random time horizon.
We study stochastic differential equations (SDEs) whose drift and diffusion coefficients are path-dependent and controlled. We construct a value process on the canonical path space, considered simultaneously under a family of singular measures, rather than the usual family of processes indexed by the controls. This val…
FinFlowRL: An Imitation-Reinforcement Learning Framework for Adaptive Stochastic Control in Financeq-fin.CP
FinFlowRL learns from experts to optimize financial control in changing markets.
problem Traditional finance control methods fail in real-world, non-stationary markets.
method Imitation-Reinforcement Learning framework that pretrains on expert strategies and finetunes in noise space.
result Consistently outperforms individually optimized experts across diverse market conditions.
Study on kinetic Langevin diffusions and their couplings, showing subtle TV bounds and new non-Markovian couplings.
problem Understanding and quantifying the TV distance between solutions of kinetic Langevin diffusions with different initial values.
method Established new non-Markovian couplings for kinetic Langevin diffusions, derived from optimal coalescence trajectories, and analyzed their TV bounds.
result No Markovian coupling can capture the asymptotic decay rate of the TV distance between solutions of kinetic Langevin diffusions with different initial values.
Partially observable environments present an important open challenge in the domain of sequential control learning with delayed rewards. Despite numerous attempts during the two last decades, the majority of reinforcement learning algorithms and associated approximate models, applied to this context, still assume Marko…
Paper solves complex game theory problems with new equations.
problem Zero-sum stochastic games with non-Markovian switching.
method New multidimensional SRE and BSDE solutions.
result Existence and uniqueness of SRE solutions.
New algorithm controls large groups of devices to match energy demand signals.
problem Controlling large populations of electrical devices to match energy demand signals.
method Developed MD-MFC algorithm for finite horizon Markovian mean field control problem.
result MD-MFC provides theoretical guarantees for convex and Lipschitz objective functions.
Breaks down complex nonlinear dynamics into simpler components.
problem Control of nonlinear dynamical systems remains challenging.
method Inspired by hybrid switching systems, decomposes dynamics into simpler stochastic switching linear dynamical systems.
result Extracts hierarchies of Markovian and auto-regressive locally linear controllers from nonlinear experts.
Macroscopic Market Makingq-fin.MF
A new macroscopic market making model connects market making and optimal execution.
problem Connecting market making and optimal execution problems.
method Using continuous processes for orders, the model bridges the gap between market making and optimal execution.
result Demonstrates the model's effectiveness through various noise and intensity function scenarios.
Markovian RNN adapts to nonstationary data using HMM for better time series prediction.
problem Nonstationary sequential data in real-life applications.
method Markovian RNN with HMM for regime switching and end-to-end optimization.
result Significant performance gains over vanilla RNN and Markov Switching ARIMA.
New model controls memory in seq2seq tasks, revealing learning regimes.
problem Understanding memory in seq2seq tasks using neural networks.
method Introducing a stochastic switching-Ornstein-Uhlenbeck (SSOU) model to control memory and a measure of non-Markovianity.
result Two learning regimes emerge from the interplay of time scales in the SSOU process.
Novel signature approach for pricing and hedging path-dependent options with market frictions.
problem Pricing and hedging path-dependent options with market frictions.
method Signature approach, mean-quadratic variation criterion, non-standard infinite-dimensional Riccati equations, time-augmented signature, non-Markovian stochastic control problem.
result Effective hedging strategies in frictional markets with low-truncated signature approximations.
We study an optimal control problem related to swing option pricing in a general non-Markovian setting in continuous time. As a main result we show that the value process solves a first-order non-linear backward stochastic partial differential equation. Based on this result we can characterize the set of optimal contro…
Paper analyzes CLT for TTSA with Markovian noise, broadening its applications.
problem Analyzing asymptotic behavior of TTSA under Markovian noise.
method Central Limit Theorem applied to TTSA with Markovian noise.
result Uncovered coupled dynamics of TTSA influenced by Markov chain.
In this paper we propose a new methodology for solving an uncertain stochastic Markovian control problem in discrete time. We call the proposed methodology the adaptive robust control. We demonstrate that the uncertain control problem under consideration can be solved in terms of associated adaptive robust Bellman equa…
New approach finds solutions to games with unbounded controls.
problem Existence of equilibrium in mean-field games with unbounded controls.
method Weak formulation and new existence/stability results for quadratic-growth generalized McKean-Vlasov BSDEs.
result Existence of equilibrium result for non-Markovian mean-field games with unbounded control space.
Study optimizes portfolio to minimize relative drawdown duration, penalizing unfavorable performance states.
problem Minimizing relative drawdown duration in portfolio optimization relative to a benchmark.
method Introduces a benchmark-relative drawdown-duration criterion penalizing unfavorable performance states. Uses a one-dimensional Markovian representation and Hamilton-Jacobi-Bellman equation.
result Derives explicit projection-based characterization of the optimal feedback control and identifies geometric settings for unique strong solutions.
We provide a probabilistic solution of a not necessarily Markovian control problem with a state constraint by means of a Backward Stochastic Differential Equation (BSDE). The novelty of our solution approach is that the BSDE possesses a singular terminal condition. We prove that a solution of the BSDE exists, thus part…
Paper proves convergence of Markovian iteration for FBSDEs with fully coupled drift and Z process.
problem Proving convergence of Markovian iteration for FBSDEs with fully coupled drift and Z process.
method Differentiation-based approach to handle Z process, uniformly controlling Lipschitz continuity of decoupling fields.
result Proves convergence of Markovian iteration method for FBSDEs with fully coupled drift and Z process.
New bounds for SA with arbitrary norm contractions and Markovian noise.
problem Finite-time analysis of two-time-scale stochastic approximation with arbitrary norm contractions and Markovian noise.
method Use of generalized Moreau envelope for arbitrary norm contractions and solutions of Poisson equation for Markovian noise.
result Mean square error decays at rates of and under different conditions.
Empirical studies indicate the presence of multi-scales in the volatility of underlying assets: a fast-scale on the order of days and a slow-scale on the order of months. In our previous works, we have studied the portfolio optimization problem in a Markovian setting under each single scale, the slow one in [Fouque and…
This paper establishes the existence of a unique nonnegative continuous viscosity solution to the HJB equation associated with a Markovian linear-quadratic control problems with singular terminal state constraint and possibly unbounded cost coefficients. The existence result is based on a novel comparison principle for…
Paper introduces MVS to detect non-Markovian observations in reinforcement learning.
problem Real-world sensors violate Markov property, leading to suboptimal reinforcement learning performance.
method Uses prediction-based Markov Violation Score (MVS) combining random forest and ridge regression.
result MVS detects non-Markovian structure in observation trajectories, quantifying its impact.
This study improves convergence of two-timescale SA under Markovian noise in reinforcement learning.
problem Stability and convergence of two-timescale stochastic approximations under Markovian noise.
method Introduced a new control strategy for the fast timescale parameter.
result Established almost sure convergence of TDC with eligibility traces under off-policy learning with linear function approximation.
Langevin algorithms enhance training of deep neural networks for stochastic control problems.
problem Training acceleration for deep neural networks in stochastic control problems.
method Application of Langevin algorithms to minimize the loss of deep neural networks in stochastic control problems.
result Langevin algorithms improve training on various stochastic control problems.
Paper solves Merton's portfolio problem in a non-Markovian, non-semimartingale model.
problem Merton's portfolio optimization in a fake stationary Volterra-Heston model.
method Stochastic factor solution to a Riccati BSDE, combined with martingale optimality principle.
result Derives semi-closed form optimal strategies and value function.
Consider the problem of a government that wants to reduce the debt-to-GDP (gross domestic product) ratio of a country. The government aims at choosing a debt reduction policy which minimises the total expected cost of having debt, plus the total expected cost of interventions on the debt ratio. We model this problem as…
We give a complete solution to the problem of minimizing the expected liquidity costs in presence of a general drift when the underlying market impact model has linear transient price impact with exponential resilience. It turns out that this problem is well-posed only if the drift is absolutely continuous. Optimal str…
Investigates mean-variance portfolio selection in non-Markovian markets.
problem Continuous-time Markowitz mean-variance portfolio selection in fake stationary affine Volterra models.
method Stochastic factor solution to a Riccati BSDE, deriving explicit solutions as multi-dimensional Riccati-Volterra equations.
result Analytical closed-form expressions for optimal portfolio policies and mean-variance efficient frontier.
Resemblance of the power-law scaling behavior of a non-Markovian and nonlinear point processescond-mat.stat-mech
Non-Markovian point process shows power-law scaling, similar to nonlinear Markovian process.
problem Understanding the scaling behavior of non-Markovian point processes.
method Analyzed a confined fractional Brownian motion-driven point process and compared it to a nonlinear Markovian process.
result A nonlinear Markovian process can reproduce the power-law scaling behavior of a non-Markovian point process.
We review the problem of defining and inferring a "state" for a control system based on complex, high-dimensional, highly uncertain measurement streams such as videos. Such a state, or representation, should contain all and only the information needed for control, and discount nuisance variability in the data. It shoul…
FinFlowRL: An Imitation-Reinforcement Learning Framework for Adaptive Stochastic Control in Financeq-fin.CP
FinFlowRL combines imitation and reinforcement learning for better financial control.
problem Traditional stochastic control methods fail in real-world finance due to changing market conditions.
method FinFlowRL uses imitation learning to pretrain an adaptive meta policy, then finetunes it with reinforcement learning.
result FinFlowRL consistently outperforms individual strategies across various market conditions.
Projects Markovian processes from Itô semimartingales with jumps.
problem Modeling Itô semimartingales with jumps using Markovian projections.
method Construct Markovian projections for Itô semimartingales with jumps using non-local FPKEs.
result Markovian projections match the marginal laws of the original process.
The paper uses stochastic control to analyze interest rate markets with roll-over risk.
problem Analyzing interest rate markets with roll-over risk without classical arbitrage assumptions.
method Stochastic optimal control problems with power-type objective functionals.
result Endogenously determined funding-liquidity spread.
This paper extends Markovian projections to semimartingales with jumps.
problem Extending Markovian projections to semimartingales with jumps.
method Using Markovian projections to match marginal laws of Itô semimartingales with jumps.
result Existence of Markovian projections for Itô semimartingales with jumps.
SRMC framework reduces Monte Carlo variance by history-based sampling in high-dimensional spaces.
problem Efficient sampling in high-dimensional discrete or continuous state spaces.
method Score-Repellent Monte Carlo (SRMC) framework that summarizes history through running average of score evaluations.
result Improves estimator variance and mode coverage with constant memory usage.
The paper analyzes stability of random matrix products with Markovian noise.
problem Analyzing stability of random matrix products with Markovian noise.
method Using a super-Lyapunov drift condition and controlled growth of matrix-valued functions, the paper provides an exponential stability result for the p-th moment of random matrix product.
result Finite-time p-th moment bounds for linear stochastic approximation and TD learning algorithms.
This paper solves the inversion problem for jump processes using Markovian projections.
problem Calibrating jump-diffusion models with both local and stochastic features.
method Inverting Markovian projections for pure jump processes.
result Constructs calibrated local stochastic intensity (LSI) models for credit risk applications.
Analyzes non-Markovian environments in stochastic approximation.
problem Understanding learning mechanisms in non-ergodic, non-Markovian settings.
method Analytic framework for transformer learning and continual learning.
result Proposes a new approach to transformer and continual learning.
Optimizes costs in uncertain Markov systems using risk filters.
problem Optimizing costs in systems with model uncertainty and unknown parameters.
method Risk filters and Bellman principle of optimality applied to Bayesian framework.
result Derives the Bellman principle for non-standard risk-averse control problems.