We simplify a complex volatility model to make it easier to price options.
problem The rough Bergomi model's non-Markovian nature complicates option pricing.
method We approximate the rBergomi model with a Bergomi model that is Markovian.
result The rBergomi model can be effectively approximated by a Markovian model.
Paper tackles robust offline RL for non-Markovian processes, improving efficiency and applicability.
problem Learning robust policies for non-Markovian decision processes with limited offline data.
method Proposes a novel algorithm with dataset distillation and LCB design for robust values, derived new dual forms, and introduces concentrability coefficients.
result Proves polynomial sample efficiency for finding ε-optimal robust policies.
Paper introduces PRMs to learn non-Markovian stochastic rewards for reinforcement learning.
problem Lack of structured representation for non-Markovian stochastic rewards in reinforcement learning.
method Introduces probabilistic reward machines (PRMs) and presents an algorithm to learn them from decision processes.
result Algorithm proves correct and convergent for learning PRMs from decision processes.
Paper provides exponential convergence guarantees for Iterative Markovian Fitting.
problem Addressing the Schrödinger Bridge problem in computational optimal transport and generative modeling.
method Develops non-asymptotic exponential convergence guarantees for Iterative Markovian Fitting.
result First non-asymptotic exponential convergence guarantees for IMF under mild structural assumptions.
This paper addresses parameter estimation for wave equations with Markovian switching.
problem Parameter estimation for wave equations with abrupt changes.
method Bayesian statistical framework using discrete sparse Bayesian learning.
result Strong performance in parameter estimation for variable coefficient PDEs.
Transformers struggle to learn Markovian dynamics, showing NP-hard optimization challenges.
problem Understanding transformers' limitations in learning Markovian dynamical functions.
method Investigated through a structured ICL setup, analyzing loss landscapes and parameter optimization.
result Recovering optimal transformer parameters for Markovian functions is NP-hard.
Non-Markovian point process shows power-law scaling, similar to nonlinear Markovian process.
problem Understanding the scaling behavior of non-Markovian point processes.
method Analyzed a confined fractional Brownian motion-driven point process and compared it to a nonlinear Markovian process.
result A nonlinear Markovian process can reproduce the power-law scaling behavior of a non-Markovian point process.
Projects Markovian processes from Itô semimartingales with jumps.
problem Modeling Itô semimartingales with jumps using Markovian projections.
method Construct Markovian projections for Itô semimartingales with jumps using non-local FPKEs.
result Markovian projections match the marginal laws of the original process.
DeepSynth synthesizes automata to guide deep RL agents through sparse, non-Markovian rewards.
problem Training deep RL agents with sparse, non-Markovian rewards and unknown high-level objectives.
method Employing a novel algorithm for synthesizing compact automata to uncover sequential structure from trace data.
result Reduces the number of iterations required for policy synthesis by two orders of magnitude and improves scalability.
Paper tackles non-Markovian control problems with new learning methods.
problem Non-Markovian stochastic control problems with unknown parameters.
method Off-model training and importance sampling for deep neural network approximation.
result Quantitative error bounds for adaptive learning under model uncertainty.
We analyze SA with Markovian data and nonlinear updates, overcoming prior limitations.
problem Analyzing stochastic approximation with Markovian data and nonlinear updates.
method Fine-grained analysis of SA iterates and Markovian data, leveraging smoothness and recurrence properties.
result Established weak convergence and precise asymptotic bias of SA iterates.
New algorithm solves complex stopping problems with robust optimization.
problem Solving complex stochastic optimal stopping problems.
method Simulation-based robust optimization with exact reformulation as a zero-one bilinear program.
result Developed polynomial-time heuristics and algorithms for practical solution.
This paper extends Markovian projections to semimartingales with jumps.
problem Extending Markovian projections to semimartingales with jumps.
method Using Markovian projections to match marginal laws of Itô semimartingales with jumps.
result Existence of Markovian projections for Itô semimartingales with jumps.
This paper solves the inversion problem for jump processes using Markovian projections.
problem Calibrating jump-diffusion models with both local and stochastic features.
method Inverting Markovian projections for pure jump processes.
result Constructs calibrated local stochastic intensity (LSI) models for credit risk applications.
Analyzes non-Markovian environments in stochastic approximation.
problem Understanding learning mechanisms in non-ergodic, non-Markovian settings.
method Analytic framework for transformer learning and continual learning.
result Proposes a new approach to transformer and continual learning.
In a Markovian model for a financial market, we characterize the best arbitrage with respect to the market portfolio that can be achieved using nonanticipative investment strategies, in terms of the smallest positive solution to a parabolic partial differential inequality; this is determined entirely on the basis of th…
Two deep learning algorithms solve utility maximisation problems in finance.
problem Solving utility maximisation problems in finance with deep learning.
method Two algorithms: one for Markovian problems via HJB equation and 2BSDE, the other for non-Markovian problems via adjoint BSDE.
result Highly accurate results with low computational cost, solving problems with power, log, and non-HARA utilities in various models.
Study on kinetic Langevin diffusions and their couplings, showing subtle TV bounds and new non-Markovian couplings.
problem Understanding and quantifying the TV distance between solutions of kinetic Langevin diffusions with different initial values.
method Established new non-Markovian couplings for kinetic Langevin diffusions, derived from optimal coalescence trajectories, and analyzed their TV bounds.
result No Markovian coupling can capture the asymptotic decay rate of the TV distance between solutions of kinetic Langevin diffusions with different initial values.
Paper introduces MVS to detect non-Markovian observations in reinforcement learning.
problem Real-world sensors violate Markov property, leading to suboptimal reinforcement learning performance.
method Uses prediction-based Markov Violation Score (MVS) combining random forest and ridge regression.
result MVS detects non-Markovian structure in observation trajectories, quantifying its impact.
A new model fits SPX and VIX volatility surfaces and term structures efficiently.
problem Calibrating SPX and VIX volatility models to market data.
method Gaussian polynomial volatility models, joint calibration, functional quantization, Neural Networks.
result A conventional one-factor Markovian model outperforms rough and non-rough models.
Stochastic differential equation approximation for linear TD(0) under Markovian noise
problem Temporal-difference learning with linear function approximation
method Stochastic differential equation approximation
result Explains the constant-stepsize error floor
Paper establishes convergence rates and concentration bounds for stochastic approximation and reinforcement learning with Markovian noise.
problem Analyzing convergence rates and concentration bounds for stochastic approximation and reinforcement learning with Markovian noise.
method Novel discretization of the mean ODE of stochastic approximation algorithms using intervals with diminishing length.
result First almost sure convergence rate and maximal concentration bound with exponential tails for contractive stochastic approximation algorithms with Markovian noise.
PMP extends GNNs to handle past states efficiently.
problem Efficient querying of data structures dependent on previous states.
method Persistent Message Passing (PMP) which persists past states through new nodes.
result Significantly outperforms GNNs in handling out-of-distribution data.
Study proves value of non-Markovian games with partial, asymmetric info.
problem Value of non-Markovian Dynkin games with partial and asymmetric information.
method Probabilistic and functional analytic approach based on Sion's min-max theorem.
result Existence of optimal strategies for both players in randomised stopping times.
Unified analytical tool for non-Markovian jump processes.
problem Analyzing history-dependent jump processes with non-Markovian behavior.
method Developed a standard form of master equations using Laplace-space embedding and asymptotic solution.
result Unified analytical toolset for general non-Markovian processes, leading to the GLE approximation.
Paper analyzes LSA algorithm bias and error bounds with RR extrapolation.
problem Analyzing bias and high-order error bounds of LSA with Markovian noise.
method Polyak-Ruppert averaging, linearization, Richardson-Romberg extrapolation.
result RR extrapolation effectively cancels the leading bias term.
Study improves covariance estimation for SGD under Markovian data, matching best rates.
problem Improving covariance estimation for SGD in Markovian data settings.
method Online overlapping batch-means covariance estimator for SGD under Markovian sampling.
result Established convergence rates for covariance estimation under Markovian sampling.
Financial econometrics has become an increasingly popular research field. In this paper we review a few parametric and nonparametric models and methods used in this area. After introducing several widely used continuous-time and discrete-time models, we study in detail dependence structures of discrete samples, includi…
New model for bandit problem with linear rewards and side information.
problem Hidden Markovian bandit problem with linear rewards and side information.
method Presented a model and algorithm with regret analysis for the problem.
result Logarithmic regret achieved even in high-dimensional problems with structural side information.
We develop a Markovian approximation for SVV models to compute hedging strategies.
problem Computing optimal hedging strategies for SVV models with non-Markovian noise.
method Develop a Markovian approximation of the Volterra noise kernel to compute hedging strategies.
result Error estimates for the approximation of volatility, prices, and optimal hedge.
The paper analyzes covariate shift in nonparametric regression with Markovian data.
problem Covariate shift in regression problems with Markovian data.
method Extension of nonparametric convergence rates to Markovian dependence structures, using Hölder smoothness assumptions and similarity measures.
result Precise convergence rates for Nadaraya-Watson kernel estimators under specific Markovian conditions.
This paper studies a portfolio optimization problem in a discrete-time Markovian model of a financial market, in which asset price dynamics depend on an external process of economic factors. There are transaction costs with a structure that covers, in particular, the case of fixed plus proportional costs. We prove that…
A new HOM model improves forecasting of Indian base metal prices.
problem Improving accuracy in predicting base metal prices in the Indian market.
method A Higher Order Markovian (HOM) model with varying order based on market delay.
result The HOM model consistently outperforms the standard Markovian model in forecasting.
Optimal sequential testing for Markovian data with lower and upper bounds.
problem Sequential hypothesis testing for Markovian data.
method Non-asymptotic lower bounds and optimal test design.
result Optimal test matches lower bound asymptotically.
Describes state variables in sequential decision problems, linking them to Markovian and non-Markovian models.
problem Sequential decision problems, especially in active learning and POMDPs, where decisions affect what is observed and learned.
method Canonical framework and novel two-agent perspective of POMDPs, defining state variables to claim Markovian or non-Markovian models.
result Properly modeled sequential decision problems are Markovian, while real decision problems are often non-Markovian.
Deep learning solves non-Markovian FBSDEs for utility maximization.
problem Solving utility maximization problems under rough volatility.
method Deep learning-based numerical methods for non-Markovian fully coupled FBSDEs.
result Error estimates and convergence provided for the deep learning approach.
Representation learning on networks offers a powerful alternative to the oft painstaking process of manual feature engineering, and as a result, has enjoyed considerable success in recent years. However, all the existing representation learning methods are based on the first-order network (FON), that is, the network th…
Investigates optimal consumption and investment strategies in non-Markovian markets with unbounded parameters.
problem Optimal consumption and investment strategies in non-Markovian markets with unbounded parameters.
method Martingale optimal principle and quadratic BSDEs with exponential moment.
result Establishes optimal strategies for consumption and investment.
New strategy identifies best Markovian arm with fixed confidence.
problem Identifying the best arm in Markovian bandit models with fixed confidence.
method Analyzed the Track-and-Stop strategy and derived a concentration inequality for Markov chains.
result The Track-and-Stop strategy is at most a factor of four apart from the lower bound for asymptotic performance.
Introduces MSW distances to improve SW metrics.
problem Redundant projections in SW distance.
method Imposes Markov structure on projecting directions.
result MSW distances improve SW metrics.
FDBM models use fractional Brownian motion to model complex stochastic processes.
problem Capturing memory effects and long-range dependencies in stochastic processes.
method Developed a generative diffusion bridge framework using a Markovian approximation of fractional Brownian motion.
result FDBM outperforms standard models in predicting future states and unpaired data translation.
Rough volatility models are very appealing because of their remarkable fit of both historical and implied volatilities. However, due to the non-Markovian and non-semimartingale nature of the volatility process, there is no simple way to simulate efficiently such models, which makes risk management of derivatives an int…
This paper first describes a class of uncertain stochastic control systems with Markovian switching, and derives an Itô-Liu formula for Markov-modulated processes. And we characterize an optimal control law, which satisfies the generalized Hamilton-Jacobi-Bellman (HJB) equation with Markovian switching. Then, by using …
Modeling high-frequency order book data with Hawkes-Markovian process.
problem Capturing the dynamics of high-frequency order book events.
method Hawkes process with Markovian baseline intensities, LASSO regularization, and Akaike Information Criteria.
result Effective modeling of order book dynamics with reduced parameter redundancy.
The paper develops a deep signature approach for option pricing under non-Markovian stochastic volatility models.
problem Pricing options under non-Markovian stochastic volatility models is challenging due to the dependence on historical paths.
method Reformulate the asset dynamics as a rough stochastic differential equation and represent rough paths via signatures. Apply standard analytical tools to solve the transformed equation.
result The deep signature approach provides a theoretically grounded and computationally efficient framework for option pricing.
DAMNETS generates complex network dynamics models.
problem Generating flexible and scalable models for network time series is challenging.
method Deep autoregressive model for Markovian network time series.
result DAMNETS outperforms other methods in sample quality.
Improved SGD bounds for machine learning models with Markovian noise.
problem Uniform high-probability bounds for SGD under PL condition with Markovian noise.
method Combining Poisson equation for Markovian noise and probabilistic induction for almost-sure bounds.
result Matching 1/k decay rate for expected suboptimality. Paper derives convergence rates and confidence intervals for LSA with Markovian noise.
problem Analyzing convergence rates and constructing confidence intervals for LSA with Markovian noise.
method Derives non-asymptotic Berry-Esseen bounds and multiplier block bootstrap procedure.
result Provides O(n−1/4) convergence rates and guarantees consistent inference.