On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

count % of corpus

48 results for “Markovian design”

Recursive Nested Filtering for Efficient Amortized Bayesian Experimental Designstat.ML

Paper introduces IO-NPF for efficient Bayesian experimental design.

problem Efficient Bayesian experimental design in non-exchangeable settings.

method Inside-Out Nested Particle Filter (IO-NPF) for non-Markovian state-space models.

result IO-NPF achieves

\mathcal{O}(T^2)

computational complexity, improving efficiency.

Bayesian experimental design non-exchangeable settings particle filter maximum likelihood estimation non-Markovian state-space model

2024-09-09abs ↗pdf ↗

Discrete-type approximations for non-Markovian optimal stopping problems: Part Imath.PR

In this paper, we present a discrete-type approximation scheme to solve continuous-time optimal stopping problems based on fully non-Markovian continuous processes adapted to the Brownian motion filtration. The approximations satisfy suitable variational inequalities which allow us to construct $ε$ -optimal stopping tim…

2017-07-17abs ↗pdf ↗

Learning to Steer Markovian Agents under Model Uncertaintycs.LG

Study designs incentives for adapting multi-agent systems without knowing their learning dynamics.

problem Designing incentives for an adapting population in multi-agent systems without prior knowledge of their learning dynamics.

method Introduces a model-based non-episodic Reinforcement Learning (RL) formulation for steering Markovian agents towards desired policies, focusing on history-dependent strategies to handle model uncertainty.

result Identifies conditions for the existence of steering strategies to guide agents to desired policies and provides empirical algorithms to approximately solve the objective.

Reinforcement Learning Markovian Agents Model Uncertainty Incentives Multi-Agent Systems

2024-07-14abs ↗pdf ↗

Covariate shift in nonparametric regression with Markovian designmath.ST

The paper analyzes covariate shift in nonparametric regression with Markovian data.

problem Covariate shift in regression problems with Markovian data.

method Extension of nonparametric convergence rates to Markovian dependence structures, using Hölder smoothness assumptions and similarity measures.

result Precise convergence rates for Nadaraya-Watson kernel estimators under specific Markovian conditions.

covariate shift nonparametric regression Markovian design Nadaraya-Watson kernel convergence rates

2023-07-17abs ↗pdf ↗

Robust Offline Reinforcement Learning for Non-Markovian Decision Processescs.LG

Paper tackles robust offline RL for non-Markovian processes, improving efficiency and applicability.

problem Learning robust policies for non-Markovian decision processes with limited offline data.

method Proposes a novel algorithm with dataset distillation and LCB design for robust values, derived new dual forms, and introduces concentrability coefficients.

result Proves polynomial sample efficiency for finding ε-optimal robust policies.

Offline Reinforcement Learning Non-Markovian Processes Robustness Low-Rank Structure Concentrability Coefficients

2024-11-12abs ↗pdf ↗

Central Limit Theorem for Two-Timescale Stochastic Approximation with Markovian Noise: Theory and Applicationsstat.ML

Paper analyzes CLT for TTSA with Markovian noise, broadening its applications.

problem Analyzing asymptotic behavior of TTSA under Markovian noise.

method Central Limit Theorem applied to TTSA with Markovian noise.

result Uncovered coupled dynamics of TTSA influenced by Markov chain.

Central Limit Theorem Two-Timescale Stochastic Approximation Markovian Noise Stochastic Approximation Reinforcement Learning

2024-01-17abs ↗pdf ↗

Unraveling the Interplay between Carryover Effects and Reward Autocorrelations in Switchback Experimentsstat.ML

This paper analyzes switchback experiments in A/B testing, revealing key factors affecting their effectiveness.

problem Understanding the effectiveness of different switchback designs in A/B testing.

method Comprehensive comparative analysis of various switchback designs in Markovian environments, covering state-of-the-art RL estimators.

result The effectiveness of switchback designs depends on the size of the carryover effect and reward autocorrelations.

A/B testing switchback experiments reinforcement learning carryover effects reward autocorrelations

2024-03-26abs ↗pdf ↗

Control randomisation approach for policy gradient and application to reinforcement learning in optimal switchingmath.OC

New framework for policy gradient methods in continuous time reinforcement learning.

problem Addressing policy gradient methods for continuous time reinforcement learning.

method Control randomisation technique to derive policy gradient representation for various Markovian control problems.

result Demonstrated application to optimal switching problems in the energy sector.

policy gradient reinforcement learning control randomisation Markovian control optimal switching

2024-04-27abs ↗pdf ↗

Adaptive KL-UCB based Bandit Algorithms for Markovian and i.i.d. Settingscs.LG

Adaptive KL-UCB algorithm for Markov and i.i.d. rewards.

problem Regret minimization for Markovian and i.i.d. rewards in MAB problems.

method Identifies Markovian vs. i.i.d. rewards, switches between KL-UCB variants.

result Logarithmic regret for both i.i.d. and Markovian settings.

Bandit Algorithms KL-UCB Markov Chains Regret Minimization Total Variation Distance

2020-09-14abs ↗pdf ↗

Multi-factor approximation of rough volatility modelsmath.PR

Rough volatility models are very appealing because of their remarkable fit of both historical and implied volatilities. However, due to the non-Markovian and non-semimartingale nature of the volatility process, there is no simple way to simulate efficiently such models, which makes risk management of derivatives an int…

2018-01-31abs ↗pdf ↗

Multiscale replay: A robust algorithm for stochastic variational inequalities with a Markovian buffermath.OC

MER algorithm speeds up VI solving with Markovian data.

problem Solving stochastic variational inequalities with Markovian data.

method MER algorithm using multi-scale sampling from a Markovian buffer.

result Achieves faster convergence without knowing Markov chain mixing time.

stochastic variational inequalities Markovian data experience replay temporal difference learning generalized linear models

2026-01-04abs ↗pdf ↗

HONEM: Learning Embedding for Higher Order Networkscs.LG

Representation learning on networks offers a powerful alternative to the oft painstaking process of manual feature engineering, and as a result, has enjoyed considerable success in recent years. However, all the existing representation learning methods are based on the first-order network (FON), that is, the network th…

2019-08-15abs ↗pdf ↗

Adaptive Learning via Off-Model Training and Importance Sampling for Fully Non-Markovian Optimal Stochastic Control. Complete versionstat.ML

Paper tackles non-Markovian control problems with new learning methods.

problem Non-Markovian stochastic control problems with unknown parameters.

method Off-model training and importance sampling for deep neural network approximation.

result Quantitative error bounds for adaptive learning under model uncertainty.

stochastic control non-Markovian systems deep learning importance sampling Monte Carlo methods

2026-04-14abs ↗pdf ↗

DAMNETS: A Deep Autoregressive Model for Generating Markovian Network Time Seriesstat.ML

DAMNETS generates complex network dynamics models.

problem Generating flexible and scalable models for network time series is challenging.

method Deep autoregressive model for Markovian network time series.

result DAMNETS outperforms other methods in sample quality.

network time series generative models deep learning Markovian

2022-03-28abs ↗pdf ↗

Resemblance of the power-law scaling behavior of a non-Markovian and nonlinear point processescond-mat.stat-mech

Non-Markovian point process shows power-law scaling, similar to nonlinear Markovian process.

problem Understanding the scaling behavior of non-Markovian point processes.

method Analyzed a confined fractional Brownian motion-driven point process and compared it to a nonlinear Markovian process.

result A nonlinear Markovian process can reproduce the power-law scaling behavior of a non-Markovian point process.

power-law scaling non-Markovian process fractional Brownian motion nonlinear Markovian process

2022-05-16abs ↗pdf ↗

Order-book modelling and market making strategiesq-fin.TR

Market making is one of the most important aspects of algorithmic trading, and it has been studied quite extensively from a theoretical point of view. The practical implementation of so-called "optimal strategies" however suffers from the failure of most order book models to faithfully reproduce the behaviour of real m…

2018-06-13abs ↗pdf ↗

Non-asymptotic Convergence Analysis of Two Time-scale (Natural) Actor-Critic Algorithmscs.LG

Paper analyzes convergence rates of two time-scale AC and NAC algorithms.

problem Finite-sample convergence rate analysis of two time-scale AC and NAC algorithms.

method Developed novel techniques for bias error and convergence rate analysis.

result Established non-asymptotic convergence rates for two time-scale AC and NAC.

reinforcement learning actor-critic two time-scale finite-sample convergence rate

2020-05-07abs ↗pdf ↗

Markovian projections for Itô semimartingales with jumpsmath.PR

Projects Markovian processes from Itô semimartingales with jumps.

problem Modeling Itô semimartingales with jumps using Markovian projections.

method Construct Markovian projections for Itô semimartingales with jumps using non-local FPKEs.

result Markovian projections match the marginal laws of the original process.

Markovian projections Itô semimartingales jumps Fokker-Planck-Kolmogorov equations calibrated models

2024-03-24abs ↗pdf ↗

Efficient Reinforcement Learning in Probabilistic Reward Machinesstat.ML

Efficient RL in PRMs with improved regret bound.

problem Reinforcement learning in probabilistic reward machines with non-Markovian rewards.

method Design of an algorithm with a new regret bound of

\widetilde{O}(\sqrt{HOAT} + H^2O^2A^{3/2} + H\sqrt{T})

result Improved regret bound over existing methods, matching lower bound up to a logarithmic factor.

Reinforcement Learning Probabilistic Reward Machines Non-Markovian Rewards Regret Bound Simulation Lemma

2024-08-19abs ↗pdf ↗

Markovian projections for functionals of Itô semimartingales with jumpsmath.PR

This paper extends Markovian projections to semimartingales with jumps.

problem Extending Markovian projections to semimartingales with jumps.

method Using Markovian projections to match marginal laws of Itô semimartingales with jumps.

result Existence of Markovian projections for Itô semimartingales with jumps.

Markovian projections Itô semimartingales jumps functionals differential characteristics

2025-06-01abs ↗pdf ↗

A nonparametric algorithm for optimal stopping based on robust optimizationmath.OC

New algorithm solves complex stopping problems with robust optimization.

problem Solving complex stochastic optimal stopping problems.

method Simulation-based robust optimization with exact reformulation as a zero-one bilinear program.

result Developed polynomial-time heuristics and algorithms for practical solution.

optimal stopping robust optimization stochastic dynamic programming zero-one bilinear program Markovian stopping rules

2021-03-04abs ↗pdf ↗

Inverting the Markovian projection for pure jump processesmath.PR

This paper solves the inversion problem for jump processes using Markovian projections.

problem Calibrating jump-diffusion models with both local and stochastic features.

method Inverting Markovian projections for pure jump processes.

result Constructs calibrated local stochastic intensity (LSI) models for credit risk applications.

Markovian projections jump processes calibration credit risk local stochastic intensity

2024-12-05abs ↗pdf ↗

Stochastic approximation in non-markovian environments revisitedstat.ML

Analyzes non-Markovian environments in stochastic approximation.

problem Understanding learning mechanisms in non-ergodic, non-Markovian settings.

method Analytic framework for transformer learning and continual learning.

result Proposes a new approach to transformer and continual learning.

stochastic approximation non-Markovian environments transformer learning continual learning

2026-03-22abs ↗pdf ↗

Inferring Probabilistic Reward Machines from Non-Markovian Reward Processes for Reinforcement Learningcs.LG

Paper introduces PRMs to learn non-Markovian stochastic rewards for reinforcement learning.

problem Lack of structured representation for non-Markovian stochastic rewards in reinforcement learning.

method Introduces probabilistic reward machines (PRMs) and presents an algorithm to learn them from decision processes.

result Algorithm proves correct and convergent for learning PRMs from decision processes.

reinforcement learning probabilistic reward machines non-Markovian rewards stochastic rewards reinforcement algorithms

2021-07-09abs ↗pdf ↗

Anticipatory Reinforcement Learning: From Generative Path-Laws to Distributional Value Functionscs.LG

ARL bridges non-Markovian decision processes with reinforcement learning, improving foresight and stability.

problem Inaccurate foresight in non-Markovian environments due to state-based methods' limitations.

method Lifted state space into a signature-augmented manifold, using a self-consistent field approach to anticipate future path-law.

result ARL achieves deterministic evaluation of expected returns with reduced computational complexity and variance.

Reinforcement Learning Non-Markovian Processes Path-Law Signature Manifold Self-Consistent Field

2026-04-06abs ↗pdf ↗

A Diffusion Approximation for Temporal-Difference Learning with Linear Features under Markovian Noisestat.ML

Stochastic differential equation approximation for linear TD(0) under Markovian noise

problem Temporal-difference learning with linear function approximation

method Stochastic differential equation approximation

result Explains the constant-stepsize error floor

TD learning linear function approximation Markovian noise stochastic differential equation

2026-06-16abs ↗pdf ↗

Almost Sure Convergence Rates and Concentration of Stochastic Approximation and Reinforcement Learning with Markovian Noisecs.LG

Paper establishes convergence rates and concentration bounds for stochastic approximation and reinforcement learning with Markovian noise.

problem Analyzing convergence rates and concentration bounds for stochastic approximation and reinforcement learning with Markovian noise.

method Novel discretization of the mean ODE of stochastic approximation algorithms using intervals with diminishing length.

result First almost sure convergence rate and maximal concentration bound with exponential tails for contractive stochastic approximation algorithms with Markovian noise.

stochastic approximation reinforcement learning Markovian noise convergence rates concentration bounds

2024-11-20abs ↗pdf ↗

A standard form of master equations for general non-Markovian jump processes: the Laplace-space embedding framework and asymptotic solutioncond-mat.stat-mech

Unified analytical tool for non-Markovian jump processes.

problem Analyzing history-dependent jump processes with non-Markovian behavior.

method Developed a standard form of master equations using Laplace-space embedding and asymptotic solution.

result Unified analytical toolset for general non-Markovian processes, leading to the GLE approximation.

master equations non-Markovian processes jump processes Laplace-space embedding asymptotic solution

2023-12-09abs ↗pdf ↗

Online covariance estimation for stochastic gradient descent under Markovian samplingmath.ST

Study improves covariance estimation for SGD under Markovian data, matching best rates.

problem Improving covariance estimation for SGD in Markovian data settings.

method Online overlapping batch-means covariance estimator for SGD under Markovian sampling.

result Established convergence rates for covariance estimation under Markovian sampling.

covariance estimation SGD Markovian sampling convergence rates online estimation

2023-08-03abs ↗pdf ↗

Sandwiched Volterra Volatility model: Markovian approximations and hedgingq-fin.MF

We develop a Markovian approximation for SVV models to compute hedging strategies.

problem Computing optimal hedging strategies for SVV models with non-Markovian noise.

method Develop a Markovian approximation of the Volterra noise kernel to compute hedging strategies.

result Error estimates for the approximation of volatility, prices, and optimal hedge.

Volterra Volatility Markovian Approximation Hedging Strategies SVV Models Non-Markovian Noise

2022-09-26abs ↗pdf ↗

Modeling the commodity prices of base metals in Indian commodity market using a Higher Order Markovian Approachq-fin.TR

A new HOM model improves forecasting of Indian base metal prices.

problem Improving accuracy in predicting base metal prices in the Indian market.

method A Higher Order Markovian (HOM) model with varying order based on market delay.

result The HOM model consistently outperforms the standard Markovian model in forecasting.

commodity prices base metals Indian market Higher Order Markovian forecasting

2020-10-07abs ↗pdf ↗

On State Variables, Bandit Problems and POMDPscs.LG

Describes state variables in sequential decision problems, linking them to Markovian and non-Markovian models.

problem Sequential decision problems, especially in active learning and POMDPs, where decisions affect what is observed and learned.

method Canonical framework and novel two-agent perspective of POMDPs, defining state variables to claim Markovian or non-Markovian models.

result Properly modeled sequential decision problems are Markovian, while real decision problems are often non-Markovian.

state variables sequential decision problems Markovian non-Markovian active learning

2020-02-14abs ↗pdf ↗

A Deep Learning-Based Method for Fully Coupled Non-Markovian FBSDEs with Applicationsq-fin.MF

Deep learning solves non-Markovian FBSDEs for utility maximization.

problem Solving utility maximization problems under rough volatility.

method Deep learning-based numerical methods for non-Markovian fully coupled FBSDEs.

result Error estimates and convergence provided for the deep learning approach.

deep learning FBSDEs non-Markovian utility maximization rough volatility

2025-11-11abs ↗pdf ↗

Consumption-investment optimization with Epstein-Zin utility in unbounded non-Markovian marketsq-fin.MF

Investigates optimal consumption and investment strategies in non-Markovian markets with unbounded parameters.

problem Optimal consumption and investment strategies in non-Markovian markets with unbounded parameters.

method Martingale optimal principle and quadratic BSDEs with exponential moment.

result Establishes optimal strategies for consumption and investment.

consumption-investment Epstein-Zin utility non-Markovian markets unbounded parameters martingale optimal principle

2024-07-29abs ↗pdf ↗

Explicit Mean-Square Error Bounds for Monte-Carlo and Linear Stochastic Approximationmath.PR

The paper provides mean-square error bounds for stochastic approximation algorithms.

problem Error bounds for recursive equations with Markovian disturbances.

method Analysis of mean-square error for stochastic approximation algorithms.

result Mean-square error achieves the optimal rate of

O(1/n)

under certain conditions.

stochastic approximation mean-square error Markovian disturbances Reinforcement Learning Monte Carlo

2020-02-07abs ↗pdf ↗

Markovian approximation of the rough Bergomi model for Monte Carlo option pricingq-fin.MF

We simplify a complex volatility model to make it easier to price options.

problem The rough Bergomi model's non-Markovian nature complicates option pricing.

method We approximate the rBergomi model with a Bergomi model that is Markovian.

result The rBergomi model can be effectively approximated by a Markovian model.

rough Bergomi model Monte Carlo option pricing Markovian approximation affine structure RFSV models

2020-07-04abs ↗pdf ↗

Optimal control of uncertain stochastic systems with Markovian switching and its applications to portfolio decisionsmath.OC

This paper first describes a class of uncertain stochastic control systems with Markovian switching, and derives an Itô-Liu formula for Markov-modulated processes. And we characterize an optimal control law, which satisfies the generalized Hamilton-Jacobi-Bellman (HJB) equation with Markovian switching. Then, by using …

2014-01-11abs ↗pdf ↗

Order Book Queue Hawkes-Markovian Modelingq-fin.TR

Modeling high-frequency order book data with Hawkes-Markovian process.

problem Capturing the dynamics of high-frequency order book events.

method Hawkes process with Markovian baseline intensities, LASSO regularization, and Akaike Information Criteria.

result Effective modeling of order book dynamics with reduced parameter redundancy.

Hawkes process Markovian model order book queue modeling high-frequency trading

2021-07-20abs ↗pdf ↗

Option pricing under non-Markovian stochastic volatility models: A deep signature approachq-fin.MF

The paper develops a deep signature approach for option pricing under non-Markovian stochastic volatility models.

problem Pricing options under non-Markovian stochastic volatility models is challenging due to the dependence on historical paths.

method Reformulate the asset dynamics as a rough stochastic differential equation and represent rough paths via signatures. Apply standard analytical tools to solve the transformed equation.

result The deep signature approach provides a theoretically grounded and computationally efficient framework for option pricing.

option pricing stochastic volatility non-Markovian models deep signatures rough paths

2025-08-21abs ↗pdf ↗

High-Probability Bounds for SGD under the Polyak-Lojasiewicz Condition with Markovian Noisecs.LG

Improved SGD bounds for machine learning models with Markovian noise.

problem Uniform high-probability bounds for SGD under PL condition with Markovian noise.

method Combining Poisson equation for Markovian noise and probabilistic induction for almost-sure bounds.

result Matching

1/k

decay rate for expected suboptimality.

SGD Polyak-Lojasiewicz condition Markovian noise Decentralized optimization Online system identification

2026-03-15abs ↗pdf ↗

Parameters Inference for Nonlinear Wave Equations with Markovian Switchingstat.ML

This paper addresses parameter estimation for wave equations with Markovian switching.

problem Parameter estimation for wave equations with abrupt changes.

method Bayesian statistical framework using discrete sparse Bayesian learning.

result Strong performance in parameter estimation for variable coefficient PDEs.

wave equations Markovian switching parameter inference Bayesian learning sparse learning

2024-08-12abs ↗pdf ↗

Statistical inference for Linear Stochastic Approximation with Markovian Noisestat.ML

Paper derives convergence rates and confidence intervals for LSA with Markovian noise.

problem Analyzing convergence rates and constructing confidence intervals for LSA with Markovian noise.

method Derives non-asymptotic Berry-Esseen bounds and multiplier block bootstrap procedure.

result Provides

\mathcal{O}(n^{-1/4})

convergence rates and guarantees consistent inference.

Linear Stochastic Approximation Markovian Noise Berry-Esseen Bounds Bootstrap Confidence Intervals Convergence Rates

2025-05-25abs ↗pdf ↗

Kernel Analog Forecasting: Multiscale Test Problemsmath.ST

Kernel analog forecasting studied for multiscale systems.

problem Interpreting data-driven predictions in multiscale dynamical systems.

method Kernel analog forecasting methods applied to multiscale systems with varying Markovian closures.

result Guidance provided for interpreting data-driven predictions in practice.

kernel analog forecasting multiscale systems Markovian closures data-driven prediction dynamical systems

2020-05-13abs ↗pdf ↗

Exponential Convergence Guarantees for Iterative Markovian Fittingstat.ML

Paper provides exponential convergence guarantees for Iterative Markovian Fitting.

problem Addressing the Schrödinger Bridge problem in computational optimal transport and generative modeling.

method Develops non-asymptotic exponential convergence guarantees for Iterative Markovian Fitting.

result First non-asymptotic exponential convergence guarantees for IMF under mild structural assumptions.

Schrödinger Bridge Iterative Markovian Fitting Convergence Guarantees Optimal Transport Generative Modeling

2025-10-23abs ↗pdf ↗

Optimal Best Markovian Arm Identification with Fixed Confidencemath.ST

We give a complete characterization of the sampling complexity of best Markovian arm identification in one-parameter Markovian bandit models. We derive instance specific nonasymptotic and asymptotic lower bounds which generalize those of the IID setting. We analyze the Track-and-Stop strategy, initially proposed for th…

2019-12-02abs ↗pdf ↗

Effectiveness of Constant Stepsize in Markovian LSA and Statistical Inferencestat.ML

Paper examines constant stepsize in LSA for Markovian data inference.

problem Improving statistical inference with constant stepsize in LSA for Markovian data.

method Established CLT, used averaged LSA iterates, applied Richardson-Romberg extrapolation.

result Constant stepsize leads to better CI coverage, especially with limited data.

constant stepsize linear stochastic approximation Markovian data statistical inference Richardson-Romberg extrapolation

2023-12-18abs ↗pdf ↗

First Order Methods with Markovian Noise: from Acceleration to Variational Inequalitiesmath.OC

Unified approach for first-order methods with Markovian noise in stochastic optimization and variational inequalities.

problem Stochastic optimization problems with Markovian noise.

method Unified theoretical analysis of first-order gradient methods using randomized batching and multilevel Monte Carlo.

result Optimal (linear) dependence on the mixing time of the noise sequence, eliminating previous limiting assumptions.

stochastic optimization first-order methods Markovian noise variational inequalities randomized batching

2023-05-25abs ↗pdf ↗

Least squares Monte Carlo methods in stochastic Volterra rough volatility modelsq-fin.PR

Developed scalable Monte Carlo method for VIX option pricing.

problem VIX option pricing in stochastic Volterra rough volatility models with non-Markovian vol-of-vol.

method Infinite dimensional Markovian representation to devise scalable least squares Monte Carlo.

result Efficient VIX option pricing method for generalized models.

Monte Carlo methods VIX pricing rough volatility Markovianity non-Markovian vol-of-vol

2021-05-10abs ↗pdf ↗