This paper extends Markovian projections to semimartingales with jumps.
problem Extending Markovian projections to semimartingales with jumps.
method Using Markovian projections to match marginal laws of Itô semimartingales with jumps.
result Existence of Markovian projections for Itô semimartingales with jumps.
Paper tackles robust offline RL for non-Markovian processes, improving efficiency and applicability.
problem Learning robust policies for non-Markovian decision processes with limited offline data.
method Proposes a novel algorithm with dataset distillation and LCB design for robust values, derived new dual forms, and introduces concentrability coefficients.
result Proves polynomial sample efficiency for finding ε-optimal robust policies.
Paper introduces PRMs to learn non-Markovian stochastic rewards for reinforcement learning.
problem Lack of structured representation for non-Markovian stochastic rewards in reinforcement learning.
method Introduces probabilistic reward machines (PRMs) and presents an algorithm to learn them from decision processes.
result Algorithm proves correct and convergent for learning PRMs from decision processes.
We give a complete characterization of the sampling complexity of best Markovian arm identification in one-parameter Markovian bandit models. We derive instance specific nonasymptotic and asymptotic lower bounds which generalize those of the IID setting. We analyze the Track-and-Stop strategy, initially proposed for th…
Unified analytical tool for non-Markovian jump processes.
problem Analyzing history-dependent jump processes with non-Markovian behavior.
method Developed a standard form of master equations using Laplace-space embedding and asymptotic solution.
result Unified analytical toolset for general non-Markovian processes, leading to the GLE approximation.
Deep learning solves non-Markovian FBSDEs for utility maximization.
problem Solving utility maximization problems under rough volatility.
method Deep learning-based numerical methods for non-Markovian fully coupled FBSDEs.
result Error estimates and convergence provided for the deep learning approach.
Investigates optimal consumption and investment strategies in non-Markovian markets with unbounded parameters.
problem Optimal consumption and investment strategies in non-Markovian markets with unbounded parameters.
method Martingale optimal principle and quadratic BSDEs with exponential moment.
result Establishes optimal strategies for consumption and investment.
This work analyzes nonexpansive stochastic approximations with Markovian noise, proving convergence in reinforcement learning.
problem Applying stochastic approximation to reinforcement learning settings with nonexpansive operators.
method Investigates nonexpansive stochastic approximations with Markovian noise, providing asymptotic and finite sample analysis.
result First-time proof of convergence for classical tabular average reward temporal difference learning.
Developed scalable Monte Carlo method for VIX option pricing.
problem VIX option pricing in stochastic Volterra rough volatility models with non-Markovian vol-of-vol.
method Infinite dimensional Markovian representation to devise scalable least squares Monte Carlo.
result Efficient VIX option pricing method for generalized models.
Kernel analog forecasting studied for multiscale systems.
problem Interpreting data-driven predictions in multiscale dynamical systems.
method Kernel analog forecasting methods applied to multiscale systems with varying Markovian closures.
result Guidance provided for interpreting data-driven predictions in practice.
Non-Markovian point process shows power-law scaling, similar to nonlinear Markovian process.
problem Understanding the scaling behavior of non-Markovian point processes.
method Analyzed a confined fractional Brownian motion-driven point process and compared it to a nonlinear Markovian process.
result A nonlinear Markovian process can reproduce the power-law scaling behavior of a non-Markovian point process.
New method for conformal prediction under Markovian data reduces coverage gap.
problem Reducing coverage gap in conformal prediction for Markovian data.
method Split Conformal Prediction method adapted to Markovian data, with K-split CP for improved performance.
result Coverage gap typically scales as √(t_mix * ln(n) / n) for general Markov chains, and can be reduced to t_mix / (n * ln(n)) with K-split CP.
Paper examines constant stepsize in LSA for Markovian data inference.
problem Improving statistical inference with constant stepsize in LSA for Markovian data.
method Established CLT, used averaged LSA iterates, applied Richardson-Romberg extrapolation.
result Constant stepsize leads to better CI coverage, especially with limited data.
Projects Markovian processes from Itô semimartingales with jumps.
problem Modeling Itô semimartingales with jumps using Markovian projections.
method Construct Markovian projections for Itô semimartingales with jumps using non-local FPKEs.
result Markovian projections match the marginal laws of the original process.
Markovian RNN adapts to nonstationary data using HMM for better time series prediction.
problem Nonstationary sequential data in real-life applications.
method Markovian RNN with HMM for regime switching and end-to-end optimization.
result Significant performance gains over vanilla RNN and Markov Switching ARIMA.
This paper solves the inversion problem for jump processes using Markovian projections.
problem Calibrating jump-diffusion models with both local and stochastic features.
method Inverting Markovian projections for pure jump processes.
result Constructs calibrated local stochastic intensity (LSI) models for credit risk applications.
Adaptive KL-UCB algorithm for Markov and i.i.d. rewards.
problem Regret minimization for Markovian and i.i.d. rewards in MAB problems.
method Identifies Markovian vs. i.i.d. rewards, switches between KL-UCB variants.
result Logarithmic regret for both i.i.d. and Markovian settings.
Analyzes non-Markovian environments in stochastic approximation.
problem Understanding learning mechanisms in non-ergodic, non-Markovian settings.
method Analytic framework for transformer learning and continual learning.
result Proposes a new approach to transformer and continual learning.
Path signatures improve hedging of exotic derivatives in non-Markovian models.
problem Hedging exotic derivatives under non-Markovian stochastic volatility models.
method Investigates path signatures in deep and shallow learning contexts, comparing neural networks and regression approaches.
result Path signatures outperform LSTM in most cases and yield more accurate results in hedging.
This paper studies the equilibrium pricing of asset shares in the presence of dynamic private information. The market consists of a risk-neutral informed agent who observes the firm value, noise traders, and competitive market makers who set share prices using the total order flow as a noisy signal of the insider's inf…
Study on bias and extrapolation in LSA with Markovian data, showing bias reduction with Richardson-Romberg extrapolation.
problem Bias in LSA with constant stepsizes and Markovian data.
method Viewing LSA as a Markov chain, proving convergence and bias expansion, and applying Richardson-Romberg extrapolation.
result Bias is proportional to the stepsize up to higher order terms, and Richardson-Romberg extrapolation reduces the bias.
Stochastic differential equation approximation for linear TD(0) under Markovian noise
problem Temporal-difference learning with linear function approximation
method Stochastic differential equation approximation
result Explains the constant-stepsize error floor
Paper establishes convergence rates and concentration bounds for stochastic approximation and reinforcement learning with Markovian noise.
problem Analyzing convergence rates and concentration bounds for stochastic approximation and reinforcement learning with Markovian noise.
method Novel discretization of the mean ODE of stochastic approximation algorithms using intervals with diminishing length.
result First almost sure convergence rate and maximal concentration bound with exponential tails for contractive stochastic approximation algorithms with Markovian noise.
Paper proves convergence of Markovian iteration for FBSDEs with fully coupled drift and Z process.
problem Proving convergence of Markovian iteration for FBSDEs with fully coupled drift and Z process.
method Differentiation-based approach to handle Z process, uniformly controlling Lipschitz continuity of decoupling fields.
result Proves convergence of Markovian iteration method for FBSDEs with fully coupled drift and Z process.
A new method scales Gaussian process variational autoencoders to handle high-dimensional time series.
problem Scalability issue in Gaussian process variational autoencoders (GPVAEs).
method Introducing Markovian GPs and using Kalman filtering and smoothing for linear time training.
result MGPVAE outperforms existing approaches in various tasks with high scalability.
Study improves covariance estimation for SGD under Markovian data, matching best rates.
problem Improving covariance estimation for SGD in Markovian data settings.
method Online overlapping batch-means covariance estimator for SGD under Markovian sampling.
result Established convergence rates for covariance estimation under Markovian sampling.
We develop a Markovian approximation for SVV models to compute hedging strategies.
problem Computing optimal hedging strategies for SVV models with non-Markovian noise.
method Develop a Markovian approximation of the Volterra noise kernel to compute hedging strategies.
result Error estimates for the approximation of volatility, prices, and optimal hedge.
A new HOM model improves forecasting of Indian base metal prices.
problem Improving accuracy in predicting base metal prices in the Indian market.
method A Higher Order Markovian (HOM) model with varying order based on market delay.
result The HOM model consistently outperforms the standard Markovian model in forecasting.
Study on regression with Markovian data, establishing limits and proposing an improved algorithm.
problem Least squares regression with dependent Markovian data.
method Sharp information theoretic lower bounds, analysis of SGD-DD and SGD, experience replay algorithm.
result Experience replay algorithm outperforms SGD-DD in Markovian data regression.
A new measure predicts deep learning model performance.
problem Predicting the generalization error of deep learning models.
method 2sED measure based on effective dimension, layerwise iterative approximation.
result 2sED correlates well with training error and generalization error.
New estimator reduces bias-variance tradeoff in Markovian interference experiments.
problem Estimating impact of interventions in systems with limited resources.
method Differences-In-Q (DQ) estimator for on-policy policy evaluation.
result DQ estimator has exponentially smaller variance than off-policy methods.
We simplify a complex volatility model to make it easier to price options.
problem The rough Bergomi model's non-Markovian nature complicates option pricing.
method We approximate the rBergomi model with a Bergomi model that is Markovian.
result The rBergomi model can be effectively approximated by a Markovian model.
Optimal sequential testing for Markovian data with lower and upper bounds.
problem Sequential hypothesis testing for Markovian data.
method Non-asymptotic lower bounds and optimal test design.
result Optimal test matches lower bound asymptotically.
This study improves convergence of two-timescale SA under Markovian noise in reinforcement learning.
problem Stability and convergence of two-timescale stochastic approximations under Markovian noise.
method Introduced a new control strategy for the fast timescale parameter.
result Established almost sure convergence of TDC with eligibility traces under off-policy learning with linear function approximation.
This paper first describes a class of uncertain stochastic control systems with Markovian switching, and derives an Itô-Liu formula for Markov-modulated processes. And we characterize an optimal control law, which satisfies the generalized Hamilton-Jacobi-Bellman (HJB) equation with Markovian switching. Then, by using …
Modeling high-frequency order book data with Hawkes-Markovian process.
problem Capturing the dynamics of high-frequency order book events.
method Hawkes process with Markovian baseline intensities, LASSO regularization, and Akaike Information Criteria.
result Effective modeling of order book dynamics with reduced parameter redundancy.
Paper introduces IO-NPF for efficient Bayesian experimental design.
problem Efficient Bayesian experimental design in non-exchangeable settings.
method Inside-Out Nested Particle Filter (IO-NPF) for non-Markovian state-space models.
result IO-NPF achieves O ( T 2 ) \mathcal{O}(T^2) O ( T 2 ) computational complexity, improving efficiency. In recent years the possibility of relaxing the so-called Faithfulness assumption in automated causal discovery has been investigated. The investigation showed (1) that the Faithfulness assumption can be weakened in various ways that in an important sense preserve its power, and (2) that weakening of Faithfulness may h…
The paper develops a deep signature approach for option pricing under non-Markovian stochastic volatility models.
problem Pricing options under non-Markovian stochastic volatility models is challenging due to the dependence on historical paths.
method Reformulate the asset dynamics as a rough stochastic differential equation and represent rough paths via signatures. Apply standard analytical tools to solve the transformed equation.
result The deep signature approach provides a theoretically grounded and computationally efficient framework for option pricing.
Improved SGD bounds for machine learning models with Markovian noise.
problem Uniform high-probability bounds for SGD under PL condition with Markovian noise.
method Combining Poisson equation for Markovian noise and probabilistic induction for almost-sure bounds.
result Matching 1 / k 1/k 1/ k decay rate for expected suboptimality. This paper addresses parameter estimation for wave equations with Markovian switching.
problem Parameter estimation for wave equations with abrupt changes.
method Bayesian statistical framework using discrete sparse Bayesian learning.
result Strong performance in parameter estimation for variable coefficient PDEs.
Paper derives convergence rates and confidence intervals for LSA with Markovian noise.
problem Analyzing convergence rates and constructing confidence intervals for LSA with Markovian noise.
method Derives non-asymptotic Berry-Esseen bounds and multiplier block bootstrap procedure.
result Provides O ( n − 1 / 4 ) \mathcal{O}(n^{-1/4}) O ( n − 1/4 ) convergence rates and guarantees consistent inference. We use Series' Markovian coding for words in Fuchsian groups and the Bowen-Series coding of limit sets to prove an ergodic theorem for Cesaro averages of spherical averages in a Fuchsian group.
Paper provides exponential convergence guarantees for Iterative Markovian Fitting.
problem Addressing the Schrödinger Bridge problem in computational optimal transport and generative modeling.
method Develops non-asymptotic exponential convergence guarantees for Iterative Markovian Fitting.
result First non-asymptotic exponential convergence guarantees for IMF under mild structural assumptions.
MER algorithm speeds up VI solving with Markovian data.
problem Solving stochastic variational inequalities with Markovian data.
method MER algorithm using multi-scale sampling from a Markovian buffer.
result Achieves faster convergence without knowing Markov chain mixing time.
New approach tackles non-Markovian behavior in maternal health programs.
problem Improving adherence and engagement in maternal and child healthcare programs.
method Extending RMABs to non-Markovian settings, using time-series forecasting and TARI policy.
result Significant increase in engagement and content listened compared to existing methods.
Optimizes state monitoring in Markovian systems with cost constraints.
problem Balancing state queries with prediction costs in Markovian systems.
method Greedy policy and SGD-based learning variant for optimal predict-query tradeoff.
result Greedy policy is suboptimal but performs close to optimal under certain conditions.
Paper analyzes convergence of decentralized algorithms with noise and bias.
problem Finite time convergence analysis of decentralized stochastic approximation schemes.
method Separated iterates into consensual parts and consensus error; bounded consensus error in terms of stationarity.
result Decentralized SA scheme converges at O ( log T / T ) {\cal O}(\log T/ \sqrt{T} ) O ( log T / T ) rate.