arXiv research

New neural processes use stacked Markov operators to improve flexibility.

We study discretizations of polynomial processes using finite state Markov processes satisfying suitable moment matching conditions. The states of these Markov processes together with their transition probabilities can be interpreted as Markov cubature rules. The polynomial property allows us to study such rules using …

We prove that the variance swap rate (fair strike) equals the price of a co-terminal European-style contract when the underlying is an exponential Markov process, time-changed by an arbitrary continuous stochastic clock, which has arbitrary correlation with the driving Markov process, provided that the payoff function …

This paper applies AMP theory to improve learning tasks.

New neural method for inferring Markov jump processes.

Study of Markov-modulated affine processes for richer models in finance.

The Viterbi process can be extended indefinitely in a pairwise Markov model.

We introduce Markov substitute processes, a new model at the crossroad of statistics and formal grammars, and prove its main property : Markov substitute processes with a given support form an exponential family.

We review some developments concerning Markov and Feller processes with jumps in geometric settings. These include stochastic differential equations in Markus canonical form, the Courrège theorem on Lie groups, and invariant Markov processes on manifolds under both transitive and more general Lie group actions.

In this paper we propose a semi-Markov modulated model of interest rates. We assume that the switching process is a semi-Markov process with finite state space E and the modulated process is a diffusive process. We derive recursive equations for the higher order moments of the discount factor and we describe a Monte Ca…

Model credit ratings using economic states with Markov chains.

New model for insurance states using Markov jump processes with non-countable state space.

New algorithm solves uncertain Markov decision processes using Wasserstein uncertainty.

Paper proposes a new method for training diffusion models using Markov operators.

Study approximates financial market with discrete-time models.

Researchers calculate Shannon entropy rates of hidden Markov processes efficiently.

We introduce LAMP: the Linear Additive Markov Process. Transitions in LAMP may be influenced by states visited in the distant history of the process, but unlike higher-order Markov processes, LAMP retains an efficient parametrization. LAMP also allows the specific dependence on history to be learned efficiently from da…

This manuscript contributes a general and practical framework for casting a Markov process model of a system at equilibrium as a structural causal model, and carrying out counterfactual inference. Markov processes mathematically describe the mechanisms in the system, and predict the system's equilibrium behavior upon i…

Model reduction of Markov processes is a basic problem in modeling state-transition systems. Motivated by the state aggregation approach rooted in control theory, we study the statistical state compression of a discrete-state Markov chain from empirical trajectories. Through the lens of spectral decomposition, we study…

The generic identification problem is to decide whether a stochastic process $(X_t)$ is a hidden Markov process and if yes to infer its parameters for all but a subset of parametrizations that form a lower-dimensional subvariety in parameter space. Partial answers so far available depend on extra assumptions on the pro…

Drawdown (resp. drawup) of a stochastic process, also referred as the reflected process at its supremum (resp. infimum), has wide applications in many areas including financial risk management, actuarial mathematics and statistics. In this paper, for general time-homogeneous Markov processes, we study the joint law of …

Model detects market anomalies using a Hawkes process with hidden Markov chain.

In this paper we present an algorithm for pricing barrier options in one-dimensional Markov models. The approach rests on the construction of an approximating continuous-time Markov chain that closely follows the dynamics of the given Markov model. We illustrate the method by implementing it for a range of models, incl…

Paper presents an algorithm for optimal regret in communicating Markov decision processes.

Enhances count process modelling with Markov-modulated non-homogeneous Poisson process.

In this paper we continue the study of the simulated stock market framework defined by the driving sentiment processes. We focus on the market environment driven by the buy/sell trading sentiment process of the Markov chain type. We apply the methodology of the Hidden Markov Models and the Recurrent Neural Networks to …

Algorithm learns mixtures of Markov chains and MDPs from short trajectories.

Modeling maximum drawdown records in capital markets using PDMP.

Detects anomalies in multiple processes using hidden Markov models.

Unified approach to denoising Markov models for efficient sampling.

We are interested in understanding stability (almost sure boundedness) of stochastic approximation algorithms (SAs) driven by a `controlled Markov' process. Analyzing this class of algorithms is important, since many reinforcement learning (RL) algorithms can be cast as SAs driven by a `controlled Markov' process. In t…

Cai, Song and Kou (2015) [Cai, N., Y. Song, S. Kou (2015) A general framework for pricing Asian options under Markov processes. Oper. Res. 63(3): 540-554] made a breakthrough by proposing a general framework for pricing both discretely and continuously monitored Asian options under one-dimensional Markov processes. In …

New self-exciting random evolutions (SEREs) for modeling traffic and transport processes.

Method calculates Parisian stopping times and option prices using Markov chains.

Develops an actor-critic algorithm for risk-sensitive Markov decision processes.

Risk measures applied to dynamic Markov processes with varying risk aversion.

New method detects changes in high-dimensional Markov processes without explicit likelihood evaluation.

A new estimator for state values in reinforcement learning reduces complexity and improves convergence.

Method reconstructs hidden Markov chains from insurance data.

In most sampling algorithms, including Hamiltonian Monte Carlo, transition rates between states correspond to the probability of making a transition in a single time step, and are constrained to be less than or equal to 1. We derive a Hamiltonian Monte Carlo algorithm using a continuous time Markov jump process, and ar…

The study uses Markov chains to forecast cryptocurrency market dynamics.

Overview of risk-sensitive Markov decision processes with Optimized Certainty Equivalent.

We consider Markov models of stochastic processes where the next-step conditional distribution is defined by a kernel density estimator (KDE), similar to Markov forecast densities and certain time-series bootstrap schemes. The KDE Markov models (KDE-MMs) we discuss are nonlinear, nonparametric, fully probabilistic repr…

Optimizes control of hybrid systems with multiple switching processes.

New algorithm identifies best policy in MDPs faster.

Study optimality in safety-constrained Markov decision processes using asynchronous value iteration and modified Q-learning.

Bayesian MS-VAR process improves option pricing models.

New entropy flow method extends generalization bounds for all Markov algorithms.