A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New RL framework models continuous-time dynamics using neural ODEs.
Logarithmic regret for continuous-time reinforcement learning.
DQNs can approximate optimal Q-functions with high accuracy on compact sets.
Paper solves POMDPs in continuous time and discrete spaces.
Introduces DF framework for sampling decisions from target distributions.
Markov jump processes and continuous time Bayesian networks are important classes of continuous time dynamical systems. In this paper, we tackle the problem of inferring unobserved paths in these models by introducing a fast auxiliary variable Gibbs sampler. Our approach is based on the idea of uniformization, and sets…
Optimizes control of hybrid systems with multiple switching processes.
Study approximates financial market with discrete-time models.
New algorithm for continuous-time switching systems using variational inference.
Continuous time framework for discrete data denoising models.
New model for insurance states using Markov jump processes with non-countable state space.
Recently there have been exciting developments in Monte Carlo methods, with the development of new MCMC and sequential Monte Carlo (SMC) algorithms which are based on continuous-time, rather than discrete-time, Markov processes. This has led to some fundamentally new Monte Carlo algorithms which can be used to sample f…
New neural method for inferring Markov jump processes.
New algorithm learns value and advantage functions for continuous-time Markov processes without structural assumptions.
Optimal fund deployment strategy under uncertain deal arrivals.
Recently, Ross showed that it is possible to recover an objective measure from a risk-neutral measure. His model assumes that there is a finite-state Markov process X that drives the economy in discrete time. Many authors extended his model to a continuous-time setting with a Markov diffusion process X with state space…
We consider a Hidden Markov Model (HMM) where the integrated continuous-time Markov chain can be observed at discrete time points perturbed by a Brownian motion. The aim is to derive a filter for the underlying continuous-time Markov chain. The recursion formula for the discrete-time filter is easy to derive, however i…
Neural models learn continuous-time Markov chain transition rates from data.
In this paper we present an algorithm for pricing barrier options in one-dimensional Markov models. The approach rests on the construction of an approximating continuous-time Markov chain that closely follows the dynamics of the given Markov model. We illustrate the method by implementing it for a range of models, incl…
We solve a continuous-time game-theoretic problem for Kihlstrom-Mirman preferences.
New algorithm solves uncertain Markov decision processes using Wasserstein uncertainty.
Efficiently infers coupled hidden Markov models with noisy discrete observations.
New CTBNs with clocks allow for non-exponential survival times.
The paper analyzes RL in high-frequency market making with theoretical and practical implications.
We study the price-setting problem of market makers under risk neutrality and perfect competition in continuous time. Thereby we follow the classic Glosten-Milgrom model that defines bid and ask prices as expectations of a true value of the asset given the market makers' partial information that includes the customers …
New entropy flow method extends generalization bounds for all Markov algorithms.
Fluid approximations have seen great success in approximating the macro-scale behaviour of Markov systems with a large number of discrete states. However, these methods rely on the continuous-time Markov chain (CTMC) having a particular population structure which suggests a natural continuous state-space endowed with a…
Paper approximates rough stochastic local volatility models for efficient computation.
Cai, Song and Kou (2015) [Cai, N., Y. Song, S. Kou (2015) A general framework for pricing Asian options under Markov processes. Oper. Res. 63(3): 540-554] made a breakthrough by proposing a general framework for pricing both discretely and continuously monitored Asian options under one-dimensional Markov processes. In …
This study shows how DDPM can be represented by the OU process.
Paper presents an algorithm for optimal regret in communicating Markov decision processes.
Decision trees are flexible models that are well suited for many statistical regression problems. In a Bayesian framework for regression trees, Markov Chain Monte Carlo (MCMC) search algorithms are required to generate samples of tree models according to their posterior probabilities. The critical component of such an …
We consider the problem of estimating the transition rate matrix of a continuous-time Markov chain from a finite-duration realisation of this process. We approach this problem in an imprecise probabilistic framework, using a set of prior distributions on the unknown transition rate matrix. The resulting estimator is a …
Study optimality in safety-constrained Markov decision processes using asynchronous value iteration and modified Q-learning.
Continuous time Bayesian networks (CTBNs) describe structured stochastic processes with finitely many states that evolve over continuous time. A CTBN is a directed (possibly cyclic) dependency graph over a set of variables, each of which represents a finite state continuous time Markov process whose transition model is…
We consider models of the population or opinion dynamics which result in the non-linear stochastic differential equations (SDEs) exhibiting the spurious long-range memory. In this context, the correspondence between the description of the birth-death processes as the continuous-time Markov chains and the continuous SDE…
In most sampling algorithms, including Hamiltonian Monte Carlo, transition rates between states correspond to the probability of making a transition in a single time step, and are constrained to be less than or equal to 1. We derive a Hamiltonian Monte Carlo algorithm using a continuous time Markov jump process, and ar…
Advances in mobile computing technologies have made it possible to monitor and apply data-driven interventions across complex systems in real time. Markov decision processes (MDPs) are the primary model for sequential decision problems with a large or indefinite time horizon. Choosing a representation of the underlying…
Method infers MJPs from noisy observations without prior training.
We consider continuous time Markovian processes where populations of individual agents interact stochastically according to kinetic rules. Despite the increasing prominence of such models in fields ranging from biology to smart cities, Bayesian inference for such systems remains challenging, as these are continuous tim…
The Markov assumption (MA) is fundamental to the empirical validity of reinforcement learning. In this paper, we propose a novel Forward-Backward Learning procedure to test MA in sequential decision making. The proposed test does not assume any parametric form on the joint distribution of the observed data and plays an…
Develops an actor-critic algorithm for risk-sensitive Markov decision processes.
Study a continuous-time PA problem with private effort and consumption decisions.
Overview of risk-sensitive Markov decision processes with Optimized Certainty Equivalent.
In this paper we consider long-run risk sensitive average cost impulse control applied to a continuous-time Feller-Markov process. Using the probabilistic approach, we show how to get a solution to a suitable continuous-time Bellman equation and link it with the impulse control problem. The optimal strategy for the und…
New method infers hidden states in continuous-time phenomena better than traditional models.
Regime-switching models, in particular Hidden Markov Models (HMMs) where the switching is driven by an unobservable Markov chain, are widely-used in financial applications, due to their tractability and good econometric properties. In this work we consider HMMs in continuous time with both constant and switching volati…
Study of bandit problem with Poisson decision times and Lévy processes.