Spectral methods reduce the complexity of Markov processes.
problem Modeling and simplifying state-transition systems.
method Spectral decomposition and state aggregation.
result Developed methods to estimate low-rank Markov models.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,181 papers · 148 categories
Trend · papers per month
48 results for “Markov state model”
Markov Observation Modelsstat.ML
Expands Hidden Markov Model to include Markov chain observations.
problem Handling Markov chain observations in Hidden Markov Models.
method Developed Expectation-Maximization algorithm and Viterbi algorithm analogs.
result Estimates transition probabilities for hidden states and observations.
Thiele's Differential Equation Based on Markov Jump Processes with Non-countable State Spaceq-fin.RM
New model for insurance states using Markov jump processes with non-countable state space.
problem Modeling insurance states with non-countable state spaces.
method Developed a new Thiele's differential equation for continuous time rehabilitation rates.
result Allows for consistent calculation of reserves in disability insurance.
Generalizes bits back coding for time-series models with latent Markov structures.
problem Efficiently compressing time-series data with latent Markov structures.
method Extends bits back coding to time-series models with latent Markov structures, including HMMs and LGSSMs.
result Effective for small scale models, promising for larger scale settings like video compression.
A new asset allocation model uses Markov states from clustered efficient frontier coefficients.
problem Characterizing market regimes using efficient frontiers for better asset allocation.
method Hierarchical clustering of monthly efficient frontier coefficients to define states, then a Markov process on these states for portfolio optimization.
result The model significantly outperforms benchmark portfolios empirically.
A new MLSS model approximates high-order Markov chains efficiently.
problem Efficiently approximating high-order Markov chains.
method Decaying mixture over past states, simple sampling algorithm.
result Approximates high-order Markov chains with fixed time and memory costs.
The method approximates stationary distributions of Markov models by truncating irrelevant states.
problem Computing the stationary distribution of complex Markov models is computationally challenging.
method A state-space lumping scheme that aggregates states in a grid structure, iteratively refining the state-space.
result The method provides a well-justified finite-state projection tailored to the stationary behavior of Markov models.
Model credit ratings using economic states with Markov chains.
problem Credit rating migration influenced by economic state changes.
method Developed a Markov chain model for credit ratings conditional on economic states.
result Derived asymptotic behavior of the rating process using Markov theory.
This paper models time-series data with a mixture of Markov chains, automatically determining the number of components.
problem Tackles the inability of common Markov state modeling frameworks to discern heterogeneities in complex data.
method Uses a mixture of Markov chains and variational expectation-maximization algorithm for automatic component selection.
result Achieves performance consistent with theoretically optimal error scaling, identifying meaningful heterogeneities in various data sets.
Paper proposes a new Markov model for efficient PLC system design.
problem Efficient estimation of Markov model parameters for bursty error channels.
method Introduced a Block Diagonal Markov model and a modified Baum-Welch algorithm.
result Efficient estimation of state transition matrix for PLC system design.
Kernel density estimators enhance Markov models with hidden states for complex data.
problem Modeling complex, non-Markovian processes with short-term dependencies.
method Kernel Density Estimation (KDE) for conditional distributions, hidden states for long-term dependencies.
result KDE-HMMs outperform traditional models on held-out data.
Study of multi-armed bandits with state-switching rewards using Markov models.
problem Multi-armed bandit problem with state-switching rewards.
method Spectral method-of-moments estimations for hidden Markov models, belief error control, upper-confidence-bound methods.
result Upper bound of for the learning algorithm performance.
We study discretizations of polynomial processes using finite state Markov processes satisfying suitable moment matching conditions. The states of these Markov processes together with their transition probabilities can be interpreted as Markov cubature rules. The polynomial property allows us to study such rules using …
A new GARCH model uses a two-dimensional Markov chain to capture long memory in volatility.
problem Capturing long-term volatility persistence in financial data.
method A GARCH-type model with state-dependent decay of past shocks using a two-dimensional Markov chain.
result The model successfully captures substantial volatility persistence and outperforms forecasts using only a two-dimensional state.
The paper introduces a method to learn Markov state abstractions for reinforcement learning.
problem Learning Markov state representations in complex environments.
method The paper introduces a novel set of conditions and a training procedure combining inverse model estimation and temporal contrastive learning.
result The approach learns representations that capture the underlying structure of the domain and improve sample efficiency.
A low-rank tensor model simplifies multi-dimensional Markov chains.
problem Simplifying the dynamics of multi-dimensional Markov chains.
method Low-rank tensor decomposition for multi-dimensional state spaces.
result Our tensor model requires fewer parameters and samples than conventional methods.
The study uses Bayesian Hidden Markov Models to predict cryptocurrency returns.
problem Predicting the volatility and trends of cryptocurrencies.
method Bayesian Hidden Markov Models with four states to capture different return characteristics.
result The NHHM model with four states outperforms other models in predicting cryptocurrency returns.
The paper estimates key metrics for linear models with Markov or hidden Markov sources.
problem Estimating free energy, mutual information, and MMSE for linear models with specific signal priors.
method Replica analysis in statistical physics, focusing on Markov and hidden Markov sources.
result The linear model with Markov or hidden Markov sources can be simplified into decoupled AWGN channels.
Detects anomalies in multiple processes using hidden Markov models.
problem Detecting an anomalous process among many with hidden states.
method Sequential search strategy using ADHM algorithm.
result ADHM algorithm effectively leverages temporal correlations.
Improved MCMC for rare events in hidden Markov models.
problem Slow inference and prediction for rare latent states in hidden Markov models.
method Targeted sub-sampling (TASS) over-samples rare latent states, reducing variance in gradient estimation.
result Substantial gains in predictive and inferential accuracy on real and synthetic examples.
Hidden Markov Model Estimation-Based Q-learning for Partially Observable Markov Decision Processcs.LG
Paper proposes an HMM-based Q-learning for POMDPs.
problem Q-learning struggles with POMDPs due to incomplete state observation.
method Formulates POMDP estimation as HMM estimation, proposing a recursive algorithm to concurrently estimate POMDP parameters and Q function.
result Algorithm converges to optimal Q function and POMDP parameters.
Novel non-parametric tree model learns tree distributions.
problem Learning distributions for tree-structured data.
method Bottom-up hidden tree Markov model with infinite states.
result Novel non-parametric generalization of hidden tree Markov model.
A new estimator for state values in reinforcement learning reduces complexity and improves convergence.
problem Estimating state values in reinforcement learning with Markov reward processes.
method Loop estimator exploiting regenerative structure of Markov reward processes.
result Instance-dependent convergence rate of for estimating state values.
New method infers hidden states in continuous-time phenomena better than traditional models.
problem Traditional HSMM's are limited to discrete time grids and cannot handle irregularly spaced data.
method Formulated integro-differential forward and backward equations for CTSMC's, introduced scalable Viterbi-type algorithm.
result Efficiently solved equations for posterior marginals and path estimates.
VAMPnets uses deep learning to model molecular kinetics from simulations.
problem Computing relevant molecular kinetics from simulations requires expert modeling.
method VAMPnets employs variational approach for Markov processes within neural networks.
result VAMPnets produces accurate kinetic models without requiring manual steps.
The study discovers digital biomarkers for Parkinson's Disease using optimized transitions and emissions in HSMM.
problem Identifying digital biomarkers for Parkinson's Disease.
method Proposed a Hidden Semi-Markov Model (HSMM) to model Parkinson's Disease patients' step and stride periodic cycles.
result The HSMM allows for more informative characterization of Parkinson's Disease patients/controls by considering the duration spent in each state.
Deep neural network learns discrete state abstractions for efficient planning.
problem Efficient sequential decision making in large state spaces.
method Information bottleneck method for learning approximate bisimulations using deep neural encoders and action-conditioned HMM.
result Trained method efficiently plans for unseen goals in multi-goal reinforcement learning.
This paper reviews recent advances in Bayesian nonparametric techniques for constructing and performing inference in infinite hidden Markov models. We focus on variants of Bayesian nonparametric hidden Markov models that enhance a posteriori state-persistence in particular. This paper also introduces a new Bayesian non…
The paper introduces FMCI and hybrid decoding for hidden Markov models.
problem Computing distributions and decoding hidden state sequences in HMMs.
method Finite Markov chain imbedding (FMCI) and hybrid decoding.
result Hybrid decoding improves performance over traditional methods.
Indexed Markov Chains for financial data: testing for the number of states of the index processq-fin.ST
New method uses Indexed Markov Chains for financial data analysis.
problem Testing the number of states in financial index processes.
method Change-point approach for Markov chains to determine state space.
result Explicit formula for first change of state of index process.
Users form information trails as they browse the web, checkin with a geolocation, rate items, or consume media. A common problem is to predict what a user might do next for the purposes of guidance, recommendation, or prefetching. First-order and higher-order Markov chains have been widely used methods to study such se…
Jump Markov linear models consists of a finite number of linear state space models and a discrete variable encoding the jumps (or switches) between the different linear models. Identifying jump Markov linear models makes for a challenging problem lacking an analytical solution. We derive a new expectation maximization …
DeepGenMSM models complex dynamical systems for accurate trajectory prediction.
problem Inference and prediction of metastable dynamical systems.
method Deep learning framework with probabilistic encoder, Markov chain, and generative part.
result Accurate long-time kinetics estimation and generation of realistic structures.
Infinite Hidden Markov Models (iHMM's) are an attractive, nonparametric generalization of the classical Hidden Markov Model which can automatically infer the number of hidden states in the system. However, due to the infinite-dimensional nature of transition dynamics performing inference in the iHMM is difficult. In th…
Paper develops a method for compact Markov modeling of time series data.
problem Compact representation of time-series data with reduced memory.
method Symbolic dynamics for partitioning, hierarchical clustering for state representation, Bayesian inference for parameter identification.
result Reduced-order Markov models capture system dynamics with minimal memory.
Modified asymmetric hidden Markov models for time series with autoregressive components.
problem Dynamic relationships between variables in time series data.
method Introducing an asymmetric autoregressive component to recent asymmetric hidden Markov models.
result The model can choose the optimal autoregressive order for better likelihood.
The Viterbi process can be extended indefinitely in a pairwise Markov model.
problem Estimating hidden chains in pairwise Markov models.
method Construction of barriers to ensure Viterbi path goes through states.
result The Viterbi process is regenerative in the PMM.
Efficiently learns sparse low-dimensional Markov chain representations.
problem Learning low-dimensional representations for large-scale Markov chains with sparse structures.
method Formulates as constrained nonnegative matrix factorization and uses gradient descent.
result Proves the effectiveness of the proposed method through convergence analysis.
New algorithms for RL in Markov games with independent linear function approximation, breaking the curse of multiagents.
problem Tackles the challenge of learning Markov equilibria in large state space Markov games with multiple agents.
method Proposes independent linear Markov games and designs new algorithms for learning Markov coarse correlated equilibria and Markov correlated equilibria with polynomial sample complexity.
result Breaks the curse of multiagents by achieving sample complexity bounds that scale polynomially with each agent's function class complexity.
The partially observable hidden Markov model is an extension of the hidden Markov Model in which the hidden state is conditioned on an independent Markov chain. This structure is motivated by the presence of discrete metadata, such as an event type, that may partially reveal the hidden state but itself emanates from a …
Paper proposes an algorithm to estimate state aggregation from Markov transition data.
problem Estimating probabilistic aggregation map from system's trajectory.
method Two-step algorithm: spectral decomposition and linear transformation of singular vectors.
result Sharp error bounds for estimating aggregation and disaggregation distributions.
We introduce LAMP: the Linear Additive Markov Process. Transitions in LAMP may be influenced by states visited in the distant history of the process, but unlike higher-order Markov processes, LAMP retains an efficient parametrization. LAMP also allows the specific dependence on history to be learned efficiently from da…
We describe a generalization of the Hierarchical Dirichlet Process Hidden Markov Model (HDP-HMM) which is able to encode prior information that state transitions are more likely between "nearby" states. This is accomplished by defining a similarity function on the state space and scaling transition probabilities by pai…
Markov jump processes and continuous time Bayesian networks are important classes of continuous time dynamical systems. In this paper, we tackle the problem of inferring unobserved paths in these models by introducing a fast auxiliary variable Gibbs sampler. Our approach is based on the idea of uniformization, and sets…
Study evaluates initialization strategies for infinite hidden Markov models.
problem Limited attention to initialization in infinite hidden Markov models.
method Systematically evaluated distance-based clustering, model-based, and uniform initializations.
result Distance-based clustering initializations consistently outperform other methods.
We introduce imprecise Markov semigroups to handle uncertainty in Markov processes.
problem Uncertainty in transition probabilities and invariant measures of Markov processes.
method Topology, geometry, and probability techniques to analyze ergodic limits under model uncertainty.
result Uniform long-term bounds collapse asymptotically in certain regimes.
Neural Markov models improve time series analysis by balancing deep learning and classical models.
problem Modeling non-stationary time series with high data sparsity.
method Hybrid approach using neural networks to parameterize stochastic matrices, estimating time-inhomogeneous Markov chains.
result Reduction of Chapman-Kolmogorov discrepancy and superior likelihood in financial markets.
In this letter we borrow from the inference techniques developed for unbounded state-cardinality (nonparametric) variants of the HMM and use them to develop a tuning-parameter free, black-box inference procedure for Explicit-state-duration hidden Markov models (EDHMM). EDHMMs are HMMs that have latent states consisting…