This paper models time-series data with a mixture of Markov chains, automatically determining the number of components.
problem Tackles the inability of common Markov state modeling frameworks to discern heterogeneities in complex data.
method Uses a mixture of Markov chains and variational expectation-maximization algorithm for automatic component selection.
result Achieves performance consistent with theoretically optimal error scaling, identifying meaningful heterogeneities in various data sets.
This paper compares HMM and LSTM for time series forecasting.
problem Comparing HMM and LSTM for time series forecasting.
method Discretization techniques and various training methods.
result HMM can outperform LSTM with limited labeled data.
Study improves generalization bounds for equivariant networks on Markov data.
problem Challenges in integrating equivariance with Markov dependencies in neural networks.
method Applied McDiarmid's inequality and computed covering number using group theory.
result Derived upper bound on Rademacher complexity for equivariant neural networks on Markov datasets.
New neural method for inferring Markov jump processes.
problem Inference in Markov jump processes is challenging.
method Variational inference using neural ODEs and backpropagation.
result Trains neural representations of data to approximate process rates.
Hidden Markov Neural Networks balance adaptation and forgetting in time-series data.
problem Balancing adaptation to new data and forgetting outdated information in time-series forecasting.
method Modeling weights as hidden states of a Hidden Markov model, using a filtering algorithm for learning a variational approximation of the posterior distribution over weights, and employing sequential Bayes by Backprop with variational DropConnect for regularization.
result Achieves strong predictive performance and effective uncertainty quantification on various tasks.
In this paper we describe three stochastic models based on a semi-Markov chains approach and its generalizations to study the high frequency price dynamics of traded stocks. The three models are: a simple semi-Markov chain model, an indexed semi-Markov chain model and a weighted indexed semi-Markov chain model. We show…
Method reconstructs hidden Markov chains from insurance data.
problem Recovering hidden Markov chains from incomplete insurance data.
method Neural architecture to explicitly provide transition probabilities.
result Neural model successfully validates decompression of insurance information.
Paper analyzes symbolic-dynamics inspired Markov modeling for time-series data.
problem Capturing temporal patterns in sequential data for statistical learning.
method Two-step process: discretization of continuous attributes and estimation of temporal memory.
result Effective Markov modeling depends on accurate discretization and memory estimation.
Stochastic kernel based dimensionality reduction approaches have become popular in the last decade. The central component of many of these methods is a symmetric kernel that quantifies the vicinity between pairs of data points and a kernel-induced Markov chain on the data. Typically, the Markov chain is fully specified…
Bayesian method detects Markov order in network paths more reliably.
problem Detecting Markov order in constrained categorical sequences.
method Multi-order Bayesian modelling framework.
result Bayesian method detects correct Markov order more reliably than competing methods.
The Markov assumption (MA) is fundamental to the empirical validity of reinforcement learning. In this paper, we propose a novel Forward-Backward Learning procedure to test MA in sequential decision making. The proposed test does not assume any parametric form on the joint distribution of the observed data and plays an…
This paper presents a technique for reduced-order Markov modeling for compact representation of time-series data. In this work, symbolic dynamics-based tools have been used to infer an approximate generative Markov model. The time-series data are first symbolized by partitioning the continuous measurement space of the …
We study the high frequency price dynamics of traded stocks by a model of returns using a semi-Markov approach. More precisely we assume that the intraday return are described by a discrete time homogeneous semi-Markov process and the overnight returns are modeled by a Markov chain. Based on this assumptions we derived…
The article proposes a deep learning method to test and infer the Markov property in time series data.
problem Testing and inferring the Markov property in high-dimensional time series data.
method Deep conditional generative learning to estimate conditional density functions and derive a doubly robust test statistic.
result The test controls the type-I error asymptotically and has power approaching one.
Users form information trails as they browse the web, checkin with a geolocation, rate items, or consume media. A common problem is to predict what a user might do next for the purposes of guidance, recommendation, or prefetching. First-order and higher-order Markov chains have been widely used methods to study such se…
A new HMM model captures kernel dependencies using context-specific Bayesian networks.
problem Traditional HMMs struggle with non-Gaussian data and independence assumptions.
method Kernel density estimation with context-specific Bayesian networks.
result The proposed model outperforms related HMMs in likelihood and classification accuracy.
Generalized Precision Matrix for scalable estimation of nonparametric Markov networks.
problem Estimating conditional independence structure in general distributions for all data types.
method Generalized Precision Matrix (GPM) for mixed-type variables, regularized score matching framework for scalability.
result Validated theoretical results and demonstrated scalability in various settings.
We present two methodologies on the estimation of rating transition probabilities within Markov and non-Markov frameworks. We first estimate a continuous-time Markov chain using discrete (missing) data and derive a simpler expression for the Fisher information matrix, reducing the computational time needed for the Wald…
We consider Markov models of stochastic processes where the next-step conditional distribution is defined by a kernel density estimator (KDE), similar to Markov forecast densities and certain time-series bootstrap schemes. The KDE Markov models (KDE-MMs) we discuss are nonlinear, nonparametric, fully probabilistic repr…
Paper uses genome Markov structure for outlier detection and read classification.
problem Identifying outliers and classifying reads in genome databases.
method Applying second-order Markov models to triplet base distributions.
result Improved accuracy in outlier identification and read classification.
The paper bounds generalization errors for deep neural networks with Markov datasets.
problem Bounding generalization errors for deep learning with Markov datasets.
method Developed new symmetrization inequalities for Markov chains, using spectral gap of the infinitesimal generator.
result Derived upper bounds on generalization errors for deep neural networks with Markov datasets.
This work speeds up fHMM analysis by tensor algebra.
problem Scalability issues in analyzing factorial hidden Markov models.
method Tensorized algorithms and scalable filtering methods.
result Significant improvement in computational performance.
Matrix Chernoff bound for Markov chains applied to co-occurrence matrices.
problem Analyzing the behavior of co-occurrence statistics in sequential data.
method Proved a matrix Chernoff-type bound for sums of matrix-valued random variables sampled via a regular Markov chain.
result Achieved exponentially fast convergence rate and sample complexity analysis for co-occurrence matrices.
New method estimates convergence bounds for nonlinear Markov chains.
problem Difficulty in describing properties of nonlinear Markov chains.
method Coupling Markov chains to reconstitute distribution relationships and estimate convergence bounds.
result Estimation of convergence bounds is more precise than existing results.
Paper proposes semi-supervised learning with triplet Markov chains.
problem Lack of labels in training data.
method Variational Bayesian inference for semi-supervised learning.
result Derives semi-supervised algorithms for various sequential models.
Enhanced Markov chain sampler learns network statistics faster.
problem Learning network statistics efficiently.
method Integrates graph Forman curvature into Markov chain transition probabilities and stationary distribution.
result Curved Markov chain Monte Carlo achieves faster convergence.
New methods for Markov Blanket discovery using MML outperform existing approaches.
problem Causal discovery from large datasets.
method Developed three new methods of Markov Blanket discovery using Minimum Message Length.
result Our best MML method is consistently competitive and has advantageous features.
Improved Bayesian regression for large datasets using multilevel Gibbs sampling.
problem Efficiently handling large-scale Bayesian regression with complex posterior distributions.
method Developed a multilevel Gibbs sampler for linear mixed models, incorporating data clustering and correlated samples for variance reduction.
result Significant speed-up achieved for Bayesian regression without sacrificing predictive performance.
We introduce LAMP: the Linear Additive Markov Process. Transitions in LAMP may be influenced by states visited in the distant history of the process, but unlike higher-order Markov processes, LAMP retains an efficient parametrization. LAMP also allows the specific dependence on history to be learned efficiently from da…
New algorithm for collective Gaussian hidden Markov models inference.
problem Inference of collective Gaussian hidden Markov models from aggregate data.
method Collective Gaussian forward-backward algorithm, extending Sinkhorn belief propagation.
result Convergence guarantee and applicability to single individual Kalman filter.
Unified framework for MCMC and machine learning problems.
problem Intersection of MCMC and machine learning problems.
method Unified framework integrating various MCMC and machine learning techniques.
result Translation and generalization of theory and methods.
Model reduction of Markov processes is a basic problem in modeling state-transition systems. Motivated by the state aggregation approach rooted in control theory, we study the statistical state compression of a discrete-state Markov chain from empirical trajectories. Through the lens of spectral decomposition, we study…
Enhances count process modelling with Markov-modulated non-homogeneous Poisson process.
problem Count data modelling challenges, especially in complex scenarios.
method Introduces a flexible frequency perturbation measure into Markov-modulated Poisson process framework.
result Natural incorporation of observed event arrivals and latent factors.
The paper shows that relaxing assumptions about causal graphs can lead to exponentially large equivalence classes.
problem The size of Markov equivalence classes under relaxed assumptions.
method Analytical proofs for three settings: sparse random directed acyclic graphs, uniformly random acyclic directed mixed graphs, and uniformly random directed cyclic graphs.
result Exponentially large lower bounds for the expected size of Markov equivalence classes.
A new model separates persistence and transition priors in HDP-HMM.
problem Limitation of sticky HDP-HMM in expressing different persistence strengths.
method Developed a disentangled sticky HDP-HMM (DS-HDP-HMM) with novel Gibbs sampling algorithms.
result DS-HDP-HMM outperforms sticky HDP-HMM and HDP-HMM on synthetic and real data.
This paper proposes a stochastic model using the concept of Markov chains for the inter-state transitions of the millisecond order quasi-stable phase synchronized patterns or synchrostates, found in multi-channel Electroencephalogram (EEG) signals. First and second order transition probability matrices are estimated fo…
Neural Markov models improve time series analysis by balancing deep learning and classical models.
problem Modeling non-stationary time series with high data sparsity.
method Hybrid approach using neural networks to parameterize stochastic matrices, estimating time-inhomogeneous Markov chains.
result Reduction of Chapman-Kolmogorov discrepancy and superior likelihood in financial markets.
New algorithm solves uncertain Markov decision processes using Wasserstein uncertainty.
problem Solving Markov decision processes with uncertain transition probabilities.
method Distributionally robust Q-learning algorithm for Wasserstein uncertainty. result Convergence of the algorithm proved and demonstrated with real data.
MLDL preserves manifold geometry in vector transformations.
problem Geometric deterioration in neural network transformations.
method Locally isometric smoothness (LIS) and Markov random field (MRF) encoding.
result Enhanced vector transformations into well-behaved metric homeomorphisms.
Paper proposes a new method for training diffusion models using Markov operators.
problem Training efficiency and accuracy in diffusion models.
method Operator-informed score matching using spectral decomposition of Markov operators.
result Improved score matching for both low and high-dimensional distributions.
Unbiased gradient estimation for Markov chains
problem Estimating gradients of stationary means in Markov chains
method Propose new unbiased estimators
result Improves efficiency for slow mixing Markov chains
Model detects market anomalies using a Hawkes process with hidden Markov chain.
problem Detecting high-frequency market manipulation in cryptocurrency trades.
method Developed a Markov-modulated Hawkes process with piecewise constant excitation kernels.
result Demonstrated the model's effectiveness in detecting suspicious trading activities.
Pessimistic model-based algorithm finds Nash equilibria in zero-sum Markov games from offline data.
problem Learning Nash equilibria in two-player zero-sum Markov games from limited data.
method Pessimistic model-based algorithm with Bernstein-style lower confidence bounds (VI-LCB-Game).
result Proves sample complexity no larger than (1−γ)3ε2Cclipped⋆S(A+B), achieving minimax optimality. A new sampler improves the inference of causal structures from observational data.
problem Inferring causal relationships from observational data when DAGs are Markov equivalent.
method Developed a non-reversible Markov chain, Causal Zig-Zag sampler, targeting Markov Equivalence Classes of DAGs.
result The sampler improves mixing and offers efficient algorithms for DAG inference.
New empirical PAC-Bayes bound for Markov chains with finite state space.
problem Lack of empirical bounds for Markov chains with temporal dependence.
method Proved a new PAC-Bayes bound for Markov chains, providing an empirical pseudo-spectral gap.
result First fully empirical PAC-Bayes bound for Markov chains with finite state space.
Researchers created a continuous Markov martingale that mimics Brownian motion but lacks the strong Markov property.
problem Constructing a continuous Markov martingale with Brownian marginals that misses the strong Markov property.
method Developed a new approach to create a continuous Markov martingale that differs from Brownian motion in terms of the strong Markov property.
result A continuous Markov martingale with Brownian marginals that lacks the strong Markov property was successfully constructed.
Hierarchical hidden Markov models predict market trends in financial time series.
problem Misinterpretation of short-term price fluctuations as long-term trend changes.
method Hierarchical hidden Markov models to capture both short- and long-term trends.
result Hierarchical models provide a comprehensive picture of financial markets.
Bayesian Context Trees improve change-point detection in discrete data.
problem Detecting and segmenting change-points in discrete time series data.
method Bayesian Context Trees framework, Markov chain Monte Carlo sampling.
result Effective sampling from posterior distribution of change-points.