In this note we consider sampling from (non-homogeneous) strongly Rayleigh probability measures. As an important corollary, we obtain a fast mixing Markov Chain sampler for Determinantal Point Processes.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Non-homogeneous Markov Chains”
New algorithm identifies best policy in MDPs faster.
problem Identifying the best policy in Markov Decision Processes.
method Problem-dependent lower bound and first algorithm with instance-specific sample complexity.
result First algorithm with reduced exploration rate for faster convergence.
Discrete-time hidden Markov models are a broadly useful class of latent-variable models with applications in areas such as speech recognition, bioinformatics, and climate data analysis. It is common in practice to introduce temporal non-homogeneity into such models by making the transition probabilities dependent on ti…
Enhances count process modelling with Markov-modulated non-homogeneous Poisson process.
problem Count data modelling challenges, especially in complex scenarios.
method Introduces a flexible frequency perturbation measure into Markov-modulated Poisson process framework.
result Natural incorporation of observed event arrivals and latent factors.
In this paper we propose a multi-state model for the evaluation of the conversion option contract. The multi-state model is based on age-indexed semi-Markov chains that are able to reproduce many important aspects that influence the valuation of the option such as the duration problem, the time non-homogeneity and the …
An adaptive simulated annealing EM algorithm for inference on non-homogeneous hidden Markov modelsstat.ML
New algorithm optimizes complex model selection for non-homogeneous hidden Markov models.
problem Complex combinatorial optimization problem for model selection in NHHMMs.
method Adaptive simulated annealing EM algorithm (ASA-EM).
result Joint optimization of models and parameters for a criterion of interest.
It is a well known fact that recovery rates tend to go down when the number of defaults goes up in economic downturns. We demonstrate how the loss given default model with the default and recovery dependent via the latent systematic risk factor can be estimated using Bayesian inference methodology and Markov chain Mont…
Extends Gaussian process regression for non-Gaussian data.
problem Inadequate modeling of uncertainty and over-smoothing in non-Gaussian datasets.
method Time-changed Gaussian processes with Lévy processes.
result Improved modeling of heavy-tailed non-Gaussian behaviors.
AIS uses a suboptimal extended target distribution, which this paper improves using SGM.
problem Improving the efficiency of Annealed Importance Sampling for marginal likelihood estimation.
method Leveraging score-based generative modeling to approximate the optimal extended target distribution.
result Demonstrated novel, differentiable AIS procedures on synthetic and real-world data.
The paper provides concentration inequalities for Markov chain variance estimators.
problem Estimating the variance of Markov chains with concentration properties.
method Martingale decomposition method for uniformly geometrically ergodic Markov chains.
result Explicit control of the p-th moment of the OBM estimator difference and dependence on p and mixing time.
A new method simulates a lazy version of a Markov chain for empirical inference.
problem Estimating and testing unknown Markov chains with limited data.
method Simulates an α-lazy version of an unknown Markov chain, making it ergodic.
result The pseudo spectral gap can be applied to non-ergodic Markov chains.
Reduces identity testing of reversible Markov chains to simpler symmetric chain tests.
problem Testing identity of reversible Markov chains from a single trajectory.
method Using lumping-congruent Markov embeddings, the problem is simplified to testing symmetric chains over a larger state space.
result Achieves state-of-the-art sample complexity for identity testing.
Stochastic gradient methods are the workhorse (algorithms) of large-scale optimization problems in machine learning, signal processing, and other computational sciences and engineering. This paper studies Markov chain gradient descent, a variant of stochastic gradient descent where the random samples are taken on the t…
Study Markov chain gradient descent in Hilbert spaces for quadratic loss.
problem Approximating optimal solutions for quadratic loss functions.
method Developed a Markov chain-based stochastic gradient algorithm in Hilbert spaces.
result Established probabilistic upper bounds on convergence.
Study nonparametric estimator for Markov chain transition matrices in offline setting.
problem Estimating transition matrices of finite controlled Markov chains from logged data.
method Developed sample complexity bounds and conditions for minimaxity.
result Achieving certain statistical risk requires balancing mixing properties and sample size.
This study shows how DDPM can be represented by the OU process.
problem Designing optimal noise schedules for DDPM.
method Formal equivalence between DDPM and OU process, heuristic designs based on Fisher Information.
result Fisher-Information-motivated schedule corresponds to cosine noise schedule.
Enhanced Markov chain sampler learns network statistics faster.
problem Learning network statistics efficiently.
method Integrates graph Forman curvature into Markov chain transition probabilities and stationary distribution.
result Curved Markov chain Monte Carlo achieves faster convergence.
Introducing user-prescribed constraints in Markov chains for nonlinear dimensionality reductioncs.LG
Stochastic kernel based dimensionality reduction approaches have become popular in the last decade. The central component of many of these methods is a symmetric kernel that quantifies the vicinity between pairs of data points and a kernel-induced Markov chain on the data. Typically, the Markov chain is fully specified…
Markov Observation Modelsstat.ML
Expands Hidden Markov Model to include Markov chain observations.
problem Handling Markov chain observations in Hidden Markov Models.
method Developed Expectation-Maximization algorithm and Viterbi algorithm analogs.
result Estimates transition probabilities for hidden states and observations.
In this paper we describe three stochastic models based on a semi-Markov chains approach and its generalizations to study the high frequency price dynamics of traded stocks. The three models are: a simple semi-Markov chain model, an indexed semi-Markov chain model and a weighted indexed semi-Markov chain model. We show…
New insights into Markov chain geometry via positive transition measures.
problem Lack of statistical meaning in the space of transition probabilities.
method Constructing an extension of the space of transition probabilities using Amari's theory of positive measures.
result Introduction of a new dually flat structure for the space of positive transition measures.
In his 2011 work, Maas has shown that the law of any time-reversible continuous-time Markov chain with finite state space evolves like a gradient flow of the relative entropy with respect to its stationary distribution. In this work we show the converse to the above by showing that if the relative law of a Markov chain…
New method estimates convergence bounds for nonlinear Markov chains.
problem Difficulty in describing properties of nonlinear Markov chains.
method Coupling Markov chains to reconstitute distribution relationships and estimate convergence bounds.
result Estimation of convergence bounds is more precise than existing results.
Elo ratings learn model parameters quickly using Markov chains.
problem Ranking players in online settings.
method Bradley--Terry--Luce model and Markov chain theory.
result Elo learns model parameters at a competitive rate.
DCDC calculates convergence rates for Markov chains using neural networks.
problem Computing precise convergence rates for Markov chains is hard.
method Developed a neural network-based algorithm (DCDC) to bound convergence rates in Wasserstein distance.
result Demonstrated effective convergence bounds for real-world Markov chains.
The paper studies how quickly samples from Langevin dynamics become independent.
problem Understanding the dependence between samples along Langevin dynamics and related algorithms.
method Measures dependence via -mutual information and proves strong data processing inequalities.
result The -mutual information between samples decreases exponentially to zero.
We study the problem of learning the transition matrices of a set of Markov chains from a single stream of observations on each chain. We assume that the Markov chains are ergodic but otherwise unknown. The learner can sample Markov chains sequentially to observe their states. The goal of the learner is to sequentially…
This paper models time-series data with a mixture of Markov chains, automatically determining the number of components.
problem Tackles the inability of common Markov state modeling frameworks to discern heterogeneities in complex data.
method Uses a mixture of Markov chains and variational expectation-maximization algorithm for automatic component selection.
result Achieves performance consistent with theoretically optimal error scaling, identifying meaningful heterogeneities in various data sets.
Method reconstructs hidden Markov chains from insurance data.
problem Recovering hidden Markov chains from incomplete insurance data.
method Neural architecture to explicitly provide transition probabilities.
result Neural model successfully validates decompression of insurance information.
The paper extends Hoeffding's inequality for Markov chains using a generalized concentrability condition.
problem Applying Hoeffding's inequality to non-ergodic Markov chains.
method Integrates generalized concentrability condition via IPM to extend traditional hypotheses.
result Demonstrates utility in machine learning applications such as empirical risk minimization and bandits.
Identity testing for reversible Markov chains without symmetry assumption.
problem Identity testing of reversible Markov chains.
method Using distance notion from Daskalakis et al. [2018a], testing without symmetry assumption.
result It is possible to perform identity testing under weaker assumption of reversibility.
Unbiased gradient estimation for Markov chains
problem Estimating gradients of stationary means in Markov chains
method Propose new unbiased estimators
result Improves efficiency for slow mixing Markov chains
The method of block coordinate gradient descent (BCD) has been a powerful method for large-scale optimization. This paper considers the BCD method that successively updates a series of blocks selected according to a Markov chain. This kind of block selection is neither i.i.d. random nor cyclic. On the other hand, it is…
Markov Chain Score Ascent: A Unifying Framework of Variational Inference with Markovian Gradientscs.LG
New framework improves variational inference with Markov chain methods.
problem Challenges of minimizing KL divergence with stochastic gradient descent.
method Markov chain score ascent (MCSA) methods, including parallel MCSA (pMCSA).
result Improved theoretical and empirical performance of MCSA methods.
The paper develops new inequalities for Markov chain sums, linking them to mixing time.
problem Establishing concentration inequalities for Markov chain sums.
method Developed novel concentration inequalities for geometrically ergodic Markov chains, linking bounds to mixing time constants.
result Explicit bounds for additive functionals of Markov chains, linked to Rosenthal inequality constants and mixing properties.
This paper proposes a stochastic model using the concept of Markov chains for the inter-state transitions of the millisecond order quasi-stable phase synchronized patterns or synchrostates, found in multi-channel Electroencephalogram (EEG) signals. First and second order transition probability matrices are estimated fo…
Matrix Chernoff bound for Markov chains applied to co-occurrence matrices.
problem Analyzing the behavior of co-occurrence statistics in sequential data.
method Proved a matrix Chernoff-type bound for sums of matrix-valued random variables sampled via a regular Markov chain.
result Achieved exponentially fast convergence rate and sample complexity analysis for co-occurrence matrices.
The time to converge to the steady state of a finite Markov chain can be greatly reduced by a lifting operation, which creates a new Markov chain on an expanded state space. For a class of quadratic objectives, we show an analogous behavior where a distributed ADMM algorithm can be seen as a lifting of Gradient Descent…
Policy gradient algorithm with variable learning rates achieves near-optimal performance in multi-arm bandit problems.
problem Optimizing a policy gradient algorithm for multi-arm bandit problems with variable learning rates.
method Applied Foster-Lyapunov techniques to analyze a Markov chain formed by the state of the algorithm.
result The policy gradient algorithm converges to the optimal arm with logarithmic or poly-logarithmic regret.
Algorithm learns mixtures of Markov chains and MDPs from short trajectories.
problem Learning mixtures of Markov chains and MDPs from short unlabeled trajectories.
method Subspace estimation, spectral clustering, EM algorithm, model estimation, classification.
result 96.6% average accuracy on a mixture of two MDPs in gridworld, outperforming EM algorithm with random initialization.
We study the problem of identity testing of markov chains. In this setting, we are given access to a single trajectory from a markov chain with unknown transition matrix and the goal is to determine whether for some known matrix or where is suitably defined. In r…
We study (backward) stochastic differential equations with noise coming from a finite state Markov chain. We show that, for the solutions of these equations to be `Markovian', in the sense that they are deterministic functions of the state of the underlying chain, the integrand must be of a specific form. This allows u…
Non-negative curvature affects Markov chains' mixing and expansion properties.
problem Understanding the behavior of Markov chains with non-negative curvature.
method Analyzing conductance, displacement, and cutoff phenomenon in sparse Markov chains.
result Non-negatively curved Markov chains exhibit specific, non-standard behavior in terms of mixing and expansion.
Paper proposes semi-supervised learning with triplet Markov chains.
problem Lack of labels in training data.
method Variational Bayesian inference for semi-supervised learning.
result Derives semi-supervised algorithms for various sequential models.
Marginal AMP Chain Graphsstat.ML
We present a new family of models that is based on graphs that may have undirected, directed and bidirected edges. We name these new models marginal AMP (MAMP) chain graphs because each of them is Markov equivalent to some AMP chain graph under marginalization of some of its nodes. However, MAMP chain graphs do not onl…
Gradient Descent Algorithm in Hilbert Spaces under Stationary Markov Chains with - and -Mixingstat.ML
Study on gradient descent in Hilbert spaces with Markov chains, focusing on mixing coefficients.
problem Analyzing convergence of gradient descent in Hilbert spaces with stationary Markov chains.
method Examined strictly stationary Markov chains with - and -mixing coefficients, derived probabilistic upper bounds.
result Probabilistic upper bounds on convergence behavior of gradient descent algorithm based on mixing coefficients.
The paper bounds generalization errors for deep neural networks with Markov datasets.
problem Bounding generalization errors for deep learning with Markov datasets.
method Developed new symmetrization inequalities for Markov chains, using spectral gap of the infinitesimal generator.
result Derived upper bounds on generalization errors for deep neural networks with Markov datasets.
Proposes MIVI for efficient posterior estimation and design of MCMC transitions.
problem Efficiently estimating posterior distributions in constrained time.
method Combines variational inference and MCMC with a variational distribution and optimized Markov chain.
result Optimized Markov chain improves variational distribution and vice versa, leading to more accurate posteriors.