The method of block coordinate gradient descent (BCD) has been a powerful method for large-scale optimization. This paper considers the BCD method that successively updates a series of blocks selected according to a Markov chain. This kind of block selection is neither i.i.d. random nor cyclic. On the other hand, it is…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Block Markov Chain”
Method estimates number of clusters in Block Markov Chain trajectories.
problem Challenges in choosing number of clusters for sequential data.
method Spectral embedding and density-based clustering.
result Asymptotically consistent method for estimating clusters.
Improved sampling for network community detection.
problem Inefficient sampling from network partition posterior distributions.
method Merge-split Markov chain Monte Carlo for efficient sampling.
result Significantly improved mixing time and correct sampling.
The Viterbi process can be extended indefinitely in a pairwise Markov model.
problem Estimating hidden chains in pairwise Markov models.
method Construction of barriers to ensure Viterbi path goes through states.
result The Viterbi process is regenerative in the PMM.
New concentration inequality for U-statistics of Markov chains.
problem Proving a concentration inequality for U-statistics of order two in uniformly ergodic Markov chains.
method Inductive analysis using martingale techniques, uniform ergodicity, Nummelin splitting, and Bernstein's inequality.
result Recovery of convergence rate for U-statistics of independent random variables and canonical kernels, with improved results for dependent kernels.
Graph matching in noisy environments with Markovian errors.
problem Graph matching under time-dependent Markovian noise.
method Introduced edgelighter error model and analyzed graph matching thresholds.
result Graph matching thresholds and mixing times are of order for Erdős-Rényi graphs, and for Stochastic Block Model graphs.
Suppose that one particular block in a stochastic block model is of interest, but block labels are only observed for a few of the vertices in the network. Utilizing a graph realized from the model and the observed block labels, the vertex nomination task is to order the vertices with unobserved block labels into a rank…
Model clusters networks and their communities simultaneously.
problem Clustering networks and their communities in unlabeled, heterogeneous networks.
method Nested Stochastic Block Model (NSBM) with Bayesian approach and NDP prior.
result Model accurately estimates both within and across network clustering structures.
We consider the problem of learning low-dimensional representations for large-scale Markov chains. We formulate the task of representation learning as that of mapping the state space of the model to a low-dimensional state space, called the kernel space. The kernel space contains a set of meta states which are desired …
"Mixed Data" comprising a large number of heterogeneous variables (e.g. count, binary, continuous, skewed continuous, among other data types) are prevalent in varied areas such as genomics and proteomics, imaging genetics, national security, social networking, and Internet advertising. There have been limited efforts a…
A novel MCMC method clusters data faster and more accurately.
problem Efficiently clustering large datasets with unknown number of clusters.
method Master/Worker architecture for distributed MCMC inference.
result Significant improvement in clustering accuracy and speed.
JaxSGMC simplifies SG-MCMC for Bayesian deep learning.
problem Uncertainty quantification in deep learning models.
method Modular stochastic gradient MCMC in JAX.
result Facilitates trustworthy neural network predictions.
The study explains how transformer components enable in-context learning.
problem Understanding how transformer components contribute to in-context learning.
method Analyzed a two-attention-layer transformer model trained on Markov chain data.
result Gradient flow converges to a limiting model with a copier, selector, and classifier mechanism.
Bayesian inference for biochemical reaction networks using jump-diffusion approximations.
problem Estimating hidden quantities in poorly characterized biochemical processes.
method Developed a Bayesian inference algorithm based on Markov chain Monte Carlo and sequential Monte Carlo methods.
result Numerical evaluation of the algorithm for a partially observed multi-scale birth-death process.
The paper provides concentration inequalities for Markov chain variance estimators.
problem Estimating the variance of Markov chains with concentration properties.
method Martingale decomposition method for uniformly geometrically ergodic Markov chains.
result Explicit control of the p-th moment of the OBM estimator difference and dependence on p and mixing time.
A new method simulates a lazy version of a Markov chain for empirical inference.
problem Estimating and testing unknown Markov chains with limited data.
method Simulates an α-lazy version of an unknown Markov chain, making it ergodic.
result The pseudo spectral gap can be applied to non-ergodic Markov chains.
Efficient Monte Carlo and greedy heuristic for the inference of stochastic block modelsphysics.data-an
We present an efficient algorithm for the inference of stochastic block models in large networks. The algorithm can be used as an optimized Markov chain Monte Carlo (MCMC) method, with a fast mixing time and a much reduced susceptibility to getting trapped in metastable states, or as a greedy agglomerative heuristic, w…
Reduces identity testing of reversible Markov chains to simpler symmetric chain tests.
problem Testing identity of reversible Markov chains from a single trajectory.
method Using lumping-congruent Markov embeddings, the problem is simplified to testing symmetric chains over a larger state space.
result Achieves state-of-the-art sample complexity for identity testing.
Stochastic gradient methods are the workhorse (algorithms) of large-scale optimization problems in machine learning, signal processing, and other computational sciences and engineering. This paper studies Markov chain gradient descent, a variant of stochastic gradient descent where the random samples are taken on the t…
Study Markov chain gradient descent in Hilbert spaces for quadratic loss.
problem Approximating optimal solutions for quadratic loss functions.
method Developed a Markov chain-based stochastic gradient algorithm in Hilbert spaces.
result Established probabilistic upper bounds on convergence.
Study nonparametric estimator for Markov chain transition matrices in offline setting.
problem Estimating transition matrices of finite controlled Markov chains from logged data.
method Developed sample complexity bounds and conditions for minimaxity.
result Achieving certain statistical risk requires balancing mixing properties and sample size.
Study on natural actor-critic for POMDPs with finite memory.
problem Learning in partially observed Markov decision processes with noisy observations.
method Finite actor-critic method with multi-step temporal difference learning.
result First non-asymptotic global convergence for POMDPs with function approximation.
Enhanced Markov chain sampler learns network statistics faster.
problem Learning network statistics efficiently.
method Integrates graph Forman curvature into Markov chain transition probabilities and stationary distribution.
result Curved Markov chain Monte Carlo achieves faster convergence.
Introducing user-prescribed constraints in Markov chains for nonlinear dimensionality reductioncs.LG
Stochastic kernel based dimensionality reduction approaches have become popular in the last decade. The central component of many of these methods is a symmetric kernel that quantifies the vicinity between pairs of data points and a kernel-induced Markov chain on the data. Typically, the Markov chain is fully specified…
We propose a restricted collapsed draw (RCD) sampler, a general Markov chain Monte Carlo sampler of simultaneous draws from a hierarchical Chinese restaurant process (HCRP) with restriction. Models that require simultaneous draws from a hierarchical Dirichlet process with restriction, such as infinite Hidden markov mod…
Improved Bayesian analysis for SVM models using a mixture sampler.
problem Efficient simulation-based analysis of stochastic volatility in mean models.
method Developed a generalized mixture sampler for SVM models, approximating non-central chi-squared distributions as mixtures of normal distributions.
result The proposed method outperforms other volatility models based on marginal likelihoods in empirical studies.
Markov Observation Modelsstat.ML
Expands Hidden Markov Model to include Markov chain observations.
problem Handling Markov chain observations in Hidden Markov Models.
method Developed Expectation-Maximization algorithm and Viterbi algorithm analogs.
result Estimates transition probabilities for hidden states and observations.
In this paper we describe three stochastic models based on a semi-Markov chains approach and its generalizations to study the high frequency price dynamics of traded stocks. The three models are: a simple semi-Markov chain model, an indexed semi-Markov chain model and a weighted indexed semi-Markov chain model. We show…
New insights into Markov chain geometry via positive transition measures.
problem Lack of statistical meaning in the space of transition probabilities.
method Constructing an extension of the space of transition probabilities using Amari's theory of positive measures.
result Introduction of a new dually flat structure for the space of positive transition measures.
In his 2011 work, Maas has shown that the law of any time-reversible continuous-time Markov chain with finite state space evolves like a gradient flow of the relative entropy with respect to its stationary distribution. In this work we show the converse to the above by showing that if the relative law of a Markov chain…
New method estimates convergence bounds for nonlinear Markov chains.
problem Difficulty in describing properties of nonlinear Markov chains.
method Coupling Markov chains to reconstitute distribution relationships and estimate convergence bounds.
result Estimation of convergence bounds is more precise than existing results.
Elo ratings learn model parameters quickly using Markov chains.
problem Ranking players in online settings.
method Bradley--Terry--Luce model and Markov chain theory.
result Elo learns model parameters at a competitive rate.
DCDC calculates convergence rates for Markov chains using neural networks.
problem Computing precise convergence rates for Markov chains is hard.
method Developed a neural network-based algorithm (DCDC) to bound convergence rates in Wasserstein distance.
result Demonstrated effective convergence bounds for real-world Markov chains.
The paper studies how quickly samples from Langevin dynamics become independent.
problem Understanding the dependence between samples along Langevin dynamics and related algorithms.
method Measures dependence via -mutual information and proves strong data processing inequalities.
result The -mutual information between samples decreases exponentially to zero.
We study the problem of learning the transition matrices of a set of Markov chains from a single stream of observations on each chain. We assume that the Markov chains are ergodic but otherwise unknown. The learner can sample Markov chains sequentially to observe their states. The goal of the learner is to sequentially…
Method reconstructs hidden Markov chains from insurance data.
problem Recovering hidden Markov chains from incomplete insurance data.
method Neural architecture to explicitly provide transition probabilities.
result Neural model successfully validates decompression of insurance information.
This paper models time-series data with a mixture of Markov chains, automatically determining the number of components.
problem Tackles the inability of common Markov state modeling frameworks to discern heterogeneities in complex data.
method Uses a mixture of Markov chains and variational expectation-maximization algorithm for automatic component selection.
result Achieves performance consistent with theoretically optimal error scaling, identifying meaningful heterogeneities in various data sets.
Parsimonious module inference in large networksphysics.data-an
We investigate the detectability of modules in large networks when the number of modules is not known in advance. We employ the minimum description length (MDL) principle which seeks to minimize the total amount of information required to describe the network, and avoid overfitting. According to this criterion, we obta…
The paper extends Hoeffding's inequality for Markov chains using a generalized concentrability condition.
problem Applying Hoeffding's inequality to non-ergodic Markov chains.
method Integrates generalized concentrability condition via IPM to extend traditional hypotheses.
result Demonstrates utility in machine learning applications such as empirical risk minimization and bandits.
Identity testing for reversible Markov chains without symmetry assumption.
problem Identity testing of reversible Markov chains.
method Using distance notion from Daskalakis et al. [2018a], testing without symmetry assumption.
result It is possible to perform identity testing under weaker assumption of reversibility.
Unbiased gradient estimation for Markov chains
problem Estimating gradients of stationary means in Markov chains
method Propose new unbiased estimators
result Improves efficiency for slow mixing Markov chains
Markov Chain Score Ascent: A Unifying Framework of Variational Inference with Markovian Gradientscs.LG
New framework improves variational inference with Markov chain methods.
problem Challenges of minimizing KL divergence with stochastic gradient descent.
method Markov chain score ascent (MCSA) methods, including parallel MCSA (pMCSA).
result Improved theoretical and empirical performance of MCSA methods.
Speeding up Markov Chain Monte Carlo (MCMC) for datasets with many observations by data subsampling has recently received considerable attention. A pseudo-marginal MCMC method is proposed that estimates the likelihood by data subsampling using a block-Poisson estimator. The estimator is a product of Poisson estimators,…
This paper proposes a stochastic model using the concept of Markov chains for the inter-state transitions of the millisecond order quasi-stable phase synchronized patterns or synchrostates, found in multi-channel Electroencephalogram (EEG) signals. First and second order transition probability matrices are estimated fo…
The paper develops new inequalities for Markov chain sums, linking them to mixing time.
problem Establishing concentration inequalities for Markov chain sums.
method Developed novel concentration inequalities for geometrically ergodic Markov chains, linking bounds to mixing time constants.
result Explicit bounds for additive functionals of Markov chains, linked to Rosenthal inequality constants and mixing properties.
Matrix Chernoff bound for Markov chains applied to co-occurrence matrices.
problem Analyzing the behavior of co-occurrence statistics in sequential data.
method Proved a matrix Chernoff-type bound for sums of matrix-valued random variables sampled via a regular Markov chain.
result Achieved exponentially fast convergence rate and sample complexity analysis for co-occurrence matrices.
The time to converge to the steady state of a finite Markov chain can be greatly reduced by a lifting operation, which creates a new Markov chain on an expanded state space. For a class of quadratic objectives, we show an analogous behavior where a distributed ADMM algorithm can be seen as a lifting of Gradient Descent…
Policy gradient algorithm with variable learning rates achieves near-optimal performance in multi-arm bandit problems.
problem Optimizing a policy gradient algorithm for multi-arm bandit problems with variable learning rates.
method Applied Foster-Lyapunov techniques to analyze a Markov chain formed by the state of the algorithm.
result The policy gradient algorithm converges to the optimal arm with logarithmic or poly-logarithmic regret.