RML improves generative modeling of complex distributions.
problem Learning complex distributions in applications.
method RML defines a forward process to a known distribution, then learns a reverse Markov process.
result RML efficiently captures complex distributions in simulations and climate data.
This study bridges discrete and continuous state spaces using the Ehrenfest process and diffusion models.
problem Understanding the relationship between discrete and continuous state spaces in stochastic processes.
method Investigates time-continuous Markov jump processes on discrete state spaces and their correspondence to state-continuous diffusion processes.
result The time-reversal of the Ehrenfest process converges to the time-reversed Ornstein-Uhlenbeck process, bridging discrete and continuous state spaces.
We present a nonparametric prior over reversible Markov chains. We use completely random measures, specifically gamma processes, to construct a countably infinite graph with weighted edges. By enforcing symmetry to make the edges undirected we define a prior over random walks on graphs that results in a reversible Mark…
Stochastic gradient methods are the workhorse (algorithms) of large-scale optimization problems in machine learning, signal processing, and other computational sciences and engineering. This paper studies Markov chain gradient descent, a variant of stochastic gradient descent where the random samples are taken on the t…
Generative models using PDMPs with explicit jump rates and kernels.
problem Creating efficient generative models for complex data distributions.
method Piecewise deterministic Markov processes (PDMPs) with explicit expressions for jump rates and kernels.
result Efficient training and simulation methods for PDMP-based generative models.
Reduces identity testing of reversible Markov chains to simpler symmetric chain tests.
problem Testing identity of reversible Markov chains from a single trajectory.
method Using lumping-congruent Markov embeddings, the problem is simplified to testing symmetric chains over a larger state space.
result Achieves state-of-the-art sample complexity for identity testing.
Study shows TD(0) with linear approx. converges for reversible Markov chains.
problem TD(0) divergence with off-policy and function approximation.
method Analyzes standard TD(0) with reversible Markov chains, adapting stochastic approximation framework.
result Establishes convergence with probability one for projected Bellman error = 0.
Extended elliptical slice sampling for infinite-dimensional spaces, proving reversibility.
problem Proving reversibility of elliptical slice sampling in infinite-dimensional spaces.
method Extended elliptical slice sampling to infinite-dimensional separable Hilbert spaces, providing an alternative proof of reversibility.
result The approach yields a positive semi-definite Markov operator, proving reversibility.
Identity testing for reversible Markov chains without symmetry assumption.
problem Identity testing of reversible Markov chains.
method Using distance notion from Daskalakis et al. [2018a], testing without symmetry assumption.
result It is possible to perform identity testing under weaker assumption of reversibility.
Study on policy testing in MDPs with lower bounds and new algorithm.
problem Deciding if policy value exceeds a threshold with limited samples.
method Derived lower bound, proposed new algorithm, reformulated problem, used policy optimization in reversed MDP.
result New algorithm outperforms existing methods in policy testing.
Bayesian method selects interacting regions in Markov models.
problem Estimating interacting regions in Markov Random Fields.
method Reversible Jump Monte Carlo Markov Chain algorithm with pseudoposteriors.
result Proposed method accurately selects interacting regions in simulations and real data.
This supplementary material includes three parts: some preliminary results, four examples, an experiment, three new algorithms, and all proofs of the results in the paper "Reversible MCMC on Markov equivalence classes of sparse directed acyclic graphs".
New method trains Markov kernels for efficient sampling.
problem Efficient sampling from complex probability distributions.
method Adversarial learning of involutive Metropolis-Hastings kernels.
result Minimizes total variation distance to empirical data.
New PDMP samplers tackle variable selection in models.
problem Jointly explore model space and parameter space.
method Develop reversible jump PDMP samplers.
result New samplers mix better and are more efficient.
Predictive rate-distortion analysis suffers from the curse of dimensionality: clustering arbitrarily long pasts to retain information about arbitrarily long futures requires resources that typically grow exponentially with length. The challenge is compounded for infinite-order Markov processes, since conditioning on fi…
QTD integrates quantization with diffusion for efficient data generation.
problem Challenges in continuous diffusion models, especially long-range transitions and biases.
method Quantized Transition Diffusion (QTD) integrates data quantization with discrete diffusion dynamics.
result QTD achieves efficient data generation with minimal score evaluations.
A new sampler improves the inference of causal structures from observational data.
problem Inferring causal relationships from observational data when DAGs are Markov equivalent.
method Developed a non-reversible Markov chain, Causal Zig-Zag sampler, targeting Markov Equivalence Classes of DAGs.
result The sampler improves mixing and offers efficient algorithms for DAG inference.
We introduce a new model for describing the fluctuations of a tick-by-tick single asset price. Our model is based on Markov renewal processes. We consider a point process associated to the timestamps of the price jumps, and marks associated to price increments. By modeling the marks with a suitable Markov chain, we can…
Bayesian symbolic regression uncovers missing physics from data with uncertainty quantification.
problem Incomplete knowledge of physical laws from experimental data.
method Bayesian symbolic regression using Reversible Jump Markov Chain Monte Carlo.
result Uncertainty quantification in recovered model structures.
Develops a method to model multivariate count processes with Cox processes and shot noise intensities.
problem Modeling and estimating dependent count processes using granular data.
method Multivariate Cox process with shot noise intensities, connected via Lévy copulas.
result Allows for over-dispersion, auto-correlation, and realistic features in count processes.
Study hypothesis testing under quantized samples with communication constraints, achieving near-optimal sample complexity.
problem Optimizing hypothesis testing with quantized samples and communication constraints.
method Developed a polynomial-time algorithm achieving near-optimal sample complexity under communication constraints.
result Achieved near-optimal sample complexity under communication constraints, with a logarithmic factor increase over unconstrained setting.
Unified approach to denoising Markov models for efficient sampling.
problem Designing efficient sampling algorithms for complex distributions.
method Mathematical foundation using measure transport and nonequilibrium statistical mechanics.
result Unified variational objective and backward generator construction.
Paper introduces DMPMs for efficient discrete data generation with sharp convergence bounds.
problem Efficient generation of discrete data with theoretical guarantees.
method Discrete Markov Probabilistic Models (DMPMs) operating in bit space with time-reversal process.
result Sharp convergence bounds established under minimal assumptions, competitive performance in discrete data generation.
In his 2011 work, Maas has shown that the law of any time-reversible continuous-time Markov chain with finite state space evolves like a gradient flow of the relative entropy with respect to its stationary distribution. In this work we show the converse to the above by showing that if the relative law of a Markov chain…
New MCMC method for complex models with large variables.
problem Inference on posterior model probabilities in large model spaces.
method Reversible genetically modified mode jumping Markov chain Monte Carlo (GMJMCMC).
result Introduced a proper MCMC with correct limiting distribution.
We study the problem of hypothesis testing between two discrete distributions, where we only have access to samples after the action of a known reversible Markov chain, playing the role of noise. We derive instance-dependent minimax rates for the sample complexity of this problem, and show how its dependence in time is…
The paper proves inequalities for Steklov eigenvalues on finite graphs.
problem Eigenvalues of Laplacians for reversible Markov chains and Steklov eigenvalues.
method Generalized Cheeger inequalities, convergence results, and resolvent convergence.
result Sharp estimate for the first non-trivial Steklov eigenvalue.
This work extends ME-RL using diffusion models to sample optimal policies.
problem Sampling from the optimal policy trajectory distribution in ME-RL.
method Introducing Diffusion-Augmented Markov Decision Processes (DA-MDPs) to minimize reverse KL divergence.
result DA-MDPs enable seamless integration into various ME-RL methods and outperform baselines.
A new sampler speeds up Bayesian mixture models.
problem Sampling from Bayesian finite mixture models is slow and hard.
method Introduces a non-reversible sampling scheme for Bayesian finite mixture models.
result The new sampler outperforms classical samplers in many scenarios, especially during convergence.
We address the problem of estimating the mixing time tmix of an arbitrary ergodic finite-state Markov chain from a single trajectory of length m. The reversible case was addressed by Hsu et al. [2019], who left the general case as an open problem. In the reversible case, the analysis is greatly facilita…
A new Monte Carlo sampling method derived from reverse diffusion.
problem Sampling from complex distributions, especially multi-modal ones.
method Transforming score matching into mean estimation; estimating means of regularized posterior distributions.
result rdMC can approximate sampling with any desired accuracy and is significantly faster than MCMC for complex distributions.
New algorithms learn Ising models from minimal observation of configuration changes.
problem Learning Ising models from the evolution of Markov chains with minimal observation.
method Developed algorithms that efficiently learn Ising models from minimal observation of configuration changes.
result First algorithms that efficiently learn Ising models in a more realistic observation model.
Continuous time framework for discrete data denoising models.
problem Efficient training and sampling for discrete data denoising models.
method Formulated as Continuous Time Markov Chains (CTMCs), efficient training using continuous time ELBO, high-dimensional CTMC simulation, novel theoretical error bound.
result Continuous time treatment enables novel theoretical error bound between generated and true data distributions.
In this paper we build on previous work which uses inferences techniques, in particular Markov Chain Monte Carlo (MCMC) methods, to solve parameterized control problems. We propose a number of modifications in order to make this approach more practical in general, higher-dimensional spaces. We first introduce a new tar…
Banded matrices can be used as precision matrices in several models including linear state-space models, some Gaussian processes, and Gaussian Markov random fields. The aim of the paper is to make modern inference methods (such as variational inference or gradient-based sampling) available for Gaussian models with band…
Study non-negative curvature Markov chains, proving entropy contraction.
problem Prove entropy contraction for Markov chains with non-negative curvature.
method Prove 1-step contraction in Wasserstein distance implies 1-step contraction in relative entropy.
result Prove MLSI with constant equal to minimal rate increment for mean-field zero-range process.
HDT improves MCMC on graphs with history-dependent sampling.
problem Efficient sampling from target distributions on general graphs with low computational overhead.
method History-driven target (HDT) framework that replaces the original target distribution with a history-dependent one.
result Near-zero variance performance and scalability to large graphs with memory-efficient implementation.
This paper develops tools for nonreversible MCMC with convergence guarantees.
problem Designing nonreversible MCMC kernels with convergence guarantees.
method Develops tools for nonreversible Markov kernels using conditional invertible transforms.
result Ensures nonreversible kernels have the desired invariance property and lead to convergent algorithms.
New method uses PDMPs with sub-sampling for efficient sampling from posterior distributions.
problem Efficient sampling from posterior distributions with limited data access.
method Approximate simulation of PDMPs with sub-sampling and stochastic gradient estimation.
result Stochastic-gradient PDMPs are efficient and robust compared to Langevin dynamics.
Generates random persistence diagrams for data analysis.
problem Generating random persistence diagrams for data analysis.
method Based on pairwise interacting point processes and RJ-MCMC algorithm.
result Demonstrates the efficacy and utility of RPDG in materials science.
Graphical models are popular statistical tools which are used to represent dependent or causal complex systems. Statistically equivalent causal or directed graphical models are said to belong to a Markov equivalent class. It is of great interest to describe and understand the space of such classes. However, with curren…
Flood extent mapping plays a crucial role in disaster management and national water forecasting. Unfortunately, traditional classification methods are often hampered by the existence of noise, obstacles and heterogeneity in spectral features as well as implicit anisotropic spatial dependency across class labels. In thi…
Introduces DF framework for sampling decisions from target distributions.
problem Sampling from target distributions with additional guidance.
method DF framework based on MDP and Path Integral Diffusion.
result DF enhances guided sampling across various applications.
New algorithm learns bridged diffusion processes without time-reversals.
problem Learning bridged diffusion processes efficiently and accurately.
method Score matching with Doob's h-transform, avoiding time-reversals.
result Outperforms existing methods in learning bridged diffusion processes.
Estimates Markov chain mixing time from a single trajectory.
problem Estimating mixing time of Markov chains from a single trajectory.
method Contraction with respect to total variation, inspired by Wolfer's contraction coefficient.
result Improved confidence intervals and instance-dependent rates for estimating Markov chains.
Paper analyzes stability and forgetting in score-based generative models.
problem Understanding the stability and long-time behavior of generative models.
method Quantitative bounds on sampling error using stability and forgetting properties of the Markov chain.
result Provides practical consequences of stability and contraction mechanism in sampling.
This paper simplifies OPE in large state spaces using state abstractions.
problem Accurately evaluating policies offline in large state spaces.
method Developed a backward-model-irrelevance condition and an iterative state abstraction procedure.
result Deeply-abstracted states substantially simplify OPE sample complexity.
Time reversal invariance can be summarized as follows: no difference can be measured if a sequence of events is run forward or backward in time. Because price time series are dominated by a randomness that hides possible structures and orders, the existence of time reversal invariance requires care to be investigated. …