Stochastic gradient Markov Chain Monte Carlo (SG-MCMC) has been developed as a flexible family of scalable Bayesian sampling algorithms. However, there has been little theoretical analysis of the impact of minibatch size to the algorithm's convergence rate. In this paper, we prove that under a limited computational bud…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “MCMC complexity reduction”
New sampler reduces MCMC complexity for Bayesian variable selection.
problem High-dimensional Bayesian variable selection with high computation complexity.
method Variable-complexity subset weighted-Tempered Gibbs Sampler (wTGS) with Rao-Blackwellized estimator.
result Variances of Rao-Blackwellized estimator are smaller than those of subset wTGS.
Optimizes MCMC chains with neural control variates.
problem Reducing variance in Markov Chain Monte Carlo (MCMC) simulations.
method Uses neural networks as control variates to minimize asymptotic variance.
result Derives optimal convergence rate under various ergodicity assumptions.
Stochastic particle-optimization sampling (SPOS) is a recently-developed scalable Bayesian sampling framework that unifies stochastic gradient MCMC (SG-MCMC) and Stein variational gradient descent (SVGD) algorithms based on Wasserstein gradient flows. With a rigorous non-asymptotic convergence theory developed recently…
In this paper we propose a novel variance reduction approach for additive functionals of Markov chains based on minimization of an estimate for the asymptotic variance of these functionals over suitable classes of control variates. A distinctive feature of the proposed approach is its ability to significantly reduce th…
A restricted Boltzmann machine (RBM) is a two-layer neural network with shared weights and has been extensively studied for dimensionality reduction, data representation and recommendation systems in the literature. The traditional RBM requires a probabilistic interpretation of the values on both layers and a Markov ch…
AI-driven framework optimizes MCMC-based preconditioners for faster linear system solving.
problem Slow convergence of Krylov subspace solvers for ill-conditioned matrices.
method Graph neural surrogate and Bayesian optimization for AI-tuned MCMC parameters.
result 50% reduction in iterations to convergence on unseen system.
New MCMC method for complex models with large variables.
problem Inference on posterior model probabilities in large model spaces.
method Reversible genetically modified mode jumping Markov chain Monte Carlo (GMJMCMC).
result Introduced a proper MCMC with correct limiting distribution.
Bayesian neural networks improve uncertainty quantification in non-linear dimensionality reduction.
problem Current neural network models lack adequate uncertainty quantification.
method Deploy Markov chain Monte Carlo sampling algorithms for Bayesian inference in ANN models with latent variables.
result New research directions are needed due to fundamental challenges in neural networks with latent variables.
Practitioners of Bayesian statistics have long depended on Markov chain Monte Carlo (MCMC) to obtain samples from intractable posterior distributions. Unfortunately, MCMC algorithms are typically serial, and do not scale to the large datasets typical of modern machine learning. The recently proposed consensus Monte Car…
A new method reduces complexity of normalizing flows for MCMC preconditioning.
problem Improving sampling efficiency in MCMC algorithms for complex target distributions.
method Factorized preconditioning architecture combining a linear component and a conditional NF.
result Significantly better tail samples and higher effective sample sizes on various distributions.
DPMC improves inverse problem solving with MCMC, reducing error in noisy conditions.
problem Inaccurate posterior approximation in inverse problems with high noise levels.
method DPMC uses Annealed MCMC to sample through a series of intermediate distributions, reducing accumulated error.
result DPMC outperforms DPS in various inverse problems, reducing error and evaluations.
We introduce a doubly stochastic proximal gradient algorithm for optimizing a finite average of smooth convex functions, whose gradients depend on numerically expensive expectations. Our main motivation is the acceleration of the optimization of the regularized Cox partial-likelihood (the core model used in survival an…
New MCMC method learns sparse preconditioner for high-dimensional problems.
problem High-dimensional sampling with complex correlation structures.
method Adaptive MCMC with sparse preconditioner using online PCA.
result Significant reduction in computational complexity and improved performance.
New method uses diffusion models to speed up MCMC sampling.
problem Efficiently exploring high-dimensional and multimodal posterior functions.
method Combines Metropolis-Hastings with diffusion models for global sampling.
result Significant reduction in likelihood evaluations for accurate posterior representation.
It is well known that Markov chain Monte Carlo (MCMC) methods scale poorly with dataset size. A popular class of methods for solving this issue is stochastic gradient MCMC. These methods use a noisy estimate of the gradient of the log posterior, which reduces the per iteration computational cost of the algorithm. Despi…
KSD Thinning uses KSD to thin MCMC samples efficiently.
problem Efficiently representing posterior distributions in Bayesian inference.
method KSD Thinning: retains only samples exceeding a KSD threshold.
result Established convergence and complexity tradeoffs for KSD Thinning.
Langevin MCMC samples efficiently from Riemannian manifolds with geometric Euler-Murayama analysis.
problem Efficient sampling from Gibbs distributions on Riemannian manifolds.
method Geometric Langevin MCMC, discretization error bound, contraction guarantee for Langevin Diffusion.
result Langevin MCMC iterates converge to the target distribution after a number of steps proportional to the inverse square of the desired accuracy.
AutoStep MCMC adapts step size locally for better sampling efficiency.
problem Challenging step size selection for complex, multiscale targets.
method AutoStep MCMC uses a locally adaptive step size for involutive proposals.
result AutoStep MCMC is π-invariant, irreducible, and aperiodic.
This paper develops tools for nonreversible MCMC with convergence guarantees.
problem Designing nonreversible MCMC kernels with convergence guarantees.
method Develops tools for nonreversible Markov kernels using conditional invertible transforms.
result Ensures nonreversible kernels have the desired invariance property and lead to convergent algorithms.
An ensemble of neural networks is known to be more robust and accurate than an individual network, however usually with linearly-increased cost in both training and testing. In this work, we propose a two-stage method to learn Sparse Structured Ensembles (SSEs) for neural networks. In the first stage, we run SG-MCMC wi…
Adaptive stopping in MCMC using classifier-based dynamics
problem Sampling from complex, unnormalized probability densities
method Training state-dependent neural classifiers
result Significant reduction in average trajectory lengths
The posteriors over neural network weights are high dimensional and multimodal. Each mode typically characterizes a meaningfully different representation of the data. We develop Cyclical Stochastic Gradient MCMC (SG-MCMC) to automatically explore such distributions. In particular, we propose a cyclical stepsize schedul…
Many Markov Chain Monte Carlo (MCMC) methods leverage gradient information of the potential function of target distribution to explore sample space efficiently. However, computing gradients can often be computationally expensive for large scale applications, such as those in contemporary machine learning. Stochastic Gr…
MCMC complexity matches optimization for large and .
problem Lack of theoretical understanding of MCMC complexity for large and .
method Comparison of MCMC, LA, and VI complexities for linear, logistic, and Poisson regression.
result MCMC complexity matches optimization complexity for .
We propose a novel approximate inference algorithm that approximates a target distribution by amortising the dynamics of a user-selected MCMC sampler. The idea is to initialise MCMC using samples from an approximation network, apply the MCMC operator to improve these samples, and finally use the samples to update the a…
New analysis of SGD with MCMC gradient estimator shows convergence rate and saddle point escape.
problem Analyzing SGD with MCMC gradient estimator under complex conditions.
method Introduced MCMC-SGD, analyzed convergence rate and saddle point escape using Bernstein inequality.
result Proven first order convergence rate and saddle point escape at least steps.
New model improves MCMC efficiency and multi-modal distribution exploration.
problem Inefficient and slow MCMC methods for complex distributions.
method Deep involutive generative models for Metropolis-Hastings updates.
result Deep involutive models can learn complex MCMC updates efficiently.
A new Monte Carlo sampling method derived from reverse diffusion.
problem Sampling from complex distributions, especially multi-modal ones.
method Transforming score matching into mean estimation; estimating means of regularized posterior distributions.
result rdMC can approximate sampling with any desired accuracy and is significantly faster than MCMC for complex distributions.
PolytopeWalk library efficiently samples high-dimensional polytopes.
problem Sampling from high-dimensional polytopes efficiently.
method End-to-end solution including preprocessing and MCMC algorithms.
result Improved sampling efficiency and scalability to high dimensions.
Data assimilation for subsurface flow using latent diffusion models shows that ensemble Kalman methods may overestimate posterior uncertainty, while Monte Carlo sampling is more reliable.
problem Data assimilation for subsurface flow
method Ensemble Kalman smoother and Markov chain Monte Carlo sampling
result Monte Carlo sampling is more reliable than ensemble Kalman methods
New MCMC method corrects bias without extra cost.
problem Correcting bias in MCMC algorithms without additional computational cost.
method Generalized Markov Chain Importance Sampling methods.
result Proposed methods are more efficient than Metropolis-Hastings versions.
In this paper we propose an efficient variance reduction approach for additive functionals of Markov chains relying on a novel discrete time martingale representation. Our approach is fully non-asymptotic and does not require the knowledge of the stationary distribution (and even any type of ergodicity) or specific str…
Bayesian method refines surrogate models for accurate full waveform inversion.
problem Complex input/output relations in full waveform inversion make accurate surrogate models difficult.
method Iterative refinement of surrogate models using MCMC samples and progressively expanding frequency bandwidth.
result Highly accurate surrogate model across full bandwidth enables accurate final MCMC inversion.
Centered plug-in estimators reduce bias in Wasserstein distance estimation.
problem Conservative bias in plug-in estimators of Wasserstein distances.
method Centering procedure based on linear combinations to reduce bias.
result Centered plug-in estimators provide informative upper and lower bounds on Wasserstein distances.
A new method de-randomizes MCMC dynamics using the Stein operator.
problem Estimating complex target distributions in Bayesian inference.
method De-randomized kernel-based particle samplers that discretize the fiber-gradient Hamiltonian flow.
result GSVGD de-randomizes complex MCMC dynamics, maintaining high sample quality.
New method for MCMC models without perfect or sequential samplers.
problem Bayesian inference for complex models with intractable terms.
method Utilizes tractable independence model to construct unbiased estimates.
result Scalable method for high-dimensional models.
Estimates covariance matrices using Markov chain Monte Carlo with improved sample complexity.
problem Complexity of covariance matrix estimation for Gibbs distributions.
method Uses Markov chain Monte Carlo with conditions on the chain's spectral gap and Poincaré inequality.
result Achieves similar sample complexity as i.i.d. samples with better query complexity.
Improved modeling of persistence diagrams for data analysis.
problem Determining significant outliers in persistence diagrams.
method Modification of the RST (Replicating Statistical Topology) model using MCMC Metropolis-Hastings algorithm.
result The modified RST model improves the goodness of fit in persistence diagram analysis.
We propose a generic Markov Chain Monte Carlo (MCMC) algorithm to speed up computations for datasets with many observations. A key feature of our approach is the use of the highly efficient difference estimator from the survey sampling literature to estimate the log-likelihood accurately using only a small fraction of …
Recent advances in stochastic gradient techniques have made it possible to estimate posterior distributions from large datasets via Markov Chain Monte Carlo (MCMC). However, when the target posterior is multimodal, mixing performance is often poor. This results in inadequate exploration of the posterior distribution. A…
We study the problem of sampling from a distribution , where the function is -smooth everywhere and -strongly convex outside a ball of radius , but potentially nonconvex inside this ball. We study both overdamped and underdamped Langevin MCMC and establish upper bound…
NEO combines orbits to sample and estimate complex distributions.
problem Sampling and normalizing complex distributions.
method NEO-IS and NEO-MCMC using invertible maps and orbits.
result NEO provides unbiased estimators and explores multimodal targets.
Fast Doubly-Adaptive MCMC to Estimate the Gibbs Partition Function with Weak Mixing Time Boundsstat.ML
New method reduces Gibbs partition function estimation complexity.
problem Estimating partition functions of Gibbs distributions.
method Doubly-adaptive MCMC with adaptive cooling schedule and mean estimator.
result Outperforms state-of-the-art algorithms in computational complexity and robustness.
Unified framework for MCMC algorithms simplifies their design and application.
problem Diverse MCMC algorithms with varying principles and applications.
method Involutive MCMC (iMCMC) framework unifying MCMC approaches.
result Unified view of MCMC algorithms facilitates the development of new and more efficient algorithms.
New MCMC algorithm reduces subset selection passes to 2 for optimal -dimensional subspace approximation.
problem Subset selection for -dimensional subspace approximation with -approximation.
method MCMC sampling algorithm reducing passes to 2 for case, poly(k/ε) size subset.
result Subset selection of nearly optimal size in 2 passes, approximation.
Performing exact Bayesian inference for complex models is computationally intractable. Markov chain Monte Carlo (MCMC) algorithms can provide reliable approximations of the posterior distribution but are expensive for large datasets and high-dimensional models. A standard approach to mitigate this complexity consists i…
Markov Chain Monte Carlo methods become increasingly popular in applied mathematics as a tool for numerical integration with respect to complex and high-dimensional distributions. However, application of MCMC methods to heavy tailed distributions and distributions with analytically intractable densities turns out to be…