Nonlinear MCMC improves Bayesian machine learning sampling.
problem Sampling problems in Bayesian machine learning.
method Nonlinear MCMC technique with convergence guarantees.
result Improves sampling in Bayesian neural networks.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “MCMC sampling”
The rapid development of computing power and efficient Markov Chain Monte Carlo (MCMC) simulation algorithms have revolutionized Bayesian statistics, making it a highly practical inference method in applied work. However, MCMC algorithms tend to be computationally demanding, and are particularly slow for large datasets…
PL-MCMC samples from normalizing flows' conditional distributions.
problem Sampling from complex conditional distributions learned by normalizing flows.
method Metropolis-Hastings implementation of PL-MCMC.
result PL-MCMC asymptotically samples from exact conditional distributions.
This study investigates the effects of Markov chain Monte Carlo (MCMC) sampling in unsupervised Maximum Likelihood (ML) learning. Our attention is restricted to the family of unnormalized probability densities for which the negative log density (or energy function) is a ConvNet. We find that many of the techniques used…
There has been recent interest in developing scalable Bayesian sampling methods such as stochastic gradient MCMC (SG-MCMC) and Stein variational gradient descent (SVGD) for big-data analysis. A standard SG-MCMC algorithm simulates samples from a discrete-time Markov chain to approximate a target distribution, thus samp…
This paper tackles sampling issues in latent space EBMs by introducing diffusion-based amortization.
problem Degenerate MCMC sampling quality hinders latent space EBM learning and generation quality.
method Introduces diffusion-based amortization for long-run MCMC sampling.
result The learned amortization of MCMC is a valid long-run MCMC sampler.
Paper proposes a method to improve MCMC sampling for energy-based models.
problem MCMC sampling of energy-based models is often not mixing in high-dimensional data.
method Proposes using a flow-based model as a backbone to correct the energy-based model, enabling mixing in latent space.
result MCMC sampling of the corrected EBM in the latent space mixes well and traverses modes in the data space.
Neural network MCMC sampler maximizes proposal entropy for efficient sampling.
problem Inefficient sampling from complex probability distributions.
method Proposes a neural network MCMC sampler that maximizes proposal entropy.
result Significantly higher efficiency in various sampling tasks.
Bayesian neural networks tutorial via MCMC in Python.
problem Bayesian inference for parameter estimation and uncertainty quantification in deep learning models.
method MCMC sampling methods to implement Bayesian inference, including advanced proposal distributions.
result Challenges in sampling multi-modal posterior distributions for Bayesian neural networks.
Stochastic gradient Markov chain Monte Carlo (SG-MCMC) has been increasingly popular in Bayesian learning due to its ability to deal with large data. A standard SG-MCMC algorithm simulates samples from a discretized-time Markov chain to approximate a target distribution. However, the samples are typically highly correl…
We propose a novel approximate inference algorithm that approximates a target distribution by amortising the dynamics of a user-selected MCMC sampler. The idea is to initialise MCMC using samples from an approximation network, apply the MCMC operator to improve these samples, and finally use the samples to update the a…
Sampling from posterior distributions using Markov chain Monte Carlo (MCMC) methods can require an exhaustive number of iterations, particularly when the posterior is multi-modal as the MCMC sampler can become trapped in a local mode for a large number of iterations. In this paper, we introduce the pseudo-extended MCMC…
EBM life cycle project improves MCMC for image generation, defense, and density modeling.
problem Improving MCMC for diverse EBM applications.
method Three novel MCMC initialization methods for negative samples.
result State-of-the-art performance gains across three applications.
Markov chain Monte Carlo (MCMC) methods are widely used in machine learning. One of the major problems with MCMC is the question of how to design chains that mix fast over the whole state space; in particular, how to select the parameters of an MCMC algorithm. Here we take a different approach and, similarly to paralle…
This paper proposes a method to train energy-based models using variational auto-encoders for efficient sampling.
problem Training energy-based models by maximum likelihood is challenging due to intractable partition functions and difficult sampling from the model distribution.
method The authors propose using a variational auto-encoder to initialize finite-step MCMC sampling, specifically Langevin dynamics, to train the energy-based model.
result The proposed method enables training energy-based models using maximum likelihood, generating samples comparable to GANs and EBMs.
Paper proposes first unlearning algorithm for MCMC models.
problem Enforcing right to be forgotten in AI causes high costs for data deletion.
method Converts MCMC unlearning to explicit optimization problem, designs MCMC influence function.
result MCMC unlearning does not compromise generalizability of models.
Markov chain Monte Carlo (MCMC) algorithms are widely used to sample from complicated distributions, especially to sample from the posterior distribution in Bayesian inference. However, MCMC is not directly applicable when facing the doubly intractable problem. In this paper, we discussed and compared two existing solu…
On Cyclical MCMC Samplingstat.CO
Cyclical MCMC tackles high-dimensional multimodal distributions, showing convergence under certain conditions.
problem High-dimensional multimodal posterior distributions in deep learning.
method Cyclical MCMC framework that tracks tempered versions of the target distribution over time.
result Cyclical MCMC converges to the target distribution under fast mixing kernels but fails in slow mixing cases.
This work optimizes MCMC algorithms for modern accelerators without synchronization overheads.
problem Synchronization overheads in vectorized MCMC algorithms.
method Designing single-chain MCMC algorithms as finite state machines (FSMs) to avoid synchronization.
result Achieves up to an order of magnitude speed-up in experiments.
Integration of Active Learning and MCMC Sampling for Efficient Bayesian Calibration of Mechanical Propertiesphysics.comp-ph
Study improves Bayesian calibration of mechanical properties using active learning and MCMC.
problem Inference of spatially varying material parameters in computational mechanics.
method Comprehensive comparative study of surrogate models and MCMC algorithms.
result Active learning strategy outperforms a priori trained models in posterior estimation.
New MCMC method corrects bias without extra cost.
problem Correcting bias in MCMC algorithms without additional computational cost.
method Generalized Markov Chain Importance Sampling methods.
result Proposed methods are more efficient than Metropolis-Hastings versions.
The paper proposes methods to estimate MCMC quality with couplings, bounding Wasserstein distance.
problem Improving MCMC efficiency without sacrificing asymptotic consistency.
method Estimators based on couplings of Markov chains to assess quality of asymptotically biased sampling methods.
result Empirical upper bounds of Wasserstein distance for assessing MCMC quality.
A new Monte Carlo sampling method derived from reverse diffusion.
problem Sampling from complex distributions, especially multi-modal ones.
method Transforming score matching into mean estimation; estimating means of regularized posterior distributions.
result rdMC can approximate sampling with any desired accuracy and is significantly faster than MCMC for complex distributions.
KSD Thinning uses KSD to thin MCMC samples efficiently.
problem Efficiently representing posterior distributions in Bayesian inference.
method KSD Thinning: retains only samples exceeding a KSD threshold.
result Established convergence and complexity tradeoffs for KSD Thinning.
New analysis of SGD with MCMC gradient estimator shows convergence rate and saddle point escape.
problem Analyzing SGD with MCMC gradient estimator under complex conditions.
method Introduced MCMC-SGD, analyzed convergence rate and saddle point escape using Bernstein inequality.
result Proven first order convergence rate and saddle point escape at least steps.
We propose Subsampling MCMC, a Markov Chain Monte Carlo (MCMC) framework where the likelihood function for observations is estimated from a random subset of observations. We introduce a highly efficient unbiased estimator of the log-likelihood based on control variates, such that the computing cost is much smal…
New method improves performance of Hamiltonian MCMC for log Z estimation.
problem Estimating tight bounds on log Z for unnormalized distributions.
method Uncorrected Hamiltonian Annealing (UHA) using reparameterization gradients.
result Better performance and easier parameter tuning compared to existing methods.
AutoStep MCMC adapts step size locally for better sampling efficiency.
problem Challenging step size selection for complex, multiscale targets.
method AutoStep MCMC uses a locally adaptive step size for involutive proposals.
result AutoStep MCMC is π-invariant, irreducible, and aperiodic.
Monte Carlo (MC) methods are widely used for Bayesian inference and optimization in statistics, signal processing and machine learning. A well-known class of MC methods are Markov Chain Monte Carlo (MCMC) algorithms. In order to foster better exploration of the state space, specially in high-dimensional applications, s…
Langevin MCMC samples efficiently from Riemannian manifolds with geometric Euler-Murayama analysis.
problem Efficient sampling from Gibbs distributions on Riemannian manifolds.
method Geometric Langevin MCMC, discretization error bound, contraction guarantee for Langevin Diffusion.
result Langevin MCMC iterates converge to the target distribution after a number of steps proportional to the inverse square of the desired accuracy.
Unified framework for MCMC algorithms simplifies their design and application.
problem Diverse MCMC algorithms with varying principles and applications.
method Involutive MCMC (iMCMC) framework unifying MCMC approaches.
result Unified view of MCMC algorithms facilitates the development of new and more efficient algorithms.
New MCMC method improves sampling efficiency across diverse structural models.
problem Low sampling efficiency in generic MCMC methods for specific problems.
method Adaptive Principal-Component (PC) Meta-learning Stochastic Gradient Hamiltonian Monte Carlo (APM-SGHMC) algorithm.
result Universal samplers achieve zero-shot generalization across structurally distinct models.
A new method reduces complexity of normalizing flows for MCMC preconditioning.
problem Improving sampling efficiency in MCMC algorithms for complex target distributions.
method Factorized preconditioning architecture combining a linear component and a conditional NF.
result Significantly better tail samples and higher effective sample sizes on various distributions.
Develops variational inference for Neyman-Scott processes for faster sampling.
problem Slow mixing time in MCMC for posterior sampling in Neyman-Scott processes.
method Variational inference algorithm for Neyman-Scott processes, minimizing KL divergence.
result Achieves better prediction performance than MCMC with limited computational time.
HDT improves MCMC on graphs with history-dependent sampling.
problem Efficient sampling from target distributions on general graphs with low computational overhead.
method History-driven target (HDT) framework that replaces the original target distribution with a history-dependent one.
result Near-zero variance performance and scalability to large graphs with memory-efficient implementation.
Wide Bayesian neural networks have a simple weight posterior: theory and accelerated samplingstat.ML
Wide Bayesian neural networks have a simpler weight posterior, leading to faster MCMC sampling.
problem Sampling from the posterior of wide Bayesian neural networks is challenging.
method Introducing repriorisation, a data-dependent reparameterisation that simplifies the posterior distribution.
result The repriorisation map accelerates MCMC sampling, achieving up to 50x higher effective sample size.
Accelerates MCMC sampling for large-scale problems using machine learning.
problem Efficiently sampling large-scale Bayesian inference problems with high computational cost.
method Integrates low-fidelity machine learning models into a multilevel MCMC framework.
result Significantly accelerates multilevel sampling by a factor of two with similar accuracy.
Paper proposes an alternative to MCMC for sampling in energy-based models.
problem Difficulty in generating samples from the current energy function in contrastive approaches.
method Viewing the evolution of the modeling distribution as the evolution of the energy function and samples from this distribution along a time-dependent vector field.
result The proposed method efficiently matches the current distribution in a finite time, unlike MCMC.
New method merges MCMC samples without distributional assumptions.
problem Efficiently merging MCMC samples from disjoint subsets.
method Diffusion generative modelling for density approximation.
result Outperforms existing methods on high-dimensional problems.
This review article surveys data augmentation MCMC algorithms.
problem Sampling from intractable probability distributions.
method Comprehensive study of DA MCMC algorithms, their convergence properties, and acceleration strategies.
result Synthesizes recent developments and provides insights for researchers.
BiDAG R package learns and samples Bayesian network structures efficiently.
problem Efficiently learning and sampling Bayesian network structures.
method Hybrid approach combining PC algorithm, iterative order MCMC, and partition MCMC.
result BiDAG can handle both discrete and continuous data.
New method uses diffusion models to speed up MCMC sampling.
problem Efficiently exploring high-dimensional and multimodal posterior functions.
method Combines Metropolis-Hastings with diffusion models for global sampling.
result Significant reduction in likelihood evaluations for accurate posterior representation.
JaxSGMC simplifies SG-MCMC for Bayesian deep learning.
problem Uncertainty quantification in deep learning models.
method Modular stochastic gradient MCMC in JAX.
result Facilitates trustworthy neural network predictions.
LSB is a new MCMC method for discrete spaces that reduces target evaluations.
problem Sampling in discrete domains with high efficiency and adaptability.
method Local self-balancing proposals, mutual information objective, self-balancing learning.
result LSB converges with fewer target evaluations compared to existing methods.
DPMC improves inverse problem solving with MCMC, reducing error in noisy conditions.
problem Inaccurate posterior approximation in inverse problems with high noise levels.
method DPMC uses Annealed MCMC to sample through a series of intermediate distributions, reducing accumulated error.
result DPMC outperforms DPS in various inverse problems, reducing error and evaluations.
The hierarchical Dirichlet process (HDP) has become an important Bayesian nonparametric model for grouped data, such as document collections. The HDP is used to construct a flexible mixed-membership model where the number of components is determined by the data. As for most Bayesian nonparametric models, exact posterio…
Variational inference (VI) and Markov chain Monte Carlo (MCMC) are two main approximate approaches for learning deep generative models by maximizing marginal likelihood. In this paper, we propose using annealed importance sampling for learning deep generative models. Our proposed approach bridges VI with MCMC. It gener…
Speeding up Markov Chain Monte Carlo (MCMC) for datasets with many observations by data subsampling has recently received considerable attention. A pseudo-marginal MCMC method is proposed that estimates the likelihood by data subsampling using a block-Poisson estimator. The estimator is a product of Poisson estimators,…