Paper proposes a method to improve MCMC sampling for energy-based models.
problem MCMC sampling of energy-based models is often not mixing in high-dimensional data.
method Proposes using a flow-based model as a backbone to correct the energy-based model, enabling mixing in latent space.
result MCMC sampling of the corrected EBM in the latent space mixes well and traverses modes in the data space.
Particle MCMC involves using a particle filter within an MCMC algorithm. For inference of a model which involves an unobserved stochastic process, the standard implementation uses the particle filter to propose new values for the stochastic process, and MCMC moves to propose new values for the parameters. We show how p…
Cyclical MCMC tackles high-dimensional multimodal distributions, showing convergence under certain conditions.
problem High-dimensional multimodal posterior distributions in deep learning.
method Cyclical MCMC framework that tracks tempered versions of the target distribution over time.
result Cyclical MCMC converges to the target distribution under fast mixing kernels but fails in slow mixing cases.
We propose a stochastic gradient Markov chain Monte Carlo (SG-MCMC) algorithm for scalable inference in mixed-membership stochastic blockmodels (MMSB). Our algorithm is based on the stochastic gradient Riemannian Langevin sampler and achieves both faster speed and higher accuracy at every iteration than the current sta…
Develops fast Markov chain samplers for constrained probability measures.
problem Sampling from constrained probability measures efficiently.
method Fast Markov chain samplers for Strongly Rayleigh (SR) measures and probabilistic models with hard constraints.
result Sharp polynomial bounds on the mixing time for SR measures, leading to fast MCMC samplers for DPPs.
New MCMC methods map high-dimensional problems to spheres for better mixing.
problem Mixing issues in high-dimensional distributions, especially heavy-tailed ones.
method Stereographic Markov Chain Monte Carlo (MCMC) methods that map high-dimensional problems to spheres.
result Uniformly ergodic samplers for various distributions, including heavy-tailed ones, with faster convergence in higher dimensions.
Lower bound on BART's mixing time increases with data points.
problem Slow mixing time in BART's MCMC chains.
method Simplified BART with a single tree and reduced MCMC moves.
result Mixing time grows exponentially with data points.
New MCMC algorithms speed up sampling from polytope distributions.
problem Sampling from uniform distributions over polytopes efficiently.
method Vaidya walk and John walk based on interior point methods.
result Vaidya walk mixes significantly faster than Dikin walk.
This work improves mixing rates for Bayesian CART, a key component of BART.
problem Understanding and improving mixing rates for Bayesian inference with MCMC.
method Derived upper bounds on mixing times, provided sufficient conditions for polynomial mixing, and proposed Twiggy Bayesian CART.
result Twiggy Bayesian CART achieves polynomial mixing without assuming signal connectivity.
New MCMC algorithm improves convergence of Bayesian regression trees.
problem Local mode stickiness and poor mixing in MCMC algorithms for Bayesian regression trees.
method Continuous-time birth-death MCMC algorithm for Bayesian regression tree models.
result The new algorithm dramatically improves convergence and mixing properties of MCMC.
Improves MCMC efficiency with magnetic HMC.
problem Improving Markov chain Monte Carlo sampling efficiency.
method Integrates non-canonical Hamiltonian dynamics into HMC.
result Non-canonical dynamics can lead to better sampling.
metabeta uses neural networks to speed up Bayesian mixed-effects regression.
problem Bayesian mixed-effects regression is computationally expensive.
method metabeta is a neural network model that pre-trains to estimate posterior distributions.
result metabeta achieves comparable performance to MCMC at a fraction of the time.
Improved sampling for multimodal posterior distributions using stochastic gradient methods.
problem Inadequate exploration of multimodal posterior distributions in stochastic gradient MCMC.
method Proposes a generalized kinetic function for Hamiltonian Monte Carlo to enhance mixing in stochastic gradient MCMC.
result Demonstrates superior exploration of complex multimodal posterior distributions.
Probabilistic models are conceptually powerful tools for finding structure in data, but their practical effectiveness is often limited by our ability to perform inference in them. Exact inference is frequently intractable, so approximate inference is often performed using Markov chain Monte Carlo (MCMC). To achieve the…
New algorithm speeds up sampling from complex Bayesian mixture models.
problem Sampling from non-log-concave, multi-modal posterior distributions in Bayesian Gaussian mixtures.
method Introduced Reflected Metropolis-Hastings Random Walk (RMRW) algorithm.
result Proved mixing time bound for RMRW in symmetric two-component Gaussian mixtures.
VB methods improve MMNL estimation speed and accuracy.
problem Scalable Bayesian estimation of MMNL models.
method Extending VB methods to include both fixed and random utility parameters, and conducting extensive simulations.
result VB methods, especially VB-NCVMP-Delta, are up to 16 times faster than MCMC and MSLE while maintaining similar accuracy.
Adaptive stopping in MCMC using classifier-based dynamics
problem Sampling from complex, unnormalized probability densities
method Training state-dependent neural classifiers
result Significant reduction in average trajectory lengths
New MCMC methods improve sampling efficiency and mixing.
problem Improving sampling efficiency and mixing in MCMC methods.
method Sequential proposals in MCMC methods, including HMC and bouncy particle sampler.
result New methods improve numerical efficiency and mixing in multimodal distributions.
New method improves MCMC for large data problems with better mixing rates.
problem Poor mixing rates in vanilla SG-MCMC for strong couplings and scale differences.
method Uses Quasi-Newton optimization ideas to approximate Hessian and scale linearly.
result Asymptotically unbiased and consistent with posterior expectations, achieving fast convergence rates.
Improved MCMC using BP for faster and more accurate inference in graphical models.
problem Combining MCMC and BP for faster and more accurate inference in graphical models.
method Proposed a polynomial-time approximation MCMC scheme using the Worm algorithm and cycle basis.
result The proposed MCMC schemes outperform direct MCMC and bare BP schemes.
New split-merge MCMC proposals improve efficiency and speed.
problem Scaling issues in split-merge MCMC for large datasets.
method Locality Sensitive Sampling (LSS) combined with weighted MinHash.
result Significantly faster than state-of-the-art methods on large datasets.
Improves MCMC sampling for multi-modal distributions.
problem Inefficient MCMC sampling in multi-modal posterior distributions.
method Pseudo-extended MCMC method using auxiliary variables.
result Improved MCMC sampling over Hamiltonian Monte Carlo.
MMM model clusters mixed-type longitudinal data efficiently.
problem Challenges in clustering multivariate longitudinal mixed-type data.
method MMM model reorganizes data into a three-way structure, using a mixture of matrix-variate normal distributions.
result MMM model handles various data types (continuous, ordinal, binary, nominal, count) and temporal dependence.
The paper explores a non-convergent MCMC method for EBM learning.
problem Learning energy-based models using traditional methods is challenging.
method The paper uses a non-convergent, non-mixing, and non-persistent short-run MCMC to learn EBM parameters.
result The learned short-run MCMC can generate realistic images and reconstruct/interpolate between images.
We study the computational complexity of Markov chain Monte Carlo (MCMC) methods for high-dimensional Bayesian linear regression under sparsity constraints. We first show that a Bayesian approach can achieve variable-selection consistency under relatively mild conditions on the design matrix. We then demonstrate that t…
MixFlows uses a mixture of flows for efficient variational inference.
problem Efficient and reliable variational inference for complex models.
method A new variational family of mixed flows with efficient algorithms and convergence guarantees.
result MixFlows provides more reliable posterior approximations and comparable sample quality to MCMC methods.
The hierarchical Dirichlet process (HDP) has become an important Bayesian nonparametric model for grouped data, such as document collections. The HDP is used to construct a flexible mixed-membership model where the number of components is determined by the data. As for most Bayesian nonparametric models, exact posterio…
New bounds for SMC show its advantage over MCMC in multimodal distributions.
problem Estimating expectations under multimodal distributions with slow global mixing.
method Proves finite sample complexities for SMC with local mixing times, addressing bias through sequential resampling.
result SMC provides fully polynomial time approximation for multimodal problems.
New method uses RRTs to explore Bayesian NMF more effectively.
problem Slow mixing and mode sticking in Bayesian NMF.
method Rapidly exploring random trees (RRTs) in a Bayesian framework.
result Greater coverage of the posterior and higher ELBO values.
The paper proposes an efficient method to scale Bayesian inference for mixed multinomial logit models to very large datasets.
problem Efficiency in Bayesian inference for mixed multinomial logit models on large datasets.
method Amortized Variational Inference with stochastic backpropagation, automatic differentiation, and GPU acceleration.
result The proposed method achieves significant computational speedups over traditional methods for large datasets.
New bounds for MCMC on discrete spaces without dimension dependence.
problem High-dimensional statistical convergence analysis of MCMC methods.
method Combining multicommodity flow and single-element drift conditions.
result Informed Metropolis-Hastings algorithms achieve relaxation times independent of dimension.
The paper improves SMC algorithm for multi-modal distributions by proving variance bounds.
problem Problems with SMC on multi-modal distributions, especially in terms of mixing time.
method Proves variance bounds for SMC on multi-modal distributions using soft decomposition.
result Bounds on SMC variance depend on local rather than global mixing times.
This paper proposes a new randomized strategy for adaptive MCMC using Bayesian optimization. This approach applies to non-differentiable objective functions and trades off exploration and exploitation to reduce the number of potentially costly objective function evaluations. We demonstrate the strategy in the complex s…
We present an efficient algorithm for the inference of stochastic block models in large networks. The algorithm can be used as an optimized Markov chain Monte Carlo (MCMC) method, with a fast mixing time and a much reduced susceptibility to getting trapped in metastable states, or as a greedy agglomerative heuristic, w…
Combines local MCMC chains to speed up sampling.
problem Designing efficient MCMC chains that mix well over the whole state space.
method Combining parallel chains prioritized by kernel Stein discrepancy, combining samples using novel probability estimation.
result Significant speedups in sampling from multimodal distributions.
Enhances sampling for complex hidden Markov models using ensemble MCMC.
problem Challenges in Bayesian inference for factorial hidden Markov models due to large latent variable space.
method Introduces ensemble MCMC with parallel tempering and genetic algorithm for efficient exploration.
result Improves sampling efficiency and mixing of existing samplers in various applications.
Develops variational inference for Neyman-Scott processes for faster sampling.
problem Slow mixing time in MCMC for posterior sampling in Neyman-Scott processes.
method Variational inference algorithm for Neyman-Scott processes, minimizing KL divergence.
result Achieves better prediction performance than MCMC with limited computational time.
Semi-Implicit Variational Inference broadens variational distributions.
problem Expanding the variational family to include implicit distributions.
method Mixing variational parameters with a flexible distribution.
result SIVI provides an asymptotically exact surrogate ELBO for optimization.
New method reduces Gibbs partition function estimation complexity.
problem Estimating partition functions of Gibbs distributions.
method Doubly-adaptive MCMC with adaptive cooling schedule and mean estimator.
result Outperforms state-of-the-art algorithms in computational complexity and robustness.
New method for MCMC models without perfect or sequential samplers.
problem Bayesian inference for complex models with intractable terms.
method Utilizes tractable independence model to construct unbiased estimates.
result Scalable method for high-dimensional models.
New MCMC method tackles label-switching problem for clustering.
problem Label-switching problem impedes MCMC efficiency in discrete clustering.
method Formulated optimal transport couplings for partition space.
result Our method efficiently overcomes label-switching problem.
New algorithms sample random graph homomorphisms for network analysis.
problem Sampling random graph homomorphisms from a graph into a large network.
method Proposed two MCMC algorithms with bounds on mixing times and concentration.
result Network observables are stable under renormalized cut distance.
Wide Bayesian neural networks have a simpler weight posterior, leading to faster MCMC sampling.
problem Sampling from the posterior of wide Bayesian neural networks is challenging.
method Introducing repriorisation, a data-dependent reparameterisation that simplifies the posterior distribution.
result The repriorisation map accelerates MCMC sampling, achieving up to 50x higher effective sample size.
New algorithm speeds Bayesian nonparametric model inference.
problem Slow inference in Bayesian nonparametric models.
method Decompose random measures into finite and infinite sub-measures; use different algorithms for each.
result Hybrid algorithm improves scalability and mixing.
Paper proposes scalable method for analyzing multi-omic data.
problem Integrating high-dimensional multi-omic data for cancer subtyping.
method Mixed graphical model approach using Birth-Death MCMC algorithm.
result Our method outperforms LASSO and standard BDMCMC in computational efficiency and model selection accuracy.
Inference is typically intractable in high-treewidth undirected graphical models, making maximum likelihood learning a challenge. One way to overcome this is to restrict parameters to a tractable set, most typically the set of tree-structured parameters. This paper explores an alternative notion of a tractable set, nam…
MCMC methods for sampling from the space of DAGs can mix poorly due to the local nature of the proposals that are commonly used. It has been shown that sampling from the space of node orders yields better results [FK03, EW06]. Recently, Koivisto and Sood showed how one can analytically marginalize over orders using dyn…
This paper proposes a method to train energy-based models using variational auto-encoders for efficient sampling.
problem Training energy-based models by maximum likelihood is challenging due to intractable partition functions and difficult sampling from the model distribution.
method The authors propose using a variational auto-encoder to initialize finite-step MCMC sampling, specifically Langevin dynamics, to train the energy-based model.
result The proposed method enables training energy-based models using maximum likelihood, generating samples comparable to GANs and EBMs.