Study on Metropolis-within-Gibbs schemes for high-dimensional Bayesian models.
problem Improving the scalability of MCMC methods for complex Bayesian models.
method Relating convergence properties to conditional conductance for non-conjugate hierarchical models.
result Established dimension-free convergence results for Metropolis-within-Gibbs schemes.
Improved spectral gap for MwG with adaptive RWM proposals.
problem Improving mixing efficiency of MwG for log-concave distributions.
method Using adaptive RWM proposals tuned to match conditional variances of log-concave target distributions.
result Established a spectral gap lower bound of order O(1/κd) for MwG. New MCMC methods improve efficiency for large network inference.
problem Efficiency of Metropolis within Gibbs for large networks.
method Combination of split Hamiltonian Monte Carlo and Firefly Monte Carlo.
result New methods outperform Metropolis within Gibbs on synthetic and real networks.
DiGS improves sampling from multi-modal distributions.
problem Inadequate mixing in MCMC methods for multi-modal distributions.
method Integrates diffusion models and Gibbs sampling to create an auxiliary noisy distribution.
result DiGS exhibits better mixing for multi-modal distributions than state-of-the-art methods.
This paper improves conditional sampling for VAEs by overcoming structural issues.
problem Computational intractability of conditional sampling in VAEs.
method Proposes two methods to address pitfalls in Metropolis-within-Gibbs (MWG) for VAEs.
result Improved performance on sampling tasks.
HiSS sampling overcomes local mode traps in rugged discrete spaces.
problem Sampling multimodal discrete distributions with gradient-based methods.
method Integrates Metropolis-within-Gibbs framework with logistic convolution.
result HiSS outperforms alternatives on various tasks, including Ising models and binary neural networks.
Gibbs sampler contracts entropy under strong log-concavity, improving mixing time.
problem Improving the mixing time of Gibbs sampler under strong log-concavity.
method Analyzing Gibbs sampler contraction under strong log-concavity, providing sharp contraction rate.
result Gibbs sampler contracts entropy linearly with condition number and independent of dimension under strong log-concavity.
Proposes PG-DA for Bayesian MMNL estimation to handle non-conjugacy.
problem Non-conjugacy in the Bayesian estimation of MMNL models.
method Pólygamma data augmentation technique applied to MMNL estimation.
result Similar posterior estimates for binary choice scenarios, but empirical identification issues for J≥3 alternatives. Bayesian framework clusters time series with nonlinear dynamics.
problem Identify subsets of neurons responding similarly to stimuli.
method Dirichlet process mixture of nonlinear state-space models, Metropolis-within-Gibbs algorithm, particle-based methods.
result Framework successfully clusters time series from mouse prefrontal cortex.
LIC compiles probabilistic models to generate efficient MCMC proposals.
problem Creating accurate Metropolis-Hastings proposals for Bayesian inference.
method Integrates probabilistic graphical models and neural networks in an open-source framework to optimize proposal distributions.
result LIC produces more efficient and robust MCMC proposals compared to existing methods.
Combines MALA and Adam for efficient uncertainty quantification in deep learning.
problem Uncertainty estimation in deep neural networks.
method Integrates Metropolis Adjusted Langevin Algorithm (MALA) with momentum-based optimization (Adam) for efficient sampling from posterior distributions.
result The algorithm approximates the Gibbs posterior in total variation distance and efficiently quantifies epistemic uncertainty.
Speed up Bayesian inference for large datasets using subsampling.
problem Bayesian inference in large data problems.
method Data subsampling to speed up Sequential Monte Carlo (SMC) for static Bayesian models.
result Efficiently estimates four generalized linear models and a generalized additive model with large datasets.
New method speeds up Gibbs sampling for large graphs.
problem Efficiently sampling from large graphical models.
method Poisson-minibatching Gibbs sampling.
result Theoretical convergence rate guarantees for Poisson-minibatching Gibbs.
Inference for the stochastic blockmodel is currently of burgeoning interest in the statistical community, as well as in various application domains as diverse as social networks, citation networks, brain connectivity networks (connectomics), etc. Recent theoretical developments have shown that spectral embedding of gra…
New training method for neural nets using multilevel entropic regularization.
problem Training efficiency and generalization bounds for neural nets.
method Multilevel relative entropy, chaining mutual information, Gibbs posterior distribution.
result Proves the Gibbs posterior achieves the unique minimum of the empirical risk minimization problem.
Proposes a hierarchical model for learning discrete Bayesian networks with shrinkage.
problem Learning discrete Bayesian networks with high-order interactions and cell probabilities.
method Hierarchical Dirichlet shrinkage model with Metropolis-adjusted Langevin algorithm for sampling.
result Efficiently learns graph structure and selects between DAGs from sparse count data.
Oracle inequality for sparse neural nets adapts to unknown structure.
problem Sparse deep neural nets in nonparametric regression.
method Gibbs posterior distribution with Metropolis-adjusted Langevin algorithms and mixture of uniform priors.
result Oracle inequality showing adaptation to unknown regularity and structure, achieving minimax-optimal rate of convergence.
The paper proposes a Gibbs sampler for neural network posterior sampling.
problem Sampling from the posterior of neural networks.
method Adding noise to activations and using a Gibbs sampler.
result The Gibbs sampler achieves similar performance to MCMC methods on real and synthetic data.
We propose a solution to the image deconvolution problem where the convolution kernel or point spread function (PSF) is assumed to be only partially known. Small perturbations generated from the model are exploited to produce a few principal components explaining the PSF uncertainty in a high dimensional space. Unlike …
Proposes a new model for estimating financial volatility with jumps.
problem Estimating volatility from financial time series with jumps.
method Gibbs Sampler with exact posterior distributions.
result Model captures speculative movements and propagates jumps in volatility.
This paper analyzes MCMC algorithms on large graphs using Dirichlet forms.
problem Analyzing the behavior of MCMC algorithms in high-dimensional problems.
method Utilizes Mosco convergence of Dirichlet forms to study RWM algorithm on large graphs.
result Demonstrates the advantages of Dirichlet form approach over standard diffusion methods.
We introduce a new geometric approach that constructs a transition kernel of Markov chain. Our method always minimizes the average rejection rate and even reduce it to zero in many relevant cases, which cannot be achieved by conventional methods, such as the Metropolis-Hastings algorithm or the heat bath algorithm (Gib…
Develops algorithm to differentiate Metropolis-Hastings for optimization.
problem Optimizing intractable densities with discrete components.
method Fuses stochastic automatic differentiation with Markov chain coupling schemes.
result Unbiased and low-variance gradient estimator for intractable densities.
Bayesian model predicts circular data with fast Gibbs sampling.
problem Predicting circular data in scientific fields.
method Expressive von Mises quasi-processes with Stratonovich augmentation for posterior inference.
result Fast Gibbs sampling for posterior inference.
A new sampler tackles high-dimensional models with intractable likelihoods.
problem Statistical inference for models with computationally intractable likelihoods and high-dimensional parameters.
method Likelihood-free approximate Gibbs sampler focusing on lower-dimensional conditional distributions estimated by flexible regression models.
result The sampler enables fitting models with 13,140 parameters that are otherwise impossible with standard ABC techniques.
A new sampler for complex discrete distributions efficiently updates all variables in parallel.
problem Sampling complex high-dimensional discrete distributions efficiently and accurately.
method Discrete Langevin proposal (DLP) for parallel coordinate updates with controlled stepsize.
result DLP efficiently explores high-dimensional and strongly correlated variables with asymptotic bias of zero for log-quadratic distributions.
Optimizes Metropolis-Hastings algorithms for efficient sampling in high dimensions.
problem Efficiently sampling from complex target distributions in high-dimensional spaces.
method Analyzes and optimizes the Barker proposal and other locally-balanced algorithms.
result Derives optimal noise distribution and balancing function for the Barker proposal.
Bayesian model predicts interest rates with short-term accuracy and long-term stability.
problem Improving short- and long-term prediction of time series with temporary non-stationary behavior.
method Time-varying autoregressive model with Bayesian regularization and MCMC inference.
result Model outperforms existing methods in both short and long-term predictions.
We present a new method for conducting Monte Carlo inference in graphical models which combines explicit search with generalized importance sampling. The idea is to reduce the variance of importance sampling by searching for significant points in the target distribution. We prove that it is possible to introduce search…
Additive regression trees are flexible non-parametric models and popular off-the-shelf tools for real-world non-linear regression. In application domains, such as bioinformatics, where there is also demand for probabilistic predictions with measures of uncertainty, the Bayesian additive regression trees (BART) model, i…
This paper proposes a new method to improve the MH algorithm for Bayesian estimation.
problem Difficulty in tuning the proposal distribution for efficient convergence in MH algorithms.
method Uses damped BFGS updates to incorporate gradient and curvature information from numerical optimization.
result Empirically demonstrates improved mixing and convergence of MH algorithm realisations.
Ray tracing sampler improves neural network sampling efficiency and resilience.
problem Sampling neural network posterior distributions efficiently and robustly.
method Markov Chain Monte Carlo using ray tracing through likelihood space.
result Significantly higher resilience to gradient heating compared to HMC.
End-to-end training of DBMs with improved gradient estimation.
problem Biased gradient estimation in DBMs, especially with high-dimensional states.
method Unbiased contrastive divergence using MH coupling and local mode initialization.
result End-to-end training of DBMs without greedy pretraining, achieving FID score of 10.33 for MNIST.
We investigate a class of feature allocation models that generalize the Indian buffet process and are parameterized by Gibbs-type random measures. Two existing classes are contained as special cases: the original two-parameter Indian buffet process, corresponding to the Dirichlet process, and the stable (or three-param…
We propose an extension to Hawkes processes by treating the levels of self-excitation as a stochastic differential equation. Our new point process allows better approximation in application domains where events and intensities accelerate each other with correlated levels of contagion. We generalize a recent algorithm f…
Within the description of stochastic differential equations it is argued that the existence of Boltzmann-Gibbs type distribution in economy is independent of the time reversal symmetry in econodynamics. Both power law and exponential distributions can be accommodated by it. The demonstration is based on a mathematical …
Particle MCMC is a class of algorithms that can be used to analyse state-space models. They use MCMC moves to update the parameters of the models, and particle filters to propose values for the path of the state-space model. Currently the default is to use random walk Metropolis to update the parameter values. We show …
Bayesian framework detects symmetries in chaotic dynamical systems.
problem Detecting symmetries in chaotic attractors for insights into dynamical system structure.
method Bayesian framework using Gibbs posterior constructed from Wasserstein distances.
result Bayesian framework accurately recovers symmetries under high noise and small sample sizes.
We propose a restricted collapsed draw (RCD) sampler, a general Markov chain Monte Carlo sampler of simultaneous draws from a hierarchical Chinese restaurant process (HCRP) with restriction. Models that require simultaneous draws from a hierarchical Dirichlet process with restriction, such as infinite Hidden markov mod…
The paper tackles sampling from Gibbs measures with constrained support, providing a sampling guarantee.
problem Sampling from Gibbs measures with constrained support, especially in the pre-asymptotic regime.
method Analyzing the spectral gap of Langevin dynamics to provide a non-asymptotic sampling guarantee.
result The low-temperature Gibbs distribution concentrates on a neighborhood of its mode in the pre-asymptotic regime.
This tutorial provides a gentle introduction to the particle Metropolis-Hastings (PMH) algorithm for parameter inference in nonlinear state-space models together with a software implementation in the statistical programming language R. We employ a step-by-step approach to develop an implementation of the PMH algorithm …
Model clusters networks and their communities simultaneously.
problem Clustering networks and their communities in unlabeled, heterogeneous networks.
method Nested Stochastic Block Model (NSBM) with Bayesian approach and NDP prior.
result Model accurately estimates both within and across network clustering structures.
This paper develops a matrix-variate adaptive Markov chain Monte Carlo (MCMC) methodology for Bayesian Cointegrated Vector Auto Regressions (CVAR). We replace the popular approach to sampling Bayesian CVAR models, involving griddy Gibbs, with an automated efficient alternative, based on the Adaptive Metropolis algorith…
Monte Carlo methods are essential tools for Bayesian inference. Gibbs sampling is a well-known Markov chain Monte Carlo (MCMC) algorithm, extensively used in signal processing, machine learning, and statistics, employed to draw samples from complicated high-dimensional posterior distributions. The key point for the suc…
Bayesian framework for image inversion using regularization by denoising.
problem Image inversion and regularization in imaging tasks.
method Bayesian approach with Langevin-within-split Gibbs sampling.
result Demonstrates the effectiveness of the proposed method through numerical experiments.
We describe Venture, an interactive virtual machine for probabilistic programming that aims to be sufficiently expressive, extensible, and efficient for general-purpose use. Like Church, probabilistic models and inference problems in Venture are specified via a Turing-complete, higher-order probabilistic language desce…
New method improves blockchain analysis by handling temporal changes and scalability.
problem Limited focus on evolving nature and scalability of blockchain transaction networks.
method Incremental approach with Metropolis-Hastings random walks.
result Comparable performance in node classification tasks with reduced computational overhead.
We introduce a method that uses the Cauchy-Crofton formula and a new curvature formula from integral geometry to reweight the sampling probabilities of Metropolis-within-Gibbs algorithms in order to increase their convergence speed. We consider algorithms that sample from a probability density conditioned on a manifold…