Adaptive scan Gibbs sampler improves large-scale inference performance.
problem Efficiently updating large-scale online inference problems.
method Derives an adaptive scan Gibbs sampler that optimizes mini-batch size selection.
result Demonstrates superior performance compared to collapsed Gibbs sampler.
The Gibbs sampler is a particularly popular Markov chain used for learning and inference problems in Graphical Models (GMs). These tasks are computationally intractable in general, and the Gibbs sampler often suffers from slow mixing. In this paper, we study the Swendsen-Wang dynamics which is a more sophisticated Mark…
Improved Gibbs sampling yields higher likelihood solutions.
problem Gibbs sampling often returns suboptimal solutions due to bottlenecks.
method Interdependent Gibbs Samplers combining multiple samplers with coupling.
result High likelihood solutions significantly more often than regular Gibbs sampler.
DoGS improves Gibbs sampling quality with variable selection orders and bounds.
problem Improving Gibbs sampler scan quality.
method Using Dobrushin influence to optimize Gibbs sampling.
result DoGS delivers higher-quality inferences with smaller sampling budgets.
Improved Gibbs sampler for crossed random effects models scales better with data.
problem Complexity issues in Gibbs samplers for crossed random effects models.
method Proposed a collapsed Gibbs sampler that is provably scalable.
result The collapsed Gibbs sampler outperforms alternative algorithms significantly.
Developed a Particle-Gibbs sampler for Bayesian feature allocation models.
problem Intractable exact inference in Bayesian feature allocation models.
method Particle-Gibbs sampler for feature allocation matrix updates.
result PG sampler improves performance of feature allocation models.
Gibbs sampler contracts entropy under strong log-concavity, improving mixing time.
problem Improving the mixing time of Gibbs sampler under strong log-concavity.
method Analyzing Gibbs sampler contraction under strong log-concavity, providing sharp contraction rate.
result Gibbs sampler contracts entropy linearly with condition number and independent of dimension under strong log-concavity.
This work analyzes Gibbs samplers for Bayesian hierarchical models without dimensionality constraints.
problem Analyzing convergence properties of Gibbs samplers for Bayesian hierarchical models.
method Using Bayesian asymptotics and total variation mixing times, the study provides dimension-free convergence results.
result Dimension-free convergence results for Gibbs samplers targeting hierarchical models under random data-generating assumptions.
Gibbs sampler mixes quickly for certain smooth distributions.
problem Drawing samples from log-smooth log-concave distributions.
method Analyzes Gibbs sampler on log-smooth and strongly log-concave distributions.
result Gibbs sampler mixes in O⋆(κ2n7.5) steps. Paper uses Gibbs sampler with jump diffusion for European option pricing.
problem Estimating market parameters for jump diffusion models in option pricing.
method Gibbs sampler applied to jump diffusion model for estimating drift, volatility, jump intensity, and occurrence.
result Demonstrates impact of jump effects on European call option and annuity pricing.
Improved Gibbs sampler speeds up Bayesian exponential smoothing model.
problem Computational inefficiency of original NUTS sampler.
method Modifications to the original model and a bespoke Gibbs sampler.
result Significant improvement in sampling time by an order of magnitude.
A fundamental task in machine learning and related fields is to perform inference on Bayesian networks. Since exact inference takes exponential time in general, a variety of approximate methods are used. Gibbs sampling is one of the most accurate approaches and provides unbiased samples from the posterior but it has hi…
APG samplers use neural suff stats to improve deep model inference.
problem Efficient inference in deep generative models.
method Amortized population Gibbs, neural suff stats, KL divergence minimization.
result Significant improvement in inference accuracy.
New sampler reduces MCMC complexity for Bayesian variable selection.
problem High-dimensional Bayesian variable selection with high computation complexity.
method Variable-complexity subset weighted-Tempered Gibbs Sampler (wTGS) with Rao-Blackwellized estimator.
result Variances of Rao-Blackwellized estimator are smaller than those of subset wTGS.
A fast Gibbs sampler for Bayesian HMMs with missing data.
problem Complexity and slow mixing in EM and Gibbs samplers for HMMs with missing observations.
method Proposes a collapsed Gibbs sampler that integrates missing observations and latent states, achieving high accuracy, large ESS, and reduced computational complexity.
result The proposed sampler is faster and more efficient than existing methods, especially with many missing entries.
GIST adapts HMC by tuning parameters based on position and momentum.
problem Locally adaptive sampling in Hamiltonian Monte Carlo.
method GIST uses Gibbs sampling to adaptively tune HMC parameters.
result GIST improves sampling efficiency for high-dimensional models.
The study examines mixing times of data-augmentation Gibbs samplers for high-dimensional probit regression.
problem Investigating convergence properties of data-augmentation samplers for Bayesian probit regression.
method Using recent results on Gibbs samplers for log-concave targets, the study provides non-asymptotic bounds on mixing times.
result Explicit non-asymptotic bounds on mixing times depend on design matrix and prior precision, holding uniformly over responses.
The latent Dirichlet allocation (LDA) model is a widely-used latent variable model in machine learning for text analysis. Inference for this model typically involves a single-site collapsed Gibbs sampling step for latent variables associated with observations. The efficiency of the sampling is critical to the success o…
A new sampler tackles high-dimensional models with intractable likelihoods.
problem Statistical inference for models with computationally intractable likelihoods and high-dimensional parameters.
method Likelihood-free approximate Gibbs sampler focusing on lower-dimensional conditional distributions estimated by flexible regression models.
result The sampler enables fitting models with 13,140 parameters that are otherwise impossible with standard ABC techniques.
Develops an efficient approximation for collapsed Gibbs sampling in complex models.
problem Intractability of integrating out variables in collapsed Gibbs sampling for complex models.
method Uses expectation propagation to approximate collapsed Gibbs integrals.
result Approximate sampler enables a runtime-accuracy tradeoff in sampling complex models.
The paper proposes a Gibbs sampler for neural network posterior sampling.
problem Sampling from the posterior of neural networks.
method Adding noise to activations and using a Gibbs sampler.
result The Gibbs sampler achieves similar performance to MCMC methods on real and synthetic data.
Proposes a new model for estimating financial volatility with jumps.
problem Estimating volatility from financial time series with jumps.
method Gibbs Sampler with exact posterior distributions.
result Model captures speculative movements and propagates jumps in volatility.
Proposes PG-DA for Bayesian MMNL estimation to handle non-conjugacy.
problem Non-conjugacy in the Bayesian estimation of MMNL models.
method Pólygamma data augmentation technique applied to MMNL estimation.
result Similar posterior estimates for binary choice scenarios, but empirical identification issues for J≥3 alternatives. Nonparametric Bayesian approaches to clustering, information retrieval, language modeling and object recognition have recently shown great promise as a new paradigm for unsupervised data analysis. Most contributions have focused on the Dirichlet process mixture models or extensions thereof for which efficient Gibbs sam…
Paper introduces a Gibbs sampler for Bayesian inversion of ill-posed problems.
problem Bayesian inversion of ill-posed problems with linear transformation and additive noise.
method Gibbs algorithm based on prior diffusion model.
result Gibbs algorithm offers a guarantee of convergence in a specific situation.
Monte Carlo methods are essential tools for Bayesian inference. Gibbs sampling is a well-known Markov chain Monte Carlo (MCMC) algorithm, extensively used in signal processing, machine learning, and statistics, employed to draw samples from complicated high-dimensional posterior distributions. The key point for the suc…
Study on Metropolis-within-Gibbs schemes for high-dimensional Bayesian models.
problem Improving the scalability of MCMC methods for complex Bayesian models.
method Relating convergence properties to conditional conductance for non-conjugate hierarchical models.
result Established dimension-free convergence results for Metropolis-within-Gibbs schemes.
The Gibbs sampler is one of the most popular algorithms for inference in statistical models. In this paper, we introduce a herding variant of this algorithm, called herded Gibbs, that is entirely deterministic. We prove that herded Gibbs has an O(1/T) convergence rate for models with independent variables and for ful…
New PG samplers improve inference in coupled state-space models.
problem Bayesian inference from multiple time series with shared parameters.
method Marginalized Particle Gibbs samplers for coupled state-space models.
result Improved parameter inference through shared information.
Topic models, and more specifically the class of Latent Dirichlet Allocation (LDA), are widely used for probabilistic modeling of text. MCMC sampling from the posterior distribution is typically performed using a collapsed Gibbs sampler. We propose a parallel sparse partially collapsed Gibbs sampler and compare its spe…
Adjoint sampler targets infinite-dimensional function spaces for efficient sampling.
problem Limited theory and algorithms for sampling infinite-dimensional function spaces.
method Adjoint Sampler for infinite-dimensional function spaces based on stochastic maximum principle.
result FAS achieves superior performance in synthetic and real systems.
We present a new notion of probabilistic duality for random variables involving mixture distributions. Using this notion, we show how to implement a highly-parallelizable Gibbs sampler for weakly coupled discrete pairwise graphical models with strictly positive factors that requires almost no preprocessing and is easy …
New Gibbs sampling reduces GLMB filtering complexity to linear time.
problem NP-hard GLMB density computation in multi-object systems.
method Tempered Gibbs sampler exploiting GLMB structure.
result Linear complexity O(T(P+M)) for GLMB filtering. A new algorithm for sampling from complex distributions.
problem Sampling from high-dimensional multivariate probability densities.
method Combines kernel herding and Gibbs sampling for deterministic sampling.
result Significantly lower computation time compared to kernel herding.
A new method speeds up DPMM inference for federated learning.
problem Slow inference for large datasets in DPMMs.
method Distributed collapsed Gibbs sampler (DisCGS) for DPMMs.
result Significant reduction in execution time (200x faster) for large datasets.
New methods improve sampling from complex dynamical models.
problem Sampling from high-dimensional, non-linear latent dynamical models is computationally challenging.
method Introduce auxiliary MCMC and Particle Gibbs samplers with improved performance and parallelisation.
result Enhanced samplers maintain performance in high-dimensional latent spaces and support parallelisation.
Robots learn movement libraries by segmenting complex trajectories.
problem Segmenting complex robot movement demonstrations for library building.
method Model trajectories as Switching Linear Dynamical Systems and infer segmentation using a nonparametric Bayesian approach.
result Robots can learn movement libraries more effectively by segmenting demonstrations.
Additive regression trees are flexible non-parametric models and popular off-the-shelf tools for real-world non-linear regression. In application domains, such as bioinformatics, where there is also demand for probabilistic predictions with measures of uncertainty, the Bayesian additive regression trees (BART) model, i…
Polynomial-time Gibbs sampling generates DPP samples efficiently.
problem Sampling from continuous Determinantal Point Processes (DPPs).
method Polynomial-time Gibbs sampling algorithm.
result Gibbs sampler generates DPP samples in polynomial time.
Model clusters networks and their communities simultaneously.
problem Clustering networks and their communities in unlabeled, heterogeneous networks.
method Nested Stochastic Block Model (NSBM) with Bayesian approach and NDP prior.
result Model accurately estimates both within and across network clustering structures.
Markov jump processes and continuous time Bayesian networks are important classes of continuous time dynamical systems. In this paper, we tackle the problem of inferring unobserved paths in these models by introducing a fast auxiliary variable Gibbs sampler. Our approach is based on the idea of uniformization, and sets…
We propose a categorical data synthesizer with a quantifiable disclosure risk. Our algorithm, named Perturbed Gibbs Sampler, can handle high-dimensional categorical data that are often intractable to represent as contingency tables. The algorithm extends a multiple imputation strategy for fully synthetic data by utiliz…
Accelerated Gibbs sampling for Gaussian graphical models using dual factor graphs.
problem Improving convergence rate of Gibbs sampling for Gaussian graphical models.
method Dual normal factor graph approach to accelerate convergence.
result Universal convergence rate improvement in dual domain for all homogeneous models.
Develops inference combinators for probabilistic programs using neural networks.
problem Creating efficient proposals for probabilistic program inference.
method Inference combinators using neural network parameterization of proposals.
result Correct by construction variational methods tailored to specific models.
The K-Mean and EM algorithms are popular in clustering and mixture modeling, due to their simplicity and ease of implementation. However, they have several significant limitations. Both coverage to a local optimum of their respective objective functions (ignoring the uncertainty in the model space), require the apriori…
Paper proposes a new method for Bayesian linear regression using spike-and-slab priors.
problem Identifying predictors with similar relationships in linear regression models.
method Hierarchical Bayesian models with spike-and-slab priors and a Gibbs sampler.
result The proposed method outperforms previous methods in simulations and real data analysis.
A new co-clustering model for high-dimensional data reduces parameter complexity.
problem High-dimensional data challenges traditional co-clustering methods.
method Parameter-wise co-clustering model with SEM and Gibbs sampler for estimation.
result The model maintains parsimony while offering more flexibility.
New method speeds up inference for non-conjugate Gaussian processes.
problem Inference for non-conjugate Gaussian processes is slow and unreliable.
method Automated augmented conjugate inference method that constructs auxiliary variables to make the model conditionally conjugate.
result Our method is up to two orders of magnitude faster and more robust than existing methods.