We develop a framework for approximating collapsed Gibbs sampling in generative latent variable cluster models. Collapsed Gibbs is a popular MCMC method, which integrates out variables in the posterior to improve mixing. Unfortunately for many complex models, integrating out these variables is either analytically or co…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Collapsed Gibbs sampling”
Variational Bayesian inference and (collapsed) Gibbs sampling are the two important classes of inference algorithms for Bayesian networks. Both have their advantages and disadvantages: collapsed Gibbs sampling is unbiased but is also inefficient for large count values and requires averaging over many samples to reduce …
The latent Dirichlet allocation (LDA) model is a widely-used latent variable model in machine learning for text analysis. Inference for this model typically involves a single-site collapsed Gibbs sampling step for latent variables associated with observations. The efficiency of the sampling is critical to the success o…
Dense Distributions from Sparse Samples: Improved Gibbs Sampling Parameter Estimators for LDAstat.ML
We introduce a novel approach for estimating Latent Dirichlet Allocation (LDA) parameters from collapsed Gibbs samples (CGS), by leveraging the full conditional distributions over the latent variable assignments to efficiently average over multiple samples, for little more computational cost than drawing a single addit…
Topic models, and more specifically the class of Latent Dirichlet Allocation (LDA), are widely used for probabilistic modeling of text. MCMC sampling from the posterior distribution is typically performed using a collapsed Gibbs sampler. We propose a parallel sparse partially collapsed Gibbs sampler and compare its spe…
We analyze the complexity of Gibbs samplers for inference in crossed random effect models used in modern analysis of variance. We demonstrate that for certain designs the plain vanilla Gibbs sampler is not scalable, in the sense that its complexity is worse than proportional to the number of parameters and data. We thu…
Streaming variational Bayes (SVB) is successful in learning LDA models in an online manner. However previous attempts toward developing online Monte-Carlo methods for LDA have little success, often by having much worse perplexity than their batch counterparts. We present a streaming Gibbs sampling (SGS) method, an onli…
Deep generative model for healthcare data identifies coherent substructures and mutational clusters.
problem Analytical challenges in healthcare data, including sparsity, missingness, and small sample sizes.
method Proposes a deep generative Bayesian model with collapsed Gibbs sampling for multinomial count data.
result Identifies coherent substructures and biologically meaningful mutational clusters in cancer data.
A fast Gibbs sampler for Bayesian HMMs with missing data.
problem Complexity and slow mixing in EM and Gibbs samplers for HMMs with missing observations.
method Proposes a collapsed Gibbs sampler that integrates missing observations and latent states, achieving high accuracy, large ESS, and reduced computational complexity.
result The proposed sampler is faster and more efficient than existing methods, especially with many missing entries.
For large scale on-line inference problems the update strategy is critical for performance. We derive an adaptive scan Gibbs sampler that optimizes the update frequency by selecting an optimum mini-batch size. We demonstrate performance of our adaptive batch-size Gibbs sampler by comparing it against the collapsed Gibb…
Max-margin learning is a powerful approach to building classifiers and structured output predictors. Recent work on max-margin supervised topic models has successfully integrated it with Bayesian topic models to discover discriminative latent semantic structures and make accurate predictions for unseen testing data. Ho…
We review three algorithms for Latent Dirichlet Allocation (LDA). Two of them are variational inference algorithms: Variational Bayesian inference and Online Variational Bayesian inference and one is Markov Chain Monte Carlo (MCMC) algorithm -- Collapsed Gibbs sampling. We compare their time complexity and performance.…
We propose a restricted collapsed draw (RCD) sampler, a general Markov chain Monte Carlo sampler of simultaneous draws from a hierarchical Chinese restaurant process (HCRP) with restriction. Models that require simultaneous draws from a hierarchical Dirichlet process with restriction, such as infinite Hidden markov mod…
Group factor analysis (GFA) methods have been widely used to infer the common structure and the group-specific signals from multiple related datasets in various fields including systems biology and neuroimaging. To date, most available GFA models require Gibbs sampling or slice sampling to perform inference, which prev…
Supervised topic models with a logistic likelihood have two issues that potentially limit their practical use: 1) response variables are usually over-weighted by document word counts; and 2) existing variational inference methods make strict mean-field assumptions. We address these issues by: 1) introducing a regulariz…
Nested Chinese Restaurant Process (nCRP) topic models are powerful nonparametric Bayesian methods to extract a topic hierarchy from a given text corpus, where the hierarchical structure is automatically determined by the data. Hierarchical Latent Dirichlet Allocation (hLDA) is a popular instance of nCRP topic models. H…
Bayesian graphical models have been shown to be a powerful tool for discovering uncertainty and causal structure from real-world data in many application fields. Current inference methods primarily follow different kinds of trade-offs between computational complexity and predictive accuracy. At one end of the spectrum,…
Distributed Collapsed Gibbs Sampler for Dirichlet Process Mixture Models in Federated Learningstat.ML
A new method speeds up DPMM inference for federated learning.
problem Slow inference for large datasets in DPMMs.
method Distributed collapsed Gibbs sampler (DisCGS) for DPMMs.
result Significant reduction in execution time (200x faster) for large datasets.
In this technical report, we present jLDADMM---an easy-to-use Java toolkit for conventional topic models. jLDADMM is released to provide alternatives for topic modeling on normal or short texts. It provides implementations of the Latent Dirichlet Allocation topic model and the one-topic-per-document Dirichlet Multinomi…
Gibbs sampling is a Markov chain Monte Carlo method that is often used for learning and inference on graphical models. Minibatching, in which a small random subset of the graph is used at each iteration, can help make Gibbs sampling scale to large graphical models by reducing its computational cost. In this paper, we p…
Novel connections between Neyman-Scott processes and Bayesian nonparametric mixture models enable scalable inference.
problem Efficiently modeling and detecting clusters in spatiotemporal data.
method Adapting collapsed Gibbs sampling for Neyman-Scott processes via connections to mixture of finite mixture models.
result Demonstrated scalability and effectiveness on neural spike trains and document streams.
A common approach to analyze a covariate-sample count matrix, an element of which represents how many times a covariate appears in a sample, is to factorize it under the Poisson likelihood. We show its limitation in capturing the tendency for a covariate present in a sample to both repeat itself and excite related ones…
As the emergence and the thriving development of social networks, a huge number of short texts are accumulated and need to be processed. Inferring latent topics of collected short texts is useful for understanding its hidden structure and predicting new contents. Unlike conventional topic models such as latent Dirichle…
Diffusive Gibbs Samplingstat.ML
DiGS improves sampling from multi-modal distributions.
problem Inadequate mixing in MCMC methods for multi-modal distributions.
method Integrates diffusion models and Gibbs sampling to create an auxiliary noisy distribution.
result DiGS exhibits better mixing for multi-modal distributions than state-of-the-art methods.
A new algorithm for sampling from complex distributions.
problem Sampling from high-dimensional multivariate probability densities.
method Combines kernel herding and Gibbs sampling for deterministic sampling.
result Significantly lower computation time compared to kernel herding.
New Gibbs sampling method improves MCMC efficiency.
problem Improving efficiency of Gibbs sampling.
method Non-uniform random scan with selection probability optimization.
result Non-uniform scan improves mixing time of Markov chain.
Introduces HMC method for sampling Gibbs densities.
problem Sampling from Gibbs densities efficiently.
method Hamiltonian Monte Carlo (HMC) method based on Hamiltonian dynamics.
result Idealized HMC preserves the target distribution and converges under certain conditions.
The pairwise influence matrix of Dobrushin has long been used as an analytical tool to bound the rate of convergence of Gibbs sampling. In this work, we use Dobrushin influence as the basis of a practical tool to certify and efficiently improve the quality of a discrete Gibbs sampler. Our Dobrushin-optimized Gibbs samp…
New Gamma-Poisson model improves topic selection for short text.
problem Topic modelling for short text using Poisson distribution.
method Gamma-Poisson mixture model with collapsed Gibbs sampler.
result Gamma-Poisson model selects more accurate number of topics.
Gibbs sampling is the de facto Markov chain Monte Carlo method used for inference and learning on large scale graphical models. For complicated factor graphs with lots of factors, the performance of Gibbs sampling can be limited by the computational cost of executing a single update step of the Markov chain. This cost …
New method transfers instances between domains using Gibbs Sampling and RBM.
problem Transfer Learning between domains with limited target data.
method Gibbs Sampling and RBM for instance transition.
result Significant improvement in target classification.
The paper proposes a Gibbs sampler for neural network posterior sampling.
problem Sampling from the posterior of neural networks.
method Adding noise to activations and using a Gibbs sampler.
result The Gibbs sampler achieves similar performance to MCMC methods on real and synthetic data.
Sampling from Constrained Gibbs Measures: with Applications to High-Dimensional Bayesian Inferencemath.ST
The paper tackles sampling from Gibbs measures with constrained support, providing a sampling guarantee.
problem Sampling from Gibbs measures with constrained support, especially in the pre-asymptotic regime.
method Analyzing the spectral gap of Langevin dynamics to provide a non-asymptotic sampling guarantee.
result The low-temperature Gibbs distribution concentrates on a neighborhood of its mode in the pre-asymptotic regime.
The Gibbs sampler is one of the most popular algorithms for inference in statistical models. In this paper, we introduce a herding variant of this algorithm, called herded Gibbs, that is entirely deterministic. We prove that herded Gibbs has an convergence rate for models with independent variables and for ful…
New Gibbs sampling reduces GLMB filtering complexity to linear time.
problem NP-hard GLMB density computation in multi-object systems.
method Tempered Gibbs sampler exploiting GLMB structure.
result Linear complexity for GLMB filtering.
Bayesian inference for Levy density with Gibbs posterior in discrete sampling.
problem Inference on Levy density for financial models with jumps.
method Gibbs posterior framework using a loss function for intractable likelihood.
result Gibbs posterior achieves nearly optimal rate of convergence under certain conditions.
Improved Bayesian regression for large datasets using multilevel Gibbs sampling.
problem Efficiently handling large-scale Bayesian regression with complex posterior distributions.
method Developed a multilevel Gibbs sampler for linear mixed models, incorporating data clustering and correlated samples for variance reduction.
result Significant speed-up achieved for Bayesian regression without sacrificing predictive performance.
Estimating Gibbs free energies via isobaric-isothermal flowsphysics.comp-ph
New model estimates Gibbs free energies using machine learning and isobaric-isothermal flows.
problem Estimating Gibbs free energies for complex systems.
method Normalizing flows trained to sample isobaric-isothermal ensemble.
result Excellent agreement with established baselines for water phases.
Monte Carlo methods are essential tools for Bayesian inference. Gibbs sampling is a well-known Markov chain Monte Carlo (MCMC) algorithm, extensively used in signal processing, machine learning, and statistics, employed to draw samples from complicated high-dimensional posterior distributions. The key point for the suc…
A fundamental task in machine learning and related fields is to perform inference on Bayesian networks. Since exact inference takes exponential time in general, a variety of approximate methods are used. Gibbs sampling is one of the most accurate approaches and provides unbiased samples from the posterior but it has hi…
Gibbs sampler mixes quickly for certain smooth distributions.
problem Drawing samples from log-smooth log-concave distributions.
method Analyzes Gibbs sampler on log-smooth and strongly log-concave distributions.
result Gibbs sampler mixes in steps.
GDiff tackles blind denoising with Gibbs sampling and Monte Carlo inference.
problem Blind denoising of signals with unknown noise parameters.
method Gibbs Diffusion (GDiff) method that alternates sampling steps from a conditional diffusion model and a Monte Carlo sampler.
result GDiff achieves blind denoising of natural images and cosmic microwave background data.
Improved MALA method for neural networks uncertainty quantification.
problem Uncertainty quantification in Bayesian neural networks.
method Corrected Stochastic MALA (csMALA) with a simplified correction term.
result Improved surrogate posterior for quantifying uncertainties in neural networks.
In binary-transaction data-mining, traditional frequent itemset mining often produces results which are not straightforward to interpret. To overcome this problem, probability models are often used to produce more compact and conclusive results, albeit with some loss of accuracy. Bayesian statistics have been widely us…
Proposes a new model for clustering passenger trips considering hierarchical and multi-dimensional data.
problem Clustering passenger trips with hierarchical and multi-dimensional data, especially in large-scale transportation systems.
method Tensor Dirichlet Process Multinomial Mixture (Tensor-DPMM) model, incorporating Dirichlet Process for automatic cluster number determination and tensor representation for multi-mode data.
result Automatic determination of the number of clusters and improved clustering quality.
GIST adapts HMC by tuning parameters based on position and momentum.
problem Locally adaptive sampling in Hamiltonian Monte Carlo.
method GIST uses Gibbs sampling to adaptively tune HMC parameters.
result GIST improves sampling efficiency for high-dimensional models.
New method improves sampling from noisy energy models.
problem Training and sampling challenges in Energy-Based Models.
method Pseudo-Gibbs sampling with moment matching.
result Effective sampling from clean model using noisy DSM-trained model.
Characterizing and Understanding the Generalization Error of Transfer Learning with Gibbs Algorithmcs.LG
We analyze Gibbs-based transfer learning algorithms using information theory.
problem Understanding the generalization error of transfer learning.
method Information-theoretic analysis focusing on -weighted-ERM and two-stage-ERM.
result Exact characterization of generalization behavior using conditional symmetrized KL information.