The latent Dirichlet allocation (LDA) model is a widely-used latent variable model in machine learning for text analysis. Inference for this model typically involves a single-site collapsed Gibbs sampling step for latent variables associated with observations. The efficiency of the sampling is critical to the success o…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,694 papers · 148 categories
Trend · papers per month
48 results for “block Gibbs sampling”
Improved sampling for Bayesian neural networks reduces vanishing acceptance rates and increases predictive accuracy.
problem Sampling inefficiency in Bayesian neural networks, especially with deep architectures and large datasets.
method Approximate blocked Gibbs sampling to partition and sample subgroups of parameters.
result Increased predictive accuracy and quantification of predictive uncertainty in classification tasks.
Machine learning models of music typically break up the task of composition into a chronological process, composing a piece of music in a single pass from beginning to end. On the contrary, human composers write music in a nonlinear fashion, scribbling motifs here and there, often revisiting choices previously made. In…
We present a new notion of probabilistic duality for random variables involving mixture distributions. Using this notion, we show how to implement a highly-parallelizable Gibbs sampler for weakly coupled discrete pairwise graphical models with strictly positive factors that requires almost no preprocessing and is easy …
New sampling method using regularized Wasserstein proximal for Gibbs distributions.
problem Sampling from Gibbs distributions with numerical stability and efficiency.
method Preconditioned regularized Wasserstein proximal operator.
result Discrete-time convergence analysis and explicit bias characterization.
Deep Temperingcs.LG
Restricted Boltzmann Machines (RBMs) are one of the fundamental building blocks of deep learning. Approximate maximum likelihood training of RBMs typically necessitates sampling from these models. In many training scenarios, computationally efficient Gibbs sampling procedures are crippled by poor mixing. In this work w…
EB-GFN models discrete data with amortized MCMC sampling.
problem Probabilistic modeling of high-dimensional discrete data.
method EB-GFN combines GFlowNets with energy-based models for efficient sampling.
result EB-GFN effectively models various discrete data tasks.
A common approach to analyze a covariate-sample count matrix, an element of which represents how many times a covariate appears in a sample, is to factorize it under the Poisson likelihood. We show its limitation in capturing the tendency for a covariate present in a sample to both repeat itself and excite related ones…
We develop amortized population Gibbs (APG) samplers, a class of scalable methods that frames structured variational inference as adaptive importance sampling. APG samplers construct high-dimensional proposals by iterating over updates to lower-dimensional blocks of variables. We train each conditional proposal by mini…
Effective implementations of sampling-based probabilistic inference often require manually constructed, model-specific proposals. Inspired by recent progresses in meta-learning for training learning agents that can generalize to unseen environments, we propose a meta-learning approach to building effective and generali…
We describe an adaptation of the simulated annealing algorithm to nonparametric clustering and related probabilistic models. This new algorithm learns nonparametric latent structure over a growing and constantly churning subsample of training data, where the portion of data subsampled can be interpreted as the inverse …
Gibbs sampling is a Markov chain Monte Carlo method that is often used for learning and inference on graphical models. Minibatching, in which a small random subset of the graph is used at each iteration, can help make Gibbs sampling scale to large graphical models by reducing its computational cost. In this paper, we p…
Model clusters networks and their communities simultaneously.
problem Clustering networks and their communities in unlabeled, heterogeneous networks.
method Nested Stochastic Block Model (NSBM) with Bayesian approach and NDP prior.
result Model accurately estimates both within and across network clustering structures.
Topic models, and more specifically the class of Latent Dirichlet Allocation (LDA), are widely used for probabilistic modeling of text. MCMC sampling from the posterior distribution is typically performed using a collapsed Gibbs sampler. We propose a parallel sparse partially collapsed Gibbs sampler and compare its spe…
THRML uses energy-based models for index tracking, reducing portfolio tracking error and improving returns.
problem NP-hard combinatorial optimization in portfolio optimization under cardinality constraints.
method THRML reformulates index tracking as probabilistic inference on an Ising Hamiltonian, using GPU-accelerated block Gibbs sampling.
result THRML achieves 4.31 percent annualized tracking error compared to 5.66-6.30 percent for baselines, with 128.63 percent total return.
New model detects hidden group structures in criminal networks.
problem Challenges in identifying group structures in criminal networks with noisy data.
method Developed an extended stochastic block model (ESBM) to infer group structures.
result Unveiled complex block structures in an Italian mafia network.
Diffusive Gibbs Samplingstat.ML
DiGS improves sampling from multi-modal distributions.
problem Inadequate mixing in MCMC methods for multi-modal distributions.
method Integrates diffusion models and Gibbs sampling to create an auxiliary noisy distribution.
result DiGS exhibits better mixing for multi-modal distributions than state-of-the-art methods.
A new algorithm for sampling from complex distributions.
problem Sampling from high-dimensional multivariate probability densities.
method Combines kernel herding and Gibbs sampling for deterministic sampling.
result Significantly lower computation time compared to kernel herding.
New Gibbs sampling method improves MCMC efficiency.
problem Improving efficiency of Gibbs sampling.
method Non-uniform random scan with selection probability optimization.
result Non-uniform scan improves mixing time of Markov chain.
Bayesian inference for biochemical reaction networks using jump-diffusion approximations.
problem Estimating hidden quantities in poorly characterized biochemical processes.
method Developed a Bayesian inference algorithm based on Markov chain Monte Carlo and sequential Monte Carlo methods.
result Numerical evaluation of the algorithm for a partially observed multi-scale birth-death process.
We propose a restricted collapsed draw (RCD) sampler, a general Markov chain Monte Carlo sampler of simultaneous draws from a hierarchical Chinese restaurant process (HCRP) with restriction. Models that require simultaneous draws from a hierarchical Dirichlet process with restriction, such as infinite Hidden markov mod…
Introduces HMC method for sampling Gibbs densities.
problem Sampling from Gibbs densities efficiently.
method Hamiltonian Monte Carlo (HMC) method based on Hamiltonian dynamics.
result Idealized HMC preserves the target distribution and converges under certain conditions.
Entity resolution (ER; also known as record linkage or de-duplication) is the process of merging noisy databases, often in the absence of unique identifiers. A major advancement in ER methodology has been the application of Bayesian generative models, which provide a natural framework for inferring latent entities with…
We develop a framework for approximating collapsed Gibbs sampling in generative latent variable cluster models. Collapsed Gibbs is a popular MCMC method, which integrates out variables in the posterior to improve mixing. Unfortunately for many complex models, integrating out these variables is either analytically or co…
The pairwise influence matrix of Dobrushin has long been used as an analytical tool to bound the rate of convergence of Gibbs sampling. In this work, we use Dobrushin influence as the basis of a practical tool to certify and efficiently improve the quality of a discrete Gibbs sampler. Our Dobrushin-optimized Gibbs samp…
Unified sampling approach for Bayesian imaging problems.
problem Sampling from complex prior and posterior distributions in Bayesian imaging.
method Gaussian latent machine model for efficient prior and posterior sampling.
result Unified and generalized sampling algorithms for various imaging problems.
The study examines MCMC methods for arbitrary objectives and finds likelihood sharpness impacts performance and regularization.
problem Limitations of MCMC methods for arbitrary objective functions.
method Two-block MCMC framework with Metropolis-Hastings and Gibbs sampling, exploring likelihood curvature and sharpness.
result Likelihood sharpness governs in-sample performance and regularization inferred by training data.
Gibbs sampling is the de facto Markov chain Monte Carlo method used for inference and learning on large scale graphical models. For complicated factor graphs with lots of factors, the performance of Gibbs sampling can be limited by the computational cost of executing a single update step of the Markov chain. This cost …
CAVI speeds up Bayesian MIDAS regression by 107x-1,772x with similar accuracy.
problem Efficiently estimating Bayesian MIDAS regression models with many predictors.
method Coordinate Ascent Variational Inference (CAVI) for linear MIDAS regression.
result CAVI produces posterior means nearly identical to Gibbs sampling with significant speedup.
New method transfers instances between domains using Gibbs Sampling and RBM.
problem Transfer Learning between domains with limited target data.
method Gibbs Sampling and RBM for instance transition.
result Significant improvement in target classification.
The paper proposes a Gibbs sampler for neural network posterior sampling.
problem Sampling from the posterior of neural networks.
method Adding noise to activations and using a Gibbs sampler.
result The Gibbs sampler achieves similar performance to MCMC methods on real and synthetic data.
Sampling from Constrained Gibbs Measures: with Applications to High-Dimensional Bayesian Inferencemath.ST
The paper tackles sampling from Gibbs measures with constrained support, providing a sampling guarantee.
problem Sampling from Gibbs measures with constrained support, especially in the pre-asymptotic regime.
method Analyzing the spectral gap of Langevin dynamics to provide a non-asymptotic sampling guarantee.
result The low-temperature Gibbs distribution concentrates on a neighborhood of its mode in the pre-asymptotic regime.
We present a new approach to estimating the interdependence of industries in an economy by applying data science solutions. By exploiting interfirm buyer--seller network data, we show that the problem of estimating the interdependence of industries is similar to the problem of uncovering the latent block structure in n…
The Gibbs sampler is one of the most popular algorithms for inference in statistical models. In this paper, we introduce a herding variant of this algorithm, called herded Gibbs, that is entirely deterministic. We prove that herded Gibbs has an convergence rate for models with independent variables and for ful…
New Gibbs sampling reduces GLMB filtering complexity to linear time.
problem NP-hard GLMB density computation in multi-object systems.
method Tempered Gibbs sampler exploiting GLMB structure.
result Linear complexity for GLMB filtering.
Bayesian inference for Levy density with Gibbs posterior in discrete sampling.
problem Inference on Levy density for financial models with jumps.
method Gibbs posterior framework using a loss function for intractable likelihood.
result Gibbs posterior achieves nearly optimal rate of convergence under certain conditions.
Dealing with collinearity in large-scale linear system identification using Bayesian regularizationmath.DS
Bayesian regularization tackles collinearity in large-scale systems with correlated inputs.
problem Collinearity in large-scale linear systems identification due to correlated inputs.
method Bayesian regularization with stable spline covariance and Markov chain Monte Carlo scheme.
result Efficient reconstruction of impulse responses with high correlation among inputs.
Improved Bayesian regression for large datasets using multilevel Gibbs sampling.
problem Efficiently handling large-scale Bayesian regression with complex posterior distributions.
method Developed a multilevel Gibbs sampler for linear mixed models, incorporating data clustering and correlated samples for variance reduction.
result Significant speed-up achieved for Bayesian regression without sacrificing predictive performance.
Estimating Gibbs free energies via isobaric-isothermal flowsphysics.comp-ph
New model estimates Gibbs free energies using machine learning and isobaric-isothermal flows.
problem Estimating Gibbs free energies for complex systems.
method Normalizing flows trained to sample isobaric-isothermal ensemble.
result Excellent agreement with established baselines for water phases.
Monte Carlo methods are essential tools for Bayesian inference. Gibbs sampling is a well-known Markov chain Monte Carlo (MCMC) algorithm, extensively used in signal processing, machine learning, and statistics, employed to draw samples from complicated high-dimensional posterior distributions. The key point for the suc…
A fundamental task in machine learning and related fields is to perform inference on Bayesian networks. Since exact inference takes exponential time in general, a variety of approximate methods are used. Gibbs sampling is one of the most accurate approaches and provides unbiased samples from the posterior but it has hi…
Gibbs sampler mixes quickly for certain smooth distributions.
problem Drawing samples from log-smooth log-concave distributions.
method Analyzes Gibbs sampler on log-smooth and strongly log-concave distributions.
result Gibbs sampler mixes in steps.
GDiff tackles blind denoising with Gibbs sampling and Monte Carlo inference.
problem Blind denoising of signals with unknown noise parameters.
method Gibbs Diffusion (GDiff) method that alternates sampling steps from a conditional diffusion model and a Monte Carlo sampler.
result GDiff achieves blind denoising of natural images and cosmic microwave background data.
Improved MALA method for neural networks uncertainty quantification.
problem Uncertainty quantification in Bayesian neural networks.
method Corrected Stochastic MALA (csMALA) with a simplified correction term.
result Improved surrogate posterior for quantifying uncertainties in neural networks.
Streaming variational Bayes (SVB) is successful in learning LDA models in an online manner. However previous attempts toward developing online Monte-Carlo methods for LDA have little success, often by having much worse perplexity than their batch counterparts. We present a streaming Gibbs sampling (SGS) method, an onli…
GIST adapts HMC by tuning parameters based on position and momentum.
problem Locally adaptive sampling in Hamiltonian Monte Carlo.
method GIST uses Gibbs sampling to adaptively tune HMC parameters.
result GIST improves sampling efficiency for high-dimensional models.
New method improves sampling from noisy energy models.
problem Training and sampling challenges in Energy-Based Models.
method Pseudo-Gibbs sampling with moment matching.
result Effective sampling from clean model using noisy DSM-trained model.
Variational Bayesian inference and (collapsed) Gibbs sampling are the two important classes of inference algorithms for Bayesian networks. Both have their advantages and disadvantages: collapsed Gibbs sampling is unbiased but is also inefficient for large count values and requires averaging over many samples to reduce …