arXiv research

New algorithm trains deep neural networks without global optimization.

In this work, we introduce a novel probabilistic representation of deep learning, which provides an explicit explanation for the Deep Neural Networks (DNNs) in three aspects: (i) neurons define the energy of a Gibbs distribution; (ii) the hidden layers of DNNs formulate Gibbs distributions; and (iii) the whole architec…

The Gibbs sampler is one of the most popular algorithms for inference in statistical models. In this paper, we introduce a herding variant of this algorithm, called herded Gibbs, that is entirely deterministic. We prove that herded Gibbs has an $O(1/T)$ convergence rate for models with independent variables and for ful…

Gibbs sampling is a Markov chain Monte Carlo method that is often used for learning and inference on graphical models. Minibatching, in which a small random subset of the graph is used at each iteration, can help make Gibbs sampling scale to large graphical models by reducing its computational cost. In this paper, we p…

Ergodicity, this is to say, dynamics whose time averages coincide with ensemble averages, naturally leads to Boltzmann-Gibbs (BG) statistical mechanics, hence to standard thermodynamics. This formalism has been at the basis of an enormous success in describing, among others, the particular stationary state correspondin…

One of the fundamental tasks of science is to find explainable relationships between observed phenomena. One approach to this task that has received attention in recent years is based on probabilistic graphical modelling with sparsity constraints on model structures. In this paper, we describe two new approaches to Bay…

Novel connections between Neyman-Scott processes and Bayesian nonparametric mixture models enable scalable inference.

We prove a large deviation principle for a sequence of point processes defined by Gibbs probability measures on a Polish space. This is obtained as a consequence of a more general Laplace principle for the non-normalized Gibbs measures. We consider three main applications: Conditional Gibbs measures on compact spaces, …

We develop a framework for approximating collapsed Gibbs sampling in generative latent variable cluster models. Collapsed Gibbs is a popular MCMC method, which integrates out variables in the posterior to improve mixing. Unfortunately for many complex models, integrating out these variables is either analytically or co…

The pairwise influence matrix of Dobrushin has long been used as an analytical tool to bound the rate of convergence of Gibbs sampling. In this work, we use Dobrushin influence as the basis of a practical tool to certify and efficiently improve the quality of a discrete Gibbs sampler. Our Dobrushin-optimized Gibbs samp…

We review a simple model of closed economy, where the economic agents make money transactions and a saving criterion is present. We observe the Gibbs distribution for zero saving propensity, and non-Gibbs distributions otherwise. While the exact solution in the case of zero saving propensity is already known to be give…

For large scale on-line inference problems the update strategy is critical for performance. We derive an adaptive scan Gibbs sampler that optimizes the update frequency by selecting an optimum mini-batch size. We demonstrate performance of our adaptive batch-size Gibbs sampler by comparing it against the collapsed Gibb…

A plethora of natural, artificial and social systems exist which do not belong to the Boltzmann-Gibbs (BG) statistical-mechanical world, based on the standard additive entropy $S_{BG}$ and its associated exponential BG factor. Frequent behaviors in such complex systems have been shown to be closely related to $q$-stati…

New Gibbs sampling method improves MCMC efficiency.

We prove a generalization of the fundamental inequality of Guivarc'h relating entropy, drift and critical exponent to Gibbs measures on geometrically finite quotients of CAT(-1) metric spaces. For random walks with finite superexponential moment, we show that the equality is achieved if and only if the Gibbs density is…

Gibbs sampler contracts entropy under strong log-concavity, improving mixing time.

Developed a Particle-Gibbs sampler for Bayesian feature allocation models.

The notion of Berman-Gibbs stability was originally introduced by Robert Berman for $\mathbb{Q}$-Fano varieties $X$. We show that the pair $(X, -K_X)$ is K-stable (resp. K-semistable) provided that $X$ is Berman-Gibbs stable (resp. semistable).

Souriau studies Gibbs states for symplectic manifolds with group actions.

New model estimates Gibbs free energies using machine learning and isobaric-isothermal flows.

Modified Gibbs-Helmholtz equation geometric models for thermodynamics.

Study on Metropolis-within-Gibbs schemes for high-dimensional Bayesian models.

Introduces HMC method for sampling Gibbs densities.

New method improves uncertainty quantification in latent variable models.

DiGS improves sampling from multi-modal distributions.

The Gibbs sampler is a particularly popular Markov chain used for learning and inference problems in Graphical Models (GMs). These tasks are computationally intractable in general, and the Gibbs sampler often suffers from slow mixing. In this paper, we study the Swendsen-Wang dynamics which is a more sophisticated Mark…

Improved MALA method for neural networks uncertainty quantification.

This work analyzes Gibbs samplers for Bayesian hierarchical models without dimensionality constraints.

Bayesian inference for Levy density with Gibbs posterior in discrete sampling.

Flexible nonlinear Hawkes processes for time-varying systems.

Gibbs sampling is the de facto Markov chain Monte Carlo method used for inference and learning on large scale graphical models. For complicated factor graphs with lots of factors, the performance of Gibbs sampling can be limited by the computational cost of executing a single update step of the Markov chain. This cost …

A new algorithm for sampling from complex distributions.

Gibbs sampler mixes quickly for certain smooth distributions.

Study non-asymptotic Langevin Monte Carlo for Gibbs distributions.

The Gibbs algorithm's generalization error is bounded, improving with prior volume in low temperatures.

We investigate a class of feature allocation models that generalize the Indian buffet process and are parameterized by Gibbs-type random measures. Two existing classes are contained as special cases: the original two-parameter Indian buffet process, corresponding to the Dirichlet process, and the stable (or three-param…

Gibbs sampling, as a model learning method, is known to produce the most accurate results available in a variety of domains, and is a de facto standard in these domains. Yet, it is also well known that Gibbs random walks usually have bottlenecks, sometimes termed "local maxima", and thus samplers often return suboptima…

New bounds tighten the generalization error of Gibbs algorithm.

New Gibbs sampling reduces GLMB filtering complexity to linear time.

New diagnostic tool for assessing approximate Bayesian inference.

A fundamental task in machine learning and related fields is to perform inference on Bayesian networks. Since exact inference takes exponential time in general, a variety of approximate methods are used. Gibbs sampling is one of the most accurate approaches and provides unbiased samples from the posterior but it has hi…

New method transfers instances between domains using Gibbs Sampling and RBM.

A result about projections of Gibbs measures from a particular class arising in economic modeling is proved.

The paper tackles sampling from Gibbs measures with constrained support, providing a sampling guarantee.

We analyze Gibbs-based transfer learning algorithms using information theory.

Gibbs pruning optimizes neural networks by combining physics and regularization.

The paper proposes a Gibbs sampler for neural network posterior sampling.

GDiff tackles blind denoising with Gibbs sampling and Monte Carlo inference.