arXiv research

Bayesian inference in state-space models is challenging due to high-dimensional state trajectories. A viable approach is particle Markov chain Monte Carlo, combining MCMC and sequential Monte Carlo to form "exact approximations" to otherwise intractable MCMC methods. The performance of the approximation is limited to t…

New methods improve sampling from complex dynamical models.

Accelerates sampling from Gibbs distributions using ARWP method.

Bayesian feature allocation models are a popular tool for modelling data with a combinatorial latent structure. Exact inference in these models is generally intractable and so practitioners typically apply Markov Chain Monte Carlo (MCMC) methods for posterior inference. The most widely used MCMC strategies rely on an e…

Particle Markov chain Monte Carlo techniques rank among current state-of-the-art methods for probabilistic program inference. A drawback of these techniques is that they rely on importance resampling, which results in degenerate particle trajectories and a low effective sample size for variables sampled early in a prog…

We present a novel method in the family of particle MCMC methods that we refer to as particle Gibbs with ancestor sampling (PG-AS). Similarly to the existing PG with backward simulation (PG-BS) procedure, we use backward sampling to (considerably) improve the mixing of the PG kernel. Instead of using separate forward a…

New model estimates Gibbs free energies using machine learning and isobaric-isothermal flows.

New method uses Coulomb gases for Monte Carlo integration with reduced errors.

Bayesian model learns multiscale interactions in complex systems.

Particle Markov chain Monte Carlo (PMCMC) is a systematic way of combining the two main tools used for Monte Carlo statistical inference: sequential Monte Carlo (SMC) and Markov chain Monte Carlo (MCMC). We present a novel PMCMC algorithm that we refer to as particle Gibbs with ancestor sampling (PGAS). PGAS provides t…

New sampling method using regularized Wasserstein proximal for Gibbs distributions.

A new algorithm reduces bias in estimating model parameters.

Infinite Hidden Markov Models (iHMM's) are an attractive, nonparametric generalization of the classical Hidden Markov Model which can automatically infer the number of hidden states in the system. However, due to the infinite-dimensional nature of transition dynamics performing inference in the iHMM is difficult. In th…

Additive regression trees are flexible non-parametric models and popular off-the-shelf tools for real-world non-linear regression. In application domains, such as bioinformatics, where there is also demand for probabilistic predictions with measures of uncertainty, the Bayesian additive regression trees (BART) model, i…

New PG samplers improve inference in coupled state-space models.

We present the particle stochastic approximation EM (PSAEM) algorithm for learning of dynamical systems. The method builds on the EM algorithm, an iterative procedure for maximum likelihood inference in latent variable models. By combining stochastic approximation EM and particle Gibbs with ancestor sampling (PGAS), PS…

We propose a new algorithm to do posterior sampling of Kingman's coalescent, based upon the Particle Markov Chain Monte Carlo methodology. Specifically, the algorithm is an instantiation of the Particle Gibbs Sampling method, which alternately samples coalescent times conditioned on coalescent tree structures, and tree…

Particle MCMC involves using a particle filter within an MCMC algorithm. For inference of a model which involves an unobserved stochastic process, the standard implementation uses the particle filter to propose new values for the stochastic process, and MCMC moves to propose new values for the parameters. We show how p…

dSMC improves parallel processing of state-space models.

New method for efficient conditional sampling from diffusion models.

In this study, we present a multi-class graphical Bayesian predictive classifier that incorporates the uncertainty in the model selection into the standard Bayesian formalism. For each class, the dependence structure underlying the observed features is represented by a set of decomposable Gaussian graphical models. Emp…

Study birth-death dynamics for sampling Gibbs measures with nonconvex potentials.

Bayesian inference for biochemical reaction networks using jump-diffusion approximations.

Optimal income crossover found using particle swarm optimization.

We study the non-asymptotic behavior of a Coulomb gas on a compact Riemannian manifold. This gas is a symmetric n-particle Gibbs measure associated to the two-body interaction energy given by the Green function. We encode such a particle system by using an empirical measure. Our main result is a concentration inequalit…

Develops a new algorithm for estimating model parameters using interacting particle systems.

A new method for generating samples without training, using smoothed score matching.

In the present paper and the companion paper [9] a probabilistic (statistical-mechanical) approach to the construction of canonical metrics on a complex algebraic varieties X is introduced, by sampling "temperature deformed" determinantal point processes. The main new ingredient is a large deviation principle for Gibbs…

We show how to speed up Sequential Monte Carlo (SMC) for Bayesian inference in large data problems by data subsampling. SMC sequentially updates a cloud of particles through a sequence of distributions, beginning with a distribution that is easy to sample from such as the prior and ending with the posterior distributio…

Combines MALA and Adam for efficient uncertainty quantification in deep learning.

We propose a general statistical framework for clustering multiple time series that exhibit nonlinear dynamics into an a-priori-unknown number of sub-groups. Our motivation comes from neuroscience, where an important problem is to identify, within a large assembly of neurons, subsets that respond similarly to a stimulu…

New MCMC method speeds up quantum physics simulations by a factor of 100.

Novel analysis of EFP for finite-sum problems in neural networks.

Dual training method for EBMs with overparametrized neural networks.

We provide an exact solution to the ideal-gas-like models studied in econophysics to understand the microscopic origin of Pareto-law. In these class of models the key ingredient necessary for having a self-organized scale-free steady-state distribution is the trading or collision rule where agents or particles save a d…

The Gibbs sampler is one of the most popular algorithms for inference in statistical models. In this paper, we introduce a herding variant of this algorithm, called herded Gibbs, that is entirely deterministic. We prove that herded Gibbs has an $O(1/T)$ convergence rate for models with independent variables and for ful…

Gibbs sampling is a Markov chain Monte Carlo method that is often used for learning and inference on graphical models. Minibatching, in which a small random subset of the graph is used at each iteration, can help make Gibbs sampling scale to large graphical models by reducing its computational cost. In this paper, we p…

We develop a novel advanced Particle Markov chain Monte Carlo algorithm that is capable of sampling from the posterior distribution of non-linear state space models for both the unobserved latent states and the unknown model parameters. We apply this novel methodology to five population growth models, including models …

Advocates for a new posterior that predicts better than classical and generalised Bayes.

We prove a large deviation principle for a sequence of point processes defined by Gibbs probability measures on a Polish space. This is obtained as a consequence of a more general Laplace principle for the non-normalized Gibbs measures. We consider three main applications: Conditional Gibbs measures on compact spaces, …

We develop a framework for approximating collapsed Gibbs sampling in generative latent variable cluster models. Collapsed Gibbs is a popular MCMC method, which integrates out variables in the posterior to improve mixing. Unfortunately for many complex models, integrating out these variables is either analytically or co…

The pairwise influence matrix of Dobrushin has long been used as an analytical tool to bound the rate of convergence of Gibbs sampling. In this work, we use Dobrushin influence as the basis of a practical tool to certify and efficiently improve the quality of a discrete Gibbs sampler. Our Dobrushin-optimized Gibbs samp…

We review a simple model of closed economy, where the economic agents make money transactions and a saving criterion is present. We observe the Gibbs distribution for zero saving propensity, and non-Gibbs distributions otherwise. While the exact solution in the case of zero saving propensity is already known to be give…

For large scale on-line inference problems the update strategy is critical for performance. We derive an adaptive scan Gibbs sampler that optimizes the update frequency by selecting an optimum mini-batch size. We demonstrate performance of our adaptive batch-size Gibbs sampler by comparing it against the collapsed Gibb…

New Gibbs sampling method improves MCMC efficiency.

We prove a generalization of the fundamental inequality of Guivarc'h relating entropy, drift and critical exponent to Gibbs measures on geometrically finite quotients of CAT(-1) metric spaces. For random walks with finite superexponential moment, we show that the equality is achieved if and only if the Gibbs density is…

Gibbs sampler contracts entropy under strong log-concavity, improving mixing time.

The notion of Berman-Gibbs stability was originally introduced by Robert Berman for $\mathbb{Q}$-Fano varieties $X$. We show that the pair $(X, -K_X)$ is K-stable (resp. K-semistable) provided that $X$ is Berman-Gibbs stable (resp. semistable).