arXiv research

Improved phylogenetic inference using VBPI-Mixtures for tree topology and branch length.

A new method improves posterior approximation for complex distributions.

Paper introduces deep structured mixtures of Gaussian processes for scalable GP approximations.

SFSVI uses Gaussian mixtures to approximate neural network outputs for continual learning.

Improves posterior approximation speed for Dirichlet process mixture models.

We propose a general algorithm for approximating nonstandard Bayesian posterior distributions. The algorithm minimizes the Kullback-Leibler divergence of an approximating distribution to the intractable posterior distribution. Our method can be used to approximate any posterior distribution, provided that it is given i…

A new method uses mixture approximations to improve diffusion models for Bayesian inverse problems.

We present a Dirichlet process mixture model over discrete incomplete rankings and study two Gibbs sampling inference techniques for estimating posterior clusterings. The first approach uses a slice sampling subcomponent for estimating cluster parameters. The second approach marginalizes out several cluster parameters …

We propose a black-box variational inference method to approximate intractable distributions with an increasingly rich approximating class. Our method, termed variational boosting, iteratively refines an existing variational approximation by solving a sequence of optimization problems, allowing the practitioner to trad…

The paper develops efficient algorithms for variational inference with mixtures of isotropic Gaussians.

Enhances RL with function approximation, improving regret bounds.

Discriminative classifier for compositional data using hierarchical mixture of Generalized Dirichlet models.

Boosting variational inference (BVI) approximates an intractable probability density by iteratively building up a mixture of simple component distributions one at a time, using techniques from sparse convex optimization to provide both computational scalability and approximation error guarantees. But the guarantees hav…

AMF-VI uses adaptive mixtures of flows for robust VI across diverse distributions.

New method improves generative model performance by fully conditioning variational posteriors.

Variational inference (VI) provides fast approximations of a Bayesian posterior in part because it formulates posterior approximation as an optimization problem: to find the closest distribution to the exact posterior over some family of distributions. For practical reasons, the family of distributions in VI is usually…

A new recursive mixture estimation algorithm improves VAE inference efficiency and accuracy.

Study on reducing forgetting in neural networks using compression theory.

New method learns complex, multimodal distributions in ADVI.

This paper computes exact posterior distributions of mixture weights in hierarchical Bayesian models.

Fast Bayesian inference with adaptable priors for real-time applications.

Rank-1 BNNs improve efficiency and scalability of Bayesian neural nets.

We develop methods for efficient amortized approximate Bayesian inference over posterior distributions of probabilistic clustering models, such as Dirichlet process mixture models. The approach is based on mapping distributed, symmetry-invariant representations of cluster arrangements into conditional probabilities. Th…

Nonparametric mixture models based on the Dirichlet process are an elegant alternative to finite models when the number of underlying components is unknown, but inference in such models can be slow. Existing attempts to parallelize inference in such models have relied on introducing approximations, which can lead to in…

MixFlows uses a mixture of flows for efficient variational inference.

Probabilistic clustering models (or equivalently, mixture models) are basic building blocks in countless statistical models and involve latent random variables over discrete spaces. For these models, posterior inference methods can be inaccurate and/or very slow. In this work we introduce deep network architectures tra…

New method for summarizing Bayesian mixture models using sliced Wasserstein distances.

Variational inference is a popular technique to approximate a possibly intractable Bayesian posterior with a more tractable one. Recently, boosting variational inference has been proposed as a new paradigm to approximate the posterior by a mixture of densities by greedily adding components to the mixture. However, as i…

We study convergence rates of variational posterior distributions for nonparametric and high-dimensional inference. We formulate general conditions on prior, likelihood, and variational class that characterize the convergence rates. Under similar "prior mass and testing" conditions considered in the literature, the rat…

Natural-gradient methods enable fast and simple algorithms for variational inference, but due to computational difficulties, their use is mostly limited to \emph{minimal} exponential-family (EF) approximations. In this paper, we extend their application to estimate \emph{structured} approximations such as mixtures of E…

Posterior regularization enhances Bayesian hierarchical mixture clustering by improving node separation.

S-VBMC improves VBMC's exploration of complex posterior distributions.

Improved VAE estimation from incomplete data using variational mixtures.

We propose a simple and general variant of the standard reparameterized gradient estimator for the variational evidence lower bound. Specifically, we remove a part of the total derivative with respect to the variational parameters that corresponds to the score function. Removing this term produces an unbiased gradient …

Solves label switching in mixture models using optimal transport.

We describe computationally efficient methods for learning mixtures in which each component is a directed acyclic graphical model (mixtures of DAGs or MDAGs). We argue that simple search-and-score algorithms are infeasible for a variety of problems, and introduce a feasible approach in which parameter and structure sea…

Improved SBI with neural networks for complex models.

Mixture models and topic models generate each observation from a single cluster, but standard variational posteriors for each observation assign positive probability to all possible clusters. This requires dense storage and runtime costs that scale with the total number of clusters, even though typically only a few clu…

BBNN improves neural network accuracy and uncertainty quantification.

Bayesian approach learns nonparametric mixture components from heterogeneous data.

Posterior conformal prediction improves prediction interval validity for subgroups.

Bayesian methods are appealing in their flexibility in modeling complex data and ability in capturing uncertainty in parameters. However, when Bayes' rule does not result in tractable closed-form, most approximate inference algorithms lack either scalability or rigorous guarantees. To tackle this challenge, we propose …

Dynamic trees are mixtures of tree structured belief networks. They solve some of the problems of fixed tree networks at the cost of making exact inference intractable. For this reason approximate methods such as sampling or mean field approaches have been used. However, mean field approximations assume a factorized di…

Method for initializing Gaussian mixtures for variational inference with multi-modal distributions.

Flexible empirical Bayes for large-scale multiple linear regression.

A new method normalizes flow mixtures for better inference across different data types.

New method selects FMM components via variational Bayes.

We consider a binary unsupervised classification problem where each observation is associated with an unobserved label that we want to retrieve. More precisely, we assume that there are two groups of observation: normal and abnormal. The `normal' observations are coming from a known distribution whereas the distributio…