Improved VAE estimation from incomplete data using variational mixtures.
problem Estimating VAEs from incomplete data increases posterior complexity.
method Introducing variational mixtures based on finite and imputation distributions.
result Variational mixtures improve VAE estimation accuracy from incomplete data.
Variational boosting refines posterior approximations through iterative optimization.
problem Approximating intractable distributions with rich approximations.
method Iteratively solves optimization problems to refine variational approximations.
result Posterior inferences using variational boosting are more accurate and efficient.
Improved phylogenetic inference using VBPI-Mixtures for tree topology and branch length.
problem Multimodality of tree-topology posterior distributions in phylogenetic inference.
method VBPI-Mixtures algorithm that uses mixture learning within the BBVI framework.
result VBPI-Mixtures captures tree-topology distributions better than VBPI.
SFSVI uses Gaussian mixtures to approximate neural network outputs for continual learning.
problem Learning new tasks without forgetting old ones in neural networks.
method Sequential function-space variational inference with Gaussian mixture approximation.
result Gaussian mixture SFSVI outperforms other methods in continual learning.
Boosting variational inference improves posterior approximations using mixtures of base distributions.
problem VI approximations do not converge to exact posterior due to constrained distribution family.
method Develops BVI algorithm that iteratively mixes current approximation with new components from base distribution family.
result BVI yields progressively more accurate posterior approximations, capturing multimodality and nonstandard shapes.
AMF-VI uses adaptive mixtures of flows for robust VI across diverse distributions.
problem Inconsistent behavior of single-flow models across different distributions.
method Sequential expert training of individual flows and adaptive global weight estimation via likelihood-driven updates.
result AMF-VI achieves lower negative log-likelihood and stable gains in transport metrics across various posterior families.
Paper proposes VampPrior to improve VAEs.
problem Improving variational auto-encoders (VAEs) with a new prior.
method Integrates a Variational Mixture of Posteriors (VampPrior) into VAEs.
result The hierarchical VampPrior architecture learns better models and avoids latent dimension issues.
The paper proves consistency of GVI posteriors under minimal conditions.
problem Consistency of generalized variational inference posteriors.
method Proves consistency using Γ-convergence theory. result GVI posteriors are consistent and collapse to the population-optimal parameter value.
The paper develops efficient algorithms for variational inference with mixtures of isotropic Gaussians.
problem Efficiently approximating multimodal Bayesian posteriors.
method Develops a variational framework and efficient algorithms for mixtures of isotropic Gaussians.
result The approach provides accurate approximations of multimodal Bayesian posteriors while being memory and computationally efficient.
UBVI improves variational inference by preventing degeneracy and improving scalability.
problem Degeneracy and scalability issues in variational inference.
method Exploits Hellinger metric geometry to prevent degeneracy, simplifies weight optimization, and uses scalable exponential family mixture components.
result Output of UBVI converges to best possible approximation in any mixture family, even when misspecified.
Paper presents a reparameterized DP-DLGMM for clustering.
problem Non-parametric DP priors in DLGMM are hard to couple with variational inference.
method Closed-form updates for DP-DLGMM's variational posterior.
result Model generates realistic samples and performs competitively in semi-supervised settings.
A new recursive mixture estimation algorithm improves VAE inference efficiency and accuracy.
problem Inaccurate posterior approximation in traditional VAEs.
method Recursive mixture estimation algorithm using functional gradient approach for iterative component selection.
result Significantly higher test data likelihood compared to state-of-the-art methods on benchmark datasets.
Boosting variational inference uses optimization to improve posterior approximations.
problem Improving the tractability of Bayesian posterior approximations.
method An optimization-based approach to greedily add components to a mixture of densities.
result Established convergence properties and rates for the boosting variational inference algorithm.
We extend natural-gradient methods to mixtures of exponential-family distributions, improving inference speed.
problem Complex, multimodal posterior distributions are difficult to approximate with simple exponential-family distributions.
method We use minimal conditional-EF representations and derive simple natural-gradient updates.
result Our natural-gradient method converges faster than black-box methods with reparameterization gradients.
Study convergence rates of variational posterior distributions for inference.
problem Characterize convergence rates of variational posterior distributions for nonparametric and high-dimensional inference.
method Formulate general conditions on prior, likelihood, and variational class to characterize convergence rates. Propose novel prior mass conditions for specific prior distributions.
result The convergence rate of variational posterior distributions is the sum of the convergence rate of the true posterior and the variational approximation error.
Simplifies gradient estimation for variational inference with reduced variance.
problem Reducing variance in gradient estimation for variational inference.
method Proposes a new gradient estimator by removing the score function term from the reparameterized gradient.
result The new estimator has zero variance as the approximate posterior approaches the exact posterior.
Elliptical processes extend Gaussian models with heavier tails.
problem Regression and classification with non-Gaussian likelihoods or heavy tails.
method Spline normalizing flow for variational inference of elliptical distributions.
result Elliptical processes outperform Gaussian processes in non-Gaussian settings.
Improved vector quantization using Gaussian mixtures for better codebook utilization.
problem Training instability and information loss in discrete vector quantization.
method Generalized vector quantization with Gaussian mixture model and aggregated categorical posterior evidence lower bound.
result GM-VQ improves codebook utilization and reduces information loss without heuristics.
A new method improves posterior approximation for complex distributions.
problem Difficulty in capturing multimodal and heavy-tailed posteriors with standard normalizing flows.
method StiCTAF: stick-breaking mixture base with component-wise tail adaptation.
result Improved tail recovery and better mode coverage compared to benchmarks.
Improves posterior approximation speed for Dirichlet process mixture models.
problem Inefficiency of stochastic variational inference in large datasets.
method Uses stochastic gradient ascent with adaptive stepsize optimization.
result Adaptive stepsize improves speed and performance of posterior approximation.
A new framework enriches variational family with auxiliary variables.
problem Challenges in maximizing ELBO with complex variational distributions.
method Proposes a novel framework using auxiliary variables to enrich variational family.
result Flexible inference model built from probabilistic mixture of simple variational posteriors.
New method improves generative model performance by fully conditioning variational posteriors.
problem Inaccurate inference due to partial conditioning of variational posteriors in sequential LVMs.
method Introduces fully-conditioned approximate posteriors to improve generative model performance.
result Improves generative modelling and multi-step prediction performance.
Discriminative classifier for compositional data using hierarchical mixture of Generalized Dirichlet models.
problem Classifying compositional data, especially in spam detection and color space identification.
method Hierarchical mixture of discriminative Generalized Dirichlet classifiers, using variational approximation for parameter learning.
result First time a variational upper-bound for Generalized Dirichlet mixture is proposed in literature.
Study on reducing forgetting in neural networks using compression theory.
problem Catastrophic forgetting in neural networks.
method Defined forgetting as increased description lengths, compared variational posterior approaches to prequential coding methods.
result Proposed a new continual learning method combining ML plug-in and Bayesian mixture codes.
MixFlows uses a mixture of flows for efficient variational inference.
problem Efficient and reliable variational inference for complex models.
method A new variational family of mixed flows with efficient algorithms and convergence guarantees.
result MixFlows provides more reliable posterior approximations and comparable sample quality to MCMC methods.
New method learns complex, multimodal distributions in ADVI.
problem Learning unimodal approximate posteriors limits ADVI's ability to capture complex data structures.
method Use stratified sampling to allow mixture distributions as approximate posteriors, derive a tighter evidence lower bound.
result SIWAE objective allows ADVI to learn more complex, multimodal distributions, improving accuracy and calibration.
Paper formulates particle flow using variational inference and Fisher-Rao gradient flow.
problem Estimating posterior densities in probabilistic models.
method Variational formulation of particle flow, Fisher-Rao gradient flow, Gaussian and Gaussian mixture approximations.
result Gaussian and Gaussian mixture approximations of Fisher-Rao particle flow reduce to Exact Daum and Huang particle flow under linear Gaussian assumptions.
Mixture models and topic models generate each observation from a single cluster, but standard variational posteriors for each observation assign positive probability to all possible clusters. This requires dense storage and runtime costs that scale with the total number of clusters, even though typically only a few clu…
Bayesian mixture models inference in federated learning
problem Inference of Bayesian mixture models in federated learning
method Consensus Monte Carlo approach
result Recovery of small clusters with greater accuracy than standard MCMC
A scalable method for efficient inference in Gaussian process regression networks.
problem Intractable inference in Gaussian process regression networks (GPRN).
method Tensorization of output space, tensor/matrix-normal variational posteriors, joint optimization, and exploiting Kronecker product structure.
result Captures posterior dependencies and improves inference quality for large number of outputs.
VAEs struggle with surjective multimodal data, especially class labels describing images.
problem VAEs struggle to capture variability in surjective multimodal data.
method Theoretical and empirical demonstration of VAEs with a mixture of experts posterior.
result VAEs with a mixture of experts posterior can disregard variation in surjective multimodal data.
SMI uses mixture models to improve SVGD's performance in Bayesian inference.
problem Variance collapse in SVGD for Bayesian inference, especially with small models.
method Generalizes SVGD to Stein mixture models, optimizing an ELBO lower bound.
result SMI avoids variance collapse and accurately estimates uncertainty for small BNNs.
BBNN improves neural network accuracy and uncertainty quantification.
problem Overfitting and lack of interpretability in probabilistic neural networks.
method Boosted Bayesian Neural Networks (BBNN) using Boosting Variational Inference (BVI).
result BBNN achieves ~5% higher accuracy and superior uncertainty quantification.
This paper models time-series data with a mixture of Markov chains, automatically determining the number of components.
problem Tackles the inability of common Markov state modeling frameworks to discern heterogeneities in complex data.
method Uses a mixture of Markov chains and variational expectation-maximization algorithm for automatic component selection.
result Achieves performance consistent with theoretically optimal error scaling, identifying meaningful heterogeneities in various data sets.
Method for initializing Gaussian mixtures for variational inference with multi-modal distributions.
problem Challenges in variational inference with Gaussian mixtures due to multimodality and nonconvex loss functions.
method Optimization to find local maxima, local Gaussian approximations, and constrained least squares regression.
result Robust initialization improves variational inference performance and scalability.
OPVI uses operators to optimize variational objectives, improving scalability and approximation quality.
problem Statistical properties of classical variational inference can be undesirable.
method OPVI redefines variational inference using operators to optimize variational objectives.
result OPVI enables data subsampling and variational programs, improving scalability and approximation quality.
We propose a general algorithm for approximating nonstandard Bayesian posterior distributions. The algorithm minimizes the Kullback-Leibler divergence of an approximating distribution to the intractable posterior distribution. Our method can be used to approximate any posterior distribution, provided that it is given i…
A new method normalizes flow mixtures for better inference across different data types.
problem Inference failure across diverse posterior geometries in normalizing flows.
method Introduces a two-stage framework with a stable global weighting mechanism based on sEMA.
result Achieves consistent NLL improvements and stable weight trajectories over baselines.
Improves variational inference for sparse models using mixtures of exponential families.
problem Intractability of posterior distributions in Bayesian sparse models.
method Flexible mean field variational inference using mixtures of non-overlapping exponential families.
result Mixtures of exponential families with non-overlapping support form an exponential family, enabling analytical updates.
New method selects FMM components via variational Bayes.
problem Selecting the correct number of components in finite mixture models.
method Variational Bayes with mean-field approximation.
result Consistency of model selection based on ELBO maximization.
NVA combines variational posteriors, annealing, and natural-gradient learning for multimodal optimization.
problem Finding multiple global and local modes in nonconvex objectives.
method NVA integrates variational posteriors, annealing, and natural-gradient learning.
result NVA outperforms gradient descent and evolution strategies on simulations and real-world problems.
A new method uncovers discrete and continuous factors in gene expression data.
problem Jointly identifying discrete and continuous factors of variability without supervision.
method cpl-mixVAE framework using multiple interacting networks.
result The method successfully uncovers discrete and continuous factors in gene expression data.
HyperVAE encodes distributions of distributions using variational inference.
problem Modeling distributions of distributions efficiently and preserving information.
method Variational inference with Gaussian mixture models and matrix-network decoders.
result HyperVAE encodes parameters of a VAE in a low-dimensional Gaussian distribution, preserving more information.
S-VBMC improves VBMC's exploration of complex posterior distributions.
problem Efficient inference for computationally expensive models with complex posterior distributions.
method Stacking multiple independent VBMC runs to create a robust global posterior approximation.
result Significant improvements in posterior approximation quality across various applications.
We consider a binary unsupervised classification problem where each observation is associated with an unobserved label that we want to retrieve. More precisely, we assume that there are two groups of observation: normal and abnormal. The `normal' observations are coming from a known distribution whereas the distributio…
We derive a novel variational expectation maximization approach based on truncated posterior distributions. Truncated distributions are proportional to exact posteriors within subsets of a discrete state space and equal zero otherwise. The treatment of the distributions' subsets as variational parameters distinguishes …
A new model predicts multivariate regression using similarities to data points.
problem Complex, high-dimensional input-output relationships.
method Bayesian mixture-of-experts with conditional Gaussian mixtures and variational Bayes.
result Outperforms competitors in high-dimensional settings.
A new model combines normalizing flows with mixture components for better density estimation.
problem Lack of explicit probability density functions in deep generative models.
method Variational mixture of normalizing flows, using variational inference and neural network parameters.
result The model can perform density estimation, semi-supervised learning, and clustering.