arXiv research

Theoretical analysis of entropy approximation for Gaussian mixtures.

Study finds the minimum number of finite Gaussian mixtures for best approximation.

New method uses subtractive mixture models for approximate inference.

Parameter estimation for model-based clustering using a finite mixture of normal inverse Gaussian (NIG) distributions is achieved through variational Bayes approximations. Univariate NIG mixtures and multivariate NIG mixtures are considered. The use of variational Bayes approximations here is a substantial departure fr…

Conditional diffusion models can approximate target distributions well with Gaussian-mixture reverse kernels.

Improved phylogenetic inference using VBPI-Mixtures for tree topology and branch length.

Information-theoretic measures such as the entropy, cross-entropy and the Kullback-Leibler divergence between two mixture models is a core primitive in many signal processing tasks. Since the Kullback-Leibler divergence of mixtures provably does not admit a closed-form formula, it is in practice either estimated using …

Discriminative classifier for compositional data using hierarchical mixture of Generalized Dirichlet models.

SFSVI uses Gaussian mixtures to approximate neural network outputs for continual learning.

We consider unsupervised estimation of mixtures of discrete graphical models, where the class variable corresponding to the mixture components is hidden and each mixture component over the observed variables can have a potentially different Markov graph structure and parameters. We propose a novel approach for estimati…

Method for initializing Gaussian mixtures for variational inference with multi-modal distributions.

Bayesian approach approximates probability functions of Gaussian mixtures.

Paper proposes new costs for learning multiple centers in MDNs.

Paper analyzes approximation and learning of MoEs with (P)ReLU activation.

New method improves Gaussian Mixture Model fitting speed.

A novel optimization-based Gaussian mixture reduction method using composite transportation divergence.

A new method uses mixture approximations to improve diffusion models for Bayesian inverse problems.

Gaussian Processes (GPs) are powerful non-parametric Bayesian regression models that allow exact posterior inference, but exhibit high computational and memory costs. In order to improve scalability of GPs, approximate posterior inference is frequently employed, where a prominent class of approximation techniques is ba…

Improves learning of spectral mixture kernels with approximate Bayesian inference.

Boosting variational inference (BVI) approximates an intractable probability density by iteratively building up a mixture of simple component distributions one at a time, using techniques from sparse convex optimization to provide both computational scalability and approximation error guarantees. But the guarantees hav…

Latent Dirichlet Allocation models discrete data as a mixture of discrete distributions, using Dirichlet beliefs over the mixture weights. We study a variation of this concept, in which the documents' mixture weight beliefs are replaced with squashed Gaussian distributions. This allows documents to be associated with e…

Improved Bayesian analysis for SVM models using a mixture sampler.

New sampling and identity-testing methods for mixtures of distributions that don't satisfy approximate tensorization of entropy.

A new method improves posterior approximation for complex distributions.

The paper develops efficient algorithms for variational inference with mixtures of isotropic Gaussians.

How can we train a statistical mixture model on a massive data set? In this work we show how to construct coresets for mixtures of Gaussians. A coreset is a weighted subset of the data, which guarantees that models fitting the coreset also provide a good fit for the original data set. We show that, perhaps surprisingly…

Plug-and-play L-GM-AMP improves CS recovery for any i.i.d. source prior.

Learning word representations has garnered greater attention in the recent past due to its diverse text applications. Word embeddings encapsulate the syntactic and semantic regularities of sentences. Modelling word embedding as multi-sense gaussian mixture distributions, will additionally capture uncertainty and polyse…

Diffusion models achieve high-quality samples from complex high-dimensional Gaussian mixtures without scaling with dimension.

Enhances RL with function approximation, improving regret bounds.

Improves posterior approximation speed for Dirichlet process mixture models.

A new method interpolates between sampling and variational inference using stochastic mixtures.

Paper uses Gaussian mixture models and Wasserstein distance for schema matching.

In binary-transaction data-mining, traditional frequent itemset mining often produces results which are not straightforward to interpret. To overcome this problem, probability models are often used to produce more compact and conclusive results, albeit with some loss of accuracy. Bayesian statistics have been widely us…

Natural-gradient methods enable fast and simple algorithms for variational inference, but due to computational difficulties, their use is mostly limited to \emph{minimal} exponential-family (EF) approximations. In this paper, we extend their application to estimate \emph{structured} approximations such as mixtures of E…

The two most extended density-based approaches to clustering are surely mixture model clustering and modal clustering. In the mixture model approach, the density is represented as a mixture and clusters are associated to the different mixture components. In modal clustering, clusters are understood as regions of high d…

Clustering of data sets is a standard problem in many areas of science and engineering. The method of spectral clustering is based on embedding the data set using a kernel function, and using the top eigenvectors of the normalized Laplacian to recover the connected components. We study the performance of spectral clust…

Transformers learn to cluster Gaussian mixtures as well as the EM algorithm.

New method selects FMM components via variational Bayes.

New algorithm learns optimal path in reinforcement learning with linear approximations.

In this paper we propose a new class of Dynamic Mixture Models (DAMMs) being able to sequentially adapt the mixture components as well as the mixture composition using information coming from the data. The information driven nature of the proposed class of models allows to exactly compute the full likelihood and to avo…

A novel PP algorithm using GMMs and GAs for detecting informative structures.

The paper tackles learning mixtures of two multinomial logits, showing identifiability and presenting an algorithm.

Estimates Gaussian mixtures from weighted samples efficiently.

Mixture model-based clustering has become an increasingly popular data analysis technique since its introduction over fifty years ago, and is now commonly utilized within a family setting. Families of mixture models arise when the component parameters, usually the component covariance (or scale) matrices, are decompose…

Paper hypothesizes MLP layers in LLMs can be approximated by sparse Mixture of Experts.

Paper proposes a new model and methods for robustly de-interleaving HMP mixtures.

New slicing methods speed up Gaussian mixture Wasserstein distance computations.