arXiv research

The study bounds the stability of Gaussian mixtures under small perturbations.

Bayesian approach approximates probability functions of Gaussian mixtures.

Corrected whitening restores orthogonality in high-dimensional spherical Gaussian mixtures.

New method clusters non-spherical Gaussian mixtures with fewer samples and time.

This work provides a computationally efficient and statistically consistent moment-based estimator for mixtures of spherical Gaussians. Under the condition that component means are in general position, a simple spectral decomposition technique yields consistent parameter estimates from low-order observable moments, wit…

Gradient descent on DDPM objective learns Gaussian mixtures efficiently.

We consider the problem of clustering data points in high dimensions, i.e. when the number of data points may be much smaller than the number of dimensions. Specifically, we consider a Gaussian mixture model (GMM) with non-spherical Gaussian components, where the clusters are distinguished by only a few relevant dimens…

EM algorithm achieves optimal sample complexity for well-separated Gaussian mixtures.

Tensor decomposition recovers Gaussian mixtures from moments.

Improved sample complexity for Gaussian Mixture Models using Pair Correlation Factor.

We propose a greedy variational method for decomposing a non-negative multivariate signal as a weighted sum of Gaussians, which, borrowing the terminology from statistics, we refer to as a Gaussian mixture model. Notably, our method has the following features: (1) It accepts multivariate signals, i.e. sampled multivari…

Algorithm learns means of spherical Gaussian mixtures efficiently in low dimensions.

Statistical and machine-learning algorithms are frequently applied to high-dimensional data. In many of these applications data is scarce, and often much more costly than computation time. We provide the first sample-efficient polynomial-time estimator for high-dimensional spherical Gaussian mixtures. For mixtures of a…

New GMM models fit high-dimensional data with fewer parameters.

This paper studies clustering and embedding in high-dimensional Gaussian mixture block models.

Efficient algorithm learns mixture models of heavy-tailed distributions.

Lower bounds show learning mixtures of linear classifiers is nearly impossible.

Efficiently learns mixtures of Gaussians without separation assumptions.

Study cost-effective fairness audits with partial feedback, improving over random exploration.

We present a one-pass sparsified Gaussian mixture model (SGMM). Given $N$ data points in $P$ dimensions, $X$, the model fits $K$ Gaussian distributions to $X$ and (softly) classifies each point to these clusters. After paying an up-front cost of $\mathcal{O}(NP\log P)$ to precondition the data, we subsample $Q$ entries…

In this paper, we study the problem of learning a mixture of Gaussians with streaming data: given a stream of $N$ points in $d$ dimensions generated by an unknown mixture of $k$ spherical Gaussians, the goal is to estimate the model parameters using a single pass over the data stream. We analyze a streaming version of …

This paper finds the noise threshold for learning Gaussian mixture models equals channel capacity.

Develops efficient algorithms for learning latent-variable models using implicit moment tensor computation.

Study free energy in spherical spin glasses, proving universality dichotomy.

We provide guarantees for learning latent variable models emphasizing on the overcomplete regime, where the dimensionality of the latent space can exceed the observed dimensionality. In particular, we consider multiview mixtures, spherical Gaussian mixtures, ICA, and sparse coding models. We provide tight concentration…

Algorithm distinguishes Gaussian mixtures from pure Gaussians in quasi-polynomial time.

We provide two fundamental results on the population (infinite-sample) likelihood function of Gaussian mixture models with $M \geq 3$ components. Our first main result shows that the population likelihood function has bad local maxima even in the special case of equally-weighted mixtures of well-separated and spherical…

Efficient private algorithms for estimating block models and mixture models.

The paper corrects for node degree in spectral clustering using random walk Laplacian.

X in R^D has mean zero and finite second moments. We show that there is a precise sense in which almost all linear projections of X into R^d (for d < D) look like a scale-mixture of spherical Gaussians -- specifically, a mixture of distributions N(0, sigma^2 I_d) where the weight of the particular sigma component is P …

Robustly clusters mixtures of Gaussians even with outliers.

The learning of mixture models can be viewed as a clustering problem. Indeed, given data samples independently generated from a mixture of distributions, we often would like to find the {\it correct target clustering} of the samples according to which component distribution they were generated from. For a clustering pr…

We present a simple, general technique for reducing the sample complexity of matrix and tensor decomposition algorithms applied to distributions. We use the technique to give a polynomial-time algorithm for standard ICA with sample complexity nearly linear in the dimension, thereby improving substantially on previous b…

We treat the problem of estimation of orientation parameters whose values are invariant to transformations from a spherical symmetry group. Previous work has shown that any such group-invariant distribution must satisfy a restricted finite mixture representation, which allows the orientation parameter to be estimated u…

Improved VB algorithm for NIG mixtures outperforms Gaussian mixtures for non-Gaussian data.

Revisits Gaussian process model with spherical harmonics for scalable deep learning.

A new distance metric for vMF distributions simplifies spherical data analysis.

Spatially constrained Gaussian mixture models reduce covariance complexity.

The paper uses Gaussian mixture models for Bayesian networks and proposes an optimization algorithm.

Optimal transport for vector Gaussian mixtures improves efficiency and structure preservation.

Paper uses Gaussian mixture models and Wasserstein distance for schema matching.

Deep learning is a hierarchical inference method formed by subsequent multiple layers of learning able to more efficiently describe complex relationships. In this work, Deep Gaussian Mixture Models are introduced and discussed. A Deep Gaussian Mixture model (DGMM) is a network of multiple layers of latent variables, wh…

Generative models use kernel smoothing for conditioning on small example sets.

Heavy-tailed distributions are widely used in robust mixture modelling due to possessing thick tails. As a computationally tractable subclass of the stable distributions, sub-Gaussian $α$-stable distribution received much interest in the literature. Here, we introduce a type of expectation maximization algorithm that e…

Study uniform rates for estimating Gaussian mixtures without separation assumption.

Study proposes a new metric for comparing Gaussian mixtures in RKHS.

Deep neural networks converge to Gaussian mixtures as layer width increases.

Proposes a new model for mixed membership in Gaussian mixture.