arXiv research

The study bounds the stability of Gaussian mixtures under small perturbations.

New method clusters non-spherical Gaussian mixtures with fewer samples and time.

Bayesian approach approximates probability functions of Gaussian mixtures.

Corrected whitening restores orthogonality in high-dimensional spherical Gaussian mixtures.

This work provides a computationally efficient and statistically consistent moment-based estimator for mixtures of spherical Gaussians. Under the condition that component means are in general position, a simple spectral decomposition technique yields consistent parameter estimates from low-order observable moments, wit…

Gradient descent on DDPM objective learns Gaussian mixtures efficiently.

We consider the problem of clustering data points in high dimensions, i.e. when the number of data points may be much smaller than the number of dimensions. Specifically, we consider a Gaussian mixture model (GMM) with non-spherical Gaussian components, where the clusters are distinguished by only a few relevant dimens…

EM algorithm achieves optimal sample complexity for well-separated Gaussian mixtures.

Algorithm learns means of spherical Gaussian mixtures efficiently in low dimensions.

Tensor decomposition recovers Gaussian mixtures from moments.

Statistical and machine-learning algorithms are frequently applied to high-dimensional data. In many of these applications data is scarce, and often much more costly than computation time. We provide the first sample-efficient polynomial-time estimator for high-dimensional spherical Gaussian mixtures. For mixtures of a…

Improved sample complexity for Gaussian Mixture Models using Pair Correlation Factor.

We propose a greedy variational method for decomposing a non-negative multivariate signal as a weighted sum of Gaussians, which, borrowing the terminology from statistics, we refer to as a Gaussian mixture model. Notably, our method has the following features: (1) It accepts multivariate signals, i.e. sampled multivari…

Lower bounds show learning mixtures of linear classifiers is nearly impossible.

Efficient algorithm learns mixture models of heavy-tailed distributions.

New GMM models fit high-dimensional data with fewer parameters.

This paper studies clustering and embedding in high-dimensional Gaussian mixture block models.

Efficiently learns mixtures of Gaussians without separation assumptions.

In this paper, we study the problem of learning a mixture of Gaussians with streaming data: given a stream of $N$ points in $d$ dimensions generated by an unknown mixture of $k$ spherical Gaussians, the goal is to estimate the model parameters using a single pass over the data stream. We analyze a streaming version of …

Algorithm distinguishes Gaussian mixtures from pure Gaussians in quasi-polynomial time.

Study cost-effective fairness audits with partial feedback, improving over random exploration.

This paper finds the noise threshold for learning Gaussian mixture models equals channel capacity.

We present a one-pass sparsified Gaussian mixture model (SGMM). Given $N$ data points in $P$ dimensions, $X$, the model fits $K$ Gaussian distributions to $X$ and (softly) classifies each point to these clusters. After paying an up-front cost of $\mathcal{O}(NP\log P)$ to precondition the data, we subsample $Q$ entries…

Develops efficient algorithms for learning latent-variable models using implicit moment tensor computation.

X in R^D has mean zero and finite second moments. We show that there is a precise sense in which almost all linear projections of X into R^d (for d < D) look like a scale-mixture of spherical Gaussians -- specifically, a mixture of distributions N(0, sigma^2 I_d) where the weight of the particular sigma component is P …

Study free energy in spherical spin glasses, proving universality dichotomy.

We provide guarantees for learning latent variable models emphasizing on the overcomplete regime, where the dimensionality of the latent space can exceed the observed dimensionality. In particular, we consider multiview mixtures, spherical Gaussian mixtures, ICA, and sparse coding models. We provide tight concentration…

We provide two fundamental results on the population (infinite-sample) likelihood function of Gaussian mixture models with $M \geq 3$ components. Our first main result shows that the population likelihood function has bad local maxima even in the special case of equally-weighted mixtures of well-separated and spherical…

Robustly clusters mixtures of Gaussians even with outliers.

The paper corrects for node degree in spectral clustering using random walk Laplacian.

We present a simple, general technique for reducing the sample complexity of matrix and tensor decomposition algorithms applied to distributions. We use the technique to give a polynomial-time algorithm for standard ICA with sample complexity nearly linear in the dimension, thereby improving substantially on previous b…

Efficient private algorithms for estimating block models and mixture models.

We treat the problem of estimation of orientation parameters whose values are invariant to transformations from a spherical symmetry group. Previous work has shown that any such group-invariant distribution must satisfy a restricted finite mixture representation, which allows the orientation parameter to be estimated u…

The learning of mixture models can be viewed as a clustering problem. Indeed, given data samples independently generated from a mixture of distributions, we often would like to find the {\it correct target clustering} of the samples according to which component distribution they were generated from. For a clustering pr…

A new distance metric for vMF distributions simplifies spherical data analysis.

Improved VB algorithm for NIG mixtures outperforms Gaussian mixtures for non-Gaussian data.

We study high-dimensional distribution learning in an agnostic setting where an adversary is allowed to arbitrarily corrupt an $\varepsilon$-fraction of the samples. Such questions have a rich history spanning statistics, machine learning and theoretical computer science. Even in the most basic settings, the only known…

Revisits Gaussian process model with spherical harmonics for scalable deep learning.

Fourier PCA is Principal Component Analysis of a matrix obtained from higher order derivatives of the logarithm of the Fourier transform of a distribution.We make this method algorithmic by developing a tensor decomposition method for a pair of tensors sharing the same vectors in rank-$1$ decompositions. Our main appli…

The paper uses Gaussian mixture models for Bayesian networks and proposes an optimization algorithm.

Spatially constrained Gaussian mixture models reduce covariance complexity.

Deep neural networks converge to Gaussian mixtures as layer width increases.

Optimal transport for vector Gaussian mixtures improves efficiency and structure preservation.

Existence and uniqueness of spherical helicoidal surfaces in 3-sphere via spherical curves.

Theoretical analysis of entropy approximation for Gaussian mixtures.

Generative models use kernel smoothing for conditioning on small example sets.

Paper uses Gaussian mixture models and Wasserstein distance for schema matching.

Algorithm estimates mixtures of arbitrary Gaussians robustly in presence of corruptions.