Spatially constrained Gaussian mixture models reduce covariance complexity.
problem High dimensionality in finite mixture models for spatial data.
method Spatial covariance constraint with only four free parameters.
result Improves clustering of multi-way spatial data and inference of spatial patterns.
Study finds the minimum number of finite Gaussian mixtures for best approximation.
problem Finding the minimum number of finite Gaussian mixtures for best approximation.
method Local moment matching for upper bound and spectral analysis for lower bound.
result Corrects a previous lower bound in the case of Gaussian mixing distributions.
Proposes a new model for clustering with heavier tails.
problem Clustering with heavy-tailed data.
method Finite mixture of skewed sub-Gaussian stable distributions, maximum likelihood estimation, EM algorithm.
result The proposed model can robustly handle heavy-tailed data.
Improved VB algorithm for NIG mixtures outperforms Gaussian mixtures for non-Gaussian data.
problem Clustering non-Gaussian data, especially heavy-tailed and asymmetric.
method Proposed an improved VB algorithm for NIG mixture models and extended Dirichlet process mixture models.
result Outperforms Gaussian mixtures and existing NIG mixture models, especially for highly non-normative data.
Understanding separation effects on parameter estimation in finite Gaussian mixtures
problem Minimum component separation impact on convergence rates in finite Gaussian mixtures
method Developing a unified geometric framework using Hellinger lower bounds and specialized moment-extraction test functions
result Separation complexity driven by spatial configuration of mixture components
A fast Modal EM algorithm for Gaussian mixtures.
problem Clustering with Gaussian mixtures.
method Modal EM algorithm for Gaussian mixtures.
result High flexibility in various clustering contexts.
Conditional diffusion models can approximate target distributions well with Gaussian-mixture reverse kernels.
problem Approximating target distributions in conditional diffusion models.
method Using finite Gaussian mixtures with ReLU-network logits as reverse kernels, reducing the problem to static conditional density approximation.
result The resulting neural reverse-kernel class is dense in conditional KL divergence under exact terminal matching.
This paper is concerned with an important issue in finite mixture modelling, the selection of the number of mixing components. We propose a new penalized likelihood method for model selection of finite multivariate Gaussian mixture models. The proposed method is shown to be statistically consistent in determining of th…
Study learns Gaussian mixtures from censored data.
problem Learning Gaussian mixtures with incomplete data.
method Proposes an algorithm to estimate weights and means with minimal samples.
result Achieves accurate estimation with very few samples.
A new method detects outliers using ensembles of Dirichlet process mixtures.
problem Challenges in unsupervised outlier detection using Dirichlet process mixtures.
method Ensembles of Dirichlet process Gaussian mixtures with random subspace and subsampling.
result Empirically outperforms existing approaches in unsupervised outlier detection.
NPMLE estimator automatically chooses the right model complexity for Gaussian mixtures.
problem Learning mixture models and empirical Bayes estimation with non-convex likelihoods.
method Nonparametric maximum likelihood estimator (NPMLE) using complex-analytic techniques.
result NPMLE solution has O ( log n ) O(\log n) O ( log n ) atoms with high probability, improving model complexity. New kernel models multi-output Gaussian processes accurately.
problem Challenges in modelling cross-covariances for multiple-output Gaussian processes.
method Replaced Gaussian components with block components of finite bandwidth in spectral mixture kernel.
result First multi-output generalization of spectral mixture kernel that can approximate any stationary multi-output kernel to arbitrary precision.
The likelihood function of a finite mixture model is a non-convex function with multiple local maxima and commonly used iterative algorithms such as EM will converge to different solutions depending on initial conditions. In this paper we ask: is it possible to assess how far we are from the global maximum of the likel…
This work approximates finite neural networks with Gaussian processes, providing error bounds and applications in prior selection.
problem Approximating finite neural networks with Gaussian processes for error bounds and uncertainty quantification.
method Iterative approximation of neural network layers as mixtures of Gaussian processes, using optimal transport and Gaussian processes.
result The ability to return a mixture of Gaussian processes that is ε-close to the neural network at a finite set of input points.
SFSVI uses Gaussian mixtures to approximate neural network outputs for continual learning.
problem Learning new tasks without forgetting old ones in neural networks.
method Sequential function-space variational inference with Gaussian mixture approximation.
result Gaussian mixture SFSVI outperforms other methods in continual learning.
Parameter estimation for model-based clustering using a finite mixture of normal inverse Gaussian (NIG) distributions is achieved through variational Bayes approximations. Univariate NIG mixtures and multivariate NIG mixtures are considered. The use of variational Bayes approximations here is a substantial departure fr…
We will establish that the VC dimension of the class of d-dimensional ellipsoids is (d^2+3d)/2, and that maximum likelihood estimate with N-component d-dimensional Gaussian mixture models induces a geometric class having VC dimension at least N(d^2+3d)/2. Keywords: VC dimension; finite dimensional ellipsoid; Gaussian m…
Survey on Bayesian inference for Gaussian mixture models.
problem Estimating parameters of Gaussian mixture models using Bayesian methods.
method Uses Bayesian inference to estimate parameters and uncertainty of Gaussian mixture models.
result Bayesian approach provides point estimates and associated uncertainty for mixture model parameters.
The paper optimizes estimating high-dimensional Gaussian mixtures without separation conditions.
problem Estimating the mixing distribution in high-dimensional Gaussian mixtures without separation conditions.
method The method of moments and careful application of moment tensors.
result The minimax rate of estimating the mixing distribution in Wasserstein distance is Θ ( ( d / n ) 1 / 4 + n − 1 / ( 4 k − 2 ) ) Θ((d/n)^{1/4} + n^{-1/(4k-2)}) Θ (( d / n ) 1/4 + n − 1/ ( 4 k − 2 ) ) . New algorithm for tensor decomposition and Gaussian mixture models.
problem Efficiently decompose overcomplete order-3 tensors and estimate parameters of Gaussian mixtures.
method Proposes Jennrich's algorithm adapted for tensor decomposition and Gaussian mixture models.
result Efficient algorithm for decomposing symmetric overcomplete order-3 tensors and estimating parameters of Gaussian mixtures.
Model-based clustering imposes a finite mixture modelling structure on data for clustering. Finite mixture models assume that the population is a convex combination of a finite number of densities, the distribution within each population is a basic assumption of each particular model. Among all distributions that have …
Study shows private learning of mixtures of Gaussians is possible with polynomial samples.
problem Estimating mixtures of Gaussians under differential privacy constraints.
method Developed a new framework for privately learning mixtures of Gaussians without structural assumptions.
result Polynomial number of samples (poly(k,d,1/α,1/ε,log(1/δ))) sufficient for estimation up to total variation distance α with (ε, δ)-DP.
A new vine copula mixture model improves clustering accuracy for non-Gaussian data.
problem Finite mixture models struggle with asymmetric tail dependencies and non-elliptical clusters.
method Proposes a vine copula mixture model for clustering non-Gaussian data, addressing model selection and parameter estimation.
result Significant improvement in clustering accuracy for data with asymmetric tail dependencies or non-Gaussian margins.
We study the Nonparametric Maximum Likelihood Estimator (NPMLE) for estimating Gaussian location mixture densities in d d d -dimensions from independent observations. Unlike usual likelihood-based methods for fitting mixtures, NPMLEs are based on convex optimization. We prove finite sample results on the Hellinger accurac…
New approach for distributed learning of Gaussian mixtures.
problem Large datasets distributed across different centers.
method Split-and-conquer approach with MM algorithm.
result New estimator is consistent and retains root-n consistency.
In recent years, correntropy and its applications in machine learning have been drawing continuous attention owing to its merits in dealing with non-Gaussian noise and outliers. However, theoretical understanding of correntropy, especially in the statistical learning context, is still limited. In this study, within the…
In this study, a numerical quadrature for the generalized inverse Gaussian distribution is derived from the Gauss-Hermite quadrature by exploiting its relationship with the normal distribution. The proposed quadrature is not Gaussian, but it exactly integrates the polynomials of both positive and negative orders. Using…
Paper detects gradual changes in cluster structure using MC fusion.
problem Detecting gradual changes in cluster structure over time.
method MC fusion for multiple mixture numbers, examining MC transition.
result Accurately captures cluster structure during transitional periods.
Study of two-layer NNs under Gaussian mixtures data, proving polynomial models equivalent to neural networks.
problem Training and generalization performance of two-layer NNs under structured Gaussian mixture data.
method Asymptotic analysis of two-layer NNs after one gradient descent step under Gaussian mixture data assumption.
result High-order polynomial models equivalent to nonlinear neural networks under certain conditions.
EM algorithm achieves optimal sample complexity for well-separated Gaussian mixtures.
problem Estimating parameters of well-separated Gaussian mixtures.
method New EM convergence proof for well-separated Gaussian mixtures.
result EM algorithm converges with Ω ( log k ) Ω(\sqrt{\log k}) Ω ( log k ) separation, achieving O ( k d / ε 2 ) O(kd/ε^2) O ( k d / ε 2 ) samples. Finite mixture models have become a popular tool for clustering. Amongst other uses, they have been applied for clustering longitudinal data and clustering high-dimensional data. In the latter case, a latent Gaussian mixture model is sometimes used. Although there has been much work on clustering using latent variables…
Paper proposes a VB method for TS-SBP mixture models with reduced computational cost.
problem Efficiently learning tree-structured stick-breaking process mixture models.
method Utilizes Bayes coding algorithm for context tree models to calculate sums over all possible trees.
result Proposes a learning algorithm with less computational cost for TS-SBP mixture of Gaussians.
New algorithms improve Gaussian mixture model estimation in DFL with heterogeneous data.
problem Bias in EM algorithm for Gaussian mixtures in DFL with heterogeneous data.
method MNEM and semi-MNEM algorithms integrating historical and partially labeled data.
result MNEM and semi-MNEM achieve asymptotic efficiency and improved convergence.
We study high-dimensional Gaussian mixture classification using statistical physics methods.
problem Classifying high-dimensional Gaussian mixture with general covariance matrices.
method Replica method from statistical physics for asymptotic analysis of convex classifiers.
result Construction and validation of a de-biased estimator for variable selection.
The paper studies scaling limits of Wasserstein metrics on Gaussian mixture models.
problem Understanding the scaling limits of Wasserstein metrics on Gaussian mixture models.
method Scaling limit approach on Gaussian mixture models, including inhomogeneous and extended models.
result Existence of the limit of the Wasserstein metric after renormalization for GMMs with zero variance.
In many applications, a finite mixture is a natural model, but it can be difficult to choose an appropriate number of components. To circumvent this choice, investigators are increasingly turning to Dirichlet process mixtures (DPMs), and Pitman-Yor process mixtures (PYMs), more generally. While these models may be well…
Regularization helps improve classification of noisy high-dimensional data.
problem Classifying high-dimensional noisy Gaussian mixture with limited oracle knowledge.
method Analysis of regularized convex classifiers including ridge, hinge, and logistic regression.
result Regularization can reach Bayes-optimal performance under certain conditions.
Proposes a VAE with Student- t t t mixture model for authorship attribution.
problem Traditional authorship attribution in closed-set scenarios.
method Extends variational autoencoder with embedded Student- t t t mixture model. result Superior performance over existing methods on Amazon review dataset.
EM algorithm converges exponentially fast for overspecified Gaussian mixtures.
problem Convergence of EM algorithm in overspecified Gaussian mixtures.
method Structured configuration of means and weights, strong convexity, Polyak-Łojasiewicz inequality, finite-sample analysis.
result Exponential convergence rate of EM algorithm in KL distance.
This study analyzes theoretical guarantees for VI with fixed-variance Gaussian mixtures.
problem Theoretical analysis of variational inference with non-Gaussian distributions.
method Formulates variational inference as minimizing a mollified relative entropy, solving it through gradient descent on particle positions.
result Establishes descent lemma and approximation error bounds for optimization of variational inference.
This paper studies identifiability and convergence behaviors for parameters of multiple types in finite mixtures, and the effects of model fitting with extra mixing components. First, we present a general theory for strong identifiability, which extends from the previous work of Nguyen [2013] and Chen [1995] to address…
A faster EM algorithm for unsupervised Gaussian mixture models.
problem Efficiently determining the number of components in Gaussian mixture models.
method Adaptive Anderson Acceleration (AA) for EM algorithm, with novel monotonicity control and covariance matrix preservation.
result Significantly faster convergence compared to non-accelerated EM, up to 60X in some cases.
The paper uses Gaussian mixture models for Bayesian networks and proposes an optimization algorithm.
problem Modeling nodes in Bayesian networks with complex distributions.
method Gaussian mixture models combined with double iteration algorithm.
result The double iteration algorithm optimizes Gaussian mixture models effectively.
The study bounds the stability of Gaussian mixtures under small perturbations.
problem Stability of Gaussian mixtures under small changes in distribution.
method Deriving an explicit bound on parameter stability of spherical Gaussian Mixture Models (sGMM) in a pre-defined model class.
result Upper bound on parameter distance of close sGMMs to the original sGMM, dependent only on the original model.
Deep neural networks converge to Gaussian mixtures as layer width increases.
problem Understanding the distribution of outputs from deep neural networks.
method Proof and experiments with a simple model showing the convergence of neural network outputs to Gaussian mixtures.
result Neural networks converge to Gaussian mixtures as the width of the last hidden layer increases.
Optimal transport for vector Gaussian mixtures improves efficiency and structure preservation.
problem Optimal mass transport for vector-valued Gaussian mixtures.
method Vectorizing Gaussian mixture models and studying optimal mass transport problems.
result Computational efficiency and structure preservation in optimal mass transport.
Finite mixture models are among the most popular statistical models used in different data science disciplines. Despite their broad applicability, inference under these models typically leads to computationally challenging non-convex problems. While the Expectation-Maximization (EM) algorithm is the most popular approa…
Method estimates mixture components without discretizing parameters.
problem Learning from mixtures of continuous features with noise.
method Off-the-grid optimization method for continuous parameter space.
result Prediction error bound similar to Lasso predictor.