The paper develops efficient algorithms for variational inference with mixtures of isotropic Gaussians.
problem Efficiently approximating multimodal Bayesian posteriors.
method Develops a variational framework and efficient algorithms for mixtures of isotropic Gaussians.
result The approach provides accurate approximations of multimodal Bayesian posteriors while being memory and computationally efficient.
New spectral mixture representation for isotropic kernels simplifies random Fourier features.
problem Applying Random Fourier Features to complex kernels.
method Decompose isotropic kernels into scale mixtures of α-stable random vectors.
result Constructive spectral sampling formula for various kernels.
While several papers have investigated computationally and statistically efficient methods for learning Gaussian mixtures, precise minimax bounds for their statistical performance as well as fundamental limits in high-dimensional settings are not well-understood. In this paper, we provide precise information theoretic …
New GMM models fit high-dimensional data with fewer parameters.
problem Overparameterization and lack of flexibility in GMMs for high-dimensional data.
method Piecewise-constant covariance eigenvalue profiles, EM and penalized EM algorithms.
result Superior likelihood-parsimony tradeoffs in density fitting, clustering, and denoising.
Stochastic gradient descent converges to universal limits in high dimensions.
problem Statistical tasks in high dimensions with specific data projections.
method Stochastic gradient descent applied to mixture distributions, proving universality of limits.
result The ODE limits are universal for mixtures of arbitrary product distributions.
Study of two-layer NNs under Gaussian mixtures data, proving polynomial models equivalent to neural networks.
problem Training and generalization performance of two-layer NNs under structured Gaussian mixture data.
method Asymptotic analysis of two-layer NNs after one gradient descent step under Gaussian mixture data assumption.
result High-order polynomial models equivalent to nonlinear neural networks under certain conditions.
Framework uses unlabeled out-of-domain data to improve semi-supervised classification.
problem Improving generalization in semi-supervised classification problems.
method Combines Distributionally Robust Optimization (DRO) with self-supervised training.
result Significant improvement in generalization error compared to ERM.
We show that k-means (Lloyd's algorithm) is obtained as a special case when truncated variational EM approximations are applied to Gaussian Mixture Models (GMM) with isotropic Gaussians. In contrast to the standard way to relate k-means and GMMs, the provided derivation shows that it is not required to consider Gau…
One of the major shortcomings of variational autoencoders is the inability to produce generations from the individual modalities of data originating from mixture distributions. This is primarily due to the use of a simple isotropic Gaussian as the prior for the latent code in the ancestral sampling procedure for the da…
We classify translation surfaces in isotropic geometry with arbitrary constant isotropic Gaussian and mean curvature under the condition that at least one of translating curves lies in a plane.
Paper proves a theorem about constant mean curvature surfaces in isotropic 3-space.
problem Understanding constant mean curvature surfaces in isotropic 3-space.
method Value distribution theorem of Gaussian curvature applied to CMC surfaces.
result Implication of a Bernstein-type theorem for CMC surfaces in isotropic 3-space.
This paper represents a preliminary (pre-reviewing) version of a sublinear variational algorithm for isotropic Gaussian mixture models (GMMs). Further developments of the algorithm for GMMs with diagonal covariance matrices (instead of isotropic clusters) and their corresponding benchmarking results have been published…
In this paper, we study factorable surfaces in a 3-dimensional isotropic space. We classify such surfaces with constant isotropic Gaussian (K) and mean curvature (H). We provide a non-existence result related with the surfaces satisfying H/K=const. Several examples are also illustrated.
In this paper, we study the rotational surfaces in the isotropic 3-space I^3. satisfying Weingarten conditions in terms of the relative curvature K (analogue of the Gaussian curvature) and the isotropic mean curvature H. In particular, we classify such surfaces of linear Weingarten type in I^3.
Study proves only origin-centered spheres solve certain curvature problems.
problem Proving uniqueness of solutions to curvature problems.
method Using the Heintze-Karcher inequality, the study proves the uniqueness of smooth, strictly convex solutions to a class of Minkowski type problems.
result Only origin-centered spheres solve isotropic and Lp-Gaussian-Minkowski problems. Generative models use kernel smoothing for conditioning on small example sets.
problem Improving generative models' performance with limited conditioning examples.
method Showed that cross-attention conditioning is equivalent to kernel smoothing, specifically a Nadaraya--Watson kernel smoother.
result The approach predicts and confirms three failure regimes for kernel-based conditioning.
New blurring diffusion models bridge heat dissipation and denoising.
problem Developing a new generative modeling approach.
method Connecting blurring to Gaussian diffusion with non-isotropic noise.
result Proposed Blurring Diffusion Models offer the best of both Gaussian denoising and inverse heat dissipation.
Paper calculates KL divergence for isotropic Gaussian-Markov fields.
problem Measuring divergence between isotropic Gaussian-Markov fields.
method Derives closed-form KL divergence expressions.
result Develops new similarity measures in image processing.
New lower bounds show challenges in clustering in moderate dimensions.
problem Clustering points from mixtures of isotropic Gaussians in moderate dimensions.
method Established low-degree polynomial lower bounds and developed a novel non-spectral algorithm.
result New lower bounds reveal a 'non-parametric rate' in moderate dimensions.
Improved VB algorithm for NIG mixtures outperforms Gaussian mixtures for non-Gaussian data.
problem Clustering non-Gaussian data, especially heavy-tailed and asymmetric.
method Proposed an improved VB algorithm for NIG mixture models and extended Dirichlet process mixture models.
result Outperforms Gaussian mixtures and existing NIG mixture models, especially for highly non-normative data.
The isotropic 3-space I^3 which is one of the Cayley--Klein spaces is obtained from the Euclidean space by substituting the usual Euclidean distance with the isotropic distance. In the present paper, we give several classifications on the surfaces in I^3 with the constant relative curvature (analogue of the Gaussian cu…
X-VAE uses data-adaptive Gaussian priors to improve latent space modeling.
problem Limitations of standard Gaussian priors in complex datasets.
method Data-adaptive Gaussian prior derived from pretrained autoencoder latent codes.
result Improved latent space modeling and generation quality.
The paper uses Gaussian mixture models for Bayesian networks and proposes an optimization algorithm.
problem Modeling nodes in Bayesian networks with complex distributions.
method Gaussian mixture models combined with double iteration algorithm.
result The double iteration algorithm optimizes Gaussian mixture models effectively.
The study bounds the stability of Gaussian mixtures under small perturbations.
problem Stability of Gaussian mixtures under small changes in distribution.
method Deriving an explicit bound on parameter stability of spherical Gaussian Mixture Models (sGMM) in a pre-defined model class.
result Upper bound on parameter distance of close sGMMs to the original sGMM, dependent only on the original model.
Spatially constrained Gaussian mixture models reduce covariance complexity.
problem High dimensionality in finite mixture models for spatial data.
method Spatial covariance constraint with only four free parameters.
result Improves clustering of multi-way spatial data and inference of spatial patterns.
In this work, we are interested in the differential geometry of surfaces in simply isotropic I3 and pseudo-isotropic Ip3 spaces, which consists of the study of R3 equipped with a degenerate metric such as ds2=dx2±dy2. The investigation is…
Deep neural networks converge to Gaussian mixtures as layer width increases.
problem Understanding the distribution of outputs from deep neural networks.
method Proof and experiments with a simple model showing the convergence of neural network outputs to Gaussian mixtures.
result Neural networks converge to Gaussian mixtures as the width of the last hidden layer increases.
We propose a new class of structured methods for Monte Carlo (MC) sampling, called DPPMC, designed for high-dimensional nonisotropic distributions where samples are correlated to reduce the variance of the estimator via determinantal point processes. We successfully apply DPPMCs to problems involving nonisotropic distr…
Optimal transport for vector Gaussian mixtures improves efficiency and structure preservation.
problem Optimal mass transport for vector-valued Gaussian mixtures.
method Vectorizing Gaussian mixture models and studying optimal mass transport problems.
result Computational efficiency and structure preservation in optimal mass transport.
Theoretical analysis of entropy approximation for Gaussian mixtures.
problem Lack of theoretical guarantees for entropy approximation of Gaussian mixtures.
method Theoretical analysis of the error between true and approximate entropy.
result The error converges to zero as the ratios of means to variances tend to infinity, providing a guarantee for high-dimensional problems.
We study invariant surfaces generated by one-parameter subgroups of simply and pseudo isotropic rigid motions. Basically, the simply and pseudo isotropic geometries are the study of a three-dimensional space equipped with a rank 2 metric of index zero and one, respectively. We show that the one-parameter subgroups of i…
Paper uses Gaussian mixture models and Wasserstein distance for schema matching.
problem Schema matching between different datasets.
method Gaussian mixture models and Wasserstein distance for comparison.
result Derives an approximation for Wasserstein distance between Gaussian mixture models.
CCVFM uses coreset to improve generative models by refining residual flows.
problem Generating multimodal distributions from scratch is challenging.
method Augments hierarchical rectified flow with a data-informed source distribution using a coreset.
result CCVFM achieves competitive few-step generation without a learned noise-to-data map.
Algorithm estimates mixtures of arbitrary Gaussians robustly in presence of corruptions.
problem Estimating mixtures of arbitrary Gaussians in the presence of a constant fraction of arbitrary corruptions.
method Polynomial-time algorithm using partial clustering and tensor decomposition.
result Resolves the main open problem in several previous works on algorithmic robust statistics.
Estimates Gaussian mixtures from weighted samples efficiently.
problem Estimating Gaussian mixtures from weighted samples with correct weight treatment.
method Density interpretation and expectation-maximization method considering weights.
result Correctly estimates Gaussian mixtures with weighted samples.
Proposes Gaussian process priors on graph sets with geometric structure.
problem Defining Gaussian process priors on sets of graphs with geometric structure.
method Defines priors respecting graph geometric structure, analogous to Euclidean isotropic processes.
result Efficient computational technique for evaluating priors' kernels, making them usable in toolboxes.
Deep learning is a hierarchical inference method formed by subsequent multiple layers of learning able to more efficiently describe complex relationships. In this work, Deep Gaussian Mixture Models are introduced and discussed. A Deep Gaussian Mixture model (DGMM) is a network of multiple layers of latent variables, wh…
Developed a new concept of isometric surfaces in isotropic space.
problem Missing notion of isometric deformations in isotropic space.
method Using Gauss' Theorema Egregium as a condition, developed a new concept.
result Natural analogues of Euclidean space isometries found in isotropic space.
Study uniform rates for estimating Gaussian mixtures without separation assumption.
problem Estimating parameters in two-component Gaussian mixtures without separation.
method Uniform convergence rates derived using minimax lower bounds and careful analysis of polynomial equalities.
result Phase transition in optimal estimation rate based on mixture balance.
Study learns Gaussian mixtures from censored data.
problem Learning Gaussian mixtures with incomplete data.
method Proposes an algorithm to estimate weights and means with minimal samples.
result Achieves accurate estimation with very few samples.
GGMPs improve non-Gaussian conditional density estimation.
problem Multimodality, heteroscedasticity, and strong non-Gaussianity in conditional density estimation.
method GGMP combines local Gaussian mixture fitting, cross-input component alignment, and per-component heteroscedastic GP training.
result GGMPs improve distributional approximation on synthetic and real-world datasets.
Study proposes a new metric for comparing Gaussian mixtures in RKHS.
problem Comparing complex multimodal densities in RKHS.
method Wasserstein-type metric for kernel Gaussian mixtures.
result Enhanced capability to model multimodal densities.
A new method for clustering heterogeneous data using likelihood-adjusted SDP.
problem Clustering heterogeneous data with different cluster shapes and sizes.
method Iterative likelihood-adjusted semidefinite programming (iLA-SDP) method.
result iLA-SDP achieves lower mis-clustering errors compared to other methods.
Heavy-tailed distributions are widely used in robust mixture modelling due to possessing thick tails. As a computationally tractable subclass of the stable distributions, sub-Gaussian α-stable distribution received much interest in the literature. Here, we introduce a type of expectation maximization algorithm that e…
Improved sample efficiency for private learning of Gaussian mixtures.
problem Learning mixtures of Gaussians with differential privacy.
method Inverse sensitivity mechanism, sample compression, sumset volume bounds.
result Proved optimal sample complexity for private learning of mixtures of Gaussians.
The mixture of Gaussian distributions, a soft version of k-means , is considered a state-of-the-art clustering algorithm. It is widely used in computer vision for selecting classes, e.g., color, texture, and shapes. In this algorithm, each class is described by a Gaussian distribution, defined by its mean and covarianc…
Sharp inequality between TV and Hellinger distances for Gaussian mixtures.
problem Understanding the relationship between total variation and Hellinger distances for Gaussian mixtures.
method Established a general upper bound on Hellinger distance in terms of TV distance raised to a power, demonstrating sharpness with specific examples.
result The Hellinger distance between two Gaussian mixtures is bounded by the TV distance raised to a power 1−o(1), where o(1) is of order 1/loglog(1/TV). Tensor decomposition recovers Gaussian mixtures from moments.
problem Recovering Gaussian mixture models from datasets.
method Symmetric tensor decomposition of moment tensors built from empirical moments.
result Identifiable tensors with interpolation degree less than half their order.