Paper proposes MWDE for estimating finite location-scale mixtures.
problem Estimating finite location-scale mixtures using MLE is problematic.
method Investigates minimum Wasserstein distance estimators (MWDE).
result MWDE is consistent and provides a numerical solution.
New method estimates mixture model components efficiently.
problem Estimating the number of components in finite mixture models.
method Group-Sort-Fuse (GSF) procedure for simultaneous estimation of order and mixing measure.
result GSF achieves consistent estimation of true mixture order and n−1/2 convergence rate. Proposes a new model for clustering with heavier tails.
problem Clustering with heavy-tailed data.
method Finite mixture of skewed sub-Gaussian stable distributions, maximum likelihood estimation, EM algorithm.
result The proposed model can robustly handle heavy-tailed data.
Study finds the minimum number of finite Gaussian mixtures for best approximation.
problem Finding the minimum number of finite Gaussian mixtures for best approximation.
method Local moment matching for upper bound and spectral analysis for lower bound.
result Corrects a previous lower bound in the case of Gaussian mixing distributions.
Spatially constrained Gaussian mixture models reduce covariance complexity.
problem High dimensionality in finite mixture models for spatial data.
method Spatial covariance constraint with only four free parameters.
result Improves clustering of multi-way spatial data and inference of spatial patterns.
New algorithms improve spectral clustering for finite mixture models.
problem Issues with EM algorithm in spectral clustering.
method Spectral decomposition and non-parametric bootstrap sampling.
result Improved convergence and avoidance of poor solutions.
Study on Dirichlet process mixtures for clustering consistency.
problem Consistency of clustering with Dirichlet process mixtures.
method Analysis of posterior distribution as sample size increases, focusing on consistency for the number of clusters.
result Consistency for the number of clusters can be achieved with a properly adapted concentration parameter in a Bayesian setting.
TAMD prevents degeneracy in finite mixtures, offering strong guarantees but modest practical improvements.
problem Degeneracy in maximum likelihood estimation of finite mixtures.
method Transcendental regularization with analytic barrier functions.
result Strong theoretical guarantees (identifiability, consistency, robustness) but modest practical improvements.
Paper detects gradual changes in cluster structure using MC fusion.
problem Detecting gradual changes in cluster structure over time.
method MC fusion for multiple mixture numbers, examining MC transition.
result Accurately captures cluster structure during transitional periods.
This paper studies convergence behavior of latent mixing measures that arise in finite and infinite mixture models, using transportation distances (i.e., Wasserstein metrics). The relationship between Wasserstein distances on the space of mixing measures and f-divergence functionals such as Hellinger and Kullback-Leibl…
A new method uses Mean Field Games to optimize mixture models of Bernoulli and categorical distributions.
problem Optimizing parameters of finite mixture models of Bernoulli and categorical distributions.
method Mean Field Games theory applied to multi-population systems.
result The Mean Field Games approach provides a method to compute mixture model parameters.
Understanding separation effects on parameter estimation in finite Gaussian mixtures
problem Minimum component separation impact on convergence rates in finite Gaussian mixtures
method Developing a unified geometric framework using Hellinger lower bounds and specialized moment-extraction test functions
result Separation complexity driven by spatial configuration of mixture components
We propose a kernel method to identify finite mixtures of nonparametric product distributions. It is based on a Hilbert space embedding of the joint distribution. The rank of the constructed tensor is equal to the number of mixture components. We present an algorithm to recover the components by partitioning the data p…
A fast Modal EM algorithm for Gaussian mixtures.
problem Clustering with Gaussian mixtures.
method Modal EM algorithm for Gaussian mixtures.
result High flexibility in various clustering contexts.
Improved VB algorithm for NIG mixtures outperforms Gaussian mixtures for non-Gaussian data.
problem Clustering non-Gaussian data, especially heavy-tailed and asymmetric.
method Proposed an improved VB algorithm for NIG mixture models and extended Dirichlet process mixture models.
result Outperforms Gaussian mixtures and existing NIG mixture models, especially for highly non-normative data.
The likelihood function of a finite mixture model is a non-convex function with multiple local maxima and commonly used iterative algorithms such as EM will converge to different solutions depending on initial conditions. In this paper we ask: is it possible to assess how far we are from the global maximum of the likel…
NMDR estimates complex mixtures of distributions efficiently.
problem Estimating complex finite mixtures of distributions in high-dimensional settings.
method Flexible additive predictors, neural networks, and deep learning optimizers.
result Competitive performance in complex scenarios compared to existing approaches.
Finite mixtures of regression models offer a flexible framework for investigating heterogeneity in data with functional dependencies. These models can be conveniently used for unsupervised learning on data with clear regression relationships. We extend such models by imposing an eigen-decomposition on the multivariate …
Model-based clustering imposes a finite mixture modelling structure on data for clustering. Finite mixture models assume that the population is a convex combination of a finite number of densities, the distribution within each population is a basic assumption of each particular model. Among all distributions that have …
A new sampler speeds up Bayesian mixture models.
problem Sampling from Bayesian finite mixture models is slow and hard.
method Introduces a non-reversible sampling scheme for Bayesian finite mixture models.
result The new sampler outperforms classical samplers in many scenarios, especially during convergence.
Bayesian approach learns nonparametric mixture components from heterogeneous data.
problem Realistic modeling of heterogeneous data populations with nonparametric mixture components.
method Bayesian nonparametric modeling using Dirichlet process mixture priors.
result Posterior contraction rates for component densities are nearly polynomial, improving over deconvolution methods.
Improved convergence rates for MLE in mixture models using penalized log-likelihood.
problem Convergence rates for MLE in finite mixture models.
method Penalizing log-likelihood to discourage vanishing mixing weights, using Wasserstein distance and new loss functions.
result Improved convergence rates for some mixture components, faster than traditional methods.
A new method detects outliers using ensembles of Dirichlet process mixtures.
problem Challenges in unsupervised outlier detection using Dirichlet process mixtures.
method Ensembles of Dirichlet process Gaussian mixtures with random subspace and subsampling.
result Empirically outperforms existing approaches in unsupervised outlier detection.
DFMR improves robustness of learning finite mixture models in distributed settings.
problem Learning finite mixture models in distributed settings with Byzantine failures.
method DFMR leverages pairwise L2 distances to filter and retain local estimates, ensuring robust aggregation.
result DFMR achieves optimal convergence rate and asymptotic equivalence to global maximum likelihood estimate.
In many applications, a finite mixture is a natural model, but it can be difficult to choose an appropriate number of components. To circumvent this choice, investigators are increasingly turning to Dirichlet process mixtures (DPMs), and Pitman-Yor process mixtures (PYMs), more generally. While these models may be well…
New method reduces mixture model evaluation cost for large models.
problem Computational infeasibility of evaluating all mixture components.
method Combining EM and Metropolis-Hastings for stochastic sampling.
result Significantly reduced computational cost for large models.
Conditional diffusion models can approximate target distributions well with Gaussian-mixture reverse kernels.
problem Approximating target distributions in conditional diffusion models.
method Using finite Gaussian mixtures with ReLU-network logits as reverse kernels, reducing the problem to static conditional density approximation.
result The resulting neural reverse-kernel class is dense in conditional KL divergence under exact terminal matching.
Clustering of data sets is a standard problem in many areas of science and engineering. The method of spectral clustering is based on embedding the data set using a kernel function, and using the top eigenvectors of the normalized Laplacian to recover the connected components. We study the performance of spectral clust…
We give tight concentration bounds for mixtures of martingales that are simultaneously uniform over (a) mixture distributions, in a PAC-Bayes sense; and (b) all finite times. These bounds are proved in terms of the martingale variance, extending classical Bernstein inequalities, and sharpening and simplifying prior wor…
Singularities of a statistical model are the elements of the model's parameter space which make the corresponding Fisher information matrix degenerate. These are the points for which estimation techniques such as the maximum likelihood estimator and standard Bayesian procedures do not admit the root-n parametric rate…
New method uses dendrograms for better mixture model selection and clustering.
problem Selecting the correct number of components in finite mixture models.
method Hierarchical clustering tree derived from overfitted latent mixing measures.
result Consistently selects the true number of mixing components and optimal convergence rate for parameter estimation.
Identifying components and estimating mixing weights in unlabeled finite mixtures under marginal independence.
problem Identifying components and estimating mixing weights in unlabeled finite mixtures.
method Proving structural results and extending them to observable mixtures.
result Identifying components and estimating mixing weights under marginal independence.
Extended Ville's inequality for nonintegrable supermartingales.
problem Nonintegrable nonnegative supermartingales.
method Derivation of extended Ville's inequality and extension of the method of mixtures.
result Strengthened classical Ville's inequality for nonintegrable settings.
In this paper we present a new Bayesian network model for classification that combines the naive-Bayes (NB) classifier and the finite-mixture (FM) classifier. The resulting classifier aims at relaxing the strong assumptions on which the two component models are based, in an attempt to improve on their classification pe…
This paper is concerned with an important issue in finite mixture modelling, the selection of the number of mixing components. We propose a new penalized likelihood method for model selection of finite multivariate Gaussian mixture models. The proposed method is shown to be statistically consistent in determining of th…
Hybrid model combines continuous and tractable probabilistic models.
problem Intractable probabilistic inference in continuous latent-space models.
method Continuous mixtures of tractable probabilistic models with finite integration points.
result Hybrid models achieve state-of-the-art performance in density estimation.
Algorithm estimates nonparametric mixtures from grouped data.
problem Estimating identifiable nonparametric mixture models from grouped observations.
method Oracle inequality for weighted kernel density estimators and general consistency result.
result Consistent estimation of mixture components from grouped observations.
NPMLE estimator automatically chooses the right model complexity for Gaussian mixtures.
problem Learning mixture models and empirical Bayes estimation with non-convex likelihoods.
method Nonparametric maximum likelihood estimator (NPMLE) using complex-analytic techniques.
result NPMLE solution has O(logn) atoms with high probability, improving model complexity. Proposes a robust FMR model for handling sample heterogeneity.
problem Handling sample heterogeneity with a single regression model.
method Clusters samples and jointly models multiple incomplete mixed-type targets.
result Achieves state-of-the-art performance on synthetic and real-world data.
How to forecast next year's portfolio-wide credit default rate based on last year's default observations and the current score distribution? A classical approach to this problem consists of fitting a mixture of the conditional score distributions observed last year to the current score distribution. This is a special (…
This study analyzes communication constraints in MoE architectures using information theory.
problem Communication constraints in Mixture-of-Experts (MoE) architectures.
method Developed a rate-distortion characterization of finite-rate gating in MoE architectures using information theory.
result Yielded capacity-aware limits for communication-constrained MoE systems.
Parameter estimation for model-based clustering using a finite mixture of normal inverse Gaussian (NIG) distributions is achieved through variational Bayes approximations. Univariate NIG mixtures and multivariate NIG mixtures are considered. The use of variational Bayes approximations here is a substantial departure fr…
New bounds on sample size for identifying mixture models with grouped samples.
problem Identifying mixture models with minimal sample size.
method Generalized identifiability bounds for mixture models with grouped samples.
result Identifiability with (2m−1)/(k−1) samples per group, with no improvement possible. Finite mixture model is an important branch of clustering methods and can be applied on data sets with mixed types of variables. However, challenges exist in its applications. First, it typically relies on the EM algorithm which could be sensitive to the choice of initial values. Second, biomarkers subject to limits of…
Novel connections between Neyman-Scott processes and Bayesian nonparametric mixture models enable scalable inference.
problem Efficiently modeling and detecting clusters in spatiotemporal data.
method Adapting collapsed Gibbs sampling for Neyman-Scott processes via connections to mixture of finite mixture models.
result Demonstrated scalability and effectiveness on neural spike trains and document streams.
The paper extends sequences while preserving statistical properties using a mixture model.
problem Extending sequences while retaining their statistical properties.
method Auto-regressive Sequence Extension Mixture Model (SEMM) using deep learning.
result The mixture model outperforms traditional neural networks in sequence extension with statistical property retention.
New method selects FMM components via variational Bayes.
problem Selecting the correct number of components in finite mixture models.
method Variational Bayes with mean-field approximation.
result Consistency of model selection based on ELBO maximization.
When estimating finite mixture models, it is common to make assumptions on the mixture components, such as parametric assumptions. In this work, we make no distributional assumptions on the mixture components and instead assume that observations from the mixture model are grouped, such that observations in the same gro…