New method estimates mixture model components efficiently.
problem Estimating the number of components in finite mixture models.
method Group-Sort-Fuse (GSF) procedure for simultaneous estimation of order and mixing measure.
result GSF achieves consistent estimation of true mixture order and n−1/2 convergence rate. Spatially constrained Gaussian mixture models reduce covariance complexity.
problem High dimensionality in finite mixture models for spatial data.
method Spatial covariance constraint with only four free parameters.
result Improves clustering of multi-way spatial data and inference of spatial patterns.
Proposes a new model for clustering with heavier tails.
problem Clustering with heavy-tailed data.
method Finite mixture of skewed sub-Gaussian stable distributions, maximum likelihood estimation, EM algorithm.
result The proposed model can robustly handle heavy-tailed data.
Paper proposes MWDE for estimating finite location-scale mixtures.
problem Estimating finite location-scale mixtures using MLE is problematic.
method Investigates minimum Wasserstein distance estimators (MWDE).
result MWDE is consistent and provides a numerical solution.
Improved VB algorithm for NIG mixtures outperforms Gaussian mixtures for non-Gaussian data.
problem Clustering non-Gaussian data, especially heavy-tailed and asymmetric.
method Proposed an improved VB algorithm for NIG mixture models and extended Dirichlet process mixture models.
result Outperforms Gaussian mixtures and existing NIG mixture models, especially for highly non-normative data.
Paper detects gradual changes in cluster structure using MC fusion.
problem Detecting gradual changes in cluster structure over time.
method MC fusion for multiple mixture numbers, examining MC transition.
result Accurately captures cluster structure during transitional periods.
A new method uses Mean Field Games to optimize mixture models of Bernoulli and categorical distributions.
problem Optimizing parameters of finite mixture models of Bernoulli and categorical distributions.
method Mean Field Games theory applied to multi-population systems.
result The Mean Field Games approach provides a method to compute mixture model parameters.
New algorithms improve spectral clustering for finite mixture models.
problem Issues with EM algorithm in spectral clustering.
method Spectral decomposition and non-parametric bootstrap sampling.
result Improved convergence and avoidance of poor solutions.
Study on Dirichlet process mixtures for clustering consistency.
problem Consistency of clustering with Dirichlet process mixtures.
method Analysis of posterior distribution as sample size increases, focusing on consistency for the number of clusters.
result Consistency for the number of clusters can be achieved with a properly adapted concentration parameter in a Bayesian setting.
TAMD prevents degeneracy in finite mixtures, offering strong guarantees but modest practical improvements.
problem Degeneracy in maximum likelihood estimation of finite mixtures.
method Transcendental regularization with analytic barrier functions.
result Strong theoretical guarantees (identifiability, consistency, robustness) but modest practical improvements.
This paper studies convergence behavior of latent mixing measures that arise in finite and infinite mixture models, using transportation distances (i.e., Wasserstein metrics). The relationship between Wasserstein distances on the space of mixing measures and f-divergence functionals such as Hellinger and Kullback-Leibl…
Bayesian approach learns nonparametric mixture components from heterogeneous data.
problem Realistic modeling of heterogeneous data populations with nonparametric mixture components.
method Bayesian nonparametric modeling using Dirichlet process mixture priors.
result Posterior contraction rates for component densities are nearly polynomial, improving over deconvolution methods.
A new method detects outliers using ensembles of Dirichlet process mixtures.
problem Challenges in unsupervised outlier detection using Dirichlet process mixtures.
method Ensembles of Dirichlet process Gaussian mixtures with random subspace and subsampling.
result Empirically outperforms existing approaches in unsupervised outlier detection.
NMDR estimates complex mixtures of distributions efficiently.
problem Estimating complex finite mixtures of distributions in high-dimensional settings.
method Flexible additive predictors, neural networks, and deep learning optimizers.
result Competitive performance in complex scenarios compared to existing approaches.
New method reduces mixture model evaluation cost for large models.
problem Computational infeasibility of evaluating all mixture components.
method Combining EM and Metropolis-Hastings for stochastic sampling.
result Significantly reduced computational cost for large models.
New bounds on sample size for identifying mixture models with grouped samples.
problem Identifying mixture models with minimal sample size.
method Generalized identifiability bounds for mixture models with grouped samples.
result Identifiability with (2m−1)/(k−1) samples per group, with no improvement possible. Finite mixtures of regression models offer a flexible framework for investigating heterogeneity in data with functional dependencies. These models can be conveniently used for unsupervised learning on data with clear regression relationships. We extend such models by imposing an eigen-decomposition on the multivariate …
Algorithm estimates nonparametric mixtures from grouped data.
problem Estimating identifiable nonparametric mixture models from grouped observations.
method Oracle inequality for weighted kernel density estimators and general consistency result.
result Consistent estimation of mixture components from grouped observations.
A new sampler speeds up Bayesian mixture models.
problem Sampling from Bayesian finite mixture models is slow and hard.
method Introduces a non-reversible sampling scheme for Bayesian finite mixture models.
result The new sampler outperforms classical samplers in many scenarios, especially during convergence.
When estimating finite mixture models, it is common to make assumptions on the mixture components, such as parametric assumptions. In this work, we make no distributional assumptions on the mixture components and instead assume that observations from the mixture model are grouped, such that observations in the same gro…
New method uses dendrograms for better mixture model selection and clustering.
problem Selecting the correct number of components in finite mixture models.
method Hierarchical clustering tree derived from overfitted latent mixing measures.
result Consistently selects the true number of mixing components and optimal convergence rate for parameter estimation.
Improved convergence rates for MLE in mixture models using penalized log-likelihood.
problem Convergence rates for MLE in finite mixture models.
method Penalizing log-likelihood to discourage vanishing mixing weights, using Wasserstein distance and new loss functions.
result Improved convergence rates for some mixture components, faster than traditional methods.
Study finds the minimum number of finite Gaussian mixtures for best approximation.
problem Finding the minimum number of finite Gaussian mixtures for best approximation.
method Local moment matching for upper bound and spectral analysis for lower bound.
result Corrects a previous lower bound in the case of Gaussian mixing distributions.
Model-based clustering imposes a finite mixture modelling structure on data for clustering. Finite mixture models assume that the population is a convex combination of a finite number of densities, the distribution within each population is a basic assumption of each particular model. Among all distributions that have …
The likelihood function of a finite mixture model is a non-convex function with multiple local maxima and commonly used iterative algorithms such as EM will converge to different solutions depending on initial conditions. In this paper we ask: is it possible to assess how far we are from the global maximum of the likel…
Finite mixture models are statistical models which appear in many problems in statistics and machine learning. In such models it is assumed that data are drawn from random probability measures, called mixture components, which are themselves drawn from a probability measure P over probability measures. When estimating …
NPMLE estimator automatically chooses the right model complexity for Gaussian mixtures.
problem Learning mixture models and empirical Bayes estimation with non-convex likelihoods.
method Nonparametric maximum likelihood estimator (NPMLE) using complex-analytic techniques.
result NPMLE solution has O(logn) atoms with high probability, improving model complexity. In many applications, a finite mixture is a natural model, but it can be difficult to choose an appropriate number of components. To circumvent this choice, investigators are increasingly turning to Dirichlet process mixtures (DPMs), and Pitman-Yor process mixtures (PYMs), more generally. While these models may be well…
A new method for fast Bayesian mixture model estimation.
problem Estimating Bayesian mixture models is computationally challenging.
method Amortized Bayesian Inference (ABI) framework for mixture models.
result The method provides fast inference for mixture models.
The paper tackles learning mixtures of two multinomial logits, showing identifiability and presenting an algorithm.
problem Learning an arbitrary mixture of two multinomial logits.
method Reduction to solving a system of univariate quartic equations, followed by an algorithm using polynomial and linear samples.
result Identifiability of the mixture models may only fail on an algebraic variety of negligible measure.
Novel connections between Neyman-Scott processes and Bayesian nonparametric mixture models enable scalable inference.
problem Efficiently modeling and detecting clusters in spatiotemporal data.
method Adapting collapsed Gibbs sampling for Neyman-Scott processes via connections to mixture of finite mixture models.
result Demonstrated scalability and effectiveness on neural spike trains and document streams.
Conditional diffusion models can approximate target distributions well with Gaussian-mixture reverse kernels.
problem Approximating target distributions in conditional diffusion models.
method Using finite Gaussian mixtures with ReLU-network logits as reverse kernels, reducing the problem to static conditional density approximation.
result The resulting neural reverse-kernel class is dense in conditional KL divergence under exact terminal matching.
DFMR improves robustness of learning finite mixture models in distributed settings.
problem Learning finite mixture models in distributed settings with Byzantine failures.
method DFMR leverages pairwise L2 distances to filter and retain local estimates, ensuring robust aggregation.
result DFMR achieves optimal convergence rate and asymptotic equivalence to global maximum likelihood estimate.
Understanding separation effects on parameter estimation in finite Gaussian mixtures
problem Minimum component separation impact on convergence rates in finite Gaussian mixtures
method Developing a unified geometric framework using Hellinger lower bounds and specialized moment-extraction test functions
result Separation complexity driven by spatial configuration of mixture components
Hybrid model combines continuous and tractable probabilistic models.
problem Intractable probabilistic inference in continuous latent-space models.
method Continuous mixtures of tractable probabilistic models with finite integration points.
result Hybrid models achieve state-of-the-art performance in density estimation.
Singularities of a statistical model are the elements of the model's parameter space which make the corresponding Fisher information matrix degenerate. These are the points for which estimation techniques such as the maximum likelihood estimator and standard Bayesian procedures do not admit the root-n parametric rate…
Improved VAE estimation from incomplete data using variational mixtures.
problem Estimating VAEs from incomplete data increases posterior complexity.
method Introducing variational mixtures based on finite and imputation distributions.
result Variational mixtures improve VAE estimation accuracy from incomplete data.
We present the multidimensional membership mixture (M3) models where every dimension of the membership represents an independent mixture model and each data point is generated from the selected mixture components jointly. This is helpful when the data has a certain shared structure. For example, three unique means and …
How to forecast next year's portfolio-wide credit default rate based on last year's default observations and the current score distribution? A classical approach to this problem consists of fitting a mixture of the conditional score distributions observed last year to the current score distribution. This is a special (…
In this paper, we introduce and evaluate a data-driven staged mixture modeling technique for building density, regression, and classification models. Our basic approach is to sequentially add components to a finite mixture model using the structural expectation maximization (SEM) algorithm. We show that our technique i…
New concept of mixture complexity helps detect gradual clustering changes.
problem Determining the number of clusters in mixture models with overlaps and weight biases.
method Introducing mixture complexity (MC) as a new measure of cluster size, defined from information theory.
result MC can detect gradual clustering changes, allowing earlier detection and finer distinction.
Survey on Bayesian inference for Gaussian mixture models.
problem Estimating parameters of Gaussian mixture models using Bayesian methods.
method Uses Bayesian inference to estimate parameters and uncertainty of Gaussian mixture models.
result Bayesian approach provides point estimates and associated uncertainty for mixture model parameters.
Parameter estimation for model-based clustering using a finite mixture of normal inverse Gaussian (NIG) distributions is achieved through variational Bayes approximations. Univariate NIG mixtures and multivariate NIG mixtures are considered. The use of variational Bayes approximations here is a substantial departure fr…
This paper is concerned with an important issue in finite mixture modelling, the selection of the number of mixing components. We propose a new penalized likelihood method for model selection of finite multivariate Gaussian mixture models. The proposed method is shown to be statistically consistent in determining of th…
The paper extends sequences while preserving statistical properties using a mixture model.
problem Extending sequences while retaining their statistical properties.
method Auto-regressive Sequence Extension Mixture Model (SEMM) using deep learning.
result The mixture model outperforms traditional neural networks in sequence extension with statistical property retention.
Continuous-time interpolation of volatility surfaces preserving mixtures and arbitrage-free.
problem Interpolation of volatility surfaces
method Constructing a mixture-preserving, arbitrage-free interpolation
result Lifts Brigo-Mercurio to time-varying weights with additive cost
We propose a kernel method to identify finite mixtures of nonparametric product distributions. It is based on a Hilbert space embedding of the joint distribution. The rank of the constructed tensor is equal to the number of mixture components. We present an algorithm to recover the components by partitioning the data p…
In this paper we present a new Bayesian network model for classification that combines the naive-Bayes (NB) classifier and the finite-mixture (FM) classifier. The resulting classifier aims at relaxing the strong assumptions on which the two component models are based, in an attempt to improve on their classification pe…