Two approaches improve parameter learning in various mixture models.
problem Parameter learning in mixture models.
method Complex-analytic and algebraic-combinatorial methods.
result Improved sample sufficiency for parameter estimation in specific mixture models.
Robust learning mixtures of linear regressions improve robustness.
problem Improving robustness in learning mixtures of linear regressions.
method Connecting mixtures of linear regressions and mixtures of Gaussians with thresholding for a quasi-polynomial time algorithm.
result The algorithm has significantly better robustness than previous results.
Paper introduces Normalized Wasserstein measure for better handling of imbalanced mixture distributions.
problem Wasserstein distance fails for mixture distributions with imbalanced proportions.
method Introduce mixture proportions as optimization variables to normalize Wasserstein formulation.
result Normalized Wasserstein measure leads to significant performance gains for mixture distributions.
The paper tackles learning mixtures of two multinomial logits, showing identifiability and presenting an algorithm.
problem Learning an arbitrary mixture of two multinomial logits.
method Reduction to solving a system of univariate quartic equations, followed by an algorithm using polynomial and linear samples.
result Identifiability of the mixture models may only fail on an algebraic variety of negligible measure.
MixTS uses a mixture prior to analyze Thompson Sampling in multi-task learning.
problem Analyzing Thompson Sampling in environments with uncertain and multi-class problems.
method Developed MixTS by incorporating a mixture prior into Thompson Sampling and using a novel proof technique for mixture distributions.
result Proved Bayes regret bounds for MixTS in linear bandits and finite-horizon reinforcement learning.
Algorithm learns mixtures of rankings from noisy data.
problem Learning an unknown mixture of rankings from noisy samples.
method Algorithm for different noise models, including heat kernel and Mallows model.
result High accuracy learning of mixture to nO(logk) time. A new method for fitting mixture models using Boltzmann exploration.
problem Challenges in learning mixture models, especially with good initialisation.
method Boltzmann exploration expectation-maximisation (BEEM) algorithm.
result BEEM can escape local optima and is insensitive to parameter initialisation.
Study learns mixtures of smooth product distributions from samples.
problem Learning mixtures of non-parametric product distributions.
method Two-stage approach using identifiability properties of tensor decomposition and signal processing techniques.
result Recovery of component distributions under a smoothness condition.
Bayesian approach learns nonparametric mixture components from heterogeneous data.
problem Realistic modeling of heterogeneous data populations with nonparametric mixture components.
method Bayesian nonparametric modeling using Dirichlet process mixture priors.
result Posterior contraction rates for component densities are nearly polynomial, improving over deconvolution methods.
Study learns Gaussian mixtures from censored data.
problem Learning Gaussian mixtures with incomplete data.
method Proposes an algorithm to estimate weights and means with minimal samples.
result Achieves accurate estimation with very few samples.
New algorithm recovers mixture means even with many outliers.
problem Estimating mixture means when outliers overwhelm small groups.
method Proposes an algorithm for robust mixture learning with minimal list-size overhead.
result Order-optimal error guarantees for each mixture mean.
Infinite mixture prototypes adapt to complex data for few-shot learning.
problem Few-shot learning with complex data distributions.
method Adaptive representation of classes by clusters, inferring cluster number.
result 25% absolute accuracy improvement on alphabets, state-of-the-art semi-supervised clustering.
New algorithm learns Gaussian mixtures privately with optimal sample complexity.
problem Learning parameters of Gaussian mixtures under differential privacy constraints.
method Differentially private algorithm based on Achlioptas and McSherry's approach.
result Sample complexity matches non-private algorithm up to lower order terms.
We solve learning mixtures of graphs from epidemic cascades, establishing conditions and algorithms.
problem Learning the weighted edges of a balanced mixture of two undirected graphs from epidemic cascades.
method Established necessary and sufficient conditions for polynomial-time solvability, provided efficient algorithms with optimal sample complexity.
result First rigorous conditions and algorithms for learning graph mixtures from epidemic cascades.
Deep learning is a hierarchical inference method formed by subsequent multiple layers of learning able to more efficiently describe complex relationships. In this work, Deep Gaussian Mixture Models are introduced and discussed. A Deep Gaussian Mixture model (DGMM) is a network of multiple layers of latent variables, wh…
DGMEs use Gaussian mixtures to quantify uncertainty in deep learning.
problem Quantifying uncertainty in complex predictive densities.
method DGMEs use a Gaussian mixture model with an EM algorithm for parameter learning.
result DGMEs outperform state-of-the-art models in uncertainty quantification.
Proposes a mixture of expert architecture for polyhedral classifiers.
problem Learning polyhedral classifiers with high accuracy.
method Uses an expectation maximization algorithm to learn parameters.
result Generalization bounds are derived and the method performs comparably to state-of-the-art approaches.
Paper tackles learning mixture of RUMs from partial data.
problem Learning a mixture of Random Utility Models (RUMs) from pairwise comparisons.
method PCA-based spectral clustering to reduce mixture to single component.
result Algorithm correctly clusters data from a mixture of RUMs with high probability.
Improved sample efficiency for private learning of Gaussian mixtures.
problem Learning mixtures of Gaussians with differential privacy.
method Inverse sensitivity mechanism, sample compression, sumset volume bounds.
result Proved optimal sample complexity for private learning of mixtures of Gaussians.
Paper proposes MWDE for estimating finite location-scale mixtures.
problem Estimating finite location-scale mixtures using MLE is problematic.
method Investigates minimum Wasserstein distance estimators (MWDE).
result MWDE is consistent and provides a numerical solution.
Paper uses Gaussian mixture models and Wasserstein distance for schema matching.
problem Schema matching between different datasets.
method Gaussian mixture models and Wasserstein distance for comparison.
result Derives an approximation for Wasserstein distance between Gaussian mixture models.
This work learns sparse tensor representations using mixtures of separable dictionaries.
problem Learning sparse representations of tensor data with structured models.
method Proposes and explores learning a mixture of separable dictionaries with sufficient conditions for local identifiability.
result Developed computational algorithms for batch and online learning.
NMDR estimates complex mixtures of distributions efficiently.
problem Estimating complex finite mixtures of distributions in high-dimensional settings.
method Flexible additive predictors, neural networks, and deep learning optimizers.
result Competitive performance in complex scenarios compared to existing approaches.
This paper studies generalization in machine learning with mixture data.
problem Generalization performance and statistical rates in heterogeneous data.
method Characterization of heterogeneity via pairwise total variation distance, analysis of Rademacher and Gaussian complexities.
result The requirement on heterogeneity increases as function classes get more complex.
Two neural network-based mixture models with E-M learning for efficient likelihood computation.
problem Efficiently computing likelihood in mixture models with complex structures.
method Explicit mixture models with flow-based neural networks, E-M algorithm for parameter learning.
result Demonstrated efficiency in generating samples and maximum likelihood classification.
The study characterizes learning Gaussian mixtures using GLMs in high dimensions.
problem Learning Gaussian mixtures with generalised linear models in high-dimensional settings.
method Empirical risk minimization with convex loss and regularisation.
result Exact asymptotics of the ERM estimator for Gaussian mixtures in high dimensions.
Gradient descent on DDPM objective learns Gaussian mixtures efficiently.
problem Learning Gaussian mixtures using gradient-based methods.
method Gradient descent on DDPM objective, connecting to EM and spectral methods.
result Gradient descent can efficiently recover Gaussian mixture parameters under certain conditions.
The study calculates the risk of semi-supervised multitask learning on Gaussian mixtures.
problem Understanding the risk in semi-supervised multitask learning on Gaussian mixtures.
method Statistical physics methods applied to Gaussian mixture models.
result The study evaluates the performance gain of learning tasks together versus separately.
Transformers can learn optimal regression mixtures efficiently.
problem Limited adoption of tailored regression methods due to their model-specific nature.
method Constructed a generative process for a mixture of linear regressions and used transformers to learn optimal predictors.
result Transformers achieve low mean-squared error and make predictions close to the optimal procedure.
Algorithm finds best mixture of training datasets for improved validation performance.
problem Learning from mixture distributions with covariate shift.
method Combines SGD with optimistic tree search and model re-use over mixture space.
result Proves simple regret guarantees for recovering optimal mixture.
The paper solves robust learning of Gaussian mixtures with nearly optimal guarantees.
problem Learning a high-dimensional Gaussian mixture model with corrupted samples.
method Introduces a new framework called strong observability to circumvent the challenge of learning individual components.
result Achieves optimal robustness guarantees of ε in total variation distance for any constant number of components. GAT-GMM improves GANs' performance in learning Gaussian mixture models.
problem GANs struggle with multi-modal distributions like Gaussian mixtures.
method Proposes a minimax GAN framework using random linear generator and softmax-based quadratic discriminator.
result Gradient Descent Ascent method converges to an approximate minimax point.
The paper proposes a method for interpretable mixture density estimation using a tree structure.
problem Complex probability distributions in machine learning models.
method Interpretable tree structure for mixture density estimation with fast inference.
result The method achieves both high speed and interpretability for mixture density estimation.
SFSVI uses Gaussian mixtures to approximate neural network outputs for continual learning.
problem Learning new tasks without forgetting old ones in neural networks.
method Sequential function-space variational inference with Gaussian mixture approximation.
result Gaussian mixture SFSVI outperforms other methods in continual learning.
Unified framework models graph data as a mixture of graphons using graph moments.
problem Graph datasets often mix from multiple underlying distributions.
method Model graph data as a mixture of graphons, using graph moments to cluster graphs.
result Graphs from similar graphons exhibit similar motif densities, enabling principled estimation of graphon mixture components.
Mixture models are a fundamental tool in applied statistics and machine learning for treating data taken from multiple subpopulations. The current practice for estimating the parameters of such models relies on local search heuristics (e.g., the EM algorithm) which are prone to failure, and existing consistent methods …
Efficiently learns mixtures of Gaussians without separation assumptions.
problem Learning mixtures of Gaussian distributions without assuming separation.
method Reduction to score matching and use of diffusion models.
result Constructs a sampler for the target mixture with polynomial runtime and sample complexity.
We consider unsupervised estimation of mixtures of discrete graphical models, where the class variable corresponding to the mixture components is hidden and each mixture component over the observed variables can have a potentially different Markov graph structure and parameters. We propose a novel approach for estimati…
Plug-and-play L-GM-AMP improves CS recovery for any i.i.d. source prior.
problem Efficiently recovering signals from compressed measurements with unknown priors.
method Deep learning with Gaussian-mixture model to approximate source prior, combined with learned denoising.
result L-GM-AMP achieves state-of-the-art performance without prior knowledge of source distribution.
Improved phylogenetic inference using VBPI-Mixtures for tree topology and branch length.
problem Multimodality of tree-topology posterior distributions in phylogenetic inference.
method VBPI-Mixtures algorithm that uses mixture learning within the BBVI framework.
result VBPI-Mixtures captures tree-topology distributions better than VBPI.
Proposes a new prior for deep generative models to capture latent properties.
problem Complex non-linear relationships between data and latent properties.
method Factorial mixture prior with Gaussian mixture models for quantization.
result Empirically evaluated method for learning discrete properties in unsupervised or semi-supervised settings.
This work addresses identifiability of nonlinear mixtures and proposes a practical algorithm.
problem Identifiability of nonlinear mixtures under unknown nonlinear distortions.
method Identification criterion based on neural network and effective learning algorithm.
result Proposes a practical method for identifying nonlinear mixtures with identifiability guarantees.
TAMD prevents degeneracy in finite mixtures, offering strong guarantees but modest practical improvements.
problem Degeneracy in maximum likelihood estimation of finite mixtures.
method Transcendental regularization with analytic barrier functions.
result Strong theoretical guarantees (identifiability, consistency, robustness) but modest practical improvements.
New method classifies mixtures without labels, recovering latent classes.
problem Classification without reliable instance-level labels.
method Posterior simplex geometry for multiclass learning.
result Classifier trained on mixture identities recovers latent classes and their proportions.
New GMM models fit high-dimensional data with fewer parameters.
problem Overparameterization and lack of flexibility in GMMs for high-dimensional data.
method Piecewise-constant covariance eigenvalue profiles, EM and penalized EM algorithms.
result Superior likelihood-parsimony tradeoffs in density fitting, clustering, and denoising.
A new framework for robust policy learning in MDPs with linear mixture dynamics.
problem Off-dynamics challenge in real-world decision-making problems.
method Linear mixture DRMDP framework, meta algorithm for robust policy learning.
result The new framework provides a more refined representation of uncertainties.
New approach learns mixtures of linear dynamical systems without separation conditions.
problem Learning mixtures of linear dynamical systems with better fit or understanding.
method Tensor decompositions to learn mixtures of linear dynamical systems.
result Algorithm succeeds without strong separation conditions and can compete with Bayes optimal clustering.
Graph Mixture Density Networks model multimodal data on graphs.
problem Challenging conditional density estimation problems with structured data.
method Combining mixture models and graph representation learning.
result Significant improvement in likelihood of epidemic outcomes.