New method identifies latent components in PNL mixtures without strong assumptions.
problem Identifying latent components in PNL mixtures under unknown nonlinear functions.
method Carefully designed UML criterion to identify a null space associated with the mixing system.
result Identification/removal of unknown nonlinearity under minimal conditions.
New method identifies latent components in nonlinear mixtures without stringent assumptions.
problem Unraveling latent components in nonlinearly mixed data.
method Constrained autoencoder-based algorithm for identifiability under relaxed assumptions.
result Comprehensive sample complexity results and new identifiability conditions.
Method estimates bivariate causal models using normalising flows and variational Gaussian process regression.
problem Lack of explainability in AI models, especially in causal mechanisms.
method Combination of normalising flows for density estimation and variational Gaussian process regression for post-nonlinear models.
result Method better explains cause-effect pairs than simple additive noise models.
New method for learning causal relationships in PNL models.
problem Learning causal relationships from empirical observations in PNL models.
method Rank-based methods to estimate non-linear functions, disentangling from independence tests.
result Consistent method for PNL causal discovery, validated in experiments.
By taking into account the nonlinear effect of the cause, the inner noise effect, and the measurement distortion effect in the observed variables, the post-nonlinear (PNL) causal model has demonstrated its excellent performance in distinguishing the cause from effect. However, its identifiability has not been properly …
Multiview analysis aims at extracting shared latent components from data samples that are acquired in different domains, e.g., image, text, and audio. Classic multiview analysis, e.g., canonical correlation analysis (CCA), tackles this problem via matching the linearly transformed views in a certain latent domain. More…
Proposes a flexible method for learning latent causal representations.
problem Limited applicability of existing causal representation learning methods.
method Imposes constraints on function classes and relaxes identifiability conditions.
result Establishes partial identifiability results under weaker conditions.
New methods for scalable causal discovery from complex data.
problem Learning causal structures from nonlinear, continuous or mixed data.
method BF-BIC score and BF-LRT test for scalable causal discovery.
result BF-BIC score and BF-LRT test enable scalable causal discovery with competitive accuracy and runtime.
Study calculates tail risk for various mixture distributions.
problem Estimating tail risk for complex distribution mixtures.
method Analyzes tail conditional expectation for location-scale mixtures of elliptical distributions.
result Developed methods for calculating tail risk in various distributions.
Enhances mixture models with classifier-defined weights.
problem Density evaluation and sampling in mixture models.
method Introduces Classifier Weighted Mixtures (CWM) with functional weights.
result Improves expressivity in variational estimation without increasing complexity.
We consider unsupervised estimation of mixtures of discrete graphical models, where the class variable corresponding to the mixture components is hidden and each mixture component over the observed variables can have a potentially different Markov graph structure and parameters. We propose a novel approach for estimati…
Improved VB algorithm for NIG mixtures outperforms Gaussian mixtures for non-Gaussian data.
problem Clustering non-Gaussian data, especially heavy-tailed and asymmetric.
method Proposed an improved VB algorithm for NIG mixture models and extended Dirichlet process mixture models.
result Outperforms Gaussian mixtures and existing NIG mixture models, especially for highly non-normative data.
When estimating finite mixture models, it is common to make assumptions on the mixture components, such as parametric assumptions. In this work, we make no distributional assumptions on the mixture components and instead assume that observations from the mixture model are grouped, such that observations in the same gro…
Understanding proper distance measures between distributions is at the core of several learning tasks such as generative models, domain adaptation, clustering, etc. In this work, we focus on mixture distributions that arise naturally in several application domains where the data contains different sub-populations. For …
The two most extended density-based approaches to clustering are surely mixture model clustering and modal clustering. In the mixture model approach, the density is represented as a mixture and clusters are associated to the different mixture components. In modal clustering, clusters are understood as regions of high d…
Optimal mixtures of generative models outperform individual models on image datasets.
problem Selecting the best single model from a group of trained generative models.
method Formulated a quadratic optimization problem and proposed the Mixture-UCB algorithm for efficient selection.
result Mixture of generative models achieves better evaluation scores than individual models on benchmark datasets.
New bounds on sample size for identifying mixture models with grouped samples.
problem Identifying mixture models with minimal sample size.
method Generalized identifiability bounds for mixture models with grouped samples.
result Identifiability with (2m−1)/(k−1) samples per group, with no improvement possible. Bayesian approach learns nonparametric mixture components from heterogeneous data.
problem Realistic modeling of heterogeneous data populations with nonparametric mixture components.
method Bayesian nonparametric modeling using Dirichlet process mixture priors.
result Posterior contraction rates for component densities are nearly polynomial, improving over deconvolution methods.
Robust learning mixtures of linear regressions improve robustness.
problem Improving robustness in learning mixtures of linear regressions.
method Connecting mixtures of linear regressions and mixtures of Gaussians with thresholding for a quasi-polynomial time algorithm.
result The algorithm has significantly better robustness than previous results.
Spatially constrained Gaussian mixture models reduce covariance complexity.
problem High dimensionality in finite mixture models for spatial data.
method Spatial covariance constraint with only four free parameters.
result Improves clustering of multi-way spatial data and inference of spatial patterns.
A new method detects outliers using ensembles of Dirichlet process mixtures.
problem Challenges in unsupervised outlier detection using Dirichlet process mixtures.
method Ensembles of Dirichlet process Gaussian mixtures with random subspace and subsampling.
result Empirically outperforms existing approaches in unsupervised outlier detection.
The parsimonious Gaussian mixture models, which exploit an eigenvalue decomposition of the group covariance matrices of the Gaussian mixture, have shown their success in particular in cluster analysis. Their estimation is in general performed by maximum likelihood estimation and has also been considered from a parametr…
Information-theoretic measures such as the entropy, cross-entropy and the Kullback-Leibler divergence between two mixture models is a core primitive in many signal processing tasks. Since the Kullback-Leibler divergence of mixtures provably does not admit a closed-form formula, it is in practice either estimated using …
We present two different approaches for parameter learning in several mixture models in one dimension. Our first approach uses complex-analytic methods and applies to Gaussian mixtures with shared variance, binomial mixtures with shared success probability, and Poisson mixtures, among others. An example result is that …
Paper establishes universal lower bounds and optimal rates for clustering sub-exponential mixture models.
problem Achieving optimal error rates in clustering sub-exponential mixture models.
method Establishes universal lower bounds and demonstrates iterative algorithms' optimality in sub-exponential mixture models.
result Iterative algorithms achieve the universal lower bound in sub-exponential mixture models.
New method for summarizing Bayesian mixture models using sliced Wasserstein distances.
problem Estimating the mixing measure in nonparametric Bayesian mixture models.
method Decision-theoretic approach using sliced Wasserstein distances for Gaussian mixtures.
result Effective estimation of the mixing measure and mixture density.
Algorithm estimates nonparametric mixtures from grouped data.
problem Estimating identifiable nonparametric mixture models from grouped observations.
method Oracle inequality for weighted kernel density estimators and general consistency result.
result Consistent estimation of mixture components from grouped observations.
The paper tackles learning mixtures of two multinomial logits, showing identifiability and presenting an algorithm.
problem Learning an arbitrary mixture of two multinomial logits.
method Reduction to solving a system of univariate quartic equations, followed by an algorithm using polynomial and linear samples.
result Identifiability of the mixture models may only fail on an algebraic variety of negligible measure.
A new method for fast Bayesian mixture model estimation.
problem Estimating Bayesian mixture models is computationally challenging.
method Amortized Bayesian Inference (ABI) framework for mixture models.
result The method provides fast inference for mixture models.
New algorithm recovers mixture means even with many outliers.
problem Estimating mixture means when outliers overwhelm small groups.
method Proposes an algorithm for robust mixture learning with minimal list-size overhead.
result Order-optimal error guarantees for each mixture mean.
Finite mixture models are statistical models which appear in many problems in statistics and machine learning. In such models it is assumed that data are drawn from random probability measures, called mixture components, which are themselves drawn from a probability measure P over probability measures. When estimating …
Study on Dirichlet process mixtures for clustering consistency.
problem Consistency of clustering with Dirichlet process mixtures.
method Analysis of posterior distribution as sample size increases, focusing on consistency for the number of clusters.
result Consistency for the number of clusters can be achieved with a properly adapted concentration parameter in a Bayesian setting.
The paper uses Gaussian mixture models for Bayesian networks and proposes an optimization algorithm.
problem Modeling nodes in Bayesian networks with complex distributions.
method Gaussian mixture models combined with double iteration algorithm.
result The double iteration algorithm optimizes Gaussian mixture models effectively.
Deep learning is a hierarchical inference method formed by subsequent multiple layers of learning able to more efficiently describe complex relationships. In this work, Deep Gaussian Mixture Models are introduced and discussed. A Deep Gaussian Mixture model (DGMM) is a network of multiple layers of latent variables, wh…
Paper uses Gaussian mixture models and Wasserstein distance for schema matching.
problem Schema matching between different datasets.
method Gaussian mixture models and Wasserstein distance for comparison.
result Derives an approximation for Wasserstein distance between Gaussian mixture models.
Improves scalability and efficiency of mixture models in black-box variational inference.
problem Scaling mixture models in black-box variational inference leads to high parameter and time costs.
method Introduces MISVAE for amortized mixture parameter space and new ELBO estimators.
result Achieves superior estimation performance with fewer parameters and shorter inference time.
SMT trains generative models by estimating mixture scores, outperforming existing methods.
problem Training one-step generative models efficiently and effectively.
method Score-of-Mixture Training (SMT) estimates the score of mixture distributions between real and fake samples.
result SMT/SMD outperform existing methods on CIFAR-10 and ImageNet 64x64 datasets.
New method estimates mixture model components efficiently.
problem Estimating the number of components in finite mixture models.
method Group-Sort-Fuse (GSF) procedure for simultaneous estimation of order and mixing measure.
result GSF achieves consistent estimation of true mixture order and n−1/2 convergence rate. Discriminative classifier for compositional data using hierarchical mixture of Generalized Dirichlet models.
problem Classifying compositional data, especially in spam detection and color space identification.
method Hierarchical mixture of discriminative Generalized Dirichlet classifiers, using variational approximation for parameter learning.
result First time a variational upper-bound for Generalized Dirichlet mixture is proposed in literature.
Mixture modeling is a general technique for making any simple model more expressive through weighted combination. This generality and simplicity in part explains the success of the Expectation Maximization (EM) algorithm, in which updates are easy to derive for a wide class of mixture models. However, the likelihood of…
MixTS uses a mixture prior to analyze Thompson Sampling in multi-task learning.
problem Analyzing Thompson Sampling in environments with uncertain and multi-class problems.
method Developed MixTS by incorporating a mixture prior into Thompson Sampling and using a novel proof technique for mixture distributions.
result Proved Bayes regret bounds for MixTS in linear bandits and finite-horizon reinforcement learning.
Study uniform rates for estimating Gaussian mixtures without separation assumption.
problem Estimating parameters in two-component Gaussian mixtures without separation.
method Uniform convergence rates derived using minimax lower bounds and careful analysis of polynomial equalities.
result Phase transition in optimal estimation rate based on mixture balance.
The study bounds the stability of Gaussian mixtures under small perturbations.
problem Stability of Gaussian mixtures under small changes in distribution.
method Deriving an explicit bound on parameter stability of spherical Gaussian Mixture Models (sGMM) in a pre-defined model class.
result Upper bound on parameter distance of close sGMMs to the original sGMM, dependent only on the original model.
The paper examines risk aggregation under mixtures of marginals, finding that more homogeneous distributions lead to larger uncertainty.
problem Investigating the impact of mixing on risk aggregation uncertainty.
method Analyzes ordering relations and inequalities for aggregation sets under distribution and quantile mixtures.
result More homogeneous marginals result in larger aggregation sets, indicating greater model uncertainty.
New concept of mixture complexity helps detect gradual clustering changes.
problem Determining the number of clusters in mixture models with overlaps and weight biases.
method Introducing mixture complexity (MC) as a new measure of cluster size, defined from information theory.
result MC can detect gradual clustering changes, allowing earlier detection and finer distinction.
Mixture components improve VAE performance by increasing latent flexibility.
problem Improving variational autoencoder (VAE) performance through more flexible latent representations.
method Modeling mixture components with separate encoder networks and analyzing their impact on ELBO.
result Increasing the number of mixture components improves VAE performance on various datasets.
The paper develops methods to create reliable prediction sets for complex mixture models in high-dimensional data.
problem Building accurate prediction sets for high-dimensional mixture models with feature-dependent weights.
method The authors introduce a debiasing procedure and a novel interval combination strategy to construct valid prediction sets.
result The proposed method provides reliable coverage guarantees for prediction sets in high-dimensional mixture models.
Paper proposes a new method for estimating mixture proportions without irreducibility assumption.
problem Estimating mixture proportions when component distributions are not irreducible.
method Developed a resampling-based meta-algorithm that adapts existing MPE algorithms to non-irreducible settings.
result Empirical results show improved estimation performance compared to baseline methods and regrouping-based algorithms.