This work addresses identifiability of nonlinear mixtures and proposes a practical algorithm.
problem Identifiability of nonlinear mixtures under unknown nonlinear distortions.
method Identification criterion based on neural network and effective learning algorithm.
result Proposes a practical method for identifying nonlinear mixtures with identifiability guarantees.
Proposes a partially linear structure to capture nonlinear relationships in mixture of experts models.
problem Suboptimal estimates due to linearity assumption in mixture of experts models.
method Introduces a partially linear structure that incorporates unspecified functions to capture nonlinear relationships.
result Establishes the identifiability of the proposed model under mild conditions and introduces a practical estimation algorithm.
New method identifies latent components in PNL mixtures without strong assumptions.
problem Identifying latent components in PNL mixtures under unknown nonlinear functions.
method Carefully designed UML criterion to identify a null space associated with the mixing system.
result Identification/removal of unknown nonlinearity under minimal conditions.
Novel methods for splitting Gaussian mixtures improve uncertainty propagation in nonlinear systems.
problem Improving accuracy and efficiency in nonlinear uncertainty propagation.
method Preserving mean and covariance, novel heuristics for selecting splitting direction informed by initial uncertainty and nonlinear function properties.
result Improved accuracy and efficiency in uncertainty propagation compared to existing techniques.
A Gaussian mixture model improves generalization for long-tailed data.
problem Optimizing generalization for rare data in long-tailed distributions.
method Suggested Gaussian mixture model and comparison of linear vs. nonlinear classifiers.
result Nonlinear classifiers outperform linear ones for long-tailed data.
New method identifies latent sources from nonlinear mixtures without auxiliary variables.
problem Identifying latent sources from nonlinear mixtures without additional information.
method Structural Sparsity assumptions on the mixing process.
result Latent sources can be identified up to permutation and transformation.
Study shows mixtures of nonlinearities can improve deep learning performance.
problem Improving deep learning performance with large datasets and complex models.
method Analyzed random feature regression with features F=f(WX+B) for a random weight matrix W and random bias vector B. result Mixture of nonlinearities can improve both training and test errors over a single nonlinearity.
New method identifies latent components in nonlinear mixtures without stringent assumptions.
problem Unraveling latent components in nonlinearly mixed data.
method Constrained autoencoder-based algorithm for identifiability under relaxed assumptions.
result Comprehensive sample complexity results and new identifiability conditions.
Deep learning is a hierarchical inference method formed by subsequent multiple layers of learning able to more efficiently describe complex relationships. In this work, Deep Gaussian Mixture Models are introduced and discussed. A Deep Gaussian Mixture model (DGMM) is a network of multiple layers of latent variables, wh…
A mixture of factor analyzers is a semi-parametric density estimator that generalizes the well-known mixtures of Gaussians model by allowing each Gaussian in the mixture to be represented in a different lower-dimensional manifold. This paper presents a robust and parsimonious model selection algorithm for training a mi…
IMA addresses non-identifiability in nonlinear ICA by assuming orthogonal Jacobian columns.
problem Non-identifiability in nonlinear ICA.
method IMA assumes orthogonal Jacobian columns and extends to manifold settings.
result IMA circumvents non-identifiability issues and can be beneficial for higher-dimensional observations.
Causal Mosaic distinguishes cause from effect using nonlinear ICA and ensemble methods.
problem Distinguishing cause from effect in bivariate settings.
method Nonlinear ICA and ensemble framework (Causal Mosaic).
result Causal Mosaic shows state-of-the-art performance on artificial and real-world datasets.
Global optimization approach for MAP clustering under Gaussian mixtures.
problem Maximum a-posteriori clustering problem under Gaussian mixture model.
method Mixed-integer nonlinear optimization (MINLP) transformed into mixed-integer quadratic program (MIQP).
result Explicit quantification of optimality gap, leading to globally optimal solutions.
New insights into nonlinear multiview analysis for better data interpretation.
problem Identify shared latent components across different data views.
method Post-nonlinear model and multiview mixture learning.
result Identifies shared latent components under certain conditions.
StrADiff separates sources from mixtures without labels, using structured priors.
problem Blind source separation of linear and nonlinear mixtures without labeled data.
method Structured Source-Wise Adaptive Diffusion Framework with Gaussian process priors.
result StrADiff can recover latent source trajectories in an unsupervised manner, especially stable in linear mixtures.
New method for separating mixed signals with nonlinear functions.
problem Recovering source signals from nonlinear mixtures.
method Optimisation-based function approximation to minimize mutual statistical dependence.
result The method can recover source signals from nonlinear mixtures under certain conditions.
PDGMM-VAE uses adaptive priors for better ICA recovery.
problem Nonlinear ICA recovery of latent source signals.
method Adaptive per-dimension Gaussian mixture model priors in a variational autoencoder.
result PDGMM-VAE effectively recovers source-specific non-Gaussian marginals.
This research improves asset life prediction by integrating deep learning with mixture distributions.
problem Predicting residual useful life for assets with multiple failure modes.
method Integrates mixture (log)-location-scale distribution with deep learning.
result Proposed models outperform existing methods in predicting residual useful life.
New method discovers causal models from mixed time series data.
problem Discovering causal relationships from heterogeneous time series data.
method Variational inference-based framework MCD for linear and nonlinear causal models.
result Method outperforms state-of-the-art benchmarks in causal discovery tasks.
A method for identifying NPWARX models with arbitrary domains using probabilistic mixture models.
problem Identifying hybrid system models with discontinuous maps.
method Probabilistic mixture model with a neural network for nonlinear partitioning and Expectation Maximization for parameter estimation.
result Demonstrated on a nonlinear piece-wise problem with discontinuous maps.
New method reduces mixture model evaluation cost for large models.
problem Computational infeasibility of evaluating all mixture components.
method Combining EM and Metropolis-Hastings for stochastic sampling.
result Significantly reduced computational cost for large models.
For many years, a combination of principal component analysis (PCA) and independent component analysis (ICA) has been used for blind source separation (BSS). However, it remains unclear why these linear methods work well with real-world data that involve nonlinear source mixtures. This work theoretically validates that…
We examine some differential geometric approaches to finding approximate solutions to the continuous time nonlinear filtering problem. Our primary focus is a new projection method for the optimal filter infinite dimensional Stochastic Partial Differential Equation (SPDE), based on the direct L2 metric and on a family o…
A new clustering method using autoencoders for improved data representation.
problem Improving clustering of complex data like images and text.
method DAMIC algorithm based on a mixture of deep autoencoders.
result Significant improvement over state-of-the-art methods on image and text corpora.
Study minimax estimation of stratified structure from i.i.d. samples.
problem Estimating stratified structure from i.i.d. samples of stratified mixtures of immersed manifolds.
method Ascending hierarchical co-detection of points belonging to different layers, identifying number of layers and their dimensions, assigning points to layers accurately, estimating tangent spaces optimally.
result Achieves optimal estimation of mixture components at their optimal dimension-specific rates adaptively.
ROME improves algorithmic fairness by learning latent group structure robustly.
problem Latent subgroup disparities and distribution shifts in machine learning models.
method ROME uses an Expectation-Maximization algorithm for linear models and a neural Mixture-of-Experts for nonlinear settings.
result ROME significantly improves fairness compared to standard methods while maintaining average performance.
Paper analyzes a generalized EM algorithm for Gaussian mixtures in control systems.
problem Parametric distribution-based clustering in unsupervised learning.
method Proposes a generalized EM (GEM) algorithm for Gaussian mixture models, analyzing its convergence properties using control theory.
result GEM algorithm can be understood as a linear time-invariant system with feedback nonlinearity.
Unsupervised clustering is one of the most fundamental challenges in machine learning. A popular hypothesis is that data are generated from a union of low-dimensional nonlinear manifolds; thus an approach to clustering is identifying and separating these manifolds. In this paper, we present a novel approach to solve th…
Bayesian model uses simple functions to forecast macroeconomic data.
problem Forecasting large datasets in macroeconomics with complex nonlinear relationships.
method Sum of simple two-component location mixtures, logistic function threshold, conjugate priors.
result Accurate point and density forecasts in US macroeconomic aggregates.
Study reconstructs causal graph from latent variables using mixture oracles.
problem Reconstructing causal graphical model from data with latent variables.
method Reduction to mixture oracle to identify latent representations and causal structure.
result Conditions for identifying latent representations and causal model.
This paper introduces a robust mixing model to describe hyperspectral data resulting from the mixture of several pure spectral signatures. This new model not only generalizes the commonly used linear mixing model, but also allows for possible nonlinear effects to be easily handled, relying on mild assumptions regarding…
This paper introduces the kernel mixture network, a new method for nonparametric estimation of conditional probability densities using neural networks. We model arbitrarily complex conditional densities as linear combinations of a family of kernel functions centered at a subset of training points. The weights are deter…
The mixture of experts (MoE) model is a popular neural network architecture for nonlinear regression and classification. The class of MoE mean functions is known to be uniformly convergent to any unknown target function, assuming that the target function is from Sobolev space that is sufficiently differentiable and tha…
Appropriately designing the proposal kernel of particle filters is an issue of significant importance, since a bad choice may lead to deterioration of the particle sample and, consequently, waste of computational power. In this paper we introduce a novel algorithm adaptively approximating the so-called optimal proposal…
SMI uses mixture models to improve SVGD's performance in Bayesian inference.
problem Variance collapse in SVGD for Bayesian inference, especially with small models.
method Generalizes SVGD to Stein mixture models, optimizing an ELBO lower bound.
result SMI avoids variance collapse and accurately estimates uncertainty for small BNNs.
Generative model combines shape and intensity priors for left atrium segmentation.
problem Challenges in segmenting left atrium MRI images due to shape variation and multimodality.
method Generative image model with mixture of Gaussians for shape priors and autoencoders for intensity priors.
result Maximizes posterior probability using a mixture of Gaussians for shape priors and autoencoders for intensity priors.
Random feature maps are ubiquitous in modern statistical machine learning, where they generalize random projections by means of powerful, yet often difficult to analyze nonlinear operators. In this paper, we leverage the "concentration" phenomenon induced by random matrix theory to perform a spectral analysis on the Gr…
Unified approach for interpretable regression with flexible modeling.
problem Combining predictive adaptivity with interpretability in heterogeneous data.
method Combining random Fourier features, spectral feature map, principal component analysis, Gaussian mixture model, and cluster-specific generalized additive models.
result Consistently improves upon classical and black-box models across benchmark datasets.
Paper introduces MPPGA for integrating multiple PGA models on Riemannian manifolds.
problem Challenges in dimensionality reduction on Riemannian manifolds with multiple modalities.
method Develops a mixture probabilistic principal geodesic analysis (MPPGA) model.
result Demonstrates improved clustering and shape analysis using MPPGA.
A new model improves clustering by reducing redundancy in mixture of local EPCAs.
problem Redundancy in traditional mixture models causes ambiguity in clustering.
method Introduced a repulsiveness-encouraging prior and a DPP for diversity in DEPCAM model.
result The method reduces model redundancy and improves clustering performance.
Paper proposes SVM-based methods for inferring interaction networks.
problem Modeling interaction between variables in time series and high dimensions.
method Two approaches: neighborhood SVM and restricted Bayesian network for time series.
result Efficiency demonstrated through simulations with linear and nonlinear data.
We propose a practical and scalable Gaussian process model for large-scale nonlinear probabilistic regression. Our mixture-of-experts model is conceptually simple and hierarchically recombines computations for an overall approximation of a full Gaussian process. Closed-form and distributed computations allow for effici…
DEQs and explicit networks are nearly equivalent for Gaussian mixtures.
problem Understanding the equivalence between DEQs and explicit neural networks.
method Random matrix theory and analysis of kernel matrices.
result A shallow explicit network can mimic the kernel of a DEQ.
Study of two-layer NNs under Gaussian mixtures data, proving polynomial models equivalent to neural networks.
problem Training and generalization performance of two-layer NNs under structured Gaussian mixture data.
method Asymptotic analysis of two-layer NNs after one gradient descent step under Gaussian mixture data assumption.
result High-order polynomial models equivalent to nonlinear neural networks under certain conditions.
Bayesian framework for analyzing heterogeneous covariance data with a novel MoE-Wishart model.
problem Analyzing complex multivariate systems with varying covariance structures.
method Comprehensive Bayesian framework using mixture-of-experts Wishart model with predictor-dependent mixture weights.
result Accurate subpopulation recovery and estimation in heterogeneous covariance scenarios.
New approach tackles nonidentifiability in nonlinear blind source separation.
problem Nonidentifiability in nonlinear blind source separation.
method Independent mechanism analysis, incorporating causal assumptions.
result Empirical and theoretical evidence shows improved identifiability.
In this paper we introduce a projection method for the space of probability distributions based on the differential geometric approach to statistics. This method is based on a direct L2 metric as opposed to the usual Hellinger distance and the related Fisher Information metric. We explain how this apparatus can be used…
A new framework uses an Incremental Transformer to design geopolymer mixtures efficiently.
problem Designing geopolymer mixtures with limited data and physical constraints.
method Topology-aware surrogate framework guided by Incremental Transformer.
result The design space is redundant, with fewer effective mixture regimes.