Nonlinear independent component analysis (ICA) provides an appealing framework for unsupervised feature learning, but the models proposed so far are not identifiable. Here, we first propose a new intuitive principle of unsupervised deep learning from time series which uses the nonstationary structure of the data. Our l…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “nonlinear ICA”
Nonlinear independent component analysis (ICA) is a general framework for unsupervised representation learning, and aimed at recovering the latent variables in data. Recent practical methods perform nonlinear ICA by solving a series of classification problems based on logistic regression. However, it is well-known that…
New method identifies latent sources from nonlinear mixtures without auxiliary variables.
problem Identifying latent sources from nonlinear mixtures without additional information.
method Structural Sparsity assumptions on the mixing process.
result Latent sources can be identified up to permutation and transformation.
New model combines ICA and HMM for unsupervised learning of nonstationary time series.
problem Manual segmentation of non-stationary data is computationally expensive and inaccurate.
method Combines Hidden Markov Model with nonlinear ICA for unsupervised learning.
result Proves identifiability of the model for general mixing nonlinearity.
This paper reviews nonlinear ICA for disentangled representations in unsupervised learning.
problem Finding useful disentangled representations in unsupervised deep learning.
method Review of nonlinear ICA theory and algorithms for disentanglement.
result Nonlinear ICA can be shown to estimate useful disentangled representations.
Nonlinear ICA is a fundamental problem for unsupervised representation learning, emphasizing the capacity to recover the underlying latent variables generating the data (i.e., identifiability). Recently, the very first identifiability proofs for nonlinear ICA have been proposed, leveraging the temporal structure of the…
New framework for identifying spatial data components using TP latent components.
problem Identifying complex dependencies in spatial data.
method Introduces a new nonlinear ICA framework with -process latent components and develops a learning and inference algorithm.
result Identifiability of TP independent components under general conditions and Gaussian Process limit.
New methods generalize nonlinear ICA beyond structural sparsity.
problem Identify true latent sources from nonlinear mixtures without structural sparsity assumptions.
method Propose identifiability results for undercomplete, partial sparsity, and flexible grouping structures.
result Prove identifiability in general settings of undercompleteness, partial sparsity, and flexible grouping structures.
Paper connects contrastive learning to MI maximization and establishes robust methods for nonlinear ICA and subspace estimation.
problem Understanding and improving unsupervised representation learning and density ratio estimation.
method The paper connects contrastive learning to MI maximization, establishes new recovery conditions for nonlinear ICA, and proposes a practical outlier-robust method for nonlinear subspace estimation.
result The proposed methods can be seen as maximizing MI, performing nonlinear ICA, or estimating nonlinear subspaces, and are robust to outliers.
Independent Component Analysis (ICA) aims to find a coordinate system in which the components of the data are independent. In this paper we construct a new nonlinear ICA model, called WICA, which obtains better and more stable results than other algorithms. A crucial tool is given by a new efficient method of verifying…
New model improves transfer learning and semi-supervised learning.
problem Improving transfer learning and semi-supervised learning performance.
method Developed a new conditional energy-based model (ICE-BeeM) based on nonlinear ICA.
result Identifiable representations learned by ICE-BeeM improve performance in transfer learning and semi-supervised learning tasks.
Paper extends ICA to ISA with auxiliary variables for better speech representation learning.
problem Learning unsupervised speech representations with independent subspaces.
method Theoretical framework of nonlinear ISA with auxiliary variables.
result Proposes an algorithm to learn speech representations with independent subspaces.
We address the problem of distinguishing cause from effect in bivariate setting. Based on recent developments in nonlinear independent component analysis (ICA), we train nonparametrically general nonlinear causal models that allow non-additive noise. Further, we build an ensemble framework, namely Causal Mosaic, which …
Reservoir subspace injection improves online ICA by preserving injected features.
problem Discarding injected features in top- whitening can degrade performance.
method Formalized reservoir subspace injection (RSI) and developed diagnostics (IER, SSO, ρ_x) to identify and mitigate the failure mode.
result RSI controller preserves passthrough retention, improving performance by up to 2.2 dB.
Proposes -PCA to learn identifiable linear transformations without whitening.
problem Cannot identify axes with equal variances in PCA.
method Unified model for linear and nonlinear PCA, introducing a missing piece to eliminate rotational indeterminacy.
result Eliminates subspace rotational indeterminacy in PCA.
We consider the problem of recovering a common latent source with independent components from multiple views. This applies to settings in which a variable is measured with multiple experimental modalities, and where the goal is to synthesize the disparate measurements into a single unified representation. We consider t…
On the achievability of blind source separation for high-dimensional nonlinear source mixturesstat.ML
For many years, a combination of principal component analysis (PCA) and independent component analysis (ICA) has been used for blind source separation (BSS). However, it remains unclear why these linear methods work well with real-world data that involve nonlinear source mixtures. This work theoretically validates that…
IMA addresses non-identifiability in nonlinear ICA by assuming orthogonal Jacobian columns.
problem Non-identifiability in nonlinear ICA.
method IMA assumes orthogonal Jacobian columns and extends to manifold settings.
result IMA circumvents non-identifiability issues and can be beneficial for higher-dimensional observations.
Study on robustness of unsupervised representation learning in slightly misspecified settings.
problem Identify nonlinear representation learning in slightly misspecified settings.
method Formalize and investigate Independent Component Analysis (ICA) with observations generated by a mixing function close to a local isometry.
result Approximate identifiability of nonlinear ICA with almost isometric mixing functions.
New principle for disentangling latent factors using sparse regularization.
problem Disentangling latent factors from complex data.
method Sparse regularization of latent mechanisms to induce disentanglement.
result Recovery of latent variables up to permutation under certain conditions.
New framework for disentangling features from noisy data.
problem Disentangling identifiable features from noisy data.
method Structured Nonlinear Independent Component Analysis (SNICA).
result Identifiability holds even in the presence of noise of unknown distribution.
ICA accurately estimates treatment effects even with confounders.
problem Estimating treatment effects in the presence of confounding variables.
method Uses Independent Component Analysis (ICA) to identify latent sources and estimate mixing coefficients.
result Linear ICA can consistently estimate multiple treatment effects, even with Gaussian confounders, and is more sample-efficient than Orthogonal Machine Learning (OML).
Second-order Approximation of Minimum Discrimination Information in Independent Component Analysisstat.ML
A new method improves ICA performance by approximating MDI.
problem Improving F astICA's performance with nonlinear functions.
method Second-order approximation of MDI for joint maximization.
result Efficiency validated through experiments compared to other ICA algorithms.
This paper tackles sequential distribution shifts in representation learning.
problem Learning meaningful representations in a sequence of distribution shifts.
method Nonlinear Independent Component Analysis (ICA) framework for continual causal representation learning.
result The method achieves performance comparable to joint training on multiple offline distributions and shows no benefit from the incoming new distribution on all latent variables.
DDICA separates nonlinear mixed signals robustly.
problem Blind source separation of nonlinear mixed signals.
method Deep deterministic neural network with matrix-based entropy function.
result DDICA effectively separates independent components with high accuracy.
Randomized ICA and LDA Dimensionality Reduction Methods for Hyperspectral Image Classificationstat.ML
Dimensionality reduction is an important step in processing the hyperspectral images (HSI) to overcome the curse of dimensionality problem. Linear dimensionality reduction methods such as Independent component analysis (ICA) and Linear discriminant analysis (LDA) are commonly employed to reduce the dimensionality of HS…
This research improves dynamical systems understanding by identifying latent states and their nonlinear transitions.
problem Previous work on dynamical systems could not identify nonlinear transition dynamics, leading to unreliable predictions.
method Proposes a state-space modeling framework using variational auto-encoders to identify latent states and their nonlinear transition functions.
result Demonstrates high accuracy in recovering latent state dynamics and future prediction accuracy.
PDGMM-VAE uses adaptive priors for better ICA recovery.
problem Nonlinear ICA recovery of latent source signals.
method Adaptive per-dimension Gaussian mixture model priors in a variational autoencoder.
result PDGMM-VAE effectively recovers source-specific non-Gaussian marginals.
The paper discovers a hidden component in data using an autoencoder with a discriminator.
problem Discovering a single independent latent variable in data.
method An autoencoder with a discriminator is used to recover the hidden component.
result The approach can recover the hidden component up to entropy-preserving transformations.
New findings on identifying latent variables in nonlinear ICA models.
problem Identifying latent variables in nonlinear ICA models is challenging due to spurious solutions.
method Proved that conformal maps are identifiable and provided theoretical results on preventing spurious solutions.
result Conformal maps are identifiable in nonlinear ICA models, preventing spurious solutions.
The paper shows that certain learned representations are identifiable in function space.
problem Identifiability of learned representations in deep neural networks.
method Using recent advances in nonlinear ICA, the paper shows that a large family of discriminative models are identifiable in function space, up to a linear indeterminacy.
result Many models for representation learning are identifiable in function space, including text, images, and audio.
New method identifies causal graphs with limited data and noise.
problem Identifying causal graphs from observational data is generally impossible.
method Using additional data from two environments with different noise statistics, and assuming Gaussian noise.
result The entire causal graph can be uniquely identified with a constant number of environments.
Paper establishes identifiability conditions for a model with two latent vectors and auxiliary data.
problem Identifying conditions for a statistical model with two latent vectors and auxiliary data.
method Proposes a statistical model with two latent vectors and auxiliary data, establishing various identifiability conditions.
result Identifiability conditions reveal a dimensionality relation and link model indeterminacies to maximum link weights.
Causal Component Analysisstat.ML
Causal Component Analysis aims to recover latent variables with causal relationships.
problem Recover latent variables with causal relationships from observed mixtures.
method Introduces a likelihood-based approach using normalizing flows to estimate unmixing function and causal mechanisms.
result Demonstrates effectiveness through synthetic experiments in CauCA and ICA settings.
Independent component analysis (ICA), as an approach to the blind source-separation (BSS) problem, has become the de-facto standard in many medical imaging settings. Despite successes and a large ongoing research effort, the limitation of ICA to square linear transformations have not been overcome, so that general INFO…
The framework of variational autoencoders allows us to efficiently learn deep latent-variable models, such that the model's marginal distribution over observed variables fits the data. Often, we're interested in going a step further, and want to approximate the true joint distribution over observed and latent variables…
Non-linear source separation is a challenging open problem with many applications. We extend a recently proposed Adversarial Non-linear ICA (ANICA) model, and introduce Cramer-Wold ICA (CW-ICA). In contrast to ANICA we use a simple, closed--form optimization target instead of a discriminator--based independence measure…
A central question of representation learning asks under which conditions it is possible to reconstruct the true latent variables of an arbitrarily complex generative process. Recent breakthrough work by Khemakhem et al. (2019) on nonlinear ICA has answered this question for a broad class of conditional generative proc…
New model separates images into independent factors quickly and easily.
problem Separating high-dimensional data like images into independent latent factors.
method Combines bijective feature maps with linear ICA model on the Stiefel manifold.
result Models converge quickly and achieve better unsupervised latent factor discovery.
A new ICA method adds L1-regularization for better interpretability of fMRI data.
problem Improving interpretability of ICA features in high-dimensional fMRI data.
method L1-regularization added to ICA cost function, solved by DCA.
result Validated on synthetic and real fMRI data, improving feature interpretability.
The FastICA algorithm is one of the most popular iterative algorithms in the domain of linear independent component analysis. Despite its success, it is observed that FastICA occasionally yields outcomes that do not correspond to any true solutions (known as demixing vectors) of the ICA problem. These outcomes are comm…
OT-ICA uses optimal transport to find independent components, outperforming traditional methods.
problem Finding independent components from linear mixtures of signals.
method OT-ICA uses the squared Wasserstein distance to maximize non-Gaussianity, optimizing projections via gradient descent.
result OT-ICA outperforms traditional proxy-based methods in various applications.
Independent component analysis (ICA) is the most popular method for blind source separation (BSS) with a diverse set of applications, such as biomedical signal processing, video and image analysis, and communications. Maximum likelihood (ML), an optimal theoretical framework for ICA, requires knowledge of the true unde…
Recently, an extension of independent component analysis (ICA) from one to multiple datasets, termed independent vector analysis (IVA), has been the subject of significant research interest. IVA has also been shown to be a generalization of Hotelling's canonical correlation analysis. In this paper, we provide the ident…
Independent Component Analysis (ICA) is a popular model for blind signal separation. The ICA model assumes that a number of independent source signals are linearly mixed to form the observed signals. We propose a new algorithm, PEGI (for pseudo-Euclidean Gradient Iteration), for provable model recovery for ICA with Gau…
Paper proposes compressive ICA algorithms for ICA model.
problem Efficiently solving ICA model with reduced memory and computational complexity.
method Compressive learning approach to ICA model, proving existence of compressive ICA scheme, proposing two algorithms (IPG and ASD).
result Proposed algorithms achieve substantial memory gains over well-known ICA algorithms.
Independent component analysis (ICA) is a powerful method for blind source separation based on the assumption that sources are statistically independent. Though ICA has proven useful and has been employed in many applications, complete statistical independence can be too restrictive an assumption in practice. Additiona…
Independent component analysis (ICA) has become a standard data analysis technique applied to an array of problems in signal processing and machine learning. This tutorial provides an introduction to ICA based on linear algebra formulating an intuition for ICA from first principles. The goal of this tutorial is to prov…