LOL method simplifies forming linear combinations of latent variables.
problem Lack of general-purpose methods for manipulating latent variables.
method Latent Optimal Linear combinations (LOL) method.
result LOL simplifies creation of expressive low-dimensional representations.
The paper tackles causal disentanglement with linear models and interventions.
problem Identify latent variables in a causal model from observed data.
method Use linear transformations and interventions to uniquely identify latent variables.
result A single intervention on each latent variable is sufficient for identifying the latent causal model.
Study identifies parameters in causal models with latent confounding.
problem Parameter identification in linear non-Gaussian causal models with latent confounding.
method Graphical criterion for necessary and sufficient identifiability of direct causal effects, with polynomial-time algorithm.
result Developed a graphical criterion for identifying direct causal effects in latent variable models with arbitrary non-linear confounding.
New method identifies latent variables with causal dependencies from observed data.
problem Identify latent variables with causal relationships from observed data.
method Linear causal disentanglement via higher-order cumulants, with perfect and soft interventions.
result Recovery of parameters via coupled tensor decomposition and polynomial equations.
Develops a method to identify causal effects in linear models with latent variables.
problem Identifying causal effects in models with latent variables that are not independent.
method A novel graphical criterion and an integer linear program algorithm.
result Sufficient condition for identifying causal effects by rational formulas in the covariance matrix.
Improved DSSMs for easier interpretable latent variables.
problem Complex and hard-to-interpret latent variables in DSSMs.
method Simplified predictive decoder and shrinkage priors.
result Interpretable latent variables improve forecasting performance.
Bayesian non-linear latent variable modeling for complex data.
problem Inference for GPLVMs is computationally limited and often leads to overfitting or underestimates uncertainty.
method Approximate Gaussian process mappings with random Fourier features for MCMC inference.
result Generalized RFLVMs perform well on various data types and applications.
Paper proposes a new method to identify causal graphs with latent variables using higher-order cumulants.
problem Estimating causal directed acyclic graphs with latent confounders.
method Uses higher-order cumulants to identify causal structures among observed and latent variables.
result Validates the proposed algorithm through simulations and real-world data.
A semi-parametric, non-linear regression model in the presence of latent variables is applied towards learning network graph structure. These latent variables can correspond to unmodeled phenomena or unmeasured agents in a complex system of interacting entities. This formulation jointly estimates non-linearities in the…
Bayesian active learning reduces data needed for latent variable models.
problem Active learning for latent variable models, especially mixtures of linear regressions and HMMs with GLM observations.
method Maximum-mutual-information input selection for discrete latent variable regression models.
result Active learning can achieve large gains for mixtures of linear-Gaussian models and substantially reduces data needed for GLM-HMM.
The paper identifies causal effects in latent variable models using higher-order cumulants.
problem Challenges in identifying causal effects in latent variable models with latent confounders.
method Using higher-order cumulants, the paper addresses two challenging setups: a single proxy variable and underspecified instrumental variables.
result Causal effects are identifiable with a single proxy or instrument.
Enhances interpretability of linear latent spaces through automated clustering and ranking.
problem Severe interpretability issues in latent directions of PCA, ICA, CCA, and FA.
method LS-PIE framework automates clustering and ranking of latent vectors.
result Enhanced interpretability of latent vectors through LR, LS, LC, and LCON.
This work extends identifiability analysis to sequential latent variable models, focusing on Switching Dynamical Systems.
problem Identifying latent variables in sequential data models.
method Proved identifiability of Markov Switching Models and established conditions for Switching Dynamical Systems.
result Identifiability of latent variables and non-linear mappings in Switching Dynamical Systems up to affine transformations.
Extends linear structural causal models to include deterministic relations and latent confounders for causal discovery.
problem Causal discovery in linear SCMs with deterministic relations and latent confounders.
method Extended existing results to include deterministic relations and latent confounders, derived necessary and sufficient conditions for unique identifiability, proposed an algorithm for recovery.
result First work on identifiability results for causal discovery under latent confounding and deterministic relationships.
Improved robust latent variable estimation for neural dynamics.
problem Inconsistent results due to noise and nonlinearity in existing models.
method Probabilistic approach to latent variable estimation in decomposed models.
result More accurate latent variable inference in nonlinear systems with diverse noise conditions.
Proposes a new model to handle latent structure methods.
problem Lack of model inference, generative form, and unidentifiable parameters in latent structure methods.
method Generative Flexible Latent Structure Regression (GFLSR) model structure.
result Proposed model allows for model inference and leads to potential probabilistic model.
The paper reviews identifiability in linear and nonlinear models, from Gaussian to non-Gaussian.
problem Identifiability issues in latent-variable and structural-equation models, especially in nonlinear cases.
method Review of identifiability theory for linear and nonlinear models, including factor analysis and structural equation models.
result Even nonparametric nonlinear models can be estimated with additional assumptions.
In a variety of disciplines such as social sciences, psychology, medicine and economics, the recorded data are considered to be noisy measurements of latent variables connected by some causal structure. This corresponds to a family of graphical models known as the structural equation model with latent variables. While …
In a variety of disciplines such as social sciences, psychology, medicine and economics, the recorded data are considered to be noisy measurements of latent variables connected by some causal structure. This corresponds to a family of graphical models known as the structural equation model with latent variables. While …
We characterize distributional equivalence in latent-variable models with cycles.
problem Lack of an equivalence characterization for latent-variable causal models with cycles.
method Established graphical criterion for distributional equivalence and developed edge rank constraints.
result First equivalence characterization without structural assumptions for latent-variable models with cycles.
Bayesian non-parametric model selects latent dimensions automatically.
problem Non-linear, sparse latent variable selection.
method Indian buffet process prior, random Fourier approximation, MCMC sampling.
result Superior performance on synthetic, biological, and text datasets.
LeJEPA learns latent variables from nonlinear observations.
problem Learning latent variables from nonlinear observations.
method Proves linear identifiability of Gaussian latent distributions.
result Gaussian distribution uniquely guarantees linear identifiability.
This paper tackles causal representation learning with linear and general transformations.
problem Identify and recover latent causal variables and graphs under unknown transformations.
method Score-based algorithms that use gradients of log-density functions for identifiability and achievability.
result Two stochastic hard interventions per node are sufficient for identifiability of general transformations.
Develops a new method for nonlinear dimension reduction using random features.
problem Statistical challenges in generalizing Gaussian process-based latent variable models to non-Gaussian data.
method Random feature latent variable models (RFLVMs) that approximate nonlinear relationships with linear functions of random features.
result RFLVMs produce comparable results to state-of-the-art methods on various data types.
Using the linear Gaussian latent variable model as a starting point we relax some of the constraints it imposes by deriving a nonparametric latent feature Gaussian variable model. This model introduces additional discrete latent variables to the original structure. The Bayesian nonparametric nature of this new model al…
Multivariate categorical data occur in many applications of machine learning. One of the main difficulties with these vectors of categorical variables is sparsity. The number of possible observations grows exponentially with vector length, but dataset diversity might be poor in comparison. Recent models have gained sig…
New algorithm speeds up fitting GLLVMs to large datasets.
problem Efficiently fitting GLLVMs to large datasets with thousands of observations.
method Approximate model using penalized quasi-likelihood, then use Newton method and Fisher scoring.
result Significantly faster and more stable than previous methods, enabling fits to larger matrices.
A recent Cell paper [Chang and Tsao, 2017] reports an interesting discovery. For the face stimuli generated by a pre-trained active appearance model (AAM), the responses of neurons in the areas of the primate brain that are responsible for face recognition exhibit strong linear relationship with the shape variables and…
A body of recent work in modeling neural activity focuses on recovering low-dimensional latent features that capture the statistical structure of large-scale neural populations. Most such approaches have focused on linear generative models, where inference is computationally tractable. Here, we propose fLDS, a general …
Bayesian neural networks improve uncertainty quantification in non-linear dimensionality reduction.
problem Current neural network models lack adequate uncertainty quantification.
method Deploy Markov chain Monte Carlo sampling algorithms for Bayesian inference in ANN models with latent variables.
result New research directions are needed due to fundamental challenges in neural networks with latent variables.
New algorithm identifies causal effects in latent confounding models.
problem Identifying causal effects in linear non-Gaussian models with latent confounding.
method Recursive algorithm using rank conditions on higher-order cumulants.
result Algorithm achieves comparable performance to overcomplete ICA without knowing the number of latent variables.
Study identifies latent variables and models from spacecraft data.
problem Learning reliable models from spacecraft data with complex relationships.
method Inductive bias inspired by controllable canonical forms for sparse, input-dependent latent variables.
result Identifies latent variables up to scaling and determines dynamic models up to transformations for linear and affine systems.
This paper introduces a general Bayesian non- parametric latent feature model suitable to per- form automatic exploratory analysis of heterogeneous datasets, where the attributes describing each object can be either discrete, continuous or mixed variables. The proposed model presents several important properties. First…
Model criticism is usually carried out by assessing if replicated data generated under the fitted model looks similar to the observed data, see e.g. Gelman, Carlin, Stern, and Rubin [2004, p. 165]. This paper presents a method for latent variable models by pulling back the data into the space of latent variables, and c…
This work explains how linear representations in large language models arise from training objectives and gradient descent.
problem Understanding the origins of linear representations in large language models.
method A latent variable model to abstract and formalize concept dynamics, combined with analysis of the softmax cross-entropy objective and gradient descent.
result Linear representations emerge when learning from data matching the latent variable model, and this simple structure suffices to yield linear representations.
Proposes a new condition to estimate latent variable causal graphs from observed data.
problem Estimating causal structures when observed variables are not the underlying causal variables.
method Introduces Generalized Independent Noise (GIN) condition and a recursive learning algorithm.
result Shows that GIN helps locate latent variables and identify their causal structure.
Study identifies latent variables and causal relationships from multiple environments.
problem Identify latent variables and causal relationships from multiple environments.
method Proposes algorithm LiNGCReL for identifying causal graph up to surrounded-node ambiguity.
result Identifies latent variables up to surrounded-node ambiguity (SNA) in linear causal models.
The Trek Separation Theorem (Sullivant et al. 2010) states necessary and sufficient conditions for a linear directed acyclic graphical model to entail for all possible values of its linear coefficients that the rank of various sub-matrices of the covariance matrix is less than or equal to n, for any given n. In this pa…
We consider the problem of extracting a low-dimensional, linear latent variable structure from high-dimensional random variables. Specifically, we show that under mild conditions and when this structure manifests itself as a linear space that spans the conditional means, it is possible to consistently recover the struc…
Context-aware recommender systems (CARS) have gained increasing attention due to their ability to utilize contextual information. Compared to traditional recommender systems, CARS are, in general, able to generate more accurate recommendations. Latent factors approach accounts for a large proportion of CARS. Recently, …
We introduce the Locally Linear Latent Variable Model (LL-LVM), a probabilistic model for non-linear manifold discovery that describes a joint distribution over observations, their manifold coordinates and locally linear maps conditioned on a set of neighbourhood relationships. The model allows straightforward variatio…
A model learns causal representations from high-dimensional data.
problem Challenges in learning causal representations from high-dimensional data.
method Formulated a latent variable decoder model, Decoder BCD, for Bayesian causal discovery.
result Shows that using known intervention targets as labels helps in unsupervised Bayesian inference over structure and parameters.
Latent variable time-series models are among the most heavily used tools from machine learning and applied statistics. These models have the advantage of learning latent structure both from noisy observations and from the temporal ordering in the data, where it is assumed that meaningful correlation structure exists ac…
In nonlinear latent variable models or dynamic models, if we consider the latent variables as confounders (common causes), the noise dependencies imply further relations between the observed variables. Such models are then closely related to causal discovery in the presence of nonlinear confounders, which is a challeng…
Bayesian deep learning faces posterior collapse due to likelihood vs. prior competition.
problem Posterior collapse in Bayesian deep learning models.
method Identified competition between likelihood and prior regularization in a linear latent variable model.
result Posterior collapse is related to neural and dimensional collapse, suggesting a broader learning issue.
We consider the problem of learning causal models from observational data generated by linear non-Gaussian acyclic causal models with latent variables. Without considering the effect of latent variables, one usually infers wrong causal relationships among the observed variables. Under faithfulness assumption, we propos…
Causal discovery from data affected by latent confounders is an important and difficult challenge. Causal functional model-based approaches have not been used to present variables whose relationships are affected by latent confounders, while some constraint-based methods can present them. This paper proposes a causal f…
Paper recovers latent causal structure and linear transformation from indirect observations.
problem Recovering latent causal structure and linear transformation from indirect observations.
method Established sufficient conditions for DAG recovery, leveraged score function properties, and used soft/hard interventions.
result Perfect recovery of latent DAG structure and linear transformation up to scaling using soft interventions, hard interventions with additional hypothesis testing.