This paper quantifies how varying β affects disentanglement in variational autoencoders.
problem The challenge is to quantify the effects of β on disentanglement in variational autoencoders. method The approach involves training multiple variational autoencoders with the same β value and analyzing the disentanglement measures. result There is consistent variance in one disentanglement measure, indicating negative effects on discriminative ability.
This work creates a deep autoencoding model to interpret graph parameters.
problem Matching observed graph topologies with generative procedures and parameters is challenging.
method Developed a disentanglement-focused Beta-Variational Autoencoder (Beta-VAE) model.
result The model learns disentangled latent variables that represent graph parameters.
Improved reconstruction performance in disentanglement challenge.
problem Learning disentangled representations from real-world data.
method Adopted FactorVAE and improved reconstruction performance.
result Achieved 1st place in the disentanglement challenge.
Disentangled representations naturally emerge in multi-task learning.
problem Finding adaptable representations for multiple tasks.
method Empirical study of neural networks trained on automatically generated supervised tasks.
result Disentanglement naturally occurs during multi-task learning.
Study evaluates unsupervised disentanglement methods on a toy dataset.
problem Lack of clear disentanglement metrics capturing independent features.
method Empirical evaluation of six unsupervised disentanglement methods on MPI3D dataset.
result Beta-TCVAE outperforms other methods in metrics, but not in disentanglement quality.
Improves disentangled GAN training and selection without labeled data.
problem Challenges in training disentangled GANs, especially self-supervision.
method Contrastive Regularizer and ModelCentrality for unsupervised disentanglement.
result Significantly improved disentanglement scores without labeled data.
A new VAE model learns disentangled latent factors without supervision.
problem Learning disentangled latent representations in unsupervised settings.
method Relevance-Factor-VAE model using total correlation and relevance indicators.
result Demonstrates superior disentanglement performance across multiple datasets.
New insights explain why β-VAEs fail at disentanglement.
problem Disentanglement performance of β-VAEs peaks at intermediate β and collapses as regularization increases. method Formalized information-theoretic mechanism, introduced λβ-VAE to stabilize disentanglement. result Strong regularization pressure leads to mutual information collapse in β-VAEs. New method identifies latent variables with causal dependencies from observed data.
problem Identify latent variables with causal relationships from observed data.
method Linear causal disentanglement via higher-order cumulants, with perfect and soft interventions.
result Recovery of parameters via coupled tensor decomposition and polynomial equations.
WeLa-VAE learns interpretable disentangled representations with weak supervision.
problem Learning disentangled representations without strong supervision.
method Variational inference framework with shared latent variables and modified variational lower bound.
result WeLa-VAE learns alternative disentangled representations (polar) from weak labels (distance and angle) without refined supervision.
Our method enforces latent independence in deep generative models using a hyperprior.
problem Unsupervised disentanglement of latent representations in deep models.
method We augment VAE with an inverse-Wishart prior to enforce statistical independence in latent dimensions.
result Our approach outperforms state-of-the-art methods in disentanglement and reconstruction.
This paper tackles multi-modal label disentanglement in partition-based XMC.
problem Existing partition-based XMC methods create mutually exclusive clusters, which is sub-optimal for multi-modal labels.
method Formulates label assignment as an optimization problem to maximize precision rates, creating flexible and overlapped label clusters.
result Successfully disentangles multi-modal labels, leading to state-of-the-art results on XMC benchmarks.
A new method for VAEs improves latent space disentanglement without violating probability laws.
problem Improving latent space disentanglement in VAEs without violating probability laws.
method Developed a Renyi VAE with a conditional distribution not learned, using Singular Value Decomposition for evaluation.
result Improved latent space disentanglement without violating probability laws.
Improved sample complexity for Gaussian process approximations.
problem Efficiently approximating Gaussian processes with sparse spectrum.
method Improved sample complexity analysis and auto-encoding algorithm.
result Gaussian process predictions and model evidence can be well-approximated with low sample complexity.
Pruning neural networks reduces parameters without sacrificing interpretability.
problem Reducing unnecessary structure in neural networks to improve efficiency.
method Examined the effect of pruning on the number of hidden units learning disentangled representations.
result Pruning does not harm interpretability until a significant portion of parameters are removed.
Variational method learns disentangled latent factors from unlabeled data.
problem Learning disentangled latent factors from unlabeled data.
method Variational inference with a disentanglement regularizer and metric.
result Significant improvement in disentanglement and data likelihood.
New metric proposed to accurately measure disentanglement.
problem Lack of theoretical guarantees and consistent correlation for existing disentanglement metrics.
method Analysis and creation of a new metric.
result Proposed metric satisfies two desirable properties for disentanglement.
FAVAE learns disentangled representations from sequential data.
problem Learning disentangled and interpretable representations from sequential data.
method FAVAE uses the information bottleneck principle without supervision.
result FAVAE can disentangle multiple dynamic factors.
Discovering transformations for disentangled representations without prior knowledge of the underlying Lie group.
problem Discovering nonlinear transformations for disentangled representations without prior knowledge of the underlying Lie group.
method Approximating target vectors as matrix-vector products of the form \(\boldsymbol{\widetilde{y}}_i = \boldsymbol{\varphi}(t_i) \boldsymbol{x}_i\) where \(\boldsymbol{\varphi}(t_i)\) belongs to a one-parameter subgroup of \(\mathrm{GL}_n (\mathbb{R})\).
result Learning a Lie algebra from unlabeled data pairs without explicit group definition.
A new method, FactorVAE, learns disentangled representations from independent factors.
problem Unsupervised learning of disentangled representations from independent factors.
method FactorVAE encourages factorial distribution of representations to be independent across dimensions.
result FactorVAE improves disentanglement over β-VAE by better balancing disentanglement and reconstruction quality. Unified method for disentangling classes and content improves AI representation learning.
problem Current methods struggle with disentangling class and content variations.
method LORD: Latent Optimization for Representation Disentanglement with asymmetric noise regularization.
result LORD achieves superior disentanglement performance compared to existing methods.
Improved disentanglement in VAEs using aggregated feature maps.
problem Improving disentanglement in Variational Autoencoders (VAEs).
method Regionally aggregated feature maps extracted from pre-trained CNNs on ImageNet.
result 2nd place in NeurIPS 2019 disentanglement challenge.
New technique learns causally disentangled representations for better generation.
problem Learning disentangled representations for accurate generation.
method Causally Disentangled Generation (CDG) approach with supervised regularization.
result CDG is necessary and sufficient for accurate disentangled generation.
Improved disentanglement through learned feature aggregation.
problem Disentangling latent factors in images.
method Variational autoencoder trained on regionally aggregated feature maps from ImageNet.
result 2nd place in NeurIPS 2019 disentanglement challenge.
We analyze disentangled representations under a causal generative process, proposing new metrics and datasets.
problem Addressing fairness and interpretability through disentangled representations with a causal perspective.
method Work under a causal generative process, proposing new metrics and datasets to study disentanglement.
result Proposed metrics capture the desiderata of disentangled causal process.
New framework for disentangling graph node and edge features.
problem Learning disentangled representations for attributed graphs with node and edge features.
method Proposes a novel variational objective and architecture for node and edge deconvolutions to disentangle latent factors.
result Demonstrates effectiveness of the proposed model and its extensions on synthetic and real-world datasets.
A new method for unsupervised disentanglement in GANs.
problem Learning disentangled representations in generative models.
method Regularizing GANs by aligning Jacobian vectors with coordinate axes.
result Unsupervised disentanglement achieved in GANs through spectral regularization.
UDR selects disentangled models without labels.
problem Unsupervised disentangled model selection.
method UDR leverages variational autoencoder disentanglement theory to rank models.
result UDR performs comparably to supervised methods and correlates with task performance.
New framework learns disentangled causal representations from observed labels.
problem Learning meaningful disentangled causal representations from observed data.
method ICM-VAE framework using flow-based diffeomorphic functions and causal disentanglement prior.
result Induces highly disentangled causal factors and improves robustness.
Proposes a new method for disentangling data representations using topological analysis.
problem Learning disentangled representations for better model explainability and robustness.
method Integrates a multi-scale topological loss term into the training of deep learning models.
result Improves disentanglement scores compared to state-of-the-art methods.
We improve a graph generation model to accurately recover Barabási-Albert graph parameters.
problem Recover Barabási-Albert graph parameters from graph data.
method Use a disentanglement-focused deep autoencoding framework with a sequential LSTM decoder trained on graph data.
result Successfully recover Barabási-Albert graph parameters.
A new method for disentangling action sequences improves model stability.
problem Challenges in unsupervised disentanglement learning due to incomplete theories and abstract notions.
method Introducing disentangling action sequences and a novel fractional variational autoencoder (FVAE) framework.
result FVAE improves the stability of disentanglement for action sequences.
GCAE uses density estimation to achieve reliable disentanglement in latent space.
problem Disentangled learning representations suffer from reliability issues.
method GCAE uses Gaussian Channel Autoencoder with Dual Total Correlation (DTC) to avoid the curse of dimensionality.
result GCAE achieves highly competitive and reliable disentanglement scores.
The paper connects disentanglement to manifold charts and commutativity.
problem Discovering local charts of the data manifold for disentanglement.
method Interpreting disentanglement as local charts of the data manifold and studying commutativity.
result Commutativity is a central property in disentanglement, as shown in manifold, group theoretic, and probabilistic frameworks.
FreDN separates trends and periodicities in non-stationary time series forecasts.
problem Spectral entanglement and computational burden in frequency-domain methods for non-stationary time series.
method FreDN introduces a learnable Frequency Disentangler module to separate trend and periodic components directly in the frequency domain, and uses a ReIm Block to reduce complexity.
result FreDN outperforms state-of-the-art methods by up to 10% on long-term forecasting benchmarks.
The paper defines metrics for evaluating disentangled representations in learning models.
problem Evaluating disentangled representations in learning models.
method Defining semantics and metrics for disentanglement learning.
result Proposed metrics correctly characterize representations learned by different methods.
Improved VAE learns disentangled representations with less supervision.
problem Learning disentangled representations is challenging.
method Semi-supervised disentanglement learning with label replacement.
result Significant improvement in disentanglement with minimal supervision.
New metric for disentangling multivariate representations, accounting for more complex entanglements.
problem Current disentanglement metrics fail to detect entanglements involving more than two variables.
method Partial Information Decomposition framework to analyze information sharing and propose a new disentanglement metric.
result The proposed metric correctly identifies entanglements in high-dimensional spaces.
Framework learns disentangled continuous and categorical representations.
problem Learning disentangled representations of continuous and categorical data.
method Variational autoencoder with relaxed discrete distribution, controlling latent units.
result Framework disentangles continuous and categorical factors on various datasets.
This work proposes a model to disentangle image factors effectively and control their manipulation.
problem Controlling disentanglement during image editing while preserving object identity.
method Encoder-decoder architecture with decorrelation regularization and soft target representations.
result The model successfully disentangles image factors and manipulates them effectively.
Paper introduces Wasserstein total correlation for disentangled representation learning.
problem Learning disentangled representations from data.
method Adversarial training of a critic to estimate Wasserstein total correlation in variational and Wasserstein autoencoders.
result Proposed method achieves comparable disentanglement performance with less reconstruction loss.
This paper tackles disentanglement in image editing and reconstruction.
problem Learning disentangled image representations and balancing disentanglement strength and reconstruction quality.
method Distance covariance based decorrelation regularization for disentanglement, soft target representation for reconstruction, and collapsing AE decoder and GAN generator.
result The proposed model improves the disentanglement strength and perceptual quality of generated images.
Improved disentanglement of data factors using recursive training.
problem Current unsupervised disentanglement methods are inconsistent and fail to achieve levels of disentanglement seen in supervised approaches.
method Introduced PBT for VAEs, used UDR for heuristic scoring, and developed recursive rPU-VAE approach.
result Recursive training leads to robust disentanglement of data factors across multiple datasets.
Disentanglement-PyTorch library facilitates disentangled representation learning.
problem Unsupervised learning of disentangled representations.
method Modular library for variational algorithms, decoupling architectures, latent space, and training algorithms.
result Achieved 3rd rank in NeurIPS 2019 Disentanglement Challenge.
Paper proposes InfoAE for disentangled representation learning.
problem Learning disentangled representations from unlabeled data.
method InfoAE learns disentangled representation by maximizing mutual information.
result Achieved 98.9% test accuracy on MNIST with unsupervised training.
New method learns disentangled discrete representations using categorical variational autoencoders.
problem Learning disentangled representations from discrete latent spaces.
method Replaced standard Gaussian VAE with a categorical VAE to mitigate rotational invariance.
result Categorical distributions improve learning of disentangled representations.
A method for disentangling latent variables using weak supervision based on pairwise similarities.
problem Disentangling latent variables without strong supervision.
method Weak supervision through binary or real-valued similarities, applied within a Variational Autoencoder framework.
result Utilizing weak supervision improves disentanglement performance substantially.
New approach to disentangled representations using mutual information.
problem Disentangled representations lack sufficient inductive biases.
method Formulate disentanglement through mutual information and conditional independence.
result Violation of mutual information assumption leads to loss of disentanglement.