New method controls posterior collapse in VAEs without network architecture constraints.
problem Posterior collapse in VAEs reduces diversity of generated samples.
method Introduces Latent Reconstruction (LR) loss to control posterior collapse.
result Controls posterior collapse on various datasets without architectural constraints.
Bayesian deep learning faces posterior collapse due to likelihood vs. prior competition.
problem Posterior collapse in Bayesian deep learning models.
method Identified competition between likelihood and prior regularization in a linear latent variable model.
result Posterior collapse is related to neural and dimensional collapse, suggesting a broader learning issue.
Linear VAEs explain posterior collapse in VAEs via local maxima in log marginal likelihood.
problem Posterior collapse in VAEs where variational posterior matches prior for some latent variables.
method Analysis of linear VAEs and their relation to pPCA, proving ELBO does not introduce spurious local maxima.
result Linear VAEs have identifiable global maxima corresponding to principal component directions, explaining posterior collapse.
This work tackles posterior collapse in conditional and hierarchical VAEs.
problem Posterior collapse in VAEs leads to poor latent variable representations.
method Theoretical analysis of linear conditional and hierarchical VAEs, empirical validation.
result Theoretical and empirical evidence of posterior collapse causes in conditional and hierarchical VAEs.
A simple pooling technique prevents posterior collapse in sequence VAEs.
problem Posterior collapse in sequence VAEs, causing models to ignore latent variables.
method Proposed a pooling technique to prevent posterior collapse.
result Significantly better data log-likelihood achieved compared to standard sequence VAEs.
Variational autoencoders often collapse, showing latent variables are non-identifiable.
problem Posterior collapse in variational autoencoders due to non-identifiable latent variables.
method Proves latent variable non-identifiability causes posterior collapse. Proposes latent-identifiable models using Brenier maps and input convex neural networks.
result Latent-identifiable models resolve posterior collapse and provide meaningful representations.
New method controls posterior collapse in VAEs with theoretical guarantees.
problem Posterior collapse in VAEs where encoder ignores latent structure.
method Inverse Lipschitz constraint on decoder network.
result Controls degree of posterior collapse for various VAE models.
Levenshtein VAE prevents posterior collapse in text generation models.
problem Posterior collapse in VAEs where generators ignore latent variables.
method Replaces ELBO with a Levenshtein distance-based objective to prevent collapse.
result Levenshtein VAE produces more informative latent representations.
New research shows posterior collapse in VAEs isn't just about KL-divergence.
problem Posterior collapse in Variational Autoencoders (VAEs).
method Analyzes the loss surface of deep autoencoder networks and proves the existence of bad local minima.
result Posterior collapse in VAEs is caused by bad local minima, not just KL-divergence.
New critics improve VAEs by preventing latent collapse.
problem Posterior collapse in VAEs where latent variables are ignored.
method Inference critics that detect and incentivize meaningful latent representations.
result Optimizing inference critics increases mutual information between latent and observations.
High-dimensional VAEs inevitably collapse to prior, requiring large datasets for good performance.
problem Posterior collapse in VAEs leads to poor representation learning quality.
method Analyzed a minimal VAE in a high-dimensional limit, evaluating conditions for posterior collapse with respect to beta and dataset size.
result VAEs face 'inevitable posterior collapse' beyond a certain beta threshold, regardless of dataset size.
New method prevents posterior collapse in iVAE models.
problem Posterior collapse in iVAE models where observations and ICs are independent given covariates.
method Developed CI-iVAE by considering a mixture of encoder and posterior distributions in the objective function.
result Prevents posterior collapse, resulting in latent representations with more information of the observations.
KL annealing helps VAEs avoid posterior collapse and overfitting.
problem Posterior collapse and overfitting in VAEs.
method Theoretical analysis of learning dynamics with KL annealing.
result Posterior collapse is inevitable when β exceeds a threshold. VAE training can lead to posterior collapse, which this paper addresses.
problem Posterior collapse in VAEs where the model learns to ignore latent variables.
method Investigates the dynamics of VAE training and proposes a simple modification to optimize the inference network.
result The proposed modification avoids posterior collapse and improves model performance.
A new method improves molecule generation accuracy and efficiency.
problem Posterior collapse in VAEs for molecule sequence generation.
method Re-balancing reconstruction loss to avoid posterior collapse.
result Our method achieves state-of-the-art reconstruction accuracy and competitive validity.
New method prevents posterior collapse in generative models.
problem Posterior collapse weakens generative model capacity or requires complex objectives.
method Proposes δ-VAEs that constrain the posterior variational family to a minimum distance from the prior. result Achieves state-of-the-art log-likelihood on CIFAR-10 and ImageNet 32x32.
NE-VAE prevents posterior collapse in VAEs by embedding neighbors in latent space.
problem Posterior collapse in VAEs when strong decoders are used.
method Neighbor embedding in latent space to prevent collapse.
result NE-VAE produces qualitatively different latent representations with active latent dimensions.
New objective function reduces posterior collapse in generative models.
problem Posterior collapse in generative models with small datasets and high latent dimensions.
method Replaces ELBO's KL divergence with MMD, introduces latent clipping.
result μ-VAE outperforms ELBO and β-VAE models in quality and stability.
Semi-amortized approach improves generative models without posterior-collapse.
problem Posterior-collapse in VAEs for tasks like text generation.
method Hybrid approach combining AVI for initialization and SVI for refinement.
result Semi-amortized approach outperforms autoregressive and variational baselines.
Proposes NC-VAE to prevent posterior collapse in VAEs.
problem Posterior collapse in variational autoencoders (VAEs).
method Noise contrastive estimation applied to VAEs.
result Proves NC-VAE cannot reach posterior collapse and provides lower bounds.
Proposes a new VAE model to avoid posterior collapse by modeling latent variable dependencies.
problem Posterior collapse in variational autoencoders due to assumption of factorized variational posterior.
method Introduces Gaussian Copula Variational Autoencoder (GCVAE) to model latent variable dependencies explicitly.
result Empirical results show GCVAE can avoid posterior collapse while maintaining competitive performance.
A simple fix improves VAE performance without improving ELBO.
problem Posterior collapse in VAEs leads to unstable training and poor performance.
method Combining two heuristics: KL Divergence weighting and Data Augmentation.
result Significantly improves held-out likelihood, reconstruction, and latent representation learning.
SentenceMIM learns rich latent representations for variable-length language data.
problem Challenges in learning VAEs for variable-length language data, especially posterior collapse.
method Probabilistic auto-encoder trained with Mutual Information Machine (MIM) learning.
result SentenceMIM learns informative latent representations with high mutual information.
Advances deep latent variable models for more flexible text generation.
problem Limited representation power of VAEs due to Gaussian assumptions and posterior collapse.
method Develops sample-based variational distributions and an LVM to directly match aggregated posterior to prior.
result Demonstrates improved text generation in various scenarios.
Improved VAE models avoid posterior collapse in text modeling.
problem Posterior collapse in VAEs leads to poor data manifold parameterization.
method Coupled-VAE couples a VAE with a deterministic autoencoder to improve encoder and decoder parameterizations.
result Coupled-VAE consistently improves results in probability estimation and latent space richness.
A new model tackles language generation issues by using discrete variational attention.
problem Information under-representation and posterior collapse in variational autoencoders.
method Proposes a discrete variational attention model with categorical distribution over attention mechanism.
result Enhances latent space for language generation and avoids posterior collapse.
VAE framework improved with generated loss and augmented training.
problem Posterior collapse and low-precision reconstructions in VAEs.
method Introduced generated loss as a proxy metric and augmented training with generated variants.
result Repeating encoding and decoding produces more typical examples from VAEs.
Improves VAE by adding a softmax classifier to enhance mutual information.
problem Posterior collapse and blurred reconstructions in VAE.
method Integrates an auxiliary softmax multi-classifier to improve VAE training.
result VAE-AS significantly improves mutual information and solves posterior collapse.
This paper analyzes VAE approximation errors in conditional exponential families.
problem Posterior collapse and approximation errors in VAEs.
method Analysis of ELBO objective and conditional exponential families.
result The ELBO optimizer pulls away from the likelihood optimizer towards a consistent subset of models.
Improved co-clustering for robust data analysis.
problem Noise and posterior collapse in co-clustering.
method Fully variational Co-clustering framework with Compositional ELBO.
result Enhanced robustness and accuracy in co-clustering.
MusicVAE uses a hierarchical decoder to model long-term structure in music sequences.
problem Difficulty of existing recurrent VAE models in modeling long-term structure in sequential data.
method Proposes a hierarchical decoder that outputs embeddings for subsequences and uses these embeddings to generate each subsequence independently.
result Demonstrates better sampling, interpolation, and reconstruction performance than a flat baseline model.
Temporal VAE improves VaR estimation for financial portfolios.
problem Estimating VaR for large asset portfolios in finance.
method Temporal VAE with annealing regularization to avoid posterior collapse.
result Temporal VAE outperforms classical VaR estimation methods on real data.
CF-VAE models capture multi-modal distributions for better structured sequence prediction.
problem Challenges in capturing multi-modality of future states in latent variable models.
method Conditional Flow Variational Autoencoders (CF-VAE) with conditional normalizing flows.
result CF-VAE achieves state-of-the-art results on multi-modal structured sequence prediction datasets.
A framework connects VAEs to GLMs for better model initialization and performance.
problem Understanding and optimizing loss function critical points in VAEs.
method Introducing a theoretical framework based on GLM and EDFs.
result Maximum likelihood initialization improves VAE performance.
This paper reviews recent advancements in amortized Variational Inference.
problem Scalability and efficiency issues in traditional Variational Inference.
method Systematic review of various Variational Inference techniques, focusing on amortized approaches.
result Amortized Variational Inference improves scalability and efficiency for generative modeling tasks.
A new framework CyGen models joint distributions using cyclic conditionals.
problem Modeling a joint distribution using only two conditional models without relying on an uninformative prior.
method Developed a general theory for operable equivalence criteria for compatibility and sufficient conditions for determinacy. Proposed CyGen framework and methods to achieve compatibility and determinacy.
result CyGen better fits data and captures more representative features compared to models using an uninformative prior.
A new EM-based algorithm improves deep generative model training.
problem Training deep generative models with maximum likelihood is challenging.
method The paper proposes reweighted expectation maximization (REM), a new algorithm that directly maximizes the log marginal likelihood of the data.
result REM learns better generative models than the IWAE, leading to significantly better performance in density estimation benchmarks.
New method scales inference for deep discrete models to thousands of states.
problem Traditional inference limits model capacity for deep discrete models.
method Randomized Dynamic Programming (RDP) algorithms for scaling inference.
result RDP achieves low bias and variance while reducing computation.
HEBAE improves VAEs by adaptively balancing reconstruction and regularization.
problem Posterior collapse in VAEs leading to over-regularization and poor latent encoding.
method Hierarchical Empirical Bayes approach to probabilistic generative models.
result HEBAE generates higher quality samples with better FID scores.
Paper uses VAEs to detect genuine signatures.
problem Signature recognition performance plateaued at 2% error rate.
method Uses VAEs to learn latent space, then classifies unlabelled signatures.
result Method performs less well than existing alternatives but shows potential.
UT module refines VAE latent space, improving disentanglement and interpretability.
problem Irregular latent distributions cause posterior collapse and misalignment in VAEs.
method UT module uses G-KDE clustering, GM modeling, and PIT to transform latent space into uniform distribution.
result UT module enhances disentanglement and interpretability of latent representations.
MIM learns useful representations with high mutual information.
problem Learning useful representations for downstream tasks.
method Symmetric Jensen-Shannon divergence and mutual information regularizer in an encoder/decoder framework.
result MIM learns high mutual information representations without posterior collapse.
MIM learns joint distributions with mutual information and low divergence.
problem Learning joint distributions over observations and latent variables.
method Probabilistic auto-encoder with three design principles: low divergence, high mutual information, and low marginal entropy.
result MIM learns representations with high mutual information, consistent encoding and decoding distributions, effective latent clustering, and comparable data log likelihood to VAE.
This paper introduces a quantization-based regularizer for autoencoders to improve latent representations.
problem Autoencoders can overfit and collapse, leading to poor latent representations.
method The authors combine VQ-VAE and denoising methods to introduce a bottleneck Bayesian estimator that soft quantizes latent codes.
result The method results in better latent representations for supervised and clustering tasks.
Improved VAE model for discrete data through augmented training and multiscale approach.
problem Limited capability of VAE in capturing field correlations in structured data.
method Augmented training with generated variants and multiscale VAE with multiple β values.
result Improved generation quality of VAE models through these methods.
Combines diffusion maps and VAEs for scalable generative models.
problem Choosing a suitable prior for VAEs and GANs to match data topology.
method Variational Diffusion Autoencoders with Random Walk Sampling.
result Proves approximation theoretic results for dimension dependence.
Framework estimates multiple plausible solutions with uncertainty measures.
problem Machine learning models need to propose multiple plausible solutions with meaningful uncertainty.
method Discrete latent variables model one-to-many mappings, allowing effective conditional probability estimation.
result Framework outperforms state-of-the-art in uncertainty estimation and is practical.
A new method uses optimal transport to transform latent space distributions without solving a hard Min-Max problem.
problem Latent space data distribution collapse and loss of manifold structure in generative models.
method Proposes a GAN-like method to solve a minimization problem for optimal transport between a simple distribution and a latent-space data distribution.
result Experimental results show that the proposed method can handle multi-cluster distributions and is effective on MNIST and CelebA datasets.