Variational autoencoders provide a principled framework for learning deep latent-variable models and corresponding inference models. In this work, we provide an introduction to variational autoencoders and some important extensions.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Variational autoencoder”
A standard Variational Autoencoder, with a Euclidean latent space, is structurally incapable of capturing topological properties of certain datasets. To remove topological obstructions, we introduce Diffusion Variational Autoencoders with arbitrary manifolds as a latent space. A Diffusion Variational Autoencoder uses t…
We extend Stochastic Gradient Variational Bayes to perform posterior inference for the weights of Stick-Breaking processes. This development allows us to define a Stick-Breaking Variational Autoencoder (SB-VAE), a Bayesian nonparametric version of the variational autoencoder that has a latent representation with stocha…
Variational autoencoders often collapse, showing latent variables are non-identifiable.
problem Posterior collapse in variational autoencoders due to non-identifiable latent variables.
method Proves latent variable non-identifiability causes posterior collapse. Proposes latent-identifiable models using Brenier maps and input convex neural networks.
result Latent-identifiable models resolve posterior collapse and provide meaningful representations.
DAEs can generate images without additional loss terms, inheriting VAE properties.
problem Difficulty in using VAEs for practical generative modelling.
method Empirical exploration of DAEs for image generation without novel methods.
result DAEs can generate images successfully without additional loss terms.
Learning in the latent variable model is challenging in the presence of the complex data structure or the intractable latent variable. Previous variational autoencoders can be low effective due to the straightforward encoder-decoder structure. In this paper, we propose a variational composite autoencoder to sidestep th…
Study reveals the regularization effect of variational distributions in VAEs.
problem Understanding the regularization role of variational distributions in VAEs.
method Analyzed the role of variational family in VAEs and studied the regularization effect on local geometry.
result Uncovered the implicit regularizer in the -VAE objective and proposed a deterministic autoencoding objective.
Combines variational autoencoders with normalizing flows for faster training.
problem Training normalizing flow models like Glow is slow and requires deep architectures.
method Integrates Glow with a variational autoencoder to speed up training.
result The combined model achieves similar image quality and likelihood to Glow but trains faster.
Paper proposes efficient multivariate spatial Fay-Herriot models using variational autoencoders.
problem Estimating population characteristics in small areas with limited data.
method Integrates multivariate spatial Fay-Herriot model with variational autoencoders to leverage spatial structure efficiently.
result Significant computational efficiency improvements for high-dimensional datasets.
In Bayesian machine learning, the posterior distribution is typically computationally intractable, hence variational inference is often required. In this approach, an evidence lower bound on the log likelihood of data is maximized during training. Variational Autoencoders (VAE) are one important example where variation…
We present two deep generative models based on Variational Autoencoders to improve the accuracy of drug response prediction. Our models, Perturbation Variational Autoencoder and its semi-supervised extension, Drug Response Variational Autoencoder (Dr.VAE), learn latent representation of the underlying gene states befor…
Proposes variational autoencoder for efficient MMSE estimation.
problem Efficient parameterized MMSE estimation for noisy observations.
method Variational autoencoder models data distribution, approximates MMSE.
result Proposed estimator performs well compared to state-of-the-art.
In recent years Variation Autoencoders have become one of the most popular unsupervised learning of complicated distributions.Variational Autoencoder (VAE) provides more efficient reconstructive performance over a traditional autoencoder. Variational auto enocders make better approximaiton than MCMC. The VAE defines a …
Batch norm with entropic regularization turns deterministic autoencoders into generative modelscs.LG
Batch normalization with regularization turns deterministic autoencoders into generative models.
problem Creating generative models from deterministic autoencoders.
method Using batch normalization as a source of non-determinism and adding entropic regularization.
result Deterministic autoencoders can be transformed into generative models with similar performance to variational autoencoders.
A new model for multiview data analysis using graph autoencoders.
problem Nonlinear multiview canonical correlation analysis for large datasets.
method Variational approach with graph convolutional neural networks.
result Competitive performance on classification, clustering, and recommendation tasks.
Variational autoencoders learn unsupervised data representations, but these models frequently converge to minima that fail to preserve meaningful semantic information. For example, variational autoencoders with autoregressive decoders often collapse into autodecoders, where they learn to ignore the encoder input. In th…
PRI-VAE learns disentangled representations by optimizing principle-of-relevant-information.
problem Learning disentangled representations under VAE framework remains unknown.
method Proposes PRI-VAE, a novel learning objective to optimize disentanglement.
result Demonstrates effectiveness of PRI-VAE on four benchmark datasets.
Score matching fails to train VAEs robustly, revealing autoencoding loss insights.
problem Catastrophic failure of variational score matching on VAE models.
method Analysis of existing variational score matching objectives and their equivalence to autoencoding losses.
result Score matching methods fail to produce robust VAE models, predicting poor performance.
We present a variation of the Autoencoder (AE) that explicitly maximizes the mutual information between the input data and the hidden representation. The proposed model, the InfoMax Autoencoder (IMAE), by construction is able to learn a robust representation and good prototypes of the data. IMAE is compared both theore…
Variational Autoencoders are powerful models for unsupervised learning. However deep models with several layers of dependent stochastic variables are difficult to train which limits the improvements obtained using these highly expressive models. We propose a new inference model, the Ladder Variational Autoencoder, that…
New method learns disentangled discrete representations using categorical variational autoencoders.
problem Learning disentangled representations from discrete latent spaces.
method Replaced standard Gaussian VAE with a categorical VAE to mitigate rotational invariance.
result Categorical distributions improve learning of disentangled representations.
Lower bound for VAE training objective for binary data.
problem Finding a lower bound for the ELBO of Bernoulli VAE.
method Interpretable lower bound, modified initialization, faster training architecture, PCA for latent space dimension.
result Theoretical result and improved performance of new architecture.
In this paper, we provide an information-theoretic interpretation of the Vector Quantized-Variational Autoencoder (VQ-VAE). We show that the loss function of the original VQ-VAE can be derived from the variational deterministic information bottleneck (VDIB) principle. On the other hand, the VQ-VAE trained by the Expect…
VANO uses neural operators for unsupervised learning of functional data.
problem Learning operators between infinite dimensional spaces for functional data.
method Variational Autoencoding Neural Operators (VANO) approach.
result VANO can learn and reconstruct functional data without supervision.
The recognition network in deep latent variable models such as variational autoencoders (VAEs) relies on amortized inference for efficient posterior approximation that can scale up to large datasets. However, this technique has also been demonstrated to select suboptimal variational parameters, often resulting in consi…
A scalable factorized Gaussian process VAE for faster inference.
problem Inference bottlenecks in Gaussian process VAEs.
method Factorizes latent kernel across auxiliary features, leveraging independence.
result Significant speed-up in inference time (in theory and practice).
EVGAE improves VGAE's latent representation learning by mitigating over-pruning.
problem Over-pruning in VGAE limits latent variable capacity and diversity.
method EVGAE uses epitomic approach with multiple sparse VGAE models (epitomes) to increase active latent units and improve generative ability.
result EVGAE outperforms VGAE in generative ability and link prediction on citation networks.
Optimizes variational autoencoder for detecting missing data in Mars rover transmissions.
problem Detecting missing data in Mars rover transmissions to prevent volume loss and corruption.
method Applies derivative-free optimization to tune variational autoencoder.
result Improves variational autoencoder's ability to detect missing data, aiding GDSA team.
ED-VAE improves VAEs by explicitly including entropy components in ELBO.
problem Limitations of traditional VAEs with ELBO in generating high-quality samples and interpreting latent spaces.
method Introduces ED-VAE, a re-formulation of ELBO that includes entropy and cross-entropy components.
result Significantly enhances model flexibility and improves interpretability and generative performance.
This work gives an in-depth derivation of the trainable evidence lower bound obtained from the marginal joint log-Likelihood with the goal of training a Multi-Modal Variational Autoencoder (MVAE).
Develops VAEs for learning complex physical systems from data.
problem Learning low-dimensional representations of nonlinear physical systems.
method Variational Autoencoders with manifold latent spaces.
result Effective in learning nonlinear Burgers equation and constrained mechanical systems.
A new model learns latent spaces for graph data.
problem Scalability and expressivity limitations in graph generative models.
method Sequential Graph Variational Autoencoder (SGVAE) that learns latent spaces directly from graph data.
result Promising results on a cycle dataset, but need for permutation relaxation.
Automates VI divergence selection for efficient few-shot learning.
problem Efficiently selecting divergence measures for VI to improve performance.
method Meta-learning algorithm to learn optimal divergence metric and variational parameter initialization.
result Meta-learning approach outperforms standard VI methods across various tasks.
This work improves VAEs using MCMC methods for better variational bounds.
problem Improving the expressiveness of variational distributions in VAEs.
method Entropy-based adaptation for MALA/HMC chains to optimize tighter variational bounds.
result Higher held-out log-likelihoods and improved generative metrics.
Gaussian process variational autoencoders improve disentanglement in time series data.
problem Learning disentangled representations from multivariate time series data.
method Model each latent channel with a Gaussian process prior and a structured variational distribution to capture temporal dependencies.
result Competitive performance on benchmark and real-world medical time series data.
Enhances autoencoders to represent transformations explicitly.
problem Lack of explicit representation of transformations in autoencoders.
method Extended variational autoencoders to include latent transformations, using hierarchical graphical models.
result Inferred latent transformations reflect interpretable properties in the observation space.
Adversarial autoencoder improves music latent space learning.
problem Learning effective latent spaces for symbolic music data.
method Adversarial regularization with Gaussian mixtures.
result MusAE outperforms standard VAEs in reconstruction and interpolation.
This paper introduces Wasserstein variational inference, a new form of approximate Bayesian inference based on optimal transport theory. Wasserstein variational inference uses a new family of divergences that includes both f-divergences and the Wasserstein distance as special cases. The gradients of the Wasserstein var…
VDA improves disentanglement of latent representations in complex signals.
problem Learning disentangled and interpretable representations in nonstationary, high-dimensional time-evolving signals.
method Variational decomposition autoencoding (VDA) framework, incorporating signal decomposition, contrastive self-supervised task, and variational prior approximation.
result DecVAEs surpass state-of-the-art VAE-based methods in disentanglement quality and generalization.
New model improves multimodal autoencoders by learning joint and conditional distributions.
problem Limitations in recent multimodal autoencoders restrict their quality on complex datasets.
method Proposes a multistage training process with variational inference and Normalizing Flows, leveraging shared modality information.
result Achieves state-of-the-art results on benchmark datasets.
Adversarial training improves graph autoencoder generalization.
problem Improving graph autoencoder generalization.
method Formulated L2 and L1 adversarial training for graph autoencoders and variational graph autoencoders.
result Adversarial training boosts graph autoencoder and variational graph autoencoder generalization.
We develop a framework for incorporating structured graphical models in the \emph{encoders} of variational autoencoders (VAEs) that allows us to induce interpretable representations through approximate variational inference. This allows us to both perform reasoning (e.g. classification) under the structural constraints…
New GP-VAE model improves scalability and performance.
problem Inability of conventional VAEs to model correlations between data points.
method Principled sparse inference approaches to improve scalability of GP-VAEs.
result New model outperforms existing approaches in runtime and memory usage.
The paper presents the application of Variational Autoencoders (VAE) for data dimensionality reduction and explorative analysis of mass spectrometry imaging data (MSI). The results confirm that VAEs are capable of detecting the patterns associated with the different tissue sub-types with performance than standard appro…
New method disentangles shared and private latent factors in multimodal data.
problem Challenges in disentangling shared and private latent factors in multimodal data.
method Proposes a modification to existing multimodal Variational Autoencoders (MMVAE) to better handle modality-specific variation.
result Demonstrates improved robustness of modified MMVAE to modality-specific variation.
Develops a new VAE for mixed-curvature latent spaces.
problem Lack of a generic approach for VAEs in curved latent spaces.
method Mixed-curvature Variational Autoencoder (MVAE) that trains on a product of constant curvature Riemannian manifolds.
result Generalizes Euclidean VAE to curved latent spaces and recovers Euclidean VAE when curvatures are zero.
This paper presents an infinite variational autoencoder (VAE) whose capacity adapts to suit the input data. This is achieved using a mixture model where the mixing coefficients are modeled by a Dirichlet process, allowing us to integrate over the coefficients when performing inference. Critically, this then allows us t…
RTVAE uses β-divergence to detect anomalies in tabular data robustly.
problem Outliers in tabular data affect VAE training and anomaly detection.
method Robust VAE with β-divergence for mixed categorical and continuous features.
result Demonstrates effectiveness on network traffic anomaly detection.