WWAE improves deep generative models by minimizing penalized optimal transport.
problem Challenges in learning deep generative models, such as blurriness in variational auto-encoders and instability in generative adversarial networks.
method WWAE formulates as minimization of penalized optimal transport between target and generated distributions. It uses closed-form squared Wasserstein-2 distance for Gaussians.
result WWAE learns better latent structures and generates higher quality samples than VAEs and GANs.
The paper identifies saddlepoints in unsupervised auto-encoding neural nets.
problem The risk landscape of unsupervised least squares in auto-encoding neural nets.
method Established an equivalence between unsupervised least squares and principal manifolds, discussed regularization strategies for auto-encoders.
result All non-trivial critical points in auto-encoding are saddlepoints, which are degenerate in overcomplete auto-encoding.
Two graph auto-encoders decouple feature propagation from graph convolution layers.
problem Designing efficient graph auto-encoders with fixed receptive fields.
method L-GAE and L-VGAE using linear matrix computation before auto-encoder input.
result Comparable performance to VGAEs with smaller, simpler networks.
New auto-encoder handles varying noise levels without retraining.
problem Auto-encoders degrade in noisy conditions.
method Formalized auto-encoders as transform learning, derived new architecture.
result Models generalize well to different noise levels.
A new method uses persistent homology to assess auto-encoders' latent manifold quality.
problem Chaos in auto-encoders' latent manifold and failure of current distance measures.
method Persistent Homology for Wasserstein Auto-Encoders (PHom-WAE).
result PHom-WAE improves auto-encoders' performance in credit card transaction data.
A novel cross-modal auto-encoder associates different data types efficiently.
problem Cross-modal data association in heterogeneous datasets.
method Bayesian inference framework with variational auto-encoders and associators.
result Successfully associates visual and auditory data with minimal paired data.
DGA and DVGA learn disentangled graph representations to improve graph analysis.
problem Holistic graph auto-encoders fail to capture latent factors effectively.
method Design disentangled graph convolutional network and component-wise flow, impose independence constraints.
result Improved disentangled graph representations enhance graph analysis tasks.
A new triad decoder improves graph auto-encoders' performance.
problem Graph auto-encoders ignore edge interactions, leading to suboptimal predictions.
method Integrates triadic closure property to predict three edges in a local triad.
result Triad decoder leads to more accurate predictions, clustering, and graph characteristics preservation.
Recent work suggests that some auto-encoder variants do a good job of capturing the local manifold structure of the unknown data generating density. This paper contributes to the mathematical understanding of this phenomenon and helps define better justified sampling algorithms for deep learning based on auto-encoder v…
Graph attention auto-encoder reconstructs graph structure and attributes.
problem Lack of methods to reconstruct graph structure and node attributes in graph auto-encoders.
method Stacked encoder/decoder layers with self-attention mechanisms, regularized node representations to reconstruct graph structure.
result Competitive performance on node classification benchmarks, including inductive learning.
What do auto-encoders learn about the underlying data generating distribution? Recent work suggests that some auto-encoder variants do a good job of capturing the local manifold structure of data. This paper clarifies some of these previous observations by showing that minimizing a particular form of regularized recons…
Fisher auto-encoders use Fisher divergence for more robust generative modeling.
problem Model uncertainty in generative models.
method Minimizing Fisher divergence between true and modeled joint distributions.
result Fisher auto-encoders can more accurately quantify model uncertainty.
We propose the Wasserstein Auto-Encoder (WAE)---a new algorithm for building a generative model of the data distribution. WAE minimizes a penalized form of the Wasserstein distance between the model distribution and the target distribution, which leads to a different regularizer than the one used by the Variational Aut…
Paper proposes using auto-encoders to efficiently purify adversarial perturbations.
problem Vulnerability of machine learning models to adversarial examples.
method Iterative adversarial training on an auto-encoder to purify perturbations.
result The auto-encoder trained model outperforms other methods in protecting models against white-box attacks.
Generative source separation methods such as non-negative matrix factorization (NMF) or auto-encoders, rely on the assumption of an output probability density. Generative Adversarial Networks (GANs) can learn data distributions without needing a parametric assumption on the output density. We show on a speech source se…
CVAEs improve VAEs by accounting for correlations in latent representations.
problem VAEs fail to account for correlations between data points, limiting their effectiveness.
method CVAEs incorporate correlation structure into VAEs using a prior and tractable approximations.
result CVAEs outperform baseline algorithms in matching and link prediction tasks.
Paper proposes a novel auto-encoder for latent density estimation.
problem Challenges of learning generative probabilistic models due to curse of dimensionality.
method Joint dimensionality reduction and non-parametric density estimation framework using a novel estimator.
result Proposed model achieves promising results on various datasets.
Develops a new non-adversarial framework for better generative models.
problem Inaccurate approximation of target distribution in latent space.
method Tessellated Wasserstein Auto-Encoders (TWAE) using centroidal Voronoi tessellation (CVT) to tessellate latent space.
result Significantly enhances generative performance in terms of FID compared to existing models.
New distribution on simplex for auto-encoding tasks.
problem Developing a new distribution for sparsity in auto-encoding models.
method Using Kumaraswamy distribution and ordered stick-breaking process.
result The new distribution has an exact and closed form reparameterization.
Principal components analysis (PCA) is the optimal linear auto-encoder of data, and it is often used to construct features. Enforcing sparsity on the principal components can promote better generalization, while improving the interpretability of the features. We study the problem of constructing optimal sparse linear a…
CasVAE outperforms supervised methods for star-galaxy classification.
problem Challenges in machine learning for astronomy data.
method Cascade Variational Auto-Encoder (CasVAE) for unsupervised star-galaxy classification.
result CasVAE outperforms baseline models in accuracy and stability.
Auto-encoding generative adversarial networks (GANs) combine the standard GAN algorithm, which discriminates between real and model-generated data, with a reconstruction loss given by an auto-encoder. Such models aim to prevent mode collapse in the learned generative model by ensuring that it is grounded in all the ava…
SIG-VAE enhances VGAE for graph data modeling.
problem Limited flexibility in VGAE for graph data.
method Hierarchical variational framework with Bernoulli-Poisson link decoder.
result SIG-VAE outperforms state-of-the-art methods on graph tasks.
Paper proposes a new method for learning deep generative models using annealed importance sampling.
problem Learning deep generative models efficiently and accurately.
method Proposes annealed importance sampling as a bridge between variational inference and Markov chain Monte Carlo.
result Demonstrates improved density models and better trade-off between computation and model accuracy.
Adaptive graph auto-encoder improves general data clustering.
problem Extending graph convolution networks to general clustering tasks.
method Adaptive graph construction based on generative perspective, novel decoder design.
result Model performs well in weighted graph scenarios.
We provide theoretical and empirical evidence that using tighter evidence lower bounds (ELBOs) can be detrimental to the process of learning an inference network by reducing the signal-to-noise ratio of the gradient estimator. Our results call into question common implicit assumptions that tighter ELBOs are better vari…
New PAC-Bayesian bounds improve Sliced-Wasserstein distances.
problem Improving statistical properties of Sliced-Wasserstein distances.
method Leveraging PAC-Bayesian theory to provide bounds and learning procedures.
result PAC-Bayesian generalization bounds for adaptive SW distances.
CAE models learn complex manifold structures in data.
problem Flat latent spaces in auto-encoders fail to capture manifold structures.
method Proposes Chart Auto-Encoders (CAE) with a multi-chart latent space.
result CAE provides better data representation with manifold properties.
We introduce the variational graph auto-encoder (VGAE), a framework for unsupervised learning on graph-structured data based on the variational auto-encoder (VAE). This model makes use of latent variables and is capable of learning interpretable latent representations for undirected graphs. We demonstrate this model us…
The paper proposes a new method for online image decomposition using auto-encoders.
problem Building a part-based representation of image datasets for interpretation and online computation.
method Sparse, non-negative auto-encoder with deep encoder and shallow decoder for online computation.
result The method outperforms state-of-the-art online methods on MNIST and Fashion MNIST datasets.
We study the role of latent space dimensionality in Wasserstein auto-encoders (WAEs). Through experimentation on synthetic and real datasets, we argue that random encoders should be preferred over deterministic encoders. We highlight the potential of WAEs for representation learning with promising results on a benchmar…
Proposes an auto-encoder for Gaussian distributions using Lie group theory.
problem Generative models for Gaussian distributions.
method Lie group auto-encoder with UTDATs, incorporating geometric properties.
result Eliminates matrix exponential operator and derives intrinsic loss.
Improved reliability of machine learning predictions using variational auto-encoders.
problem Individual unreliability of machine learning models.
method Modified variational auto-encoders to identify a low-dimensional space for reliable classification.
result Improved reliability of predictions and robust identification of adversarial samples.
Self-supervised VAEs improve data compression and generation.
problem Efficient data compression and generation.
method Introducing self-supervised Variational Auto-Encoders with deterministic and discrete variational posteriors.
result Self-supervised VAEs simplify the objective function and improve data reconstruction.
Auto-Encoders are unsupervised models that aim to learn patterns from observed data by minimizing a reconstruction cost. The useful representations learned are often found to be sparse and distributed. On the other hand, compressed sensing and sparse coding assume a data generating process, where the observed data is g…
AER combines auto-encoder and LSTM for better time series anomaly detection.
problem Anomaly detection in time series data with limited labeled data and ambiguous definitions.
method AER (Auto-Encoder with Regression) integrates auto-encoder and LSTM for joint predictions and reconstructions.
result AER achieves the highest F1 score across 12 datasets with comparable runtime.
AutoDiff combines auto-encoder and diffusion model for realistic tabular data synthesis.
problem Generating realistic synthetic tabular data with heterogeneous features.
method Employing auto-encoder architecture to handle tabular data's complexity.
result Synthetic tables from AutoDiff show good statistical fidelity and perform well in machine learning tasks.
Unbiased gradient estimation improves VAE performance.
problem Training VAEs via maximum likelihood is difficult due to intractable integrals.
method Introduced unbiased estimators of the log-likelihood gradient using coupled Markov chains.
result Unbiased estimators lead to better predictive performance in VAEs.
FSPool improves set prediction accuracy and convergence.
problem Set prediction models struggle with simple datasets due to the responsibility problem.
method Featurewise sort pooling to construct a permutation-equivariant auto-encoder.
result FSPool improves reconstructions and representations on various datasets.
Graph auto-encoder predicts unobserved node features from biological networks and omics data.
problem Integrating biological networks and continuous node features for better prediction.
method Graph neural networks and feature auto-encoders trained on feature reconstruction.
result Graph feature auto-encoder outperforms auto-encoders trained on graph reconstruction for predicting unobserved node features.
ACVAEs improve on CVAEs by learning more flexible latent correlations.
problem Learning latent representations with correlated structure.
method Adaptive prior distribution and belief propagation.
result ACVAEs outperform CVAEs in link prediction and hierarchical clustering.
Poincaré VAEs improve hierarchical data representation.
problem Hierarchical data structures are difficult to represent in Euclidean latent spaces.
method Introducing Poincaré ball model of hyperbolic geometry as a latent space for VAEs.
result Better generalization and hierarchical structure recovery in hyperbolic space.
This research tackles automatic paraphrasing without translation.
problem Automatic paraphrasing without translation.
method Proposes a residual variant of vector-quantized variational auto-encoder trained on an unlabeled monolingual corpus.
result Monolingual paraphrasing outperforms unsupervised translation in all settings.
Convolutional auto-encoders learn natural exponential family distributions.
problem Learning latent variables for non-Gaussian data.
method Inspired by CDL, combines CNN and deep learning for natural exponential family data.
result Improved unsupervised and competitive supervised performance.
Paper explores combining auto-encoder representations to fool adversarial discriminators.
problem Combining auto-encoder representations to fool adversarial discriminators.
method Mixing function to produce interpolations of hidden states or masked combinations of latent representations.
result Shows that mixing function can produce interpolations consistent with a conditioned class label.
Quantum Earth Mover's distance improves stability and efficiency in quantum learning.
problem Quantum learning's loss landscapes often lead to poor local minima and gradients.
method Introduced the quantum Earth Mover's (EM) distance and proposed a quantum Wasserstein generative adversarial network (qWGAN).
result The quantum EM distance makes quantum learning more stable and efficient.
Given a convolutional dictionary underlying a set of observed signals, can a carefully designed auto-encoder recover the dictionary in the presence of noise? We introduce an auto-encoder architecture, termed constrained recurrent sparse auto-encoder (CRsAE), that answers this question in the affirmative. Given an input…
In this work, we propose an end-to-end block-based auto-encoder system for image compression. We introduce novel contributions to neural-network based image compression, mainly in achieving binarization simulation, variable bit rates with multiple networks, entropy-friendly representations, inference-stage code optimiz…