The paper identifies saddlepoints in unsupervised auto-encoding neural nets.
problem The risk landscape of unsupervised least squares in auto-encoding neural nets.
method Established an equivalence between unsupervised least squares and principal manifolds, discussed regularization strategies for auto-encoders.
result All non-trivial critical points in auto-encoding are saddlepoints, which are degenerate in overcomplete auto-encoding.
Two graph auto-encoders decouple feature propagation from graph convolution layers.
problem Designing efficient graph auto-encoders with fixed receptive fields.
method L-GAE and L-VGAE using linear matrix computation before auto-encoder input.
result Comparable performance to VGAEs with smaller, simpler networks.
New auto-encoder handles varying noise levels without retraining.
problem Auto-encoders degrade in noisy conditions.
method Formalized auto-encoders as transform learning, derived new architecture.
result Models generalize well to different noise levels.
A novel cross-modal auto-encoder associates different data types efficiently.
problem Cross-modal data association in heterogeneous datasets.
method Bayesian inference framework with variational auto-encoders and associators.
result Successfully associates visual and auditory data with minimal paired data.
DGA and DVGA learn disentangled graph representations to improve graph analysis.
problem Holistic graph auto-encoders fail to capture latent factors effectively.
method Design disentangled graph convolutional network and component-wise flow, impose independence constraints.
result Improved disentangled graph representations enhance graph analysis tasks.
A new triad decoder improves graph auto-encoders' performance.
problem Graph auto-encoders ignore edge interactions, leading to suboptimal predictions.
method Integrates triadic closure property to predict three edges in a local triad.
result Triad decoder leads to more accurate predictions, clustering, and graph characteristics preservation.
Recent work suggests that some auto-encoder variants do a good job of capturing the local manifold structure of the unknown data generating density. This paper contributes to the mathematical understanding of this phenomenon and helps define better justified sampling algorithms for deep learning based on auto-encoder v…
Graph attention auto-encoder reconstructs graph structure and attributes.
problem Lack of methods to reconstruct graph structure and node attributes in graph auto-encoders.
method Stacked encoder/decoder layers with self-attention mechanisms, regularized node representations to reconstruct graph structure.
result Competitive performance on node classification benchmarks, including inductive learning.
What do auto-encoders learn about the underlying data generating distribution? Recent work suggests that some auto-encoder variants do a good job of capturing the local manifold structure of data. This paper clarifies some of these previous observations by showing that minimizing a particular form of regularized recons…
Fisher auto-encoders use Fisher divergence for more robust generative modeling.
problem Model uncertainty in generative models.
method Minimizing Fisher divergence between true and modeled joint distributions.
result Fisher auto-encoders can more accurately quantify model uncertainty.
We propose the Wasserstein Auto-Encoder (WAE)---a new algorithm for building a generative model of the data distribution. WAE minimizes a penalized form of the Wasserstein distance between the model distribution and the target distribution, which leads to a different regularizer than the one used by the Variational Aut…
Auto-encoders are among the most popular neural network architecture for dimension reduction. They are composed of two parts: the encoder which maps the model distribution to a latent manifold and the decoder which maps the latent manifold to a reconstructed distribution. However, auto-encoders are known to provoke cha…
Paper proposes using auto-encoders to efficiently purify adversarial perturbations.
problem Vulnerability of machine learning models to adversarial examples.
method Iterative adversarial training on an auto-encoder to purify perturbations.
result The auto-encoder trained model outperforms other methods in protecting models against white-box attacks.
Generative source separation methods such as non-negative matrix factorization (NMF) or auto-encoders, rely on the assumption of an output probability density. Generative Adversarial Networks (GANs) can learn data distributions without needing a parametric assumption on the output density. We show on a speech source se…
Paper proposes a novel auto-encoder for latent density estimation.
problem Challenges of learning generative probabilistic models due to curse of dimensionality.
method Joint dimensionality reduction and non-parametric density estimation framework using a novel estimator.
result Proposed model achieves promising results on various datasets.
Develops a new non-adversarial framework for better generative models.
problem Inaccurate approximation of target distribution in latent space.
method Tessellated Wasserstein Auto-Encoders (TWAE) using centroidal Voronoi tessellation (CVT) to tessellate latent space.
result Significantly enhances generative performance in terms of FID compared to existing models.
New distribution on simplex for auto-encoding tasks.
problem Developing a new distribution for sparsity in auto-encoding models.
method Using Kumaraswamy distribution and ordered stick-breaking process.
result The new distribution has an exact and closed form reparameterization.
Principal components analysis (PCA) is the optimal linear auto-encoder of data, and it is often used to construct features. Enforcing sparsity on the principal components can promote better generalization, while improving the interpretability of the features. We study the problem of constructing optimal sparse linear a…
CasVAE outperforms supervised methods for star-galaxy classification.
problem Challenges in machine learning for astronomy data.
method Cascade Variational Auto-Encoder (CasVAE) for unsupervised star-galaxy classification.
result CasVAE outperforms baseline models in accuracy and stability.
Auto-encoding generative adversarial networks (GANs) combine the standard GAN algorithm, which discriminates between real and model-generated data, with a reconstruction loss given by an auto-encoder. Such models aim to prevent mode collapse in the learned generative model by ensuring that it is grounded in all the ava…
SIG-VAE enhances VGAE for graph data modeling.
problem Limited flexibility in VGAE for graph data.
method Hierarchical variational framework with Bernoulli-Poisson link decoder.
result SIG-VAE outperforms state-of-the-art methods on graph tasks.
Paper proposes a new method for learning deep generative models using annealed importance sampling.
problem Learning deep generative models efficiently and accurately.
method Proposes annealed importance sampling as a bridge between variational inference and Markov chain Monte Carlo.
result Demonstrates improved density models and better trade-off between computation and model accuracy.
Adaptive graph auto-encoder improves general data clustering.
problem Extending graph convolution networks to general clustering tasks.
method Adaptive graph construction based on generative perspective, novel decoder design.
result Model performs well in weighted graph scenarios.
We provide theoretical and empirical evidence that using tighter evidence lower bounds (ELBOs) can be detrimental to the process of learning an inference network by reducing the signal-to-noise ratio of the gradient estimator. Our results call into question common implicit assumptions that tighter ELBOs are better vari…
CAE models learn complex manifold structures in data.
problem Flat latent spaces in auto-encoders fail to capture manifold structures.
method Proposes Chart Auto-Encoders (CAE) with a multi-chart latent space.
result CAE provides better data representation with manifold properties.
We introduce the variational graph auto-encoder (VGAE), a framework for unsupervised learning on graph-structured data based on the variational auto-encoder (VAE). This model makes use of latent variables and is capable of learning interpretable latent representations for undirected graphs. We demonstrate this model us…
We study the role of latent space dimensionality in Wasserstein auto-encoders (WAEs). Through experimentation on synthetic and real datasets, we argue that random encoders should be preferred over deterministic encoders. We highlight the potential of WAEs for representation learning with promising results on a benchmar…
Improved reliability of machine learning predictions using variational auto-encoders.
problem Individual unreliability of machine learning models.
method Modified variational auto-encoders to identify a low-dimensional space for reliable classification.
result Improved reliability of predictions and robust identification of adversarial samples.
Self-supervised VAEs improve data compression and generation.
problem Efficient data compression and generation.
method Introducing self-supervised Variational Auto-Encoders with deterministic and discrete variational posteriors.
result Self-supervised VAEs simplify the objective function and improve data reconstruction.
Auto-Encoders are unsupervised models that aim to learn patterns from observed data by minimizing a reconstruction cost. The useful representations learned are often found to be sparse and distributed. On the other hand, compressed sensing and sparse coding assume a data generating process, where the observed data is g…
AER combines auto-encoder and LSTM for better time series anomaly detection.
problem Anomaly detection in time series data with limited labeled data and ambiguous definitions.
method AER (Auto-Encoder with Regression) integrates auto-encoder and LSTM for joint predictions and reconstructions.
result AER achieves the highest F1 score across 12 datasets with comparable runtime.
AutoDiff combines auto-encoder and diffusion model for realistic tabular data synthesis.
problem Generating realistic synthetic tabular data with heterogeneous features.
method Employing auto-encoder architecture to handle tabular data's complexity.
result Synthetic tables from AutoDiff show good statistical fidelity and perform well in machine learning tasks.
Unbiased gradient estimation improves VAE performance.
problem Training VAEs via maximum likelihood is difficult due to intractable integrals.
method Introduced unbiased estimators of the log-likelihood gradient using coupled Markov chains.
result Unbiased estimators lead to better predictive performance in VAEs.
FSPool improves set prediction accuracy and convergence.
problem Set prediction models struggle with simple datasets due to the responsibility problem.
method Featurewise sort pooling to construct a permutation-equivariant auto-encoder.
result FSPool improves reconstructions and representations on various datasets.
Graph auto-encoder predicts unobserved node features from biological networks and omics data.
problem Integrating biological networks and continuous node features for better prediction.
method Graph neural networks and feature auto-encoders trained on feature reconstruction.
result Graph feature auto-encoder outperforms auto-encoders trained on graph reconstruction for predicting unobserved node features.
ACVAEs improve on CVAEs by learning more flexible latent correlations.
problem Learning latent representations with correlated structure.
method Adaptive prior distribution and belief propagation.
result ACVAEs outperform CVAEs in link prediction and hierarchical clustering.
This research tackles automatic paraphrasing without translation.
problem Automatic paraphrasing without translation.
method Proposes a residual variant of vector-quantized variational auto-encoder trained on an unlabeled monolingual corpus.
result Monolingual paraphrasing outperforms unsupervised translation in all settings.
Convolutional auto-encoders learn natural exponential family distributions.
problem Learning latent variables for non-Gaussian data.
method Inspired by CDL, combines CNN and deep learning for natural exponential family data.
result Improved unsupervised and competitive supervised performance.
Given a convolutional dictionary underlying a set of observed signals, can a carefully designed auto-encoder recover the dictionary in the presence of noise? We introduce an auto-encoder architecture, termed constrained recurrent sparse auto-encoder (CRsAE), that answers this question in the affirmative. Given an input…
In this work, we propose an end-to-end block-based auto-encoder system for image compression. We introduce novel contributions to neural-network based image compression, mainly in achieving binarization simulation, variable bit rates with multiple networks, entropy-friendly representations, inference-stage code optimiz…
DCFSC uses a simple auto-encoder for subspace clustering without parameters.
problem Subspace clustering for large-scale high-dimensional datasets.
method Closed-form shallow auto-encoder for data-driven self-expressive layer, no parameters or optimization.
result Significant memory benefits over existing methods on large datasets.
AEVB improves understanding of latent variable models.
problem Training latent variable models efficiently and understanding their limitations.
method Motivates AEVB from EM, emphasizing approximate E-step and M-step.
result AEVB tightens ELBO, improving model training.
Graph auto-encoders predict stock market instability by measuring graph structure changes.
problem Forecasting stock market instability and volatility.
method Use graph auto-encoders to reconstruct graph structure and measure changes.
result Higher GAE reconstruction error correlates with higher volatility.
Variational Auto-Encoders (VAEs) are capable of learning latent representations for high dimensional data. However, due to the i.i.d. assumption, VAEs only optimize the singleton variational distributions and fail to account for the correlations between data points, which might be crucial for learning latent representa…
In this paper, a multi-layer architecture (in a hierarchical fashion) by stacking various Kernel Ridge Regression (KRR) based Auto-Encoder for one-class classification is proposed and is referred as MKOC. MKOC has many layers of Auto-Encoders to project the input features into new feature space and the last layer was r…
The paper evaluates variational auto-encoders using model criticism methods.
problem Evaluating the quality of variational auto-encoders (VAEs).
method Statistical model criticism, focusing on reproducing statistics of unknown data generating processes.
result The proposed framework offers possibilities for model selection beyond intrinsic metrics.
Proposes flexible auto-encoders for varying data dimensions.
problem Fixed latent dimensions limit data flexibility.
method Stochastic bottleneck with weighted dropouts.
result Seamless variable dimensionality reduction with high performance.
New anomaly score for generative models without manifold assumptions.
problem Reconstruction error's theoretical limitations for generative models.
method Defining a new anomaly score compatible with generative models.
result The new score is theoretically sound and practical for auto-encoders.