CRsAE auto-encoder recovers convolutional dictionary from noisy signals.
problem Recovering a convolutional dictionary from noisy signals.
method Constrained recurrent sparse auto-encoder (CRsAE) architecture.
result CRsAE successfully recovers the underlying dictionary in the presence of noise.
Principal components analysis (PCA) is the optimal linear auto-encoder of data, and it is often used to construct features. Enforcing sparsity on the principal components can promote better generalization, while improving the interpretability of the features. We study the problem of constructing optimal sparse linear a…
The paper proposes a new method for online image decomposition using auto-encoders.
problem Building a part-based representation of image datasets for interpretation and online computation.
method Sparse, non-negative auto-encoder with deep encoder and shallow decoder for online computation.
result The method outperforms state-of-the-art online methods on MNIST and Fashion MNIST datasets.
New auto-encoder handles varying noise levels without retraining.
problem Auto-encoders degrade in noisy conditions.
method Formalized auto-encoders as transform learning, derived new architecture.
result Models generalize well to different noise levels.
Auto-Encoders are unsupervised models that aim to learn patterns from observed data by minimizing a reconstruction cost. The useful representations learned are often found to be sparse and distributed. On the other hand, compressed sensing and sparse coding assume a data generating process, where the observed data is g…
Graph neural network constructs a sparse latent point cloud from dense point clouds.
problem Efficiently reconstructing and simulating point clouds with fine details.
method Irregular graph convolutional neural network with non-isotropic operations.
result The model can reconstruct dense point clouds from a sparse latent representation.
Improved sample complexity for Gaussian process approximations.
problem Efficiently approximating Gaussian processes with sparse spectrum.
method Improved sample complexity analysis and auto-encoding algorithm.
result Gaussian process predictions and model evidence can be well-approximated with low sample complexity.
RandNet learns from compressed image data, improving efficiency and accuracy.
problem Efficiency and accuracy in training neural networks with large datasets.
method RandNet uses compressed random measurements of images to train neural networks efficiently.
result RandNet achieves comparable accuracy to full data training with minimal loss.
New model reconstructs flow from sparse data with uncertainty quantification.
problem Reconstructing nonlinear flow from limited observations.
method Semi-Conditional Variational Autoencoder (SCVAE) for probabilistic flow reconstruction.
result SCVAE improves reconstruction accuracy compared to Gappy Proper Orthogonal Decomposition (GPOD).
Paper proposes VAE-BPTF for better tensor factorization of sparse, imbalanced count data.
problem Inference of Bayesian Poisson-Gamma models for sparse and imbalanced count data is challenging.
method Variational auto-encoder framework with multi-layer perceptron networks for complex update information sharing and reweighting.
result VAE-BPTF outperforms current models in reconstruction errors and latent factor coherence across real-world datasets.
AutoIDS uses auto-encoders to detect network intrusions efficiently.
problem Detecting network intrusions with high accuracy and efficiency.
method Semi-supervised machine learning with cascading encoder-decoder neural networks.
result AutoIDS achieves 90.17% accuracy on NSL-KDD dataset, superior to other methods.
This paper studies an intelligent ultimate technique for health-monitoring and prognostic of common rotary machine components, particularly bearings. During a run-to-failure experiment, rich unsupervised features from vibration sensory data are extracted by a trained sparse auto-encoder. Then, the correlation of the ex…
Proposes flexible auto-encoders for varying data dimensions.
problem Fixed latent dimensions limit data flexibility.
method Stochastic bottleneck with weighted dropouts.
result Seamless variable dimensionality reduction with high performance.
This paper introduces structure learning for autoencoder recommenders to improve performance and generalization.
problem Efficient training and generalization in sparse collaborative filtering data.
method Learn groups of related items and use this information to determine the connectivity structure of an auto-encoding neural network.
result The proposed structure learning method results in a sparse network that converges to a local optimum with smaller spectral norm and generalization error.
Develops MgCSL for discovering causal structures in high-dimensional data.
problem Discovering causal relationships from high-dimensional data with complex interplay of variables.
method MgCSL uses sparse auto-encoders for coarse-graining and multi-layer perceptrons for detailed analysis, introducing simplified acyclicity constraints.
result MgCSL outperforms existing methods and finds explainable causal connections in fMRI datasets.
In this note we present a generative model of natural images consisting of a deep hierarchy of layers of latent random variables, each of which follows a new type of distribution that we call rectified Gaussian. These rectified Gaussian units allow spike-and-slab type sparsity, while retaining the differentiability nec…
Paper proposes a method to learn structure from data similarities.
problem Learning hidden structures in data beyond low-rank and sparsity.
method Explicitly models data relation, retaining pairwise similarities.
result Framework significantly improves performance on clustering and semisupervised classification.
The paper identifies saddlepoints in unsupervised auto-encoding neural nets.
problem The risk landscape of unsupervised least squares in auto-encoding neural nets.
method Established an equivalence between unsupervised least squares and principal manifolds, discussed regularization strategies for auto-encoders.
result All non-trivial critical points in auto-encoding are saddlepoints, which are degenerate in overcomplete auto-encoding.
Unsupervised learning summarizes EHR data into a patient status vector.
problem Challenges in modeling electronic health records due to irregularities and varying procedures/diagnoses.
method Two-step unsupervised representation learning scheme using auto-encoders and forecasting tasks.
result Improved generalization performance on mortality and readmission tasks.
Two graph auto-encoders decouple feature propagation from graph convolution layers.
problem Designing efficient graph auto-encoders with fixed receptive fields.
method L-GAE and L-VGAE using linear matrix computation before auto-encoder input.
result Comparable performance to VGAEs with smaller, simpler networks.
A new method uses persistent homology to assess auto-encoders' latent manifold quality.
problem Chaos in auto-encoders' latent manifold and failure of current distance measures.
method Persistent Homology for Wasserstein Auto-Encoders (PHom-WAE).
result PHom-WAE improves auto-encoders' performance in credit card transaction data.
A novel cross-modal auto-encoder associates different data types efficiently.
problem Cross-modal data association in heterogeneous datasets.
method Bayesian inference framework with variational auto-encoders and associators.
result Successfully associates visual and auditory data with minimal paired data.
DGA and DVGA learn disentangled graph representations to improve graph analysis.
problem Holistic graph auto-encoders fail to capture latent factors effectively.
method Design disentangled graph convolutional network and component-wise flow, impose independence constraints.
result Improved disentangled graph representations enhance graph analysis tasks.
A new triad decoder improves graph auto-encoders' performance.
problem Graph auto-encoders ignore edge interactions, leading to suboptimal predictions.
method Integrates triadic closure property to predict three edges in a local triad.
result Triad decoder leads to more accurate predictions, clustering, and graph characteristics preservation.
Recent work suggests that some auto-encoder variants do a good job of capturing the local manifold structure of the unknown data generating density. This paper contributes to the mathematical understanding of this phenomenon and helps define better justified sampling algorithms for deep learning based on auto-encoder v…
Graph attention auto-encoder reconstructs graph structure and attributes.
problem Lack of methods to reconstruct graph structure and node attributes in graph auto-encoders.
method Stacked encoder/decoder layers with self-attention mechanisms, regularized node representations to reconstruct graph structure.
result Competitive performance on node classification benchmarks, including inductive learning.
What do auto-encoders learn about the underlying data generating distribution? Recent work suggests that some auto-encoder variants do a good job of capturing the local manifold structure of data. This paper clarifies some of these previous observations by showing that minimizing a particular form of regularized recons…
The recently proposed Multi-Layer Convolutional Sparse Coding (ML-CSC) model, consisting of a cascade of convolutional sparse layers, provides a new interpretation of Convolutional Neural Networks (CNNs). Under this framework, the computation of the forward pass in a CNN is equivalent to a pursuit algorithm aiming to e…
Fisher auto-encoders use Fisher divergence for more robust generative modeling.
problem Model uncertainty in generative models.
method Minimizing Fisher divergence between true and modeled joint distributions.
result Fisher auto-encoders can more accurately quantify model uncertainty.
FF layers in transformers are nearly as interpretable as sparse autoencoders.
problem Comparing interpretability of feature vectors in FF layers vs. sparse autoencoders.
method Revisited interpretability of FF layers as key-value memories using modern benchmarks.
result FF and SAE feature vectors are similarly interpretable, but FFs can be better in some aspects.
We propose the Wasserstein Auto-Encoder (WAE)---a new algorithm for building a generative model of the data distribution. WAE minimizes a penalized form of the Wasserstein distance between the model distribution and the target distribution, which leads to a different regularizer than the one used by the Variational Aut…
Paper tackles outlier detection in signals modeled by generative models with theoretical guarantees.
problem Recovering signals from linear measurements with sparse outliers.
method Proposes an iterative ADMM algorithm and gradient descent algorithm for outlier detection using ℓ1 and squared ℓ1 norm minimization. result Establishes theoretical recovery guarantees for signal reconstruction under sparse outliers.
Paper proposes using auto-encoders to efficiently purify adversarial perturbations.
problem Vulnerability of machine learning models to adversarial examples.
method Iterative adversarial training on an auto-encoder to purify perturbations.
result The auto-encoder trained model outperforms other methods in protecting models against white-box attacks.
Generative source separation methods such as non-negative matrix factorization (NMF) or auto-encoders, rely on the assumption of an output probability density. Generative Adversarial Networks (GANs) can learn data distributions without needing a parametric assumption on the output density. We show on a speech source se…
CVAEs improve VAEs by accounting for correlations in latent representations.
problem VAEs fail to account for correlations between data points, limiting their effectiveness.
method CVAEs incorporate correlation structure into VAEs using a prior and tractable approximations.
result CVAEs outperform baseline algorithms in matching and link prediction tasks.
Paper proposes a novel auto-encoder for latent density estimation.
problem Challenges of learning generative probabilistic models due to curse of dimensionality.
method Joint dimensionality reduction and non-parametric density estimation framework using a novel estimator.
result Proposed model achieves promising results on various datasets.
Paper proposes a fair auto-encoder using hierarchical VampPrior and mutual information.
problem Learning fair representations to remove biases in decision-making.
method Hierarchical Variational Auto-Encoder with mutual information regularization.
result The approach either outperforms or performs on par with the current best model in experiments.
Paper proposes a generative model approach for outlier detection in signals.
problem Recovering signals from compressed measurements with sparse outliers.
method Iterative ADMM and gradient descent algorithms for ℓ1 and squared ℓ1 norm minimization. result Established recovery guarantees for generative models in the presence of outliers.
Develops a new non-adversarial framework for better generative models.
problem Inaccurate approximation of target distribution in latent space.
method Tessellated Wasserstein Auto-Encoders (TWAE) using centroidal Voronoi tessellation (CVT) to tessellate latent space.
result Significantly enhances generative performance in terms of FID compared to existing models.
Unsupervised deep learning is one of the most powerful representation learning techniques. Restricted Boltzman machine, sparse coding, regularized auto-encoders, and convolutional neural networks are pioneering building blocks of deep learning. In this paper, we propose a new building block -- distributed random models…
WWAE improves deep generative models by minimizing penalized optimal transport.
problem Challenges in learning deep generative models, such as blurriness in variational auto-encoders and instability in generative adversarial networks.
method WWAE formulates as minimization of penalized optimal transport between target and generated distributions. It uses closed-form squared Wasserstein-2 distance for Gaussians.
result WWAE learns better latent structures and generates higher quality samples than VAEs and GANs.
New distribution on simplex for auto-encoding tasks.
problem Developing a new distribution for sparsity in auto-encoding models.
method Using Kumaraswamy distribution and ordered stick-breaking process.
result The new distribution has an exact and closed form reparameterization.
CasVAE outperforms supervised methods for star-galaxy classification.
problem Challenges in machine learning for astronomy data.
method Cascade Variational Auto-Encoder (CasVAE) for unsupervised star-galaxy classification.
result CasVAE outperforms baseline models in accuracy and stability.
Auto-encoding generative adversarial networks (GANs) combine the standard GAN algorithm, which discriminates between real and model-generated data, with a reconstruction loss given by an auto-encoder. Such models aim to prevent mode collapse in the learned generative model by ensuring that it is grounded in all the ava…
SIG-VAE enhances VGAE for graph data modeling.
problem Limited flexibility in VGAE for graph data.
method Hierarchical variational framework with Bernoulli-Poisson link decoder.
result SIG-VAE outperforms state-of-the-art methods on graph tasks.
Recommender systems (RS) help users navigate large sets of items in the search for "interesting" ones. One approach to RS is Collaborative Filtering (CF), which is based on the idea that similar users are interested in similar items. Most model-based approaches to CF seek to train a machine-learning/data-mining model b…
Paper proposes a new method for learning deep generative models using annealed importance sampling.
problem Learning deep generative models efficiently and accurately.
method Proposes annealed importance sampling as a bridge between variational inference and Markov chain Monte Carlo.
result Demonstrates improved density models and better trade-off between computation and model accuracy.
Adaptive graph auto-encoder improves general data clustering.
problem Extending graph convolution networks to general clustering tasks.
method Adaptive graph construction based on generative perspective, novel decoder design.
result Model performs well in weighted graph scenarios.