Kernel Autoencoder (KAE) encodes any data type using RKHSs.
problem Representing any data type in a compact form.
method KAE uses mappings from vv-RKHSs to minimize reconstruction error.
result KAE can autoencode any kind of data by choosing X as a RKHS.
Enhances autoencoders with kernel alignment for better data representation.
problem Lack of clear properties for autoencoders to capture in data representations.
method Aligns inner products between codes with a kernel matrix to capture topological properties.
result Effective data representations learned, preserving input data similarities.
In this paper we introduce the deep kernelized autoencoder, a neural network model that allows an explicit approximation of (i) the mapping from an input space to an arbitrary, user-specified kernel space and (ii) the back-projection from such a kernel space to input space. The proposed method is based on traditional a…
Sigmoid autoencoders can implement associative memory with certain conditions.
problem Implementing associative memory in neural networks.
method Theoretical analysis of overparameterized sigmoid autoencoders using the NTK and iterative maps.
result Overparameterized sigmoid autoencoders can have attractors in the NTK limit, leading to associative memory.
This paper explores using SSIM for better image generation in generative models.
problem Improving perceptual quality in generated images using ℓ2 norm. method Theoretical discussion and practical implementation of SSIM in generative models and autoencoders.
result SSIM can be used in generative models and autoencoders to generate better images.
Clinical measurements collected over time are naturally represented as multivariate time series (MTS), which often contain missing data. An autoencoder can learn low dimensional vectorial representations of MTS that preserve important data characteristics, but cannot deal explicitly with missing data. In this work, we …
A new autoencoder method uses empirical beta copulas for generating data.
problem Creating a generative model from an autoencoder's latent space.
method Empirical Beta Copula Autoencoder method.
result The Empirical Beta Copula Autoencoder outperforms other methods in simplicity and effectiveness.
A new generator uses kernel distance to avoid GAN weaknesses.
problem Stability and mode collapse in GANs and autoencoders.
method LCW generator (Latent Cramer-Wold generator) using kernel distance.
result Very competitive FID values.
Paper bridges VAEs and KDEs for more flexible posterior estimation.
problem Limitations of Gaussian latent space in VAEs and challenges in KL-divergence estimation.
method Approximate posterior with KDEs and derive upper bound of KL-divergence in ELBO.
result Epanechnikov kernel minimizes KL-divergence upper bound asymptotically.
Explains various PCA and SPCA methods with theory and applications.
problem No specific problem stated; focuses on explaining methods.
method Explains PCA, SPCA, kernel PCA, and kernel SPCA methods with theory and applications.
result Comprehensive coverage of PCA and SPCA methods with theory and applications.
A scalable factorized Gaussian process VAE for faster inference.
problem Inference bottlenecks in Gaussian process VAEs.
method Factorizes latent kernel across auxiliary features, leveraging independence.
result Significant speed-up in inference time (in theory and practice).
A new method integrates autoencoders with geometry regularization for manifold learning.
problem Extracting simplified low-dimensional representations that capture intrinsic geometry in data.
method Integrates autoencoders with a geometric regularization term based on diffusion potential distances.
result The method preserves intrinsic structure, enables out-of-sample extension, and faithful reconstruction.
Researchers calculate entropy of heat kernel on manifolds for very small times.
problem Estimating entropy of heat kernel on compact Riemannian manifolds for small times.
method Asymptotic expansion, polynomial expressions in curvature tensor components.
result First three coefficients of entropy expansion computed and expressed as polynomials.
Diffusion Variational Autoencoders capture topological properties of datasets.
problem Standard VAEs struggle with topological properties of certain datasets.
method Introduces Diffusion VAEs with transition kernels of Brownian motion on arbitrary manifolds.
result Diffusion VAEs can capture topological properties of synthetic datasets.
MoCA uses a novel autoencoder to analyze multi-modal health data.
problem Challenges in analyzing continuous multi-modal health data from wearable devices.
method Proposes MoCA, a self-supervised learning framework combining transformer and masked autoencoder methods.
result Demonstrates strong performance boosts across reconstruction and classification tasks.
The paper optimizes autoencoder latent spaces for one-class learning with controlled connectivity.
problem Learning representations with controllable connectivity for better upstream tasks.
method A novel loss function based on persistent homology controls the connectivity of autoencoder latent spaces.
result The controlled connectivity in latent space improves one-class learning performance, especially in low sample size scenarios.
A scalable GPVAE method using local adjacencies to approximate GP inference.
problem Scalability issues in exact GP inference for large-scale GPVAEs.
method Neighbour-driven approximation strategy that confines computations to nearest neighbours.
result Outperforms other GPVAE variants in predictive performance and computational efficiency.
New tensor kernels reduce mismatch between clustering and reconstruction objectives in deep learning.
problem Objective Function Mismatch in deep clustering.
method Proposed Unsupervised Companion Objectives (UCOs) with tensor kernels to address mismatch.
result Reduced OFM between clustering and reconstruction objectives, leading to improved clustering performance.
Unsupervised method selects genes for tumor subtype discovery.
problem High-dimensional tumor gene expression data with noisy variables and heterogeneity.
method Autoencoders for latent space learning, Multiple Kernel Learning for feature selection, clustering.
result Lower redundancy and better clustering performance compared to benchmarks.
Generalizes machine learning models using localization kernels and local means.
problem Understanding and unifying diverse machine learning models.
method Formal definition of localization method through localization kernels and local means.
result Unified theoretical lens and new methodological tools for designing flexible learning systems.
Proposes a method to ensure latent space of AutoEncoders is Gaussian without randomness.
problem Ensuring latent space of AutoEncoders is Gaussian without randomness.
method Directly optimize agreement between empirical distribution function and desired CDF for chosen properties.
result Ensures latent space of AutoEncoders is Gaussian without randomness.
KernelNet learns data-dependent kernels for deep generative models.
problem Learning kernels requires careful selection of hyperparameters.
method KernelNet constructs a data-dependent kernel using deep neural networks.
result KernelNet achieves better performance in deep generative models.
Sliced kernelized Stein discrepancy improves goodness-of-fit tests and model learning in high dimensions.
problem The curse-of-dimensionality in kernelized Stein discrepancy (KSD).
method Sliced Stein discrepancy and its scalable variants using optimal one-dimensional projections.
result Significantly outperforms KSD and baselines in goodness-of-fit tests and improves model learning.
Principal component analysis (PCA) is largely adopted for chemical process monitoring and numerous PCA-based systems have been developed to solve various fault detection and diagnosis problems. Since PCA-based methods assume that the monitored process is linear, nonlinear PCA models, such as autoencoder models and kern…
Improved autoencoders show joint training benefits over weak training.
problem Improving unsupervised learning performance with over-parameterized networks.
method Analyzing gradient dynamics of two-layer autoencoders with ReLU activation, proving linear convergence in weakly-trained and jointly-trained regimes.
result Joint training leads to better global optima and requires less over-parameterization.
The paper tackles model collapse in GPLVMs by improving kernel flexibility and projection variance.
problem Model collapse in GPLVMs leading to vague latent representations.
method Theoretical analysis of projection variance, integration of SM and RFF kernels, and variational inference.
result The advisedRFLVM outperforms competing models in informative latent representations and missing data imputation.
Generative models learn smoother densities to sample from unknown distributions.
problem Sampling from unknown distributions in high-dimensional spaces.
method Formalizes sampling problem, introduces multimeasurement noise model, derives Bayes estimator, and uses underdamped Langevin MCMC.
result Formulation leads to efficient sampling methods and theoretical connections with denoising autoencoders.
DSIVI improves variational autoencoders by optimizing a proper lower bound on ELBO.
problem Improving variational autoencoders with implicit priors.
method Introducing DSIVI, a method that optimizes a proper lower bound on ELBO for models with semi-implicit priors and posteriors.
result DSIVI improves the performance of VampPrior, a state-of-the-art prior for variational autoencoders.
A new method estimates generative model mappings using kernel transfer operators, reducing costs and improving performance.
problem Efficiently estimating mappings between known and unknown distributions in generative models.
method Adapting kernel transfer operators to estimate mappings, reducing computational costs.
result Significant runtime savings and good empirical performance compared to existing methods.
The paper analyzes variational autoencoders for state space models with risk bounds.
problem Analyzing the risk associated with variational autoencoders for state space models.
method Backward factorization of variational distributions to analyze excess risk, providing oracle inequalities and upper bounds.
result Explicit upper bounds on variational estimation error for state space models under strong mixing assumptions.
tvGP-VAE models tensor-valued latent variables with Gaussian processes for better data structure representation.
problem Agnostic latent variables in VAEs ignore data structure correlations.
method Proposes tensor-variate Gaussian process prior for variational autoencoder.
result Explicitly modeling correlation structures improves model performance in reconstruction.
UT module refines VAE latent space, improving disentanglement and interpretability.
problem Irregular latent distributions cause posterior collapse and misalignment in VAEs.
method UT module uses G-KDE clustering, GM modeling, and PIT to transform latent space into uniform distribution.
result UT module enhances disentanglement and interpretability of latent representations.
One-class support vector machine (OC-SVM) for a long time has been one of the most effective anomaly detection methods and extensively adopted in both research as well as industrial applications. The biggest issue for OC-SVM is yet the capability to operate with large and high-dimensional datasets due to optimization c…
New method uses deep learning for better monitoring of industrial processes.
problem Monitoring high-dimensional, nonlinear profiles in industrial systems.
method Variational Autoencoders (VAEs) for modeling nonlinear manifolds.
result Deep probabilistic models outperform traditional methods in residual-space.
Levenshtein VAE prevents posterior collapse in text generation models.
problem Posterior collapse in VAEs where generators ignore latent variables.
method Replaces ELBO with a Levenshtein distance-based objective to prevent collapse.
result Levenshtein VAE produces more informative latent representations.
A deep neural network for spatial time series forecasting.
problem Challenges in forecasting spatial time series with specific patterns and curse of dimensionality.
method Spatial-temporal decomposition, fuzzy clustering, multi-kernel convolution, convolution-LSTM, denoising autoencoder.
result Model outperforms baseline and state-of-the-art models in traffic flow prediction.
SKR-VAE improves VAEs for ICA with reduced computational cost.
problem Efficiently performing ICA in VAEs with large datasets.
method Structured kernel functions to avoid costly GP kernel matrix inversion.
result SKR-VAE achieves greater computational efficiency and reduced resource consumption.
This paper describes a method for learning low-dimensional approximations of nonlinear dynamical systems, based on neural-network approximations of the underlying Koopman operator. Extended Dynamic Mode Decomposition (EDMD) provides a useful data-driven approximation of the Koopman operator for analyzing dynamical syst…
New autoencoder framework learns structured latent priors.
problem Learning autoencoders with flexible priors.
method Relational regularization on latent prior, scalable algorithms.
result RAE outperforms existing autoencoders in image generation.
Variational autoencoders learn deep latent models.
problem Learning deep latent-variable models.
method Principled framework using variational inference.
result Introduction to variational autoencoders and extensions.
Feature extraction becomes increasingly important as data grows high dimensional. Autoencoder as a neural network based feature extraction method achieves great success in generating abstract features of high dimensional data. However, it fails to consider the relationships of data samples which may affect experimental…
Autoencoders compress and reconstruct data for various applications.
problem Efficiently compress and reconstruct data.
method Neural network architecture that encodes and decodes data.
result Autoencoders can be applied to various data types and applications.
Proposes a Bayesian Autoencoder with sparse Gaussian process priors to capture data correlations.
problem Autoencoders' i.i.d. assumption of latent representations fails to capture data correlations.
method Imposes fully Bayesian sparse Gaussian Process priors on the latent space of a Bayesian Autoencoder and uses stochastic gradient Hamiltonian Monte Carlo for posterior estimation.
result Consistently outperforms alternatives relying on Variational Autoencoders on various tasks.
A new clustering method using autoencoders for improved data representation.
problem Improving clustering of complex data like images and text.
method DAMIC algorithm based on a mixture of deep autoencoders.
result Significant improvement over state-of-the-art methods on image and text corpora.
Study on dynamics of non-linear autoencoders learning principal components.
problem Technical difficulty in studying non-linear autoencoders due to non-trivial correlations.
method Derive asymptotically exact equations for SGD training of shallow, non-linear autoencoders.
result Autoencoders learn principal components sequentially and tie weights are ineffective.
Chart autoencoders learn latent features preserving manifold topology and geometry, with robust denoising capabilities.
problem Learning low-dimensional latent features of high-dimensional data sampled near a manifold.
method Chart autoencoders encode data into latent features on charts, preserving manifold topology and geometry.
result Chart autoencoders achieve a squared generalization error of n−d+22log4n under proper network architectures. POTATOES improves autoencoder UOD accuracy without tuning.
problem Improving unsupervised outlier detection accuracy.
method Randomly partition data, overfit each part with an autoencoder, use max reconstruction error as anomaly score.
result Significant improvement in UOD performance for dense inlier sets.
A novel online GP model captures long-term memory in sequential data.
problem Capturing long-term memory in sequential data online.
method Integrates HiPPO framework into interdomain GP, leveraging time-varying orthogonal projections as inducing variables.
result OHSVGP outperforms existing online GP methods in predictive performance, long-term memory preservation, and computational efficiency.