Topological anomaly scores predict return curves in S&P 500 stocks
problem Detecting anomalies in financial time series
method BallMapper, decoder-conditional VAE, Function-on-Function regression
result Anomaly history carries predictive content for return curves
This work proposes an efficient autoregressive model for text generation.
problem The challenge of generating high-quality text with autoregressive models.
method Introduces a cascaded decoding approach using Markov transformers to achieve sub-linear parallel time generation.
result Shows competitive accuracy/speed tradeoff compared to existing methods on five machine translation datasets.
In this paper, we integrate VAEs and flow-based generative models successfully and get f-VAEs. Compared with VAEs, f-VAEs generate more vivid images, solved the blurred-image problem of VAEs. Compared with flow-based models such as Glow, f-VAE is more lightweight and converges faster, achieving the same performance und…
This work tackles posterior collapse in conditional and hierarchical VAEs.
problem Posterior collapse in VAEs leads to poor latent variable representations.
method Theoretical analysis of linear conditional and hierarchical VAEs, empirical validation.
result Theoretical and empirical evidence of posterior collapse causes in conditional and hierarchical VAEs.
BS-VAE separates decoder variance and beta to improve VAE performance.
problem Blurriness in VAE outputs and difficulty in analyzing model performance.
method Explicitly separates beta and decoder variance in Beta-Sigma VAE.
result Superior performance in natural image synthesis and controllable parameters.
A tutorial on VAEs explaining its derivation and applications.
problem Understanding the VAE model and its limitations.
method Explains VAE through probabilistic and information theoretic perspectives.
result Identifies two common misconceptions and their practical consequences.
New system for i-vector speaker recognition based on variational autoencoder (VAE) is investigated. VAE is a promising approach for developing accurate deep nonlinear generative models of complex data. Experiments show that VAE provides speaker embedding and can be effectively trained in an unsupervised manner. LLR est…
The paper provides convergence guarantees for VAEs using SGD and Adam.
problem Understanding theoretical convergence guarantees for VAEs.
method Derives non-asymptotic convergence rates for VAEs trained with SGD and Adam.
result Convergence rate of \(\mathcal{O}(\log n / \sqrt{n})\) with explicit hyperparameter dependencies.
VAEs help in learning latent variables for cryo-EM applications.
problem Learning latent variables for cryo-EM data.
method Used VAEs for latent variable learning, focusing on the encoder's role.
result The encoder of the VAE in cryo-EM applications resembles traditional latent variable representations.
PRI-VAE learns disentangled representations by optimizing principle-of-relevant-information.
problem Learning disentangled representations under VAE framework remains unknown.
method Proposes PRI-VAE, a novel learning objective to optimize disentanglement.
result Demonstrates effectiveness of PRI-VAE on four benchmark datasets.
C2VAE learns disentangled and coupled representations without prior knowledge.
problem Learning disentangled and coupled representations in latent space.
method Introduces C2VAE, a self-supervised VAE that factorizes posterior and uses Gaussian copula for dependencies. result Demonstrates strong effect in enhancing disentangled representation learning.
Variational autoencoders (VAEs) have ushered in a new era of unsupervised learning methods for complex distributions. Although these techniques are elegant in their approach, they are typically not useful for representation learning. In this work, we propose a simple yet powerful class of VAEs that simultaneously resul…
NE-VAE prevents posterior collapse in VAEs by embedding neighbors in latent space.
problem Posterior collapse in VAEs when strong decoders are used.
method Neighbor embedding in latent space to prevent collapse.
result NE-VAE produces qualitatively different latent representations with active latent dimensions.
The variational autoencoder (VAE) framework remains a popular option for training unsupervised generative models, especially for discrete data where generative adversarial networks (GANs) require workaround to create gradient for the generator. In our work modeling US postal addresses, we show that our discrete VAE wit…
q-VAE extracts disentangled latent spaces for robot control and dynamic systems.
problem Disentangled representation learning for latent spaces in robotics.
method Proposes q-VAE based on Tsallis statistics, improving disentanglement and latent dynamics.
result Improves disentangled representation while maintaining data reconstruction accuracy.
Single training run learns optimal VAE parameters for various β values.
problem Training VAEs with varying β values for optimal trade-off between distortion and rate.
method Introduced Multi-Rate VAE (MR-VAE) using hypernetworks to map β to optimal parameters.
result MR-VAEs can construct the full rate-distortion curve without additional training.
VAEs analyzed using harmonic analysis, showing how variance controls frequency content and robustness.
problem Understanding and optimizing VAEs for robustness and frequency control.
method Viewing VAE latent space as Gaussian space, deriving results on variance and frequency content, and demonstrating soft Lipschitz constraints.
result Increasing encoder variance reduces high frequency content and improves adversarial robustness.
SKR-VAE improves VAEs for ICA with reduced computational cost.
problem Efficiently performing ICA in VAEs with large datasets.
method Structured kernel functions to avoid costly GP kernel matrix inversion.
result SKR-VAE achieves greater computational efficiency and reduced resource consumption.
VAE improves anomaly detection for jet tagging at the LHC.
problem Anomaly detection in jet tagging at the LHC.
method Variational Autoencoder (VAE) trained on background QCD jets, with latent space learning for anomaly detection.
result Outlier Exposed VAE (OE-VAE) achieves excellent results in both sensitivity and decorrelation of jet mass.
High-dimensional VAEs inevitably collapse to prior, requiring large datasets for good performance.
problem Posterior collapse in VAEs leads to poor representation learning quality.
method Analyzed a minimal VAE in a high-dimensional limit, evaluating conditions for posterior collapse with respect to beta and dataset size.
result VAEs face 'inevitable posterior collapse' beyond a certain beta threshold, regardless of dataset size.
NeRF-VAE generates 3D scenes with geometric structure from few images.
problem Generating 3D scenes from few images with geometric consistency.
method Combines NeRF and VAE, incorporating shared geometric structure.
result NeRF-VAE can infer and render geometrically-consistent scenes from unseen environments.
Flexible VAEs using FIFs improve model likelihood on image datasets.
problem Limitations of diagonal Gaussian posteriors in VAEs.
method Regularized Free-form Injective Flow (FIF) for flexible posterior.
result Full covariance VAEs outperform diagonal Gaussian posteriors.
Improved deep hierarchical VAE with diffusion-based VampPrior.
problem Latent variable generative modeling challenges.
method Hierarchical VAE with amortized diffusion-based VampPrior.
result Better performance with fewer parameters and improved stability.
While enormous progress has been made to Variational Autoencoder (VAE) in recent years, similar to other deep networks, VAE with deep networks suffers from the problem of degeneration, which seriously weakens the correlation between the input and the corresponding latent codes, deviating from the goal of the representa…
As a general-purpose generative model architecture, VAE has been widely used in the field of image and natural language processing. VAE maps high dimensional sample data into continuous latent variables with unsupervised learning. Sampling in the latent variable space of the feature, VAE can construct new image or text…
The VAE's reconstruction ability is studied using PAC-Bayes theory.
problem Understanding the performance of VAEs for unseen data.
method PAC-Bayes theory is applied to analyze VAE's reconstruction error.
result Generalization bounds on VAE's reconstruction error are provided.
HL-VAE extends VAE for heterogeneous temporal and longitudinal data.
problem Handling heterogeneous data in temporal and longitudinal datasets.
method Proposes HL-VAE, an extension of existing VAEs for temporal and longitudinal data, incorporating likelihood models for various data types.
result HL-VAE achieves competitive performance in missing value imputation and predictive accuracy.
NegBio-VAE models neural spike counts with negative binomial distribution.
problem Limited biological plausibility of continuous latent variables in VAEs for neural spike modeling.
method Proposes a negative binomial latent-variable model with a dispersion parameter for overdispersed spike count modeling.
result NegBio-VAE outperforms competing models in reconstruction and generation tasks.
A new method for effective VAE training using calibrated decoders.
problem Training VAEs requires hyperparameter tuning, leading to inefficiency.
method Calibrated decoders that learn uncertainty and automatically determine information retention.
result Calibrated decoders can simplify VAE training without heuristic modifications.
A new method for VAEs improves latent space disentanglement without violating probability laws.
problem Improving latent space disentanglement in VAEs without violating probability laws.
method Developed a Renyi VAE with a conditional distribution not learned, using Singular Value Decomposition for evaluation.
result Improved latent space disentanglement without violating probability laws.
Exemplar VAEs link generative models with nearest neighbor retrieval and data augmentation.
problem Improving generative model performance and data augmentation effectiveness.
method Exemplar VAEs with Parzen window prior, retrieval augmented training, exemplar leave-one-out, and subsampling.
result Generative data augmentation reduces classification error on MNIST and Fashion MNIST.
Proposes a method to learn conditional VAEs from datasets with missing covariates.
problem Learning conditional VAEs from datasets with missing covariates.
method Augments conditional VAEs with a prior distribution for missing covariates and estimates their posterior using amortised variational inference.
result The proposed method outperforms previous methods in learning conditional VAEs from non-temporal, temporal, and longitudinal datasets.
Certifiably robust VAEs are trained with bounds on input perturbations.
problem Ensuring VAEs are robust to adversarial attacks.
method Derive bounds on minimal perturbation size, control parameters, and train VAEs to meet criteria.
result Certifiably robust VAEs are more robust to attacks than standard VAEs.
Half-AVAE enhances VAE for underdetermined ICA with adversarial training.
problem Challenges in ICA under underdetermined conditions.
method Encoder-free VAE with adversarial networks and EE terms.
result Half-AVAE outperforms baseline models in underdetermined ICA.
GUIDE-VAE generates user-guided data with improved realism and performance.
problem Generating data points for multi-user datasets while considering user information.
method Conditional generative model that integrates user embeddings and a pattern dictionary-based covariance composition.
result GUIDE-VAE outperforms conventional VAEs in multi-user settings, especially under data imbalance.
Integrates inductive biases into VAEs using intermediary latent variables.
problem Ineffective mechanisms for incorporating inductive biases into VAEs.
method InteL-VAEs use an intermediary latent space to control encoding, with a parametric function to enforce desired properties.
result InteL-VAEs lead to better generative models and representations.
Jigsaw-VAE tackles feature imbalance in VAE latent variables, improving generalization across environments.
problem Feature imbalance in VAE latent variables leads to poor generalization and biased sample generation.
method Proposes a regularization scheme to balance features in VAE latent variables and introduces a metric to measure balance.
result The regularization scheme substantially addresses feature imbalance, leading to improved generalization and diverse sample generation.
C-VAE improves VAE by resolving prior issues and generating better samples.
problem Low-quality samples from VAE due to prior issues.
method Formulates VAE as OT, allows flexible priors, and uses OT formulations.
result C-VAE generates higher quality samples and latent representations.
ED-VAE improves VAEs by explicitly including entropy components in ELBO.
problem Limitations of traditional VAEs with ELBO in generating high-quality samples and interpreting latent spaces.
method Introduces ED-VAE, a re-formulation of ELBO that includes entropy and cross-entropy components.
result Significantly enhances model flexibility and improves interpretability and generative performance.
VAEs struggle with low-dimensional data; this paper shows they can learn the correct manifold.
problem Training VAEs on low-dimensional data.
method Two-stage training algorithm based on conjecture of 0 variance generator.
result VAEs can learn generators with support equal to the ground truth manifold.
The variational autoencoder (VAE) is a generative model with continuous latent variables where a pair of probabilistic encoder (bottom-up) and decoder (top-down) is jointly learned by stochastic gradient variational Bayes. We first elaborate Gaussian VAE, approximating the local covariance matrix of the decoder as an o…
Posterior collapse in Variational Autoencoders (VAEs) arises when the variational posterior distribution closely matches the prior for a subset of latent variables. This paper presents a simple and intuitive explanation for posterior collapse through the analysis of linear VAEs and their direct correspondence with Prob…
Improved VAE model enhances uncertainty estimation for out-of-distribution samples.
problem VAEs assign higher likelihood to out-of-distribution inputs.
method INCPVAE integrates noise contrastive prior into VAEs for reliable uncertainty estimation.
result INCPVAE outperforms standard VAEs in uncertainty estimation for OOD inputs.
Integrates VAEs into EM for deep clustering and generation.
problem Clustering and generating new samples from complex distributions.
method Combines VAEs and EM, updating model parameters and refining cluster assignments.
result Superior clustering performance on MNIST and FashionMNIST.
Variational autoencoders (VAEs), as well as other generative models, have been shown to be efficient and accurate for capturing the latent structure of vast amounts of complex high-dimensional data. However, existing VAEs can still not directly handle data that are heterogenous (mixed continuous and discrete) or incomp…
Proposes VAE-KRnet for density estimation and variational Bayes.
problem Challenges of density estimation and posterior approximation in high dimensions.
method Combines VAE and KRnet for density estimation, incorporating mutual information maximization.
result Improves posterior approximation in high-dimensional cases.
Proposes a statistical test for VAE-based anomaly detection reliability.
problem Ensuring reliability of anomaly detection in high-stakes applications.
method Variance Autoencoder (VAE) Test based on selective inference.
result Validates VAE-based anomaly detection with p-values controlling false detection probability.
Improved image reconstruction using VAEs with Student's t-prior.
problem Improving the robustness of VAEs in image reconstruction.
method Proposed a VAE with Student's t-distribution as prior, trained all distribution parameters.
result Better image reconstruction achieved with Student's t-prior compared to Gaussian priors.