We study the role of latent space dimensionality in Wasserstein auto-encoders (WAEs). Through experimentation on synthetic and real datasets, we argue that random encoders should be preferred over deterministic encoders. We highlight the potential of WAEs for representation learning with promising results on a benchmar…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “latent encoding”
Low-rank Characteristic Tensor Density Estimation Part II: Compression and Latent Density Estimationstat.ML
Paper proposes a novel auto-encoder for latent density estimation.
problem Challenges of learning generative probabilistic models due to curse of dimensionality.
method Joint dimensionality reduction and non-parametric density estimation framework using a novel estimator.
result Proposed model achieves promising results on various datasets.
The key idea of variational auto-encoders (VAEs) resembles that of traditional auto-encoder models in which spatial information is supposed to be explicitly encoded in the latent space. However, the latent variables in VAEs are vectors, which can be interpreted as multiple feature maps of size 1x1. Such representations…
A new method for disentangled latent spaces in VAEs that can manipulate attributes.
problem Disentangled representation of attributes in latent spaces of VAEs.
method Attribute-based regularization loss to enforce monotonic relationships between attributes and latent codes.
result Manipulation of attributes in latent spaces post-training.
Improves latent space structure for better data representation.
problem Limited ability of conventional priors to encode data manifold structure.
method Introduces an Encoded Prior Sliced Wasserstein AutoEncoder with iterative training and geodesic interpolation.
result Learned manifold encoding preserves topological and geometric properties of data.
DGA and DVGA learn disentangled graph representations to improve graph analysis.
problem Holistic graph auto-encoders fail to capture latent factors effectively.
method Design disentangled graph convolutional network and component-wise flow, impose independence constraints.
result Improved disentangled graph representations enhance graph analysis tasks.
Generative Adversarial Networks (GANs) play an increasingly important role in machine learning. However, there is one fundamental issue hindering their practical applications: the absence of capability for encoding real-world samples. The conventional way of addressing this issue is to learn an encoder for GAN via Vari…
A new method, REC, compresses images by encoding their latent representations efficiently.
problem Efficiently compressing single images with latent representations.
method Relative Entropy Coding (REC) that directly encodes latent representations with codelength close to relative entropy.
result REC is more efficient for single image compression compared to previous methods and is competitive for lossy compression.
A new method directly encodes data into latent space using gradient flow.
problem Suboptimal representations in physical sciences due to encoder inversion.
method Decoder-only approach using gradient flow and ODEs, avoiding integrals.
result Superior data efficiency and explicit encoding compared to traditional autoencoders.
Unified principle LZN unifies generative modeling, representation learning, and classification.
problem Disjoint solutions for generative modeling, representation learning, and classification.
method LZN creates a shared latent space for all tasks, using encoders and decoders for each data type.
result LZN improves FID on CIFAR10 and outperforms existing methods in representation learning and joint generation/classification.
VED framework learns low-dimensional latent representations of physical systems.
problem Learning latent representations of complex physical systems.
method Variational Encoder-Decoder (VED) framework with KL divergence and covariance regularization.
result VED achieves lower-dimensional latent representations with improved feature disentanglement.
Deep learning models have shown state-of-the-art performance in many inverse reconstruction problems. However, it is not well understood what properties of the latent representation may improve the generalization ability of the network. Furthermore, limited models have been presented for inverse reconstructions over ti…
We introduce the Mutual Information Machine (MIM), a probabilistic auto-encoder for learning joint distributions over observations and latent variables. MIM reflects three design principles: 1) low divergence, to encourage the encoder and decoder to learn consistent factorizations of the same underlying distribution; 2…
VAEs help in learning latent variables for cryo-EM applications.
problem Learning latent variables for cryo-EM data.
method Used VAEs for latent variable learning, focusing on the encoder's role.
result The encoder of the VAE in cryo-EM applications resembles traditional latent variable representations.
Gradient flow autoencoder improves data efficiency over traditional autoencoders.
problem Sub-optimal latent space representations in autoencoders.
method Gradient flow through ODE with adaptive step size for optimization.
result Gradient flow autoencoder achieves higher data efficiency.
New encoding improves volatility surface generation and risk management.
problem Generating accurate synthetic volatility surfaces from limited data.
method PCA variational auto-encoder model to encode surface descriptors into a latent space.
result Better scenario generation, volatility extrapolation, and direct stock surface inference.
This paper proposes a method to improve VAEs by extracting latent spaces from pre-trained diffusion models.
problem VAEs struggle with generating high-quality images due to unrealistic Gaussian assumptions.
method Optimizes an encoder to maximize marginal data log-likelihood and derives a decoder analytically.
result The method enhances VAE performance by discarding Gaussian assumptions and training a separate decoder network.
Paper proposes Adversarial CCA for multi-view alignment and generation.
problem Aligning multiple views in cross-view data analysis problems.
method Bayesian perspective, adversarial training for consistent latent encodings.
result ACCA model achieves superior performance in multi-view alignment and generation.
We proposed a novel graph convolutional neural network that could construct a coarse, sparse latent point cloud from a dense, raw point cloud. With a novel non-isotropic convolution operation defined on irregular geometries, the model then can reconstruct the original point cloud from this latent cloud with fine detail…
Auto-encoders are among the most popular neural network architecture for dimension reduction. They are composed of two parts: the encoder which maps the model distribution to a latent manifold and the decoder which maps the latent manifold to a reconstructed distribution. However, auto-encoders are known to provoke cha…
New method learns graphical models with latent variables for extreme events.
problem Learning graphical models with latent variables for multivariate extremes.
method Tractable convex program exttt{eglatent} for Hüsler-Reiss models.
result Consistently recovers conditional graph and latent variables.
Adaptive framework for learning latent space dimensions in GANs.
problem Inadequate latent space dimensions lead to poor generative models for complex data.
method Proposes a novel framework (LWGAN) that adaptively learns latent dimensions of data manifolds.
result Proves that the estimated intrinsic dimension is a consistent estimate of the true data manifold dimension.
Probabilistic Contrastive Learning Recovers the Correct Aleatoric Uncertainty of Ambiguous Inputscs.LG
Contrastive learning recovers latent distributions for ambiguous inputs, including aleatoric uncertainty.
problem Real-world observations often have inherent ambiguities, making the true posterior probabilistic with heteroscedastic uncertainty.
method Extended InfoNCE objective and encoders to predict latent distributions, proving they recover the correct posteriors, including aleatoric uncertainty.
result Contrastive learning encoders can recover the correct posteriors of data-generating processes, including aleatoric uncertainty, up to a rotation of the latent space.
Paper proposes a new Autoencoder for robustly encoding white matter streamlines.
problem Limited Autoencoder architectures ignore global streamline geometry and lack interpretability.
method Introduces Differentiable Vector Quantized Variational Autoencoder (D-VQ-VAE) for entire streamline bundles.
result Demonstrates superior performance in encoding and synthesis compared to state-of-the-art Autoencoders.
Modeling Causal Mechanisms with Diffusion Models for Interventional and Counterfactual Queriesstat.ML
DCM uses diffusion models to answer causal queries from observational data.
problem Answering causal queries from observational data alone.
method Diffusion models to learn causal mechanisms and generate latent encodings.
result Significant improvements over existing methods for causal query answering.
CIPNN model tackles continuous latent variables, solving intractable posterior problems.
problem Solving intractable posterior calculation for continuous latent variables.
method Derives analytical solution for posterior of continuous latent variables, proposes CIPNN and CIPAE.
result CIPNN model demonstrates great classification capability, solving problems for continuous latent variables.
The manifold hypothesis states that many kinds of high-dimensional data are concentrated near a low-dimensional manifold. If the topology of this data manifold is non-trivial, a continuous encoder network cannot embed it in a one-to-one manner without creating holes of low density in the latent space. This is at odds w…
FlexAE addresses bias-variance trade-off in RAEs by learning latent priors.
problem Improving generation quality of deterministic AE models.
method Introducing flexibly learnable latent priors in WAEs to optimize the latent distribution.
result FlexAE achieves state-of-the-art performance in AE-based generative models.
Unbiased gradient estimation improves VAE performance.
problem Training VAEs via maximum likelihood is difficult due to intractable integrals.
method Introduced unbiased estimators of the log-likelihood gradient using coupled Markov chains.
result Unbiased estimators lead to better predictive performance in VAEs.
Half-AVAE enhances VAE for underdetermined ICA with adversarial training.
problem Challenges in ICA under underdetermined conditions.
method Encoder-free VAE with adversarial networks and EE terms.
result Half-AVAE outperforms baseline models in underdetermined ICA.
LLMs encode latent topic distributions, suggesting Bayesian inference.
problem Capturing topic structure from large language models.
method Connecting LLM optimization to implicit Bayesian inference and de Finetti's theorem.
result LLMs recover latent topic distributions, matching LDA-generated topics.
We introduce the variational graph auto-encoder (VGAE), a framework for unsupervised learning on graph-structured data based on the variational auto-encoder (VAE). This model makes use of latent variables and is capable of learning interpretable latent representations for undirected graphs. We demonstrate this model us…
Non-negative tensor factorization models enable predictive analysis on count data. Among them, Bayesian Poisson-Gamma models can derive full posterior distributions of latent factors and are less sensitive to sparse count data. However, current inference methods for these Bayesian models adopt restricted update rules f…
Humans are able to imagine a person's voice from the person's appearance and imagine the person's appearance from his/her voice. In this paper, we make the first attempt to develop a method that can convert speech into a voice that matches an input face image and generate a face image that matches the voice of the inpu…
Transformers encode latent distributions in text, improving performance in out-of-distribution cases.
problem What should embeddings from language models represent?
method Connecting autoregressive prediction to sufficient statistics, identifying three settings.
result Transformers encode latent generating distributions, improving performance.
A new model encodes distances and topology in latent variables.
problem Modeling dissimilarity data with latent variables and invariances.
method Isometric Gaussian Process Latent Variable Model using Riemannian geometry and variational inference.
result The model can encode invariances in learned manifolds.
Generative autoencoders offer a promising approach for controllable text generation by leveraging their latent sentence representations. However, current models struggle to maintain coherent latent spaces required to perform meaningful text manipulations via latent vector operations. Specifically, we demonstrate by exa…
Integrates inductive biases into VAEs using intermediary latent variables.
problem Ineffective mechanisms for incorporating inductive biases into VAEs.
method InteL-VAEs use an intermediary latent space to control encoding, with a parametric function to enforce desired properties.
result InteL-VAEs lead to better generative models and representations.
In this work, we propose a method for learning driver models that account for variables that cannot be observed directly. When trained on a synthetic dataset, our models are able to learn encodings for vehicle trajectories that distinguish between four distinct classes of driver behavior. Such encodings are learned wit…
Proposes flexible auto-encoders for varying data dimensions.
problem Fixed latent dimensions limit data flexibility.
method Stochastic bottleneck with weighted dropouts.
result Seamless variable dimensionality reduction with high performance.
AEVB improves understanding of latent variable models.
problem Training latent variable models efficiently and understanding their limitations.
method Motivates AEVB from EM, emphasizing approximate E-step and M-step.
result AEVB tightens ELBO, improving model training.
DDPM encoder matches optimal transport for natural images.
problem Understanding theoretical properties of DDPM latent space.
method Showed DDPM encoder matches optimal transport for common distributions.
result DDPM encoder map coincides with optimal transport map for natural images.
This study rethinks the latent space in generative modeling, improving performance with less complex models.
problem Determining the optimal latent space for generative models and understanding its impact on model complexity.
method Proposed a new distance metric between latent and data distributions, and a two-stage training strategy called Decoupled Autoencoder (DAE).
result Improves generative performance with less complex models, as shown by comprehensive experiments on various models.
Deep generative models have made tremendous advances in image and signal representation learning and generation. These models employ the full Euclidean space or a bounded subset as the latent space, whose flat geometry, however, is often too simplistic to meaningfully reflect the manifold structure of the data. In this…
EXoN creates an explainable latent space for semi-supervised learning.
problem Creating an explainable latent space for semi-supervised learning.
method EXoN combines VAE with SCI (Soft-label Consistency Interpolation) to create an explainable latent space.
result EXoN reduces the cost of investigating representation patterns on the latent space.
Improved graph embedding through refined linear transformation and community recovery.
problem Identifying meaningful latent communities in graph data.
method Refined graph encoder embedding via linear transformation, self-training, and latent community recovery.
result Improved vertex embedding and better decision boundaries for vertex classification.
Proposes a geometry-aware VAE for better latent space modeling.
problem Lack of meaningful latent space structure in VAEs for small datasets.
method Introduces a Riemannian Hamiltonian VAE with a learned metric.
result Improves latent space structure leading to better interpolations and data generation.
After deep generative models were successfully applied to image generation tasks, learning disentangled latent variables of data has become a crucial part of deep generative model research. Many models have been proposed to learn an interpretable and factorized representation of latent variable by modifying their objec…