Deep generative networks have been widely used for learning mappings from a low-dimensional latent space to a high-dimensional data space. In many cases, data transformations are defined by linear paths in this latent space. However, the Euclidean structure of the latent space may be a poor match for the underlying lat…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “latent space transformation”
UT module refines VAE latent space, improving disentanglement and interpretability.
problem Irregular latent distributions cause posterior collapse and misalignment in VAEs.
method UT module uses G-KDE clustering, GM modeling, and PIT to transform latent space into uniform distribution.
result UT module enhances disentanglement and interpretability of latent representations.
Geometrically transforms word embeddings into a common space for better comparison.
problem Comparing embeddings from different sources is challenging.
method Applies orthogonal rotations and Mahalanobis scaling to transform embeddings into a shared latent space.
result The method improves word similarity and analogy tasks.
We extend the framework of variational autoencoders to represent transformations explicitly in the latent space. In the family of hierarchical graphical models that emerges, the latent space is populated by higher order objects that are inferred jointly with the latent representations they act on. To explicitly demonst…
The paper provides convergence guarantees for ODE-based generative models using transformers.
problem Theoretical guarantees for ODE-based generative models.
method A pre-trained autoencoder maps inputs to a latent space, and a transformer predicts the velocity field.
result The distribution of samples generated via estimated ODE flow converges to the target distribution in Wasserstein-2 distance.
Noting the importance of the latent variables in inference and learning, we propose a novel framework for autoencoders based on the homeomorphic transformation of latent variables, which could reduce the distance between vectors in the transformed space, while preserving the topological properties of the original space…
Unified framework for learning function representations using INRs and Transformers.
problem Scalability and efficiency limitations in existing generative models.
method Integrates INRs and Transformer-based hypernetworks into latent variable models.
result Improved scalability, expressiveness, and generalization over existing models.
VAELLS learns latent manifold structure to improve VAE model accuracy.
problem VAEs struggle with mismatched latent structure and global structure.
method Integrates learnable manifold model into latent space of VAE.
result Improves model accuracy by matching prior to data manifold structure.
Neural Canonical Transformation with Symplectic Flowscond-mat.stat-mech
Canonical transformation plays a fundamental role in simplifying and solving classical Hamiltonian systems. We construct flexible and powerful canonical transformations as generative models using symplectic neural networks. The model transforms physical variables towards a latent representation with an independent harm…
Deep latent variable models are powerful tools for representation learning. In this paper, we adopt the deep information bottleneck model, identify its shortcomings and propose a model that circumvents them. To this end, we apply a copula transformation which, by restoring the invariance properties of the information b…
Paper recovers latent causal structure and linear transformation from indirect observations.
problem Recovering latent causal structure and linear transformation from indirect observations.
method Established sufficient conditions for DAG recovery, leveraged score function properties, and used soft/hard interventions.
result Perfect recovery of latent DAG structure and linear transformation up to scaling using soft interventions, hard interventions with additional hypothesis testing.
Deep generative models are universal tools for learning data distributions on high dimensional data spaces via a mapping to lower dimensional latent spaces. We provide a study of latent space geometries and extend and build upon previous results on Riemannian metrics. We show how a class of heuristic measures gives mor…
Variational Auto-Encoders enforce their learned intermediate latent-space data distribution to be a simple distribution, such as an isotropic Gaussian. However, this causes the posterior collapse problem and loses manifold structure which can be important for datasets such as facial images. A GAN can transform a simple…
The Generator of a Generative Adversarial Network (GAN) is trained to transform latent vectors drawn from a prior distribution into realistic looking photos. These latent vectors have been shown to encode information about the content of their corresponding images. Projecting input images onto the latent space of a GAN…
Latent DiTs improve data distribution recovery and inference efficiency under low-dimensional latent space.
problem Improving data distribution recovery and inference efficiency in latent DiTs.
method Investigates statistical and computational limits of latent DiTs under low-dimensional latent space assumption, deriving approximation error bounds, sample complexity, and efficient inference and training algorithms.
result Latent DiTs can bypass high dimensionality challenges and achieve almost-linear time inference and training.
New model learns symmetry transformations from complex data.
problem Learning symmetry transformations in complex domains like chemical space.
method Two latent subspaces, deep information bottleneck, continuous mutual information regularizer.
result Model outperforms state-of-the-art methods on artificial and molecular datasets.
StrTransformer recovers sources without labels by optimizing latent matrices and enforcing structural constraints.
problem Unsupervised blind source recovery in signal processing.
method Source-wise structured Transformer framework with latent source matrix optimization, structural regularization, and branch-specific weights.
result StrTransformer learns distinct temporal-scale structures and recovers source-aligned latent trajectories.
A new model for time series using discrete latent states.
problem Efficiently modeling time series data with discrete latent states.
method A Markov chain-based model for training high-dimensional discrete latent data.
result Improved performance on time series datasets.
Fractal Flow enhances normalizing flows with interpretable latent space and hierarchical modeling.
problem High-dimensional density estimation and generative modeling challenges.
method Integrates topic modeling (LDA) and fractal strategy into normalizing flows.
result Achieves latent clustering, controllable generation, and superior estimation accuracy.
Unsupervised learning of probabilistic models is a central yet challenging problem in machine learning. Specifically, designing models with tractable learning, sampling, inference and evaluation is crucial in solving this task. We extend the space of such models using real-valued non-volume preserving (real NVP) transf…
Method discovers symmetries in data with neural networks.
problem Discovering symmetries in high-dimensional data.
method Fully connected neural networks trained with a loss function for symmetry.
result Generators for symmetries in latent space.
Recent studies identified that sequential Recommendation is improved by the attention mechanism. By following this development, we propose Relation-Aware Kernelized Self-Attention (RKSA) adopting a self-attention mechanism of the Transformer with augmentation of a probabilistic model. The original self-attention of Tra…
LSI enables joint learning of latent variables and generative models.
problem Joint optimization of latent variables and generative models.
method Developed a principled ELBO objective in continuous time for joint learning.
result LSI learns effective latent representations and generative transformations.
Learning multiple tasks across heterogeneous domains is a challenging problem since the feature space may not be the same for different tasks. We assume the data in multiple tasks are generated from a latent common domain via sparse domain transforms and propose a latent probit model (LPM) to jointly learn the domain t…
Proposes a new latent variable model for hyperspherical latent spaces.
problem Efficiently modeling heavy-tailed distributions in hyperspherical latent spaces.
method Introduces spherical Cauchy (spCauchy) latent variables and applies Möbius transformations.
result Shows spCauchy recovers vMF geometry in high-concentration limits and avoids complex evaluations.
This paper clarifies VAE's property through geometric and information-theoretic interpretations.
problem The transparency of VAE model is an underlying issue.
method Quantitative understanding of VAE through differential geometry and information theory.
result VAE can be mapped to an implicit isometric embedding with a scale factor derived from the posterior parameter.
Neural samplers such as variational autoencoders (VAEs) or generative adversarial networks (GANs) approximate distributions by transforming samples from a simple random source---the latent space---to samples from a more complex distribution represented by a dataset. While the manifold hypothesis implies that the densit…
New normalizing flows in hyperbolic space improve posterior modeling for hierarchical data.
problem Limited flexibility of existing normalizing flows in Euclidean space for hierarchical data.
method Elevated normalizing flows to hyperbolic spaces using coupling transforms and Wrapped Hyperboloid Coupling.
result Improved performance on density estimation and hierarchical graph data.
Improved DSSMs for easier interpretable latent variables.
problem Complex and hard-to-interpret latent variables in DSSMs.
method Simplified predictive decoder and shrinkage priors.
result Interpretable latent variables improve forecasting performance.
In this paper, we propose a two-step training procedure for source separation via a deep neural network. In the first step we learn a transform (and it's inverse) to a latent space where masking-based separation performance using oracles is optimal. For the second step, we train a separation module that operates on the…
A new model designs molecules with desired properties.
problem Finding molecules with optimal chemical or biological properties.
method A latent prompt Transformer model with three components: latent vector, molecule generation, and property prediction.
result The model achieves state-of-the-art performance on molecule design tasks.
Paper tackles reinforcement learning with complex observations and simple latent dynamics.
problem Understanding reinforcement learning with complex observations and simple latent dynamics.
method Statistical and algorithmic analysis of reinforcement learning under general latent dynamics.
result Identifies latent pushforward coverability as a condition for statistical tractability.
Rate-Distortion Optimization Guided Autoencoder for Isometric Embedding in Euclidean Latent Spacecs.LG
To analyze high-dimensional and complex data in the real world, deep generative models, such as variational autoencoder (VAE) embed data in a low-dimensional space (latent space) and learn a probabilistic model in the latent space. However, they struggle to accurately reproduce the probability distribution function (PD…
Researchers analyze and improve latent space in NAR models.
problem Latent space resolution and range issues in NAR models.
method Detailed analysis of GNN latent space structure, proposing and testing solutions.
result Improvements in majority of algorithms on CLRS-30 benchmark.
Elicitation without Backpropagation: Steering Model Behavior by Optimizing the Latent Posteriorcs.LG
PPT optimizes transformer behavior by steering its latent posterior using prior samples.
problem Eliciting desired behavior from transformers without backpropagation.
method Posterior Prefix Tuning (PPT) uses predictive Monte Carlo (PMC) samples and importance sampling to optimize the latent posterior.
result PPT optimizes transformer behavior without backpropagation, achieving high utility across different utility functions.
Towards the Latent Transcriptomeq-bio.GN
In this work we propose a method to compute continuous embeddings for kmers from raw RNA-seq data, without the need for alignment to a reference genome. The approach uses an RNN to transform kmers of the RNA-seq reads into a 2 dimensional representation that is used to predict abundance of each kmer. We report that our…
A new method for discrete data normalizing flows using latent transformations.
problem Challenges in parameterizing bijective transformations for discrete data.
method Predict a distribution over latent transformations to make the marginal likelihood differentiable.
result Discrete-data normalizing flows can be trained using gradient-based learning with unbiased score function estimation.
Study reveals RL game's embedding space is stratified, not a manifold.
problem Understanding the structure of RL game embeddings.
method Adapted Robinson's volume growth transform for RL setting.
result Token embedding space is stratified, not a manifold.
Generative model disentangles dark matter halo properties.
problem Entangling physical factors in generative model latent spaces.
method Auxiliary-variable-guided framework with halo mass and concentration.
result Reveals mass-concentration scaling relation and identifies unusual halo formation.
Identifying customer segments in retail banking portfolios with different risk profiles can improve the accuracy of credit scoring. The Variational Autoencoder (VAE) has shown promising results in different research domains, and it has been documented the powerful information embedded in the latent space of the VAE. We…
New method uses cycle consistency to enforce invariance in latent space.
problem Learning meaningful and independent factors of variation in datasets.
method Two separate latent subspaces, cycle consistency constraints, deep information bottleneck.
result Identifies more meaningful factors leading to sparser and interpretable models.
Generative adversarial networks (GANs) transform latent vectors into visually plausible images. It is generally thought that the original GAN formulation gives no out-of-the-box method to reverse the mapping, projecting images back into latent space. We introduce a simple, gradient-based technique called stochastic cli…
TK-GCN forecasts spatiotemporal dynamics using Koopman-enhanced graph convolutional networks.
problem Forecasting complex spatiotemporal dynamics over irregular domains.
method Two-stage framework: Koopman-enhanced Graph Convolutional Network (K-GCN) for spatial encoding and Transformer for temporal modeling.
result TK-GCN outperforms state-of-the-art methods in spatiotemporal cardiac dynamics forecasting.
Transformers encode latent distributions in text, improving performance in out-of-distribution cases.
problem What should embeddings from language models represent?
method Connecting autoregressive prediction to sufficient statistics, identifying three settings.
result Transformers encode latent generating distributions, improving performance.
Researchers apply concept-based explainability to EEG data.
problem Understanding the internal states of complex EEG transformer models.
method Concept Activation Vectors (CAVs) adapted for EEG data, using externally labeled datasets and anatomically defined concepts.
result Both approaches to concept formation yield valuable insights into EEG model representations.
Improved sampling quality with RBM-Flow and D-Flow models.
problem Efficient sampling of complex data distributions using invertible flows.
method Implement RBM-Flow and D-Flow models with discrete latent variables.
result Significant improvement in sampling quality over baseline models.
Extends Gaussian process regression for non-Gaussian data.
problem Inadequate modeling of uncertainty and over-smoothing in non-Gaussian datasets.
method Time-changed Gaussian processes with Lévy processes.
result Improved modeling of heavy-tailed non-Gaussian behaviors.
This study rethinks the latent space in generative modeling, improving performance with less complex models.
problem Determining the optimal latent space for generative models and understanding its impact on model complexity.
method Proposed a new distance metric between latent and data distributions, and a two-stage training strategy called Decoupled Autoencoder (DAE).
result Improves generative performance with less complex models, as shown by comprehensive experiments on various models.