New autoencoder generates better images by estimating latent distribution.
problem Improving image generation quality.
method Directly estimate latent distribution using latent density estimator.
result Generative model produces higher quality images.
The study examines issues with latent distributions in generative models and proposes using Cauchy distribution.
problem Issues with latent distributions causing mismatch in sampled regions during linear interpolations.
method Proposed using multidimensional Cauchy distribution and two methods for creating non-linear interpolations.
result Linear interpolations may generate unrealistic data due to the Central Limit Theorem, and Cauchy distribution mitigates this issue.
Improves NF for complex data distributions with multiple modes.
problem Difficulty in handling data distributions with multiple isolated modes.
method Proposes a new framework using variational latent representation to improve NF.
result Significantly more powerful for generating data distributions with multiple modes.
New method improves robustness in partially observable domains by training against latent distribution shifts.
problem Challenges in robustness under latent distribution shift in partially observable reinforcement learning.
method Formalizes adversarial latent-initial-state POMDP, proves minimax principle, derives best-response inequalities.
result Reduces robustness gaps from 10.3 to 3.1 shots with targeted exposure to shifted latent distributions.
Unified framework for self-supervised learning via latent distribution matching.
problem Lack of a unifying theoretical framework for diverse SSL methods.
method Casting SSL as latent distribution matching (LDM): maximizing alignment and uniformity.
result Derives a Bayesian filtering model and proves identifiable latent representations.
A new method uses optimal transport to transform latent space distributions without solving a hard Min-Max problem.
problem Latent space data distribution collapse and loss of manifold structure in generative models.
method Proposes a GAN-like method to solve a minimization problem for optimal transport between a simple distribution and a latent-space data distribution.
result Experimental results show that the proposed method can handle multi-cluster distributions and is effective on MNIST and CelebA datasets.
Framework LiLY recovers latent causal variables from time-series data under distribution shifts.
problem Learning and correcting models under unknown distribution shifts in time-series data.
method LiLY framework that recovers latent causal variables and identifies their relations from temporal data under different distribution shifts.
result The framework reliably identifies time-delayed latent causal influences from observed variables under different distribution changes.
LLMs encode latent topic distributions, suggesting Bayesian inference.
problem Capturing topic structure from large language models.
method Connecting LLM optimization to implicit Bayesian inference and de Finetti's theorem.
result LLMs recover latent topic distributions, matching LDA-generated topics.
Proposes a method to ensure latent space of AutoEncoders is Gaussian without randomness.
problem Ensuring latent space of AutoEncoders is Gaussian without randomness.
method Directly optimize agreement between empirical distribution function and desired CDF for chosen properties.
result Ensures latent space of AutoEncoders is Gaussian without randomness.
Transformers encode latent distributions in text, improving performance in out-of-distribution cases.
problem What should embeddings from language models represent?
method Connecting autoregressive prediction to sufficient statistics, identifying three settings.
result Transformers encode latent generating distributions, improving performance.
Improves autoencoder reconstruction quality by approximating latent space with adversarial learning.
problem Ambiguity in autoencoder reconstructions and difficulty in matching true distribution.
method Adversarially Approximated Autoencoder (AAAE) using GAN for flexible latent space approximation.
result Generates more faithful reconstructions and maintains latent manifold structure.
AI learns to classify and represent univariate distributions in a 2D latent space.
problem Classifying and representing univariate empirical distributions.
method Unsupervised beta variational autoencoder (beta-VAE) to separate and represent distributions in a 2D latent space.
result The latent space representation separates distributions of different shapes while overlapping similar ones.
Proposes SQUAD for better predictive uncertainty in deep latent models.
problem Intractable inference in deep latent variable models lead to overconfident predictions.
method Introduces Stochastic Quantized Activation Distributions (SQUAD) for flexible yet tractable latent variable distributions.
result The model provides competitive quality predictive uncertainty and learns non-linearities.
Study on reliability of latent reuse in diffusion models under distribution shift.
problem When can latent spaces from a source dataset be reused for a target dataset with different distributions?
method Considered a source-target setting with approximately low-dimensional datasets near different subspaces. Analyzed the target-domain score error due to principal-angle misalignment and target ambient noise.
result Latent reuse is reliable only if the source and target subspaces are close and the target ambient noise is not too amplified.
LSDM uses unpaired data to match latent space distributions for generative modeling.
problem Generating high-quality images with limited paired data.
method Two-stage approach: latent space learning from paired and unpaired data, followed by joint distribution matching.
result LSDM enhances geometric fidelity in generated outputs and provides theoretical insights into LDMs.
Proposes a new approach to MSDA by introducing latent covariate shift to handle varying label distributions.
problem Challenges of conventional MSDA approaches in real-world settings where label distributions vary across domains.
method Introduces latent covariate shift (LCS) and a causal generative model with latent noises, latent content variable, and latent style variable.
result Identifies latent content variable up to block identifiability, enabling more nuanced label distribution recovery.
New framework TDRL identifies latent causal variables from sequential data.
problem Identify latent causal variables from sequential data.
method Proposes TDRL framework to recover time-delayed latent causal variables and identify their relations from measured sequential data.
result Identifies latent causal variables reliably from sequential data.
Method evaluates disentanglement in DLVMs, including those not aligned with latent axes.
problem Evaluate disentanglement in DLVMs, especially those not aligned with latent axes.
method Proposes a statistical method to discover generative factors of a dataset.
result Empirically demonstrates the advantage of the method on two datasets.
UT module refines VAE latent space, improving disentanglement and interpretability.
problem Irregular latent distributions cause posterior collapse and misalignment in VAEs.
method UT module uses G-KDE clustering, GM modeling, and PIT to transform latent space into uniform distribution.
result UT module enhances disentanglement and interpretability of latent representations.
Unified framework for disentangling latent variables.
problem Unidentifiability of deep latent-variable models.
method Variational autoencoders and nonlinear ICA, with a factorized prior conditioned on an observed variable.
result Identification of true joint distribution over observed and latent variables is possible up to simple transformations.
Proxy methods adapt to distribution shifts without explicitly modeling latent confounders.
problem Adapting to distribution shifts under latent variable confounding.
method Proximal causal learning, two-stage kernel estimation.
result Proxy methods outperform other methods in adapting to complex distribution shifts.
This study improves GANs by learning latent distributions and pushforward maps.
problem Improving the performance of GANs with optimal transport metrics.
method Focuses on the interplay between latent distribution and generator complexity.
result Learning latent distributions and pushforward maps can significantly reduce sample complexity.
Generative model improves latent space convexity through adversarial training on interpolations.
problem Improving latent space convexity in generative models.
method Adversarial training on latent space interpolations within an AE-GAN architecture.
result Convex latent distribution of generated images, preserving realistic resemblances.
Adaptive framework for learning latent space dimensions in GANs.
problem Inadequate latent space dimensions lead to poor generative models for complex data.
method Proposes a novel framework (LWGAN) that adaptively learns latent dimensions of data manifolds.
result Proves that the estimated intrinsic dimension is a consistent estimate of the true data manifold dimension.
We focus on generative autoencoders, such as variational or adversarial autoencoders, which jointly learn a generative model alongside an inference model. Generative autoencoders are those which are trained to softly enforce a prior on the latent distribution learned by the inference model. We call the distribution to …
Contrastive learning recovers latent distributions for ambiguous inputs, including aleatoric uncertainty.
problem Real-world observations often have inherent ambiguities, making the true posterior probabilistic with heteroscedastic uncertainty.
method Extended InfoNCE objective and encoders to predict latent distributions, proving they recover the correct posteriors, including aleatoric uncertainty.
result Contrastive learning encoders can recover the correct posteriors of data-generating processes, including aleatoric uncertainty, up to a rotation of the latent space.
Discrete-AIR model identifies objects in images with interpretable latent codes.
problem Identifying objects in images without labeled data.
method Recurrent Auto-Encoder with structured latent distributions for discrete, continuous, and spatial attention.
result Discrete-AIR model uses minimal latent variables for efficient inference.
Graphical models are commonly used tools for modeling multivariate random variables. While there exist many convenient multivariate distributions such as Gaussian distribution for continuous data, mixed data with the presence of discrete variables or a combination of both continuous and discrete variables poses new cha…
Unsupervised matching method for relational data without alignment info.
problem Matching objects in different relational datasets without correspondence info.
method Model latent vectors, estimate by likelihood, project onto shared space.
result Preserves structural information in latent vectors across datasets.
Enhances learning of structured distributions using nonlinear denoising score matching.
problem Learning structured distributions from noisy data.
method Latent Nonlinear Denoising Score Matching (LNDSM) integrating nonlinear dynamics with VAE-based latent score matching.
result LNDSM achieves superior sample quality and variability compared to structure-agnostic methods.
Auto-decoder synthesizes graphs from latent codes.
problem Creating new graph structures from specified distributions.
method Generative model learns latent codes from empirical distribution. Self-attention identifies likely connectivity patterns. Graph-based normalizing flows sample latent codes.
result Model outperforms state of the art by 1.5x in accuracy and 2x in speed.
New approach models noisy labels without assuming a unique true label.
problem Aggregating noisy labels in ambiguous scenarios.
method Introduces a novel latent distribution assumption.
result The new assumption is more suitable for tasks with ambiguous labels.
LSI enables joint learning of latent variables and generative models.
problem Joint optimization of latent variables and generative models.
method Developed a principled ELBO objective in continuous time for joint learning.
result LSI learns effective latent representations and generative transformations.
We stabilize the Kumaraswamy distribution for efficient sampling and differentiation.
problem Numerical instabilities in the Kumaraswamy distribution's inverse CDF and log-pdf.
method Identified and resolved numerical issues, introduced a stabilized KS distribution.
result Stabilized Kumaraswamy distribution supports efficient sampling and differentiation.
Paper introduces a new generative learning model using Schrödinger bridge diffusion in latent space.
problem Learning distributions from divergent data distributions.
method Pre-training with large-scale models, Schrödinger bridge diffusion model in latent space.
result Effective control of second-order Wasserstein distance between generated and target distributions.
SLOGAN improves GANs' conditional generation by balancing latent attribute distributions.
problem Imbalanced latent attribute distributions in GANs.
method Stein latent optimization with a Gaussian mixture prior and contrastive loss.
result SLOGAN achieves state-of-the-art unsupervised conditional generation performance.
New findings suggest latent regularization is unnecessary for high-quality image generation.
problem Improving image generation quality without latent regularization.
method Investigated the effect of latent regularization on image generation using learned priors.
result In the case of a sufficiently expressive prior, latent regularization is not necessary and may harm image quality.
A novel MCMC method clusters data faster and more accurately.
problem Efficiently clustering large datasets with unknown number of clusters.
method Master/Worker architecture for distributed MCMC inference.
result Significant improvement in clustering accuracy and speed.
Generative models learn latent process to match target distributions.
problem Training flow-matching models with auxiliary stochastic dynamics.
method Introduces latent process generator matching, treating generative state as a deterministic image of a Markov process.
result Learn generator of a stochastic process with same marginal distributions.
New method uses Fisher-Rao metric for non-Gaussian decoders.
problem Existing latent space geometry theory only works for Gaussian decoders.
method Pull back Fisher-Rao metric to latent space for non-Gaussian decoders.
result Achieves meaningful latent geometries for various non-Gaussian decoders.
Proposes a multi-view VAE for imputing missing data from correlated sources.
problem Imputing missing data from multi-view sources with latent space correlation.
method Enforces a joint prior with latent space correlation between VAEs trained on each view.
result More strongly correlated latent spaces are uncovered, enabling effective imputation.
DPA preserves data distribution in reduced dimensions.
problem Loss of data distribution in dimension reduction.
method DPA combines encoder and decoder to match data distribution.
result DPA successfully reconstructs data distribution.
PGA maps data and target distributions to a latent space for better generative model training.
problem Training generative models is difficult due to mismatch between data and target distributions.
method Train generator to match target distribution in latent space using autoencoder.
result PGA achieves state-of-the-art FID scores on CIFAR-10 and CelebA.
New LVMs optimize any exponential family distribution without specific assumptions.
problem Optimizing latent variable models with non-Gaussian observables.
method Generic optimization using EM approach for exponential family distributions.
result Concise parameter update equations applicable to various data types.
PRGDS models count tensors with sparsity and burstiness.
problem Modeling sequential count data with sparsity and burstiness.
method Poisson-randomized gamma dynamical system with alternating Poisson and gamma latent states.
result Sparse PRGDS often outperforms other models in predicting count data.
Generative models such as Variational Auto Encoders (VAEs) and Generative Adversarial Networks (GANs) are typically trained for a fixed prior distribution in the latent space, such as uniform or Gaussian. After a trained model is obtained, one can sample the Generator in various forms for exploration and understanding,…
Paper proposes a fast method for learning deep latent variable models.
problem Learning deep generative models with hierarchical latent variables.
method Noise initialized short run MCMC with variational optimization of step size.
result The method outperforms VAE in reconstruction and synthesis quality.
We characterize distributional equivalence in latent-variable models with cycles.
problem Lack of an equivalence characterization for latent-variable causal models with cycles.
method Established graphical criterion for distributional equivalence and developed edge rank constraints.
result First equivalence characterization without structural assumptions for latent-variable models with cycles.