SDE Matching eliminates simulation for training Latent SDEs, achieving similar performance.
problem Training Latent SDEs with adjoint sensitivity methods is computationally expensive and limited.
method SDE Matching, inspired by Score- and Flow Matching, eliminates simulation for training Latent SDEs.
result SDE Matching achieves performance comparable to adjoint sensitivity methods while reducing computational complexity.
Generative models learn latent process to match target distributions.
problem Training flow-matching models with auxiliary stochastic dynamics.
method Introduces latent process generator matching, treating generative state as a deterministic image of a Markov process.
result Learn generator of a stochastic process with same marginal distributions.
Unified framework for self-supervised learning via latent distribution matching.
problem Lack of a unifying theoretical framework for diverse SSL methods.
method Casting SSL as latent distribution matching (LDM): maximizing alignment and uniformity.
result Derives a Bayesian filtering model and proves identifiable latent representations.
LSDM uses unpaired data to match latent space distributions for generative modeling.
problem Generating high-quality images with limited paired data.
method Two-stage approach: latent space learning from paired and unpaired data, followed by joint distribution matching.
result LSDM enhances geometric fidelity in generated outputs and provides theoretical insights into LDMs.
Polynomial-time algorithm matches correlated random graphs with non-vanishing correlation.
problem Matching correlated random graphs with non-vanishing edge correlation.
method Iterative algorithm for polynomial-time recovery of latent matching.
result Algorithm succeeds in recovering latent matching as long as edge correlation is non-vanishing.
FMI uses matching to mimic interventions for causal feature learning.
problem Challenges in causal discovery from observational data.
method Feature Matching Intervention (FMI) using matching to emulate perfect interventions.
result FMI outperforms in identifying causal features from observational data.
A latent-variable model is introduced for text matching, inferring sentence representations by jointly optimizing generative and discriminative objectives. To alleviate typical optimization challenges in latent-variable models for text, we employ deconvolutional networks as the sequence decoder (generator), providing l…
We propose an unsupervised object matching method for relational data, which finds matchings between objects in different relational datasets without correspondence information. For example, the proposed method matches documents in different languages in multi-lingual document-word networks without dictionaries nor ali…
Enhances learning of structured distributions using nonlinear denoising score matching.
problem Learning structured distributions from noisy data.
method Latent Nonlinear Denoising Score Matching (LNDSM) integrating nonlinear dynamics with VAE-based latent score matching.
result LNDSM achieves superior sample quality and variability compared to structure-agnostic methods.
Study sharpens threshold for matching correlated graphs without labels.
problem Matching latent vertex correspondences in correlated random graphs.
method Analyzes information-theoretic limits for correct vertex matching in sub-sampled graphs.
result Establishes a sharp information-theoretic threshold for vertex matching recovery.
Decentralized learning for matching markets with time-varying preferences.
problem Matching between competing agents and supply arms with time-varying preferences.
method Linear contextual bandit framework, learning algorithms to identify latent environment and stable matchings.
result Achieve instance-dependent logarithmic regret, applicable for large markets.
VAELLS learns latent manifold structure to improve VAE model accuracy.
problem VAEs struggle with mismatched latent structure and global structure.
method Integrates learnable manifold model into latent space of VAE.
result Improves model accuracy by matching prior to data manifold structure.
Framework generates causal probabilities from observational data.
problem Generating causal probabilities from observational data.
method Moment-matching graph-networks for causal inference.
result Automated sampling of latent space conditional probability distributions.
The paper provides convergence guarantees for ODE-based generative models using transformers.
problem Theoretical guarantees for ODE-based generative models.
method A pre-trained autoencoder maps inputs to a latent space, and a transformer predicts the velocity field.
result The distribution of samples generated via estimated ODE flow converges to the target distribution in Wasserstein-2 distance.
In many statistical problems, a more coarse-grained model may be suitable for population-level behaviour, whereas a more detailed model is appropriate for accurate modelling of individual behaviour. This raises the question of how to integrate both types of models. Methods such as posterior regularization follow the id…
PFM generates novel samples on data manifolds using pullback geometry.
problem Generating novel samples on complex data manifolds.
method Pullback Flow Matching framework leveraging pullback geometry and isometric learning.
result PFM achieves improved manifold learning and generative performance.
Method evaluates disentanglement in DLVMs, including those not aligned with latent axes.
problem Evaluate disentanglement in DLVMs, especially those not aligned with latent axes.
method Proposes a statistical method to discover generative factors of a dataset.
result Empirically demonstrates the advantage of the method on two datasets.
A new method for generative modeling of discrete data using geometric latent subspaces.
problem Learning generative models for discrete data with statistical dependencies.
method Geometric latent-subspace framework in exponential parameter space of product manifolds of categorical distributions.
result Low-dimensional latent space encodes statistical dependencies and accurately models high-dimensional discrete data.
LOL method simplifies forming linear combinations of latent variables.
problem Lack of general-purpose methods for manipulating latent variables.
method Latent Optimal Linear combinations (LOL) method.
result LOL simplifies creation of expressive low-dimensional representations.
Permutations and matchings are core building blocks in a variety of latent variable models, as they allow us to align, canonicalize, and sort data. Learning in such models is difficult, however, because exact marginalization over these combinatorial objects is intractable. In response, this paper introduces a collectio…
Early stopping improves sample quality in latent diffusion models.
problem Latent diffusion models degrade sample quality with conventional early stopping.
method Analyzed the interaction between latent dimension and stopping time under Gaussian framework.
result Lower-dimensional representations benefit from earlier termination, higher-dimensional spaces require later stopping.
Paper introduces Latent-CLIP for efficient text-image comparison in latent space.
problem Efficiently compare text and images in latent space without costly decoding.
method Trains CLIP model in latent space, uses Latent-CLIP rewards for noise optimization, and guides generation away from harmful content.
result Latent-CLIP matches CLIP performance on text-image classification and harmful content detection.
A new approach to protein language models combines latent space prediction with masked language modeling.
problem Improving protein language models by predicting amino acid identities at masked positions.
method A variant of masked language modeling that predicts latent targets only at masked positions, retaining the MLM cross-entropy.
result The new approach outperforms pure masked language modeling on 11 out of 16 downstream tasks.
New method uses latent variables to estimate treatment effects from single-arm trials.
problem Estimating treatment effects from single-arm trials due to lack of external control groups.
method Latent-variable modeling with amortized variational inference for patient matching and direct effect estimation.
result Improved performance in direct treatment effect estimation and effect estimation via patient matching compared to previous methods.
CPFM integrates dimensionality reduction and reconstruction with flow networks.
problem Learning coupled continuous flows for data and embeddings.
method Coupled flow matching framework with Gromov-Wasserstein objective and dual-conditional flow network.
result CPFM preserves and recovers residual information in latent space.
Deep neural networks with discrete latent variables offer the promise of better symbolic reasoning, and learning abstractions that are more useful to new tasks. There has been a surge in interest in discrete latent variable models, however, despite several recent improvements, the training of discrete latent variable m…
Humans are able to imagine a person's voice from the person's appearance and imagine the person's appearance from his/her voice. In this paper, we make the first attempt to develop a method that can convert speech into a voice that matches an input face image and generate a face image that matches the voice of the inpu…
Algorithm matches vertices of correlated Erdős-Rényi graphs efficiently.
problem Graph matching in correlated Erdős-Rényi graphs.
method Counting chandeliers to extract graph correlation.
result Correctly matches all but a vanishing fraction of vertices with high probability.
Efficiently matches random graphs with inhomogeneous edge probabilities.
problem Matching latent vertex correspondence between two correlated random graphs with inhomogeneous edge probabilities.
method Inspired by Ding et al. (2021), an efficient matching algorithm is developed with conditions on minimal average degree and minimal correlation.
result An efficient matching algorithm is obtained as long as the minimal average degree is at least Ω(log2n) and the minimal correlation is at least 1−O(log−2n). A new method for inferring latent states in Markov jump processes.
problem Intractable exact inference for Markov jump processes.
method Entropic matching within expectation propagation.
result Superior performance in approximating the mean of the posterior process.
Modern generative models are usually designed to match target distributions directly in the data space, where the intrinsic dimension of data can be much lower than the ambient dimension. We argue that this discrepancy may contribute to the difficulties in training generative models. We therefore propose to map both th…
Fast approximate inference for non-Gaussian data.
problem Efficient inference for non-Gaussian data.
method Laplace Matching for fast approximate inference in latent Gaussian models.
result Achieves high approximation quality with low computational cost.
This paper tackles matching two complete graphs with correlated edge weights in geometric models.
problem Matching two complete graphs with edge weights correlated through latent geometries.
method Derives an approximate maximum likelihood estimator for recovering hidden vertex correspondence.
result The estimator provably achieves perfect recovery under certain noise conditions.
Synthetic tabular data synthesis models balance utility and risk.
problem Generating synthetic tabular data for regulated domains.
method Latent flow models with various learning targets, paths, and sampling methods.
result Velocity and posterior matching objectives yield higher utility, while score and noise matching achieve lower risk.
We analyze a new spectral graph matching algorithm, GRAph Matching by Pairwise eigen-Alignments (GRAMPA), for recovering the latent vertex correspondence between two unlabeled, edge-correlated weighted graphs. Extending the exact recovery guarantees established in the companion paper for Gaussian weights, in this work,…
A new metric assesses latent variable models using data and model moments.
problem Difficulty in assessing the quality of unsupervised learning models.
method A moment-matching metric using matrix norms to compare data and model moments.
result The proposed metric is faster and has less variance than alternative methods.
Proposes a non-adversarial method for distribution matching.
problem Stability and optimization challenges in adversarial matching methods.
method Non-adversarial VAE-based matching method with alignment upper bounds.
result Demonstrates applicability of non-adversarial matching methods without modifying original architectures.
We present a neural network architecture based upon the Autoencoder (AE) and Generative Adversarial Network (GAN) that promotes a convex latent distribution by training adversarially on latent space interpolations. By using an AE as both the generator and discriminator of a GAN, we pass a pixel-wise error function acro…
Proposes a VAE with a discrete bottleneck for better text generation.
problem VAEs struggle with latent variable auto-regressive decoding in text generation.
method Introduces a discretized bottleneck to enforce latent feature matching in a compact space.
result Demonstrates improved text generation capabilities across various tasks.
Generalizes adversarial learning for better latent variable inference in GANs.
problem Improving latent variable inference in GANs for diverse applications.
method Adversarial learning with multiple feedback layers, self-supervision, and auxiliary tasks.
result Achieves global optimum matching multiple joint distributions.
We introduce a novel discriminative latent variable model for bilingual lexicon induction. Our model combines the bipartite matching dictionary prior of Haghighi et al. (2008) with a representation-based approach (Artetxe et al., 2017). To train the model, we derive an efficient Viterbi EM algorithm. We provide empiric…
Paper presents variational estimates for EBLVMs without structural assumptions.
problem Challenges in learning and evaluating EBLVMs due to intractable true posteriors and partition functions.
method Variational estimates of the score function and its gradient (VaES and VaGES) in a general EBLVM.
result The estimates can be applied to KSD and SM-based methods to learn EBLVMs and estimate Fisher divergence.
PLIs improve classifier performance by fine-tuning latent representations.
problem Difficult interpretation of high-dimensional latent representations in neural networks.
method Back-propagation of manual changes to low-dimensional embeddings using t-distributed stochastic neighbourhood embeddings.
result Manual separation of class clusters in latent space enhances classifier performance.
We consider moment matching techniques for estimation in Latent Dirichlet Allocation (LDA). By drawing explicit links between LDA and discrete versions of independent component analysis (ICA), we first derive a new set of cumulant-based tensors, with an improved sample complexity. Moreover, we reuse standard ICA techni…
SPLICE generates accurate time-series imputations with reliable prediction intervals.
problem Lack of reliability guarantees in time-series imputation models.
method Modular framework combining latent generative imputation with distribution-free prediction intervals.
result SPLICE achieves lowest mean Load-only MSE and best CRPS on various datasets.
Deep generative networks have been widely used for learning mappings from a low-dimensional latent space to a high-dimensional data space. In many cases, data transformations are defined by linear paths in this latent space. However, the Euclidean structure of the latent space may be a poor match for the underlying lat…
Increasingly many real world tasks involve data in multiple modalities or views. This has motivated the development of many effective algorithms for learning a common latent space to relate multiple domains. However, most existing cross-view learning algorithms assume access to paired data for training. Their applicabi…
DPA preserves data distribution in reduced dimensions.
problem Loss of data distribution in dimension reduction.
method DPA combines encoder and decoder to match data distribution.
result DPA successfully reconstructs data distribution.