NLGS optimizes latent geometry for better model performance.
problem Improving machine learning model performance by aligning latent space geometry with data structure.
method NLGS uses product manifolds with Gromov-Hausdorff distance for latent geometry search.
result NLGS finds optimal latent geometry with query-efficient Bayesian optimization.
New method uses Fisher-Rao metric for non-Gaussian decoders.
problem Existing latent space geometry theory only works for Gaussian decoders.
method Pull back Fisher-Rao metric to latent space for non-Gaussian decoders.
result Achieves meaningful latent geometries for various non-Gaussian decoders.
This research examines the geometry of latent spaces in push-forward generative models.
problem Tendency of deep generative models to output samples outside target distribution support.
method Geometric measure theory and truncation method to enforce simplicial cluster structure.
result Proves sufficient condition for optimality in latent space geometry.
Paper analyzes latent space geometry in generative models using Fisher information.
problem Understanding the structure of latent spaces in generative models.
method Reconstructs Fisher information metric from generated samples and posterior distribution.
result Reveals fractal structure and abrupt changes in Fisher metric at phase boundaries.
A new method for generative modeling of discrete data using geometric latent subspaces.
problem Learning generative models for discrete data with statistical dependencies.
method Geometric latent-subspace framework in exponential parameter space of product manifolds of categorical distributions.
result Low-dimensional latent space encodes statistical dependencies and accurately models high-dimensional discrete data.
Ensemble decoders to capture latent space topology in deep generative models.
problem Topological mismatch between latent space geometry and data manifolds.
method Using ensembles of decoders to compute geodesics on the expected manifold.
result Ensemble approach provides a simple and reliable way to capture model uncertainty in latent space.
Proposes a geometry-aware VAE for better latent space modeling.
problem Lack of meaningful latent space structure in VAEs for small datasets.
method Introduces a Riemannian Hamiltonian VAE with a learned metric.
result Improves latent space structure leading to better interpolations and data generation.
PFM generates novel samples on data manifolds using pullback geometry.
problem Generating novel samples on complex data manifolds.
method Pullback Flow Matching framework leveraging pullback geometry and isometric learning.
result PFM achieves improved manifold learning and generative performance.
Latent representations are the essence of deep generative models and determine their usefulness and power. For latent representations to be useful as generative concept representations, their latent space must support latent space interpolation, attribute vectors and concept vectors, among other things. We investigate …
Generative autoencoders offer a promising approach for controllable text generation by leveraging their latent sentence representations. However, current models struggle to maintain coherent latent spaces required to perform meaningful text manipulations via latent vector operations. Specifically, we demonstrate by exa…
A new method learns manifold-valued latents without an encoder.
problem Distorting data with intrinsic non-Euclidean structure.
method Riemannian generative decoder that learns latents directly.
result Learned representations respect the prescribed geometry and capture intrinsic non-Euclidean structure.
A new model encodes distances and topology in latent variables.
problem Modeling dissimilarity data with latent variables and invariances.
method Isometric Gaussian Process Latent Variable Model using Riemannian geometry and variational inference.
result The model can encode invariances in learned manifolds.
Online distributional prediction with latent cluster geometry
problem Predicting the full data-generating distribution in non-stationary streams
method Representing candidate laws as latent cluster geometry and using Gibbs quasi-posterior
result Achieving sublinear cumulative Wasserstein regret under bounded support and stable latent geometry
Study on reliability of latent reuse in diffusion models under distribution shift.
problem When can latent spaces from a source dataset be reused for a target dataset with different distributions?
method Considered a source-target setting with approximately low-dimensional datasets near different subspaces. Analyzed the target-domain score error due to principal-angle misalignment and target ambient noise.
result Latent reuse is reliable only if the source and target subspaces are close and the target ambient noise is not too amplified.
This paper improves probabilistic latent models on hyperbolic spaces.
problem Uncertainty in predictions due to geodesics crossing low-data regions.
method Augmenting hyperbolic manifold with a pullback metric for probabilistic pullback metrics.
result Geodesics on pullback metric respect both geometry and data distribution, reducing uncertainty.
The Fisher information metric is an important foundation of information geometry, wherein it allows us to approximate the local geometry of a probability distribution. Recurrent neural networks such as the Sequence-to-Sequence (Seq2Seq) networks that have lately been used to yield state-of-the-art performance on speech…
Hyperbolic geometry reveals financial network structure and systemic importance.
problem Understanding the structure and importance of financial networks.
method Data from European banking stress tests, hyperbolic geometry analysis.
result Latent dimensions of `popularity' and `similarity' are strongly associated with systemic importance.
SPLICE method disentangles shared and private latent variables from multi-view data.
problem Lack of methods to characterize nonlinear relationships and preserve geometric information in multi-view data.
method Neural network-based approach to infer disentangled, interpretable representations of shared and private latent variables.
result SPLICE yields more interpretable representations by preserving geometry and is more robust to incorrect latent dimensionality.
Deep generative models have made tremendous advances in image and signal representation learning and generation. These models employ the full Euclidean space or a bounded subset as the latent space, whose flat geometry, however, is often too simplistic to meaningfully reflect the manifold structure of the data. In this…
Deep generative models are tremendously successful in learning low-dimensional latent representations that well-describe the data. These representations, however, tend to much distort relationships between points, i.e. pairwise distances tend to not reflect semantic similarities well. This renders unsupervised tasks, s…
A new model integrates covariates with grade of membership analysis for better latent structure recovery.
problem Improving latent structure recovery in multivariate categorical data analysis.
method Covariate-assisted grade of membership model exploiting shared low-rank simplex geometry.
result Auxiliary covariates can provably improve latent structure recovery, leading to faster convergence rates.
A new method integrates autoencoders with geometry regularization for manifold learning.
problem Extracting simplified low-dimensional representations that capture intrinsic geometry in data.
method Integrates autoencoders with a geometric regularization term based on diffusion potential distances.
result The method preserves intrinsic structure, enables out-of-sample extension, and faithful reconstruction.
Bayesian models for networks are often misspecified, leading to overconfident inference.
problem Real-world networks violate assumptions of geometry and link function in latent space models.
method Proposes a generalized posterior framework for random geometric graphs, using Link-Sequential R-SafeBayes to adaptively tune posterior regularization.
result Improved calibration and better link prediction performance demonstrated on synthetic and real-world networks.
SEMASIA provides a large dataset of latent representations for model comparison.
problem Difficulty in comparing semantic structures across different neural network models.
method Collection of latent representations from 1700 pretrained models across various benchmarks.
result Consistent semantic organization across models and datasets.
Enhances graph modeling with hyperbolic geometry and variational inference.
problem Challenges in modeling relational data with complex dependencies.
method Semi-implicit hierarchical variational Bayes with Poincaré embedding and mutual information regularization.
result Improves graph representation quality and flexibility in edge prediction and node classification.
New framework models neural systems with random architecture on manifolds.
problem Complex, uncertain systems with non-Gaussian outputs.
method Latent random field on compact manifold generates neural architecture and weights.
result Synthetic neural systems can produce stochastic outputs for deterministic inputs.
The paper analyzes the latent geometry of generative diffusion models.
problem The manifold overfitting phenomenon in generative models.
method Statistical physics approach to analyze the spectrum of eigenvalues and singular values of the Jacobian of the score function.
result Three distinct qualitative phases during the generative process: trivial, manifold coverage, and consolidation phases.
Paper proposes a probabilistic alignment method for domain adaptation.
problem Latent distribution mismatch and miscalibrated uncertainty in adapting large-scale models.
method Bayesian latent transport framework with PAC-Bayesian regularization.
result Reduction in latent manifold discrepancy and improved uncertainty calibration.
We investigate the geometrical structure of probabilistic generative dimensionality reduction models using the tools of Riemannian geometry. We explicitly define a distribution over the natural metric given by the models. We provide the necessary algorithms to compute expected metric tensors where the distribution over…
GeoFunFlow tackles inverse problems on complex geometries with efficient learning.
problem Challenges in inverse problems governed by PDEs, especially on irregular geometries.
method Combines geometric function autoencoder and latent diffusion model trained via rectified flow.
result Achieves state-of-the-art reconstruction accuracy and efficient inference.
Bayesian modelling of dynamic systems must achieve a compromise between providing a complete mechanistic specification of the process while retaining the flexibility to handle those situations in which data is sparse relative to model complexity, or a full specification is hard to motivate. Latent force models achieve …
Affinity propagation is one of the most effective unsupervised pattern recognition algorithms for data clustering in high-dimensional feature space. However, the numerous attempts to test its performance for community detection in complex networks have been attaining results very far from the state of the art methods s…
A new tree-Wasserstein distance for high-dimensional data with latent feature hierarchy.
problem Finding meaningful distances between high-dimensional data samples with latent feature hierarchy.
method Proposes a new tree-Wasserstein distance (TWD) for high-dimensional data with a latent feature hierarchy, using diffusion geometry and tree decoding.
result The proposed TWD effectively recovers the latent feature hierarchy and is efficient and scalable.
Generative models use Riemannian manifolds to improve latent space interpretation.
problem Generative models often bias latent space interpretations.
method Use Riemannian manifolds to define latent space paths that respect ambient geometry.
result Improves interpretability of learned representations for both stochastic and deterministic generators.
Paper calculates the exact error of LDA models.
problem Bayesian generalization error in Latent Dirichlet Allocation (LDA).
method Theoretical analysis of learning coefficient using algebraic geometry.
result Exact asymptotic form of LDA's generalization error.
The paper predicts responses on out-of-sample nodes using latent positions on unknown curves.
problem Predicting responses on out-of-sample nodes with latent positions on unknown curves.
method Manifold learning and graph embedding technique using latent positions.
result Convergence guarantees for predicting responses on out-of-sample nodes.
New findings show disentangled latent representations are not enough for robust compositional generalization.
problem Deep learning models struggle with compositional generalization, especially in out-of-distribution samples.
method Investigated a 2D Gaussian generation task with fully disentangled inputs, then forced disentangled latent representations into full-dimensional output space.
result Forcing disentangled latent representations into full-dimensional output space enables robust compositional generalization.
It can be difficult to tell whether a trained generative model has learned to generate novel examples or has simply memorized a specific set of outputs. In published work, it is common to attempt to address this visually, for example by displaying a generated example and its nearest neighbor(s) in the training set (in,…
Cryo-em images are found to be low-dimensional.
problem Understanding the geometric structure of cryo-em data.
method Applied manifold learning techniques to CryoSBI representations.
result Cryo-em data inherently populate low-dimensional manifolds.
Conformal Autoencoders infer intrinsic dimensionality and impose invariance.
problem Detecting intrinsic dimensionality and imposing invariance in nonlinear manifold data.
method Imposing orthogonality conditions on latent variables to infer intrinsic dimensionality and build coordinate invariance.
result The method can infer intrinsic dimensionality and build coordinate invariance on submanifolds.
A basic problem in machine learning is to find a mapping f from a low dimensional latent space Y to a high dimensional observation space X. Modern tools such as deep neural networks are capable to represent general non-linear mappings. A learner can easily find a mapping which perfectly fits a…
A novel GPUM constructs Gaussian Processes for unknown manifolds with probabilistic metrics.
problem High-dimensional data on unknown manifolds with non-Euclidean geometry.
method Bayesian Gaussian Processes latent variable models (BGPLVM), Riemannian geometry, probabilistic metric tensor, Brownian Motion.
result GPUM provides more accurate predictions on unknown manifolds compared to traditional methods.
We develop VAE-DLM for dynamics with geometric flows in latent space.
problem Learning latent geometric properties for dynamics in high-dimensional data.
method Riemannian approaches to VAEs with a geometric flow in latent space, reformulating ELBO loss.
result Improved performance and robust learning for external dynamics, reducing OOD error.
TALBO optimizes latent spaces for evolving design objectives.
problem Temporal drift in design objectives.
method GP-prior variational autoencoder for time-varying latent space.
result Consistently outperforms LSBO baselines across varying drift speeds and objectives.
In most cases deep learning architectures are trained disregarding the amount of operations and energy consumption. However, some applications, like embedded systems, can be resource-constrained during inference. A popular approach to reduce the size of a deep learning architecture consists in distilling knowledge from…
I-BBS identifies latent sub-manifolds from distance matrices, robust to noise.
problem Identifying latent sub-manifolds from distance matrices in high-dimensional spaces.
method Coordinate-free inference using random distance matrix theory and generative noise models.
result Recovering latent geometry from integer-stable signatures of eigenvalues.
This paper improves spectral embedding for multipartite networks, revealing latent subspaces and providing consistent node representations.
problem Improving spectral embedding for multipartite networks to better represent node types.
method Developed a follow-on step to spectral embedding that recovers node representations in their intrinsic rather than ambient dimension, proving consistency under a specific model.
result Node representations in multipartite networks lie near type-specific subspaces, and the proposed method recovers these representations consistently.
In this paper we explore the richness of information captured by the latent space of a vision-based generative model. The model combines unsupervised generative learning with a task-based performance predictor to learn and to exploit task-relevant object affordances given visual observations from a reaching task, invol…