New method trains normalizing flows using entropy-regularized transport.
problem Training continuous normalizing flows efficiently.
method Formulates flows as gradients of scalar potentials, training only these potentials.
result Trains normalizing flows without explicit flow computation during training.
Paper introduces Categorical Normalizing Flows for better handling of categorical data.
problem Limited application of normalizing flows on categorical data due to lack of intrinsic order.
method Categorical Normalizing Flows use continuous transformations to model latent relations in categorical data, optimizing both continuous representation and model likelihood.
result GraphCNF, a permutation-invariant generative model, outperforms state-of-the-art on molecule generation.
Defines mass for non-smooth hyperbolic spaces using a modified flow.
problem Defining mass for non-smooth, asymptotically hyperbolic spaces.
method Normalized Ricci-DeTurck flow with scalar curvature lower bound.
result Mass function well-defined for continuous metrics.
New path-gradient estimator for continuous normalizing flows.
problem Limitation of simple Gaussian variational distributions in complex applications.
method Proposed a path-gradient estimator for continuous normalizing flows.
result Empirical evidence of superior performance of the new estimator.
Riemannian flows model curved spaces better.
problem Normalizing flows misspecified on curved spaces.
method Riemannian continuous normalizing flows using ODE solutions.
result Improves modeling on spheres, torii, and hyperbolic spaces.
Study on Gaussian interpolation flows for generative modeling.
problem Theoretical properties and regularizing effect of Gaussian denoising in continuous normalizing flows.
method Unified framework of Gaussian interpolation flow, Lipschitz regularity, existence and uniqueness of flow, stability analysis.
result Established theoretical properties of Gaussian interpolation flows, including Lipschitz continuity and existence of flow.
Proves strong solutions for graphical Brakke flows with L2 normal velocity.
problem Proving strong solutions for graphical Brakke flows with specific velocity conditions.
method Combining L2 normal velocity with parabolic regularity theory. result Graphical Brakke flows with forcing term in Lp,q and C0,α are strong and classical solutions. We show that the isoperimetric profile hg(t)(ξ) of a compact Riemannian manifold (M,g) is jointly continuous when metrics g(t) vary continuously. We also show that, when M is a compact surface and g(t) evolves under normalized Ricci flow, hg(t)2(ξ) is uniform Lipschitz continuous and hence $h_{g(t)}(…
CRAFT improves on existing methods for sampling complex distributions.
problem Sampling from complex probability distributions.
method Combines SMC with variational inference using normalizing flows.
result Improves on Annealed Flow Transport Monte Carlo and MCMC-based Stochastic Normalizing Flows.
This paper uses normalizing flows to approximate transport maps between densities.
problem Approximating transport maps between given densities.
method Construct time-dependent controls using normalizing flows.
result Provides bounds on the number of switches for piecewise constant approximations.
Neural Manifold ODEs improve manifold data modeling.
problem Adapting deep generative models to non-Euclidean spaces.
method Introducing Neural Manifold ODEs for manifold generalization and continuous probability computation.
result Improves density estimation and downstream tasks on arbitrary manifolds.
EWFM trains continuous flows with only energy evaluations, improving sample quality with fewer computations.
problem Efficiently sampling from complex, high-dimensional Boltzmann distributions using only energy evaluations.
method Energy-Weighted Flow Matching (EWFM) using importance sampling and iterative/annealed training.
result Improved sample quality with up to 3 orders of magnitude fewer energy evaluations compared to existing methods.
Hybrid model avoids forgetting across tasks and classes.
problem Challenges of continual learning in modern deep learning.
method Hybrid generative-discriminative approach using normalizing flows.
result Strong performance on various continual learning benchmarks.
Extends normalizing flows to arbitrary smooth manifolds.
problem Current normalizing flows are limited to basic geometries and cannot handle complex real-world data.
method Uses Neural ODEs and geometric control theory to extend flows to arbitrary smooth manifolds.
result Demonstrates scalable unbiased estimator for divergence in generalized setting.
Flow based models such as Real NVP are an extremely powerful approach to density estimation. However, existing flow based models are restricted to transforming continuous densities over a continuous input space into similarly continuous distributions over continuous latent variables. This makes them poorly suited for m…
OT-Flow uses optimal transport to improve CNFs for faster and more accurate density estimation.
problem Computational challenges in continuous normalizing flows.
method OT-Flow leverages optimal transport to regularize CNFs and uses exact trace computation.
result OT-Flow achieves competitive performance with one-fourth the number of weights and significant speedups.
A new method for normalizing flows using stochastic interpolants simplifies likelihood estimation and improves efficiency.
problem Efficient and scalable likelihood estimation for complex probability distributions.
method Inference of velocity field from time-dependent density interpolating between base and target densities.
result Simplified quadratic loss for velocity estimation, leading to faster and more efficient training.
A new method learns quantization boundaries in continuous space using tessellation.
problem Mapping between discrete and continuous distributions is difficult.
method Constructs normalizing flows on convex polytopes with exact likelihood evaluations.
result Improves likelihood evaluation and quantization learning across various data modalities.
Bayesian Gaussian Process ODEs enhanced with normalizing flows for improved flexibility and accuracy.
problem Limitations of standard Gaussian Process ODEs in modeling complex scenarios.
method Introducing normalizing flows to reparameterize the ODE vector field, developing a data-driven variational learning algorithm.
result Improved accuracy and uncertainty estimates for Bayesian Gaussian Process ODEs.
Continuity of solutions to complex Monge-Ampère equations on compact Kähler spaces proved.
problem Continuity of solutions to complex Monge-Ampère equations on compact Kähler spaces.
method Analyzing bounded solutions on reduced, locally irreducible compact Kähler spaces.
result Proves continuity of solutions, affirming conjectures and solving open problems.
Paper proposes a new generative model for discrete distributions using flows on submanifolds.
problem Discretization issues and complex statistical dependencies in discrete data.
method Continuous normalizing flows on factorizing discrete measures, geodesic flow matching.
result Efficient training and broad applicability demonstrated through experiments.
Paper proposes efficient training for normalizing flows in Boltzmann generators.
problem Training normalizing flows for Boltzmann generators is computationally challenging and unstable.
method Regression Training of Normalizing Flows (RegFlow) using ℓ2-regression. result RegFlow enables efficient and stable training of normalizing flows for Boltzmann generators.
CLPF models continuous time-series data with improved representational power and variational approximations.
problem Fitting continuous time-series data with existing models faces challenges in representational power and variational quality.
method CLPF uses a time-dependent normalizing flow driven by a stochastic differential equation to decode continuous latent processes into continuous observables. Maximum likelihood optimization is achieved through a novel variational posterior process.
result CLPF outperforms state-of-the-art baselines on synthetic and real-world time-series data.
E-NFs generate molecules and their positions while preserving Euclidean symmetries.
problem Generating molecules with their positions while preserving Euclidean symmetries.
method Integrating E(n) graph neural networks into a differential equation to create an invertible equivariant function.
result E-NFs significantly outperform baselines and existing methods in log-likelihood for particle systems and molecules.
Normalizing flows are a powerful class of generative models for continuous random variables, showing both strong model flexibility and the potential for non-autoregressive generation. These benefits are also desired when modeling discrete random variables such as text, but directly applying normalizing flows to discret…
CIFs improve VI by providing flexible posteriors for complex topologies.
problem Complex posterior distributions in VI problems.
method Using CIFs as part of an auxiliary VI scheme, exploiting conditional independence.
result CIFs yield low-variance estimators of model evidence and improve VI results.
ELF simplifies normalizing flows, making them more efficient and universal.
problem Computational inefficiency of normalizing flows.
method ELF introduces a simple, one-layer network with closed-form Lipschitz constants, combining the ease of residual flows with the performance of autoregressive flows.
result ELF is a provably universal density approximator, more efficient computationally and parameter-wise.
We accelerate CNF by reducing ODE truncation errors with polynomial regularization.
problem High computation cost of CNF due to large truncation errors in solving ODEs.
method Add polynomial regularization to approximate ODE trajectories with polynomial functions.
result 42.3% to 71.3% reduction of NFE on density estimation, 19.3% to 32.1% on variational auto-encoder.
Normalizing flows transform a simple base distribution into a complex target distribution and have proved to be powerful models for data generation and density estimation. In this work, we propose a novel type of normalizing flow driven by a differential deformation of the Wiener process. As a result, we obtain a rich …
Generative model for condensed matter using Riemannian flow matching.
problem Sampling equilibrium distributions in condensed-phase systems.
method Riemannian flow matching to incorporate periodicity, using Hutchinson's trace estimator and cumulant expansion for bias correction.
result Highly accurate free energy estimates on monatomic ice without multistage estimators.
We introduce stochastic normalizing flows, an extension of continuous normalizing flows for maximum likelihood estimation and variational inference (VI) using stochastic differential equations (SDEs). Using the theory of rough paths, the underlying Brownian motion is treated as a latent variable and approximated, enabl…
Gaussianization flows transform any random vector into a Gaussian, enabling efficient computation and sample generation.
problem Transforming any random vector into a Gaussian for efficient computation and sample generation.
method Iterative Gaussianization and normalizing flow model.
result Gaussianization flows are universal approximators and achieve better performance on tabular datasets.
Causal normalizing flows recover causal models from observational data.
problem Recovering causal models from observational data.
method Use autoregressive normalizing flows and analyze design choices.
result Causal normalizing flows can capture causal data-generating processes.
Guided Flows enhance sample quality in conditional image generation and text-to-speech.
problem Improving sample quality in conditional generative models.
method Integrating classifier-free guidance into Flow Matching (FM) models for Continuous Normalizing Flows (CNFs).
result Guided Flows significantly improve sample quality in conditional image generation and text-to-speech synthesis.
iEFM trains CNF models from unnormalized densities efficiently.
problem Training generators from energy functions or unnormalized densities.
method Iterated energy-based flow matching (iEFM) with simulation-free objective.
result iEFM outperforms existing methods in probabilistic modeling.
Muon dynamics study uses spectral Wasserstein flow for optimization stability.
problem Optimizing deep learning models with gradient normalization.
method Introduces Spectral Wasserstein distances for matrix flows, proving equivalence with Benamou--Brenier formulation.
result Gradient-flow interpretation of mean-field normalized training dynamics.
Improved sampling efficiency for molecular systems using path gradients after Flow Matching.
problem Improving sampling efficiency for complex molecular systems.
method Hybrid approach combining Flow Matching and path gradients.
result Up to a threefold increase in sampling efficiency for molecular systems.
Paper introduces geometry-aware normalizing flows for improved causal inference.
problem Disparity between sample and population distributions in causal inference.
method Integrates continuous normalizing flows with parametric submodels, employing Wasserstein gradient flows and optimal transport.
result Significantly reduces parameter estimation bias and variance in finite-sample settings.
Analyzing and interpreting time-dependent stochastic data requires accurate and robust density estimation. In this paper we extend the concept of normalizing flows to so-called temporal Normalizing Flows (tNFs) to estimate time dependent distributions, leveraging the full spatio-temporal information present in the data…
This work interprets diffusion score matching using normalizing flows for better model training and evaluations.
problem Limitations of diffusion score matching when dealing with certain types of distributions.
method The approach involves interpreting the diffusion matrix using normalizing flows to provide better interpretation and usage of diffusion score matching.
result Diffusion score matching is equivalent to the original score matching evaluated in the transformed space defined by the normalizing flow.
Existence and uniqueness theorem for Ricci flow on weighted graphs proved.
problem Existence and uniqueness of solutions to Ricci flow equations on weighted graphs.
method Continuous time normalized Ricci flow approach.
result Existence and uniqueness theorem for solutions to Ricci flow on weighted graphs.
RC flow learns molecular kinetics in low dimensions.
problem Discovering interpretable low-dimensional models of molecular kinetics.
method Normalizing flow for coordinate transformation and Brownian dynamics for kinetics approximation.
result Tractable and trainable model of reduced kinetics in continuous time and space.
Two fundamental problems in unsupervised learning are efficient inference for latent-variable models and robust density estimation based on large amounts of unlabeled data. Algorithms for the two tasks, such as normalizing flows and generative adversarial networks (GANs), are often developed independently. In this pape…
Improves NF for complex data distributions with multiple modes.
problem Difficulty in handling data distributions with multiple isolated modes.
method Proposes a new framework using variational latent representation to improve NF.
result Significantly more powerful for generating data distributions with multiple modes.
A new method combines regularization and generative rehearsal for continual learning.
problem Catastrophic forgetting in neural networks over past tasks.
method Uses a normalizing flow to conditionally store past task data and regularize network embeddings.
result Performs favorably compared to state-of-the-art approaches with constant memory overhead.
A new training method for normalizing flows without samples.
problem Training normalizing flows without samples but with energy functions.
method Interpolates energy functions to find a transport vector field.
result Optimizes transport vector field and energy function to satisfy continuity equation.
A new method for categorical variational inference using discrete normalizing flows.
problem Challenges in optimizing variational approximations for discrete latent variables.
method Differentiable reparameterization using a mixture of discrete normalizing flows.
result Improves optimization of evidence lower bound and reduces sensitivity to hyperparameters.
FNFs model parameter-dependent densities by combining a fixed flow with a polynomial parameter-dependent transformation.
problem Learning a separate flow for every parameter configuration is intractable.
method Factorizable Normalizing Flows (FNFs) represent the parameter-dependent density as a fixed flow for a reference configuration and a learnable polynomial transformation factorized over parameters.
result FNFs enable the recovery of the combined effect of multiple parameters without sampling their joint space, providing a scalable and interpretable solution.