AGF explains feature learning in neural networks through alternating steps.
problem Understanding what features neural networks learn and how they learn them.
method AGF is an algorithmic framework that approximates the dynamics of feature learning in two-layer networks.
result AGF provides a unified framework to understand feature learning in neural networks, matching experimental results across various architectures.
This paper explores gradient flows for sampling distributions without normalization constants.
problem Sampling from distributions with unknown normalization constants.
method Gradient flows in the space of probability measures, focusing on Kullback-Leibler divergence, Fisher-Rao metric, and affine invariance.
result Gradient flows derived from Kullback-Leibler divergence do not depend on the normalization constant.
This paper bridges variational inference and Wasserstein gradient flows.
problem Combining variational inference and Wasserstein gradient flows for more efficient approximations.
method Recasting Bures-Wasserstein gradient flow as a Euclidean gradient flow and using path-derivative gradient estimator.
result A new gradient estimator for f-divergences that can be implemented using machine learning libraries. We construct short retractions of a CAT(1) space to its small convex subsets. This construction provides an alternative geometric description of an analytic tool introduced by Wilfrid Kendall. Our construction uses a tractrix flow which can be defined as a gradient flow for a family of functions of certain type. In an …
New construction of Fukaya-Seidel categories using complex gradient flow equation.
problem Constructing Fukaya-Seidel categories for specific models.
method Using the complex gradient flow equation and neck-stretching limits.
result Alternative proof of Seidel's spectral sequence for Lagrangian Floer cohomology.
A new ODE model explains gradient descent dynamics near edge of stability.
problem Understanding gradient-based training over non-convex landscapes.
method Rod Flow, a new ODE approximation of GD dynamics.
result Rod Flow accurately predicts critical sharpness threshold and self-stabilization in quartic potentials.
Paper introduces a differentially private generative model using gradient flow and sliced Wasserstein distance.
problem Protecting privacy in sensitive training data for generative models.
method Gradient flow in the space of probability measures, Gaussian-smoothed Sliced Wasserstein Distance, and numerical scheme for SDE.
result Demonstrates higher-fidelity data generation at low privacy budget compared to existing methods.
Gradient Boosted Normalizing Flows improve flexibility of NFs without increasing complexity.
problem Improving flexibility of normalizing flows without increasing complexity.
method Gradient Boosting applied to normalizing flows to create a mixture model structure.
result GBNFs outperform non-boosted NFs and produce better results with simpler components.
We define a family of functionals generalizing the Yang-Mills functional. We study the corresponding gradient flows and prove long-time existence and convergence results for subcritical dimensions as well as a bubbling criterion for the critical dimensions. Consequently, we have an alternate proof of the convergence of…
The paper proves convergence of normalized Ricci flow on compact manifolds.
problem Convergence of normalized Ricci flow on compact manifolds.
method Gradient inequality of Łojasiewicz type to show convergence to steady-states.
result Convergence of normalized Ricci flow to steady-states on compact manifolds.
Paper proves uniqueness of weak solutions for Plateau flow.
problem Proving uniqueness of weak solutions for Plateau flow.
method Used natural energy condition and alternative methods from Struwe.
result Proves uniqueness of weak solutions under natural condition.
New method improves MMD estimation without convexity assumptions.
problem Lack of theoretical guarantees for MMD estimation algorithms.
method Preconditioned gradient descent (PGD) scheme for MMD optimization.
result PGD scheme converges globally under specific conditions.
A new method for categorical variational inference using discrete normalizing flows.
problem Challenges in optimizing variational approximations for discrete latent variables.
method Differentiable reparameterization using a mixture of discrete normalizing flows.
result Improves optimization of evidence lower bound and reduces sensitivity to hyperparameters.
A new algorithm trains deep neural networks by adding neurons greedily.
problem Training deep neural networks efficiently and effectively.
method Neuron Pursuit (NP) algorithm, which alternates between neuron addition and loss minimization.
result The algorithm can train deep neural networks efficiently and effectively.
A new method for discrete data normalizing flows using latent transformations.
problem Challenges in parameterizing bijective transformations for discrete data.
method Predict a distribution over latent transformations to make the marginal likelihood differentiable.
result Discrete-data normalizing flows can be trained using gradient-based learning with unbiased score function estimation.
New particle algorithms optimize latent variable models.
problem Optimizing latent variable models for maximum likelihood estimation.
method Identify gradient flows associated with free energy functional and discretize them to create particle-based algorithms.
result Novel particle algorithms scale to high-dimensional settings and perform well in experiments.
We study blow-ups around fixed points at Type I singularities of the Ricci flow on closed manifolds using Perelman's W-functional. First, we give an alternative proof of the result obtained by Naber and Enders-Müller-Topping that blow-up limits are non-flat gradient shrinking Ricci solitons. Our second and main result …
We give new characterisations of sets of positive reach and show that a closed hypersurface has positive reach if and only if it is of class C1,1. These results are then used to prove new alternating Steiner formulæ for hypersurfaces of positive reach. Furthermore, it will turn out that every hypersurface that sat…
Generative model improved using Liouville PDE-based sliced-Wasserstein flow.
problem Improving generative models for fair regression.
method Transformed sliced-Wasserstein flow into Liouville PDE-based formalism, handling density estimation with normalizing flows of neural ODE.
result Outperforms in convergence and fairness with reduced variance.
The Straight-Through (ST) estimator is a widely used technique for back-propagating gradients through discrete random variables. However, this effective method lacks theoretical justification. In this paper, we show that ST can be interpreted as the simulation of the projected Wasserstein gradient flow (pWGF). Based on…
New approach to heat flow for half-harmonic maps, related to minimal surfaces.
problem Heat flow for half-harmonic maps from S1 to closed target manifolds. method Classical approach using Dirichlet-to-Neumann operator for the Laplace equation.
result Analogous results to 1985 harmonic map heat flow, valid for finite-energy data.
New framework improves EM algorithm convergence under log-Sobolev inequality.
problem Improving convergence of the EM algorithm.
method Extending gradient flow techniques to EM algorithm, using free energy representation.
result Exponential convergence of EM algorithm under log-Sobolev inequality.
New framework for Bayesian inference using neural Schrödinger-Föllmer flows.
problem Approximate Bayesian inference in large datasets.
method Stochastic control, Schrödinger bridges, SDE-based models.
result Advocates stochastic control as a finite time and low variance alternative to SGLD.
The paper studies alternating links in thickened surfaces using flow lattices and disc mutations.
problem Understanding alternating links in thickened surfaces and their invariants.
method Using integer flows on Tait graphs and disc mutations, the paper proves invariants and compares link properties.
result Found alternating knots with isometric flow lattices but different linking forms.
We study a class of fourth order curvature flows on a compact Riemannian manifold, which includes the gradient flows of a number of quadratic geometric functionals, as for instance the L2 norm of the curvature. Such flows can develop a special kind of singularities, that could not appear in the Ricci flow, namely singu…
Stable training of deep normalizing flows for high-dimensional variational inference.
problem Training deep normalizing flows for high-dimensional posterior distributions is infeasible due to high stochastic gradient variance.
method Proposed a combination of soft-thresholding of scale and bijective soft log transformation to stabilize training.
result Stable training of Real NVPs for posterior distributions with thousands of dimensions is possible.
Survey classifies singularity models in 3D Ricci flow.
problem Understanding singularity formation in 3D Ricci flow.
method Analysis of ancient κ-solutions and steady gradient Ricci solitons.
result Complete classification of singularity models in 3D.
Stochastic control-flow models (SCFMs) are a class of generative models that involve branching on choices from discrete random variables. Amortized gradient-based learning of SCFMs is challenging as most approaches targeting discrete variables rely on their continuous relaxations---which can be intractable in SCFMs, as…
We introduce the notions of `super-Ricci flows' and `Ricci flows' for time-dependent families of metric measure spaces (X,dt,mt)t∈I. The former property is proven to be stable under suitable space-time versions of mGH-convergence. Uniformly bounded families of super-Ricci flows are compact. In the spirit of t…
Improved sampling method using regularized Stein Variational Gradient Flow.
problem Improving the accuracy of sampling methods in machine learning.
method Proposed Regularized Stein Variational Gradient Flow to interpolate between SVGD and Wasserstein Gradient Flow.
result Established theoretical properties and provided preliminary numerical evidence of improved performance.
New sampling method uses gradient-free IPS with RKHS velocity field.
problem Efficient sampling from unnormalized target densities.
method Gradient-free interacting particle systems (IPS) with RKHS velocity field.
result IPS produce high-quality samples from various target distributions.
In this paper we study the steepest descent L2-gradient flow of the functional $\SW_{λ_1,λ_2}$, which is the the sum of the Willmore energy, λ1-weighted surface area, and λ2-weighted enclosed volume, for surfaces immersed in R3. This coincides with the Helfrich functional with zero `spontaneous curvature'.…
The paper studies learning dynamics in two-layer neural networks.
problem Learning dynamics and time scales in two-layer neural networks.
method Gradient flow dynamics of a wide two-layer neural network in high-dimension, with data following a single-index model.
result The learning dynamics exhibit separation of timescales and intermittency.
Proposes a method combining CNFs and rejection-resampling for sampling from unnormalized densities.
problem Sampling from unnormalized probability densities, especially multimodal ones.
method Combines continuous normalizing flows with rejection-resampling steps based on importance weights.
result The method improves sampling accuracy and performance compared to state-of-the-art methods.
Unified view of LR and RP gradients explained via divergence theorem.
problem Explaining the nature and relationship of LR and RP gradients.
method First principles approach using divergence theorem.
result Characterization of all possible estimators combining LR and RP.
Mean curvature flow is not a gradient flow on two nondegenerate metric spaces.
problem Whether mean curvature flow is a gradient flow on nondegenerate metric spaces of simple closed plane curves.
method Examined two nondegenerate metric spaces: uniformness-preserving and curvature-weighted structures.
result Mean curvature flow is not a gradient flow on either metric space.
Paper explores Fisher-Rao gradient flows and their kernel approximations.
problem Understanding and analyzing approximations of Fisher-Rao gradient flows.
method Rigorous investigation of Fisher-Rao and Wasserstein type gradient flows, focusing on kernel approximations.
result Proves evolutionary Γ-convergence for kernel-approximated Fisher-Rao flows, providing theoretical guarantees.
Flow-based generative models parameterize probability distributions through an invertible transformation and can be trained by maximum likelihood. Invertible residual networks provide a flexible family of transformations where only Lipschitz conditions rather than strict architectural constraints are needed for enforci…
Found first example of homogeneous gradient solitons for G2-Laplacian flow.
problem Existence of homogeneous gradient solitons for G2-Laplacian flow. method Provided the first known example of homogeneous gradient solitons.
result G2-Laplacian flow admits homogeneous gradient solitons on one-dimensional extensions. Alternative proof of weak solutions to mean curvature flow using minimizing movements.
problem Existence of weak solutions to mean curvature flow and volume preserving mean curvature flow.
method Proposes a new existence proof using a minimizing movements scheme and a novel proxy for distance.
result Unconditional convergence towards a De Giorgi solution for the minimizing movements scheme.
Optimization is at the heart of machine learning, statistics and many applied scientific disciplines. It also has a long history in physics, ranging from the minimal action principle to finding ground states of disordered systems such as spin glasses. Proximal algorithms form a class of methods that are broadly applica…
Proves Thom's conjecture for parabolic flows on Hilbert spaces.
problem Gradient flows on infinite-dimensional spaces and geometric flows with symmetry.
method Analytic functions, Hilbert spaces, Yang-Mills Flow, Ricci flow, critical points, Lojasiewicz inequality.
result Gradient conjecture holds for parabolic flows on Hilbert spaces, including flows with gauge symmetry.
The paper connects two clustering methods by showing gradient ascent flow can move up the cluster tree.
problem Establishing a strong correspondence between clustering methods.
method Moving up the cluster tree by following the gradient ascent flow.
result Gradient ascent flow can be used to move up the cluster tree.
Gradient flow on diffeomorphisms for image registration, with well-posedness proven.
problem Image registration with metric tensor deformation penalization.
method Gradient flow on Sobolev diffeomorphisms for a specific energy functional.
result Well-posedness of the gradient flow established.
New method for LVEBMs using saddle-point optimization and Langevin updates.
problem Expressive generative modeling of latent variables with hidden structure.
method Reformulate LVEBM training as a saddle problem, using Langevin updates and gradient flows.
result Proves existence and convergence of the algorithm under standard assumptions, with improved ELBO bounds.
Paper formulates particle flow using variational inference and Fisher-Rao gradient flow.
problem Estimating posterior densities in probabilistic models.
method Variational formulation of particle flow, Fisher-Rao gradient flow, Gaussian and Gaussian mixture approximations.
result Gaussian and Gaussian mixture approximations of Fisher-Rao particle flow reduce to Exact Daum and Huang particle flow under linear Gaussian assumptions.
Alternative construction of quasi-Fuchsian flows using vortex equations.
problem Constructing quasi-Fuchsian flows.
method Using coupled vortex equations to construct quasi-Fuchsian flows as thermostats.
result Formulas for the marked length spectrum of quasi-Fuchsian flows.
Constructs explicit solutions to Spin(7)-structures gradient flow.
problem Finding explicit solutions to Spin(7)-structures gradient flow.
method Expressed Spin(7)-torsion tensor and gradient flow in terms of torsion forms; used these formulae to find solutions.
result Found explicit solutions including a shrinking soliton on SU(3) and another on a T7-bundle over S1.