Improved sampling method using regularized Stein Variational Gradient Flow.
problem Improving the accuracy of sampling methods in machine learning.
method Proposed Regularized Stein Variational Gradient Flow to interpolate between SVGD and Wasserstein Gradient Flow.
result Established theoretical properties and provided preliminary numerical evidence of improved performance.
Paper formulates particle flow using variational inference and Fisher-Rao gradient flow.
problem Estimating posterior densities in probabilistic models.
method Variational formulation of particle flow, Fisher-Rao gradient flow, Gaussian and Gaussian mixture approximations.
result Gaussian and Gaussian mixture approximations of Fisher-Rao particle flow reduce to Exact Daum and Huang particle flow under linear Gaussian assumptions.
This paper bridges variational inference and Wasserstein gradient flows.
problem Combining variational inference and Wasserstein gradient flows for more efficient approximations.
method Recasting Bures-Wasserstein gradient flow as a Euclidean gradient flow and using path-derivative gradient estimator.
result A new gradient estimator for f-divergences that can be implemented using machine learning libraries. New path-gradient estimator for continuous normalizing flows.
problem Limitation of simple Gaussian variational distributions in complex applications.
method Proposed a path-gradient estimator for continuous normalizing flows.
result Empirical evidence of superior performance of the new estimator.
New method for constrained sampling using gradient flows.
problem Sampling from constrained domains.
method Introducing a boundary condition for gradient flow to confine particles within the domain.
result Provable continuous-time convergence in total variation for constrained sampling.
This paper shows equivalence between SVGD and BBVI using kernel gradient flows.
problem Bayesian inference methods and their equivalence.
method Formalizes equivalence between SVGD and BBVI using kernel gradient flows.
result BBVI corresponds precisely to SVGD when using the neural tangent kernel.
The paper studies curves in Riemannian manifolds using total variation flow.
problem Analyzing the evolution of curves in Riemannian manifolds using total variation.
method Defining and proving the existence of strong solutions to the flow equations, showing variational equality, and proving convergence.
result Strong solutions converge to a constant map in finite time for non-positive sectional curvature.
A new ParVI framework improves particle-based variational inference methods.
problem Non-trivial kernel design in particle-based variational inference methods.
method Proposes a generalized Wasserstein gradient descent (GWG) framework with broader regularizers.
result Demonstrates strong convergence guarantees and effectiveness on simulated and real data.
Analyzes gradient flows and EVI in metric spaces with structural properties and convergence results.
problem Characterizing and analyzing gradient flows and Evolution Variational Inequalities in metric spaces.
method Study of structural properties, equivalence with maximal slope curves, convergence of Minimizing Movement-JKO scheme.
result Equivalence with maximal slope curves and uniform convergence of Minimizing Movement-JKO scheme.
New particle-based VI algorithm expands function class and improves scalability.
problem Limited function class in particle-based VI algorithms restricts flexibility and scalability.
method Introduces a functional regularization term to expand the function class and proposes PFG algorithm.
result Proposed PFG algorithm has larger function class, improved scalability, better adaptation to ill-conditioned distributions, and provable convergence.
A new method for Gaussian filtering using gradient flows and Wasserstein metrics.
problem Approximating Gaussian and mixture-of-Gaussians filtering for complex systems.
method Variational approximation via gradient-flow representation on Wasserstein metric space.
result Competitive performance in posterior representation and parameter estimation for systems with multiplicative noise and multi-modal distributions.
This work proposes a new method for variational inference using Wasserstein gradient descent.
problem Optimizing variational parameters to match a true posterior distribution.
method Reinterpreting VI as an optimization problem over a variational parameter space, using Wasserstein gradient descent.
result The proposed Wasserstein gradient descent can be seen as a generalization of existing optimization techniques in VI.
Improved KL divergence estimators for normalizing flows lead to faster convergence and better approximations.
problem Estimating KL divergences for normalizing flows efficiently and accurately.
method Path-gradient estimators for reverse and forward KL divergences.
result Path-gradient estimators lead to faster convergence and better approximation results.
New method for variational inference without conjugacy constraints.
problem Efficient variational inference with flexible prior and approximation families.
method Wasserstein gradient flow for mean-field approximation.
result Improved convergence and efficiency of variational inference.
This work proposes new methods for variational inference using gradient flows on Gaussian measures.
problem Developing algorithmic guarantees for variational inference.
method Proposes principled methods for variational inference using gradient flows on the Bures--Wasserstein space of Gaussian measures.
result Strong theoretical guarantees for log-concave posteriors.
A new gradient flow framework for distributionally robust optimization.
problem Optimizing under uncertainty with worst-case distributional constraints.
method Gradient flow theory applied to distributionally robust optimization.
result Practical algorithms for sampling from worst-case distributions.
Proposes a new Langevin flow approach for VAEs.
problem Difficulty in constructing low variance ELBO for VAEs with large datasets.
method Integrates Langevin dynamic with quasi-symplectic integrator to improve posterior estimation.
result Shows theoretical and practical effectiveness compared to gradient flow-based methods.
SIFG uses noisy particles to efficiently sample from complex distributions.
problem Efficient sampling from complex distributions using particle-based methods.
method SIFG introduces a semi-implicit functional gradient flow with Gaussian noise to improve sampling efficiency and accuracy.
result SIFG achieves strong theoretical convergence guarantees and efficient sampling.
This study explains gradient flow dynamics in neural networks for small initialisation.
problem Understanding the training dynamics of neural networks for small initialisation.
method Analysis of gradient flow dynamics for one-hidden layer ReLU networks with orthogonal inputs.
result Gradient flow converges to zero loss and characterizes implicit bias towards minimum variation norm.
A new method for categorical variational inference using discrete normalizing flows.
problem Challenges in optimizing variational approximations for discrete latent variables.
method Differentiable reparameterization using a mixture of discrete normalizing flows.
result Improves optimization of evidence lower bound and reduces sensitivity to hyperparameters.
ASVGD accelerates SVGD for efficient sampling.
problem Slow SVGD in high-dimensional sampling.
method Accelerated gradient flow in a metric space of probability densities, using Nesterov's method and momentum-based updates.
result ASVGD outperforms SVGD and other methods in sampling efficiency.
The paper analyzes rates for a modified gradient descent method using Stein variational gradients.
problem Improving the accuracy of gradient descent methods for complex target distributions.
method Derives finite-particle rates for regularized Stein variational gradient descent (R-SVGD).
result Establishes explicit non-asymptotic bounds for time-averaged empirical measures.
This paper analyzes Stein variational gradient descent for Bayesian inference.
problem Sampling or approximating high-dimensional probability distributions.
method Iterated steepest descent steps with a reproducing kernel Hilbert space norm.
result Performance gains of certain nondifferentiable kernels with adjusted tails.
Develops a new framework for analyzing MFVI algorithms.
problem Analyzes mean field variational inference (MFVI) formulations.
method Inspired by variational Bayesian formulations, represents MFVI problem in three ways: gradient flow, Fokker-Planck-like equations, and diffusion process.
result Establishes rigorous guarantees for convergence of time-discretized coordinate ascent variational inference algorithms.
A new path gradient estimator speeds up normalizing flows without sacrificing accuracy.
problem High computational cost and limited scalability of path gradient estimators for normalizing flows.
method Proposed a fast path gradient estimator that improves computational efficiency and scalability.
result The new estimator achieves superior performance and reduced variance across various applications.
Square-root natural-gradient improves variational inference convergence.
problem Challenges in establishing theoretical convergence guarantees for natural-gradient descent.
method Square-root parameterization for Gaussian covariance.
result Establishes novel convergence guarantees for natural-gradient Gaussian inference.
Stable training of deep normalizing flows for high-dimensional variational inference.
problem Training deep normalizing flows for high-dimensional posterior distributions is infeasible due to high stochastic gradient variance.
method Proposed a combination of soft-thresholding of scale and bijective soft log transformation to stabilize training.
result Stable training of Real NVPs for posterior distributions with thousands of dimensions is possible.
Stein variational gradient descent (SVGD) is a deterministic sampling algorithm that iteratively transports a set of particles to approximate given distributions, based on an efficient gradient-based update that guarantees to optimally decrease the KL divergence within a function space. This paper develops the first th…
PVI improves SIVI by directly optimizing ELBO without parametric assumptions.
problem Intractable variational densities in SIVI methods.
method Particle Variational Inference (PVI) using empirical measures to approximate optimal mixing distributions.
result PVI directly optimizes the ELBO and performs favorably compared to other SIVI methods.
Paper analyzes convergence rates of mean-field SVGD method.
problem Establishing quantitative rates of convergence for mean-field SVGD.
method Quantitative analysis of mean-field SVGD dynamics on torus.
result Explicit polynomial convergence rates in L2-norm for Riesz-type kernels.
The paper accelerates gradient flows on probability distributions using optimal control theory.
problem Optimizing probability distributions efficiently.
method Variational formulation and Hamilton's equations for accelerated gradient flows.
result The method achieves accelerated density transport from any initial distribution to a target distribution.
Cascading flows improve variational inference in structured programs.
problem Challenges in variational inference for complex probabilistic programs.
method Integrates normalizing flows and ASVI to create cascading flows, which embed the forward-pass of probabilistic programs.
result Cascading flows outperform normalizing flows and ASVI in structured inference problems.
Paper explores Fisher-Rao gradient flows and their kernel approximations.
problem Understanding and analyzing approximations of Fisher-Rao gradient flows.
method Rigorous investigation of Fisher-Rao and Wasserstein type gradient flows, focusing on kernel approximations.
result Proves evolutionary Γ-convergence for kernel-approximated Fisher-Rao flows, providing theoretical guarantees.
CoSMIC extends flow-based SVI to transdimensional problems.
problem Bayesian structure learning and model selection with multi-model parameter spaces.
method Normalizing flows with a combined stochastic variational transdimensional inference approach.
result Improved performance on high-cardinality model spaces.
We generalize the theory of gradient flows of semi-convex functions on CAT(0)-spaces, developed by Mayer and Ambrosio--Gigli--Savaré, to CAT(1)-spaces. The key tool is the so-called "commutativity" representing a Riemannian nature of the space, and all results hold true also for metric spaces satisfying the commutativi…
New method for sampling on constrained domains using orthogonal-space gradient flow.
problem Sampling on manifolds defined by constraints is challenging.
method Orthogonal-Space Variational Gradient Descent (O-Gradient)
result O-Gradient converges to the target constrained distribution efficiently.
Study fourth-order geometric flow of shape operator for co-dimension one immersions.
problem Analyzing the geometry of isometric immersions in Riemannian manifolds.
method Introduce a moduli flow to decrease curvature variation energy.
result The flow decreases a natural energy measuring curvature variation.
Variational inference improves training of generative flow networks.
problem Training generative flow networks efficiently and accurately.
method Define variational objectives in terms of KL divergences and optimize convex combinations.
result Variational inference methods can reduce the variance of gradients in training generative flow networks.
vOED-NFs uses normalizing flows to improve Bayesian OED without likelihood evaluations.
problem Optimizing experiments to maximize information gain in model parameters.
method vOED-NFs combines variational approximations with normalizing flows for efficient EIG estimation.
result vOED-NFs achieves lower EIG estimation bias compared to previous methods.
This paper studies the combinatorial Yamabe flow on hyperbolic surfaces with boundary. It is proved by applying a variational principle that the length of boundary components is uniquely determined by the combinatorial conformal factor. The combinatorial Yamabe flow is a gradient flow of a concave function. The long ti…
New gradient flows improve high-dimensional sampling.
problem Sampling from high-dimensional target densities.
method Introducing Radon--Wasserstein gradient flows.
result Linear scaling in particles and dimensions.
The paper uses gradient flow to minimize geodesic curvature of closed planar curves.
problem Minimizing geodesic curvature of closed planar curves.
method Gradient flow to deform curves to those with least L2 variation of geodesic curvature. result Smooth convergence to a multiply-covered circle for bounded length curves.
A new method for deep generative learning using gradient flow.
problem Learning deep generative models efficiently and accurately.
method VGrow framework that evolves a distribution via the negative gradient of f-divergence.
result VGrow can generate high-fidelity images efficiently and stably.
The article reviews how gradient flow systems on hypergraphs connect to information geometry and nonequilibrium physics.
problem Understanding the geometry of perturbed gradient flow systems on hypergraphs.
method Formulating modern nonequilibrium principles within the framework of perturbed gradient flow systems on hypergraphs.
result New concepts like moduli spaces and thermodynamical area are introduced to understand speed limits.
Variational inference has experienced a recent surge in popularity owing to stochastic approaches, which have yielded practical tools for a wide range of model classes. A key benefit is that stochastic variational inference obviates the tedious process of deriving analytical expressions for closed-form variable updates…
Paper bridges VI and GFlowNets, showing their equivalence in certain cases.
problem Modeling distributions over continuous and discrete structures.
method Demonstrates equivalence between VI and GFlowNets in specific scenarios.
result GFlowNets are more suitable for off-policy training without high gradient variance.
ASVGD accelerates SVGD for efficient sampling from Gaussian targets.
problem Efficient sampling from Gaussian distributions using SVGD.
method Accelerated gradient flow in a metric space of probability densities, including momentum and Wasserstein regularization.
result ASVGD achieves optimal convergence rate for Gaussian targets, independent of covariance.
This paper explores geometric and analytic aspects of Lojasiewicz inequalities on vector bundles.
problem Understanding growth and stability conditions for real-analytic functions over vector bundles.
method Outline theory of functionals and variational problems over vector bundles, explore applications to real-analytic functionals.
result Describes the energy functional on $S^{n-1$ as a functional over a vector bundle.