This paper studies gradient flows in asymmetric metric spaces and proves existence results.
problem Investigating gradient flows in asymmetric metric spaces.
method Discrete approximation and natural convexity assumption on potential function.
result Existence of curves of maximal slope in asymmetric metric spaces.
TSGO optimizes gradients in tensor networks to avoid vanishing/exploding issues.
problem Gradient vanishing and exploding problems in deep learning models.
method TSGO rotates parameters towards gradient direction in tangent space of normalized state.
result TSGO naturally determines learning rate based on angle between parameters and gradient.
Improved VI with Price's gradient estimator for target log-density.
problem Approximating target distributions from unnormalized log-densities.
method Stochastic gradient-based variational inference with Price's gradient estimator.
result Identifies Price's gradient as the key to WVI's superior performance.
Gradient flow in parameters equals linear interpolation in outputs.
problem Understanding and optimizing training algorithms in deep learning.
method Proving equivalence between gradient flow in parameter space and linear interpolation in output space, and deriving formulas for global minima.
result Gradient flow in parameters can be transformed into linear interpolation in outputs, leading to global minima.
Study Markov chain gradient descent in Hilbert spaces for quadratic loss.
problem Approximating optimal solutions for quadratic loss functions.
method Developed a Markov chain-based stochastic gradient algorithm in Hilbert spaces.
result Established probabilistic upper bounds on convergence.
Mean curvature flow is not a gradient flow on two nondegenerate metric spaces.
problem Whether mean curvature flow is a gradient flow on nondegenerate metric spaces of simple closed plane curves.
method Examined two nondegenerate metric spaces: uniformness-preserving and curvature-weighted structures.
result Mean curvature flow is not a gradient flow on either metric space.
NES optimizes discrete structured VAEs effectively without gradient propagation.
problem Learning high-dimensional discrete latent spaces in generative models.
method Natural Evolution Strategies (NES) for gradient-free optimization of discrete structures.
result NES effectively optimizes discrete structured VAEs, comparable to gradient-based methods.
The paper derives gradient estimates for solutions of certain equations on metric measure spaces.
problem Gradient estimates for solutions of specific nonlinear and elliptic equations on metric measure spaces.
method Derives Li-Yau and Hamilton's type gradient estimates for positive solutions.
result Gradient estimates for positive solutions of the equations on complete noncompact metric measure spaces.
The paper provides gradient estimates for nonlinear heat-type equations on smooth metric measure spaces.
problem Proving gradient estimates for nonlinear heat-type equations on smooth metric measure spaces.
method Using Hamilton type and Li-Yau type estimates, the paper proves gradient estimates on positive solutions to generalized nonlinear parabolic equations on smooth metric measure spaces with compact boundary.
result Gradient estimates for nonlinear heat-type equations on smooth metric measure spaces.
Gradient estimates for hyperbolic space CMC equation solved.
problem Gradient estimates for solutions to constant mean curvature equation in hyperbolic space.
method Maximum principles theory of Φ-functions.
result Gradient estimates obtained for bounded strictly convex domains.
The paper explores almost Ricci solitons on Finsler spaces, proving conditions for their existence.
problem Characterizing almost Ricci solitons on Finsler measure spaces.
method Introducing and investigating gradient almost Ricci solitons, proving conditions for existence.
result Conditions for the existence of gradient almost Ricci solitons on Finsler measure spaces.
The paper classifies invariant gradient k-Yamabe solitons in pseudo-Euclidean spaces.
problem Characterizing invariant gradient k-Yamabe solitons in pseudo-Euclidean spaces. method Characterization through the action of an (n−1)-dimensional translation group and classification of rotational invariant solutions. result Infinitely many explicit examples of geodesically complete steady gradient k-Yamabe solitons are constructed. The paper proves gradient estimates for nonlinear parabolic equations on smooth metric measure spaces.
problem Proving gradient estimates for nonlinear parabolic equations on smooth metric measure spaces.
method Using Souplet-Zhang type estimates and properties of Bakry-Emery Ricci tensor and weighted mean curvature.
result Gradient estimates for nonlinear parabolic equations on smooth metric measure spaces with Dirichlet boundary condition.
Constructs retractions of CAT(1) spaces to convex subsets.
problem Geometric description of an analytic tool.
method Gradient flow of time-dependent locally Lipschitz semiconcave functions.
result Existence of gradient flows proved for independent interest.
Gradient inequalities for harmonic map energy proved using abstract inequalities.
problem Proving gradient inequalities for harmonic map energy.
method Applied abstract gradient inequalities to harmonic map energy function.
result Generalized Lojasiewicz--Simon gradient inequalities.
We present a version of the equivariant gradient degree defined for equivariant gradient perturbations of an equivariant unbounded self-adjoint operator with purely discrete spectrum in Hilbert space. Two possible applications are discussed.
New method for sampling on constrained domains using orthogonal-space gradient flow.
problem Sampling on manifolds defined by constraints is challenging.
method Orthogonal-Space Variational Gradient Descent (O-Gradient)
result O-Gradient converges to the target constrained distribution efficiently.
Stochastic gradient descent on manifolds improves low-rank approximation.
problem Efficiently approximate large matrices with lower rank.
method Stochastic gradient descent on a manifold.
result Algorithm outperforms Euclidean space methods on Netflix Prize data.
New method QMLE performs well in complex action spaces without policy gradients.
problem Why policy gradients outperform action-value methods in complex action spaces.
method QMLE framework for action-value methods based on three principles.
result QMLE performs comparably to policy gradient methods in complex action spaces.
Study on gradient descent in Hilbert spaces with Markov chains, focusing on mixing coefficients.
problem Analyzing convergence of gradient descent in Hilbert spaces with stationary Markov chains.
method Examined strictly stationary Markov chains with φ- and β-mixing coefficients, derived probabilistic upper bounds. result Probabilistic upper bounds on convergence behavior of gradient descent algorithm based on mixing coefficients.
In this paper, we establish a Bochner type formula on Alexandrov spaces with Ricci curvature bounded below. Yau's gradient estimate for harmonic functions is also obtained on Alexandrov spaces.
Functional central limit theorem for kernel gradient flow and infinitesimal gradient boosting
problem Fluctuations of boosting processes around their deterministic limit
method Stochastic perturbation analysis of ODEs in Banach spaces
result Rescaled deviations converge to a Gaussian process
The paper derives gradient estimates for porous medium and fast diffusion equations on metric measure spaces.
problem Gradient estimates for porous medium and fast diffusion equations on metric measure spaces.
method Derives Li-Yau and Souplet-Zhang type gradient estimates for the given equations.
result Gradient estimates for the equations on complete noncompact metric measure spaces with compact boundary.
New Banach spaces for ReLU networks enable better function approximation and gradient dynamics analysis.
problem Function approximation and gradient dynamics in multi-layer ReLU networks.
method Developed Banach spaces for ReLU networks, defined new function representations, and analyzed gradient flow dynamics.
result Gradient flow dynamics of the new representation is the continuous analog of gradient descent for ReLU networks.
Study of geometric analysis on asymmetric metric spaces, including heat flow and Sobolev spaces.
problem Analysis of geometric properties on asymmetric metric measure spaces.
method Introduction of upper gradients, q-Laplacian, and q-heat flow in asymmetric settings. result Extension of concepts from symmetric to asymmetric metric measure spaces.
A framework for natural gradient with arbitrary similarity measures.
problem Unclear metric for natural gradient in non-Euclidean spaces.
method Derive a metric for natural gradient given an arbitrary similarity measure.
result General framework for natural gradient in non-Euclidean spaces.
Paper investigates conditions for independence of weak gradients on metric spaces.
problem Dependence of weak gradients on p in arbitrary metric measure spaces. method Investigates the Bounded Interpolation Property to ensure independence of weak gradients.
result Bounded Interpolation Property guarantees independence of weak gradients.
Gradient-free method solves infinite-dimensional optimization problems.
problem Optimizing functions in infinite-dimensional spaces.
method Uses directional derivatives and a pre-basis for Hilbert space.
result Proves convergence for solving PDEs using PINNs.
Gradient flow autoencoder improves data efficiency over traditional autoencoders.
problem Sub-optimal latent space representations in autoencoders.
method Gradient flow through ODE with adaptive step size for optimization.
result Gradient flow autoencoder achieves higher data efficiency.
Gradient descent, or negative gradient flow, is a standard technique in optimization to find minima of functions. Many implementations of gradient descent rely on discretized versions, i.e., moving in the gradient direction for a set step size, recomputing the gradient, and continuing. In this paper, we present an appr…
Paper develops unbiased gradient estimator for continuous-time models.
problem Estimating unbiased gradient of log-likelihood for continuous-time models.
method Doubly randomized scheme with coupled conditional particle filter (CCPF).
result Unbiased gradient estimate facilitates gradient-based algorithms.
We prove that on compact Alexandrov spaces with curvature bounded below the gradient flow of the Dirichlet energy in the L2-space produces the same evolution as the gradient flow of the relative entropy in the L2-Wasserstein space. This means that the heat flow is well defined by either one of the two gradient fl…
We study the Wasserstein natural gradient in parametric statistical models with continuous sample spaces. Our approach is to pull back the L2-Wasserstein metric tensor in the probability density space to a parameter space, equipping the latter with a positive definite metric tensor, under which it becomes a Riemanni…
Equipped with the L^2-distortion distance, the space "X" of all metric measure spaces (X,d,m) is proven to have nonnegative curvature in the sense of Alexandrov. Geodesics and tangent spaces are characterized in detail. Moreover, classes of semiconvex functionals and their gradient flows on "X" are presented.
The paper provides new gradient estimates for solutions to a nonlinear elliptic equation on smooth metric measure spaces.
problem Gradient estimates for solutions to a specific nonlinear elliptic equation on smooth metric measure spaces.
method Nash-Moser iteration technique to obtain local gradient estimates.
result New local gradient estimates for positive solutions to the equation.
In this paper, motivated by the works of Bakry et. al in finding sharp Li-Yau type gradient estimate for positive solutions of the heat equation on complete Riemannian manifolds with nonzero Ricci curvature lower bound, we first introduce a general form of Li-Yau type gradient estimate and show that the validity of suc…
Global gradient estimates for Fisher-KPP equation on Finsler metric measure spaces.
problem Establishing gradient estimates for the Finslerian Fisher-KPP equation.
method Global gradient estimates on compact and noncompact Finsler metric measure spaces using the traditional CD(K,N) condition and new comparison theorems. result Global gradient estimates for positive solutions of the Finslerian Fisher-KPP equation.
The paper establishes inequalities and gradient estimates for harmonic functions on Finsler measure spaces.
problem Functional and geometric inequalities on Finsler measure spaces.
method Local uniform Poincaré and Sobolev inequalities, mean value inequality, Harnack inequalities, and gradient estimates.
result Global gradient estimates for positive harmonic functions on Finsler measure spaces.
Gradient descent works well for large NNs due to convexity in a transformed space.
problem Why gradient descent works well in non-convex NN optimization.
method Introduced canonical space and disparity matrix to prove convexity.
result Gradient descent converges to global minimum in large NNs.
Study proves structure results for homogeneous spaces supporting specific equations.
problem Proving structure results for homogeneous spaces supporting specific equations.
method Analyzing homogeneous spaces with non-constant solutions to two general classes of equations involving the Hessian and an invariant 2-tensor.
result Generalizes rigidity results for gradient Ricci solitons and warped product Einstein metrics.
Gradient-based framework for optimizing text prompts in diffusion models.
problem Efficiently optimizing prompts in text-to-image diffusion models with large domain space and non-differentiable embeddings.
method Formulated as discrete optimization over language space, designed compact subspaces, and introduced shortcut text gradient.
result Empirically discovered prompts that enhance or destroy image faithfulness.
Gradient-based MCMC for discrete spaces improves sampling performance.
problem Sampling in discrete spaces using traditional methods is challenging.
method Introduced new discrete Metropolis-Hastings samplers inspired by MALA, with a novel preconditioning technique.
result Demonstrated strong empirical performance across various challenging sampling problems.
We generalize the theory of gradient flows of semi-convex functions on CAT(0)-spaces, developed by Mayer and Ambrosio--Gigli--Savaré, to CAT(1)-spaces. The key tool is the so-called "commutativity" representing a Riemannian nature of the space, and all results hold true also for metric spaces satisfying the commutativi…
This paper bridges variational inference and Wasserstein gradient flows.
problem Combining variational inference and Wasserstein gradient flows for more efficient approximations.
method Recasting Bures-Wasserstein gradient flow as a Euclidean gradient flow and using path-derivative gradient estimator.
result A new gradient estimator for f-divergences that can be implemented using machine learning libraries. Direct policy gradients optimize policies in discrete action spaces using sampling.
problem Optimizing policies in discrete action spaces with direct methods.
method Combining direct optimization and A⋆ sampling for policy gradient approximation. result DirPG algorithms can incorporate domain knowledge and have higher probability of sampling informative gradients.
New rates for GLD and SGLD in infinite-dimensional spaces without dimensionality issues.
problem Gradient Langevin dynamics and SGLD convergence rates in high-dimensional spaces.
method Analysis of GLD and SGLD in infinite-dimensional Hilbert spaces, using stochastic differential equations and Markov chains.
result Derivation of dimension-free convergence rates for GLD and SGLD.
This work proposes a new method for variational inference using Wasserstein gradient descent.
problem Optimizing variational parameters to match a true posterior distribution.
method Reinterpreting VI as an optimization problem over a variational parameter space, using Wasserstein gradient descent.
result The proposed Wasserstein gradient descent can be seen as a generalization of existing optimization techniques in VI.
This paper analyzes Stein variational gradient descent for Bayesian inference.
problem Sampling or approximating high-dimensional probability distributions.
method Iterated steepest descent steps with a reproducing kernel Hilbert space norm.
result Performance gains of certain nondifferentiable kernels with adjusted tails.