Natural gradient simplification for deep learning networks.
problem Efficiency in training deep Bayesian networks.
method Analysis of two geometries of Fisher information matrix and development of a method to simplify natural gradient for the second geometry.
result A method to simplify natural gradient for deep networks using an auxiliary recognition model.
The paper discusses fundamental problems in global Finsler geometry.
problem Fundamental problems in global Finsler geometry.
method Optimization and improvement of Lie derivatives, characterization of gradient vector fields, gradient estimate.
result Gradient estimate for smooth functions on Randers manifolds.
VB uses natural gradients in information geometry.
problem Estimating or computing natural gradients in VB.
method Natural-gradient descent algorithm and Bayesian Learning Rule.
result Simplification of Bayes' rule and generalization of quadratic surrogates.
The article reviews how gradient flow systems on hypergraphs connect to information geometry and nonequilibrium physics.
problem Understanding the geometry of perturbed gradient flow systems on hypergraphs.
method Formulating modern nonequilibrium principles within the framework of perturbed gradient flow systems on hypergraphs.
result New concepts like moduli spaces and thermodynamical area are introduced to understand speed limits.
Studying various functionals and associated gradient ows are known problems in differential geometry. The perpose of this article is to provide a general overview of curvature functionals in Finsler geometry and use their information for introducing different gradient ows on Finsler manifolds.
The paper establishes sub-gradient estimates and entropy formulas for quaternionic contact geometry heat equations.
problem Developing sub-gradient estimates and entropy formulas for quaternionic contact geometry.
method Establishing sub-gradient estimates and entropy formulas for the quaternionic contact heat equation.
result Two Perelman-type entropy formulas and sub-gradient estimates for the quaternionic contact heat equation.
Riemannian stochastic gradient descent approximates a diffusion process called Riemannian stochastic modified flow.
problem Improving convergence rate of Riemannian stochastic gradient descent.
method Using stochastic differential geometry, the paper shows RSGD can be approximated by the Riemannian stochastic modified flow (RSMF).
result RSGD can be approximated by the solution to the RSMF driven by an infinite-dimensional Wiener process, increasing the order of approximation.
Flow on curves in inversive geometry converges to loxodromics.
problem Gradient flow for curve length in inversive geometry.
method Invariant gradient flow for invariant length functional.
result Solutions exist for all time and converge to loxodromic curves.
Optimizes stochastic and online optimization methods based on problem geometry.
problem Optimizing computational and statistical outcomes in stochastic and online optimization problems.
method Characterizes optimal methods based on constraint set and gradient geometry.
result Stochastic and adaptive-gradient methods are optimal for quadratically convex constraint sets.
New flows introduced for symplectic geometry.
problem No specific problem stated; focuses on new flows.
method Introduces several geometric flows on symplectic manifolds.
result Examples include the Hitchin gradient flow and dual Ricci flow.
Quantum Natural Gradient uses quantum geometry for optimization.
problem Optimizing variational quantum circuits efficiently.
method Quantum generalization of Natural Gradient Descent using Quantum Information Geometry.
result Efficient algorithm for computing metric tensor approximations.
This work explores gradient flows and Riemannian structure in Gromov-Wasserstein geometry for data with global structure.
problem Suitable geometry for tasks requiring preservation of global data structure.
method Study of gradient flows and Riemannian structure in Gromov-Wasserstein geometry for distributions on \(\mathbb{R}^d\).
result Established a Benamou-Brenier-like formula for IGW and derived the IGW gradient.
New gradient flows improve high-dimensional sampling.
problem Sampling from high-dimensional target densities.
method Introducing Radon--Wasserstein gradient flows.
result Linear scaling in particles and dimensions.
New steady gradient Ricci solitons found with specific symmetry.
problem Finding new steady gradient Ricci solitons with positive curvature.
method Utilized a procedure by Lai to construct examples with O(p)imesO(q) symmetry. result Found new examples of steady gradient Ricci solitons with O(p)imesO(q) symmetry in dimensions p+q. The paper examines the geometry and topology of Sasaki-Ricci solitons, proving they are either connected at infinity or compact.
problem Understanding the geometry and topology of Sasaki-Ricci solitons.
method Analyzing the properties of complete gradient shrinking Sasaki-Ricci solitons, proving connectedness at infinity and compactness under certain curvature conditions.
result Proves that Sasaki-Ricci solitons are either connected at infinity or compact, generalizing results from previous studies.
The paper analyzes how noise geometry influences the performance of SGD in machine learning.
problem Understanding how noise geometry affects the performance of stochastic gradient descent.
method Developed two metrics to quantify noise alignment strength and analyzed their effects on loss and subspace projection dynamics.
result Noise geometry can be used to guarantee alignment under certain conditions, aiding SGD's ability to escape from sharp minima.
Survey on gradient Ricci solitons in 4D, focusing on geometry and classification.
problem Understanding gradient Ricci solitons in four dimensions.
method Geometric analysis and classification of solitons.
result Recent results on classification and rigidity of gradient Ricci solitons in 4D.
The paper studies a gradient system on a beta statistical manifold, proving integrability and deriving explicit expressions.
problem Investigating the geometry and integrability of a gradient system on a bivariate beta statistical manifold.
method Proving the system is Hamiltonian and admitting a Lax pair representation, deriving explicit expressions using Stirling's approximation, and identifying the Hamiltonian function.
result The gradient flow is linearizable in dual affine coordinates, and the system is completely integrable.
The paper studies equations in conformal geometry with gradient and existence results.
problem Equations in conformal geometry on closed smooth Riemannian manifolds.
method Local gradient and second derivative estimates, existence result.
result Proved local gradient and second derivative estimates for solutions.
The space of probability densities is an infinite-dimensional Riemannian manifold, with Riemannian metrics in two flavors: Wasserstein and Fisher--Rao. The former is pivotal in optimal mass transport (OMT), whereas the latter occurs in information geometry---the differential geometric approach to statistics. The Rieman…
Paper introduces ICGNs to model convex gradients.
problem Modeling convex gradients efficiently.
method Integrates Jacobian-vector product in a neural network.
result Single layer ICGN outperforms single layer ICNN in fitting.
Adapts IG for better feature attributions and robustness.
problem Reliability concerns in feature attributions for deep learning models.
method Adaptation of path-based feature attribution to Riemannian geometry of data manifolds.
result IG along geodesics generates more intuitive and robust explanations.
The paper extends gradient flow and relaxation studies to non-flat Riemannian manifolds.
problem Understanding gradient flows and relaxation in non-flat Riemannian manifolds.
method Developed a criterion for comparing relaxation along gradient descent curves using non-metricity tensor.
result Revealed a universal asymmetry: warming up is faster than cooling down.
Gravilon improves gradient descent for neural networks.
problem Improving efficiency and accuracy of gradient descent methods.
method Uses geometric modification of gradient step lengths.
result Promising experimental results on MNIST classification.
The paper classifies 3D complete gradient Yamabe solitons.
problem Classifying nontrivial 3D complete gradient Yamabe solitons.
method Analyzing properties of Yamabe solitons to show rotationally symmetry.
result Nontrivial non-flat 3D complete steady gradient Yamabe solitons are rotationally symmetric.
The study classifies h-almost Ricci-Yamabe solitons in various paracontact manifolds.
problem Classifying h-almost Ricci-Yamabe solitons in paracontact geometry.
method Characterization and classification of para-Kenmotsu, para-Sasakian, and para-cosymplectic manifolds.
result Characterizations and classifications of various paracontact manifolds.
Gradient descent with geometrically adapted metrics drives L2 cost to global minimum at uniform rate.
problem Minimizing L2 cost in deep learning networks. method Adapting gradient descent to output layer metric in deep learning.
result Uniform exponential convergence to global minimum in L2 cost. Deep learning models viewed through tame geometry for convergence guarantees.
problem Understanding convergence guarantees in deep learning models.
method Introducing tame geometry concepts and tools for nonsmooth nonconvex settings.
result Illustrates tame geometry as a natural framework for AI systems, especially deep learning.
We provide some examples of harmonic unit vector fields as normalized gradients of isoparametric functions from a K-contact geometry setting.
Alternative proof for 4D shrinking Ricci solitons with constant scalar curvature.
problem Proving the structure of four-dimensional shrinking gradient Ricci solitons with constant scalar curvature.
method Analyzing the asymptotic geometry at infinity.
result Alternative proof that such solitons are finite quotients of R^2 x S^2.
In optimization, the natural gradient method is well-known for likelihood maximization. The method uses the Kullback-Leibler divergence, corresponding infinitesimally to the Fisher-Rao metric, which is pulled back to the parameter space of a family of probability distributions. This way, gradients with respect to the p…
New differential geometry perspective on orthogonal RNNs.
problem Mitigating exploding and vanishing gradients in RNNs.
method Using tools from differential geometry, parameterizing vector fields via directional derivatives of scalar functions.
result Our approach achieves comparable or better results on benchmark tasks.
Study steady gradient Ricci solitons with cylindrical tangent flows at infinity.
problem Characterize the geometry of steady gradient Ricci solitons at infinity.
method Analyze the rescaled limits of finite-time singular solutions of the Ricci flow.
result Classify the tangent flows at infinity of 4-dimensional steady soliton singularity models.
Study 4D steady gradient Ricci solitons reducing to 3D manifolds.
problem Understanding 4D steady gradient Ricci solitons that reduce to 3D.
method Analyzing asymptotic geometry and curvature properties.
result 4D solitons either reduce to spherical space forms or the 3D Bryant soliton.
Introduces new info-geometric structure for dynamics on graphs and hypergraphs.
problem Modeling dynamics on discrete structures like graphs and hypergraphs.
method Introduces two dually flat structures: one on vertex space and another on edge space.
result Extends gradient flows to include nonequilibrium dynamics.
New metric for probability measures connects physics and geometry.
problem Developing a new metric for probability measures.
method Transport Hessian metric, formulated dynamical systems.
result Connections to physics equations and mathematical models.
The paper provides a geometric framework for understanding non-equilibrium thermodynamics.
problem Unclear geometric structure of GENERIC in non-equilibrium thermodynamics.
method Cotangent lifts of dynamics, splitting into holonomic and vertical representatives, and formulation within contact geometry.
result Physical meaning and explicit formulation of the second law of thermodynamics within evolution equations.
New gradient flows for non-negative and probability measures combining optimal transport and interaction forces.
problem Optimizing non-negative and probability measures using interaction forces and optimal transport.
method Interaction-Force Transport (IFT) gradient flows and their spherical variant, developed via infimal convolution of Wasserstein and spherical MMD tensors, with a particle-based optimization algorithm.
result The spherical IFT gradient flow provides global exponential convergence guarantees for both MMD and KL energy.
This paper addresses anisotropy in Transformer models, providing geometric insights and empirical support.
problem Anisotropy phenomenon in Transformer models, challenging their geometric interpretation.
method Derive geometric arguments and use concept-based mechanistic interpretability during training.
result Activation-derived directions capture large gradient energy and a larger share of gradient anisotropy than normal controls.
Study on 4D compact Ricci solitons and their geometric properties.
problem Investigating the geometry of 4D compact gradient Ricci solitons.
method Proving the Hitchin-Thorpe inequality under specific conditions.
result 4D compact gradient Ricci solitons satisfy the Hitchin-Thorpe inequality.
Optimization with inequality constraints using embedded gradient vector field method
problem Optimization with inequality constraints
method Geometric framework using quadratic slack variables
result Derives Lagrange multiplier functions and second-order optimality conditions
For a harmonic function u on Euclidean space, this note shows that its gradient is essentially determined by the geometry of its level hypersurfaces. Specifically, the factor by which |grad(u)| changes along a gradient flow is completely determined by the mean curvature of the level hypersurfaces intersecting the flow.
Simple rules ensure gradient descent adapts to local geometry, converging for convex and nonconvex problems.
problem Minimizing convex and nonconvex functions efficiently.
method Two rules: don't increase stepsize too fast and don't overstep local curvature.
result Method converges for convex and nonconvex problems, even with infinite global smoothness.
Extends gradient estimates for heat equation under Finsler geometric flows.
problem Global gradient estimates for positive solutions to heat equation.
method General compact Finsler CD(−K,N) geometric flow. result Derives Harnack inequality for positive solutions.
Novel multisymplectic framework for pseudo-Fueter curves in Hamiltonian field theory.
problem Generalizing Floer theory to multisymplectic geometry.
method Introducing pseudo-Fueter curves in a compatible almost hyperkähler structure.
result Gradient lines of multisymplectic action functional are pseudo-Fueter curves.
CWGD measures gradient diversity weighted by curvature, improving SGD convergence.
problem Gradient noise in high-curvature directions is underestimated by standard methods.
method CWGD weights gradient diversity by the inverse square root of the Hessian.
result CWGD-Cosine reduces optimization error by up to 20% compared to standard cosine annealing.
Gradient Ricci solitons can be extended to non-gradient Ricci solitons using energy function.
problem Extending the geometry of gradient Ricci solitons to non-gradient Ricci solitons.
method Using energy function E to study the geometry. result A non-steady Ricci soliton with symmetric covariant derivative is gradient.
We study the geometry at infinity of expanding gradient Ricci solitons of dimension greater than two with finite asymptotic curvature ratio without curvature sign assumptions. We mainly prove that they have a cone structure at infinity.