GGA improves untrustworthy prediction detection in neural networks without retraining.
problem Susceptibility of neural networks to untrustworthy predictions, especially adversarial attacks and out-of-distribution data.
method Geometric Gradient Analysis (GGA) analyzes the geometry of neural network loss landscapes based on saliency maps.
result GGA outperforms existing methods in detecting untrustworthy predictions, including adversarial and out-of-distribution data.
In general, gradient estimates are very important and necessary for deriving convergence results in different geometric flows, and most of them are obtained by analytic methods. In this paper, we will apply a stochastic approach to systematically give gradient estimates for some important geometric quantities under the…
Study of geometric analysis on asymmetric metric spaces, including heat flow and Sobolev spaces.
problem Analysis of geometric properties on asymmetric metric measure spaces.
method Introduction of upper gradients, q-Laplacian, and q-heat flow in asymmetric settings. result Extension of concepts from symmetric to asymmetric metric measure spaces.
SGD converges with perturbed forward-backward passes, explained by geometric amplification.
problem Analyzing convergence of SGD with perturbed forward-backward passes in composite optimization.
method Characterized propagation and amplification of perturbations, derived convergence guarantees for non-convex and PL objectives.
result Perturbations cascade through the computational graph, affecting convergence order under specific conditions.
This is a survey article on recent progress of comparison geometry and geometric analysis on Finsler manifolds of weighted Ricci curvature bounded below. Our purpose is two-fold: Give a concise and geometric review on the birth of weighted Ricci curvature and its applications; Explain recent results from a nonlinear an…
LEGO estimates tangent spaces more robustly than LPCA in noisy data.
problem Estimating tangent spaces in high-noise settings.
method Spectral method using graph Laplacian eigenvectors and gradient orthogonization.
result LEGO yields more robust tangent space estimates than LPCA.
New rates for GLD and SGLD in infinite-dimensional spaces without dimensionality issues.
problem Gradient Langevin dynamics and SGLD convergence rates in high-dimensional spaces.
method Analysis of GLD and SGLD in infinite-dimensional Hilbert spaces, using stochastic differential equations and Markov chains.
result Derivation of dimension-free convergence rates for GLD and SGLD.
This paper explores geometric and analytic aspects of Lojasiewicz inequalities on vector bundles.
problem Understanding growth and stability conditions for real-analytic functions over vector bundles.
method Outline theory of functionals and variational problems over vector bundles, explore applications to real-analytic functionals.
result Describes the energy functional on $S^{n-1$ as a functional over a vector bundle.
Gradient clipping helps private SGD converge despite potential bias.
problem Gradient clipping in private SGD can bias convergence.
method Theoretical analysis and empirical evaluation of gradient clipping effects.
result Gradient clipping can prevent convergence to stationary points and introduces bias.
Geometric analysis improves convergence of variational inference.
problem Challenges in analyzing convergence of variational inference due to non-convexity and non-smoothness.
method Exploits exponential family structure and Bregman divergences to geometrically analyze the optimization landscape.
result Establishes non-asymptotic convergence rates for gradient descent algorithms.
New geometric analysis shows L2 score error is flawed for diffusion models.
problem Score matching errors in diffusion models do not fully capture distributional quality.
method Decomposed score errors into gradient and solenoidal components, focusing on gradient's role in Fokker-Planck dynamics.
result Only gradient component affects marginal distributional quality; solenoidal component is structurally invisible.
The paper studies geometric properties of group equivariant operators and their Riemannian structure.
problem Understanding the geometric structure of group equivariant operators.
method Endowing the space of group equivariant non-expansive operators with a Riemannian manifold structure and using gradient descent methods.
result Gradient descent methods can be applied to minimize cost functions on the space of group equivariant non-expansive operators.
Gradient flow on diffeomorphisms for image registration, with well-posedness proven.
problem Image registration with metric tensor deformation penalization.
method Gradient flow on Sobolev diffeomorphisms for a specific energy functional.
result Well-posedness of the gradient flow established.
In this paper, we will study the (linear) geometric analysis on metric measure spaces. We will establish a local Li-Yau's estimate for weak solutions of the heat equation and prove a sharp Yau's gradient gradient for harmonic functions on metric measure spaces, under the Riemannian curvature-dimension condition $RCD^*(…
BBVI with STL converges geometrically under perfect specification, with quadratic variance bound.
problem Convergence rate of BBVI with STL estimator.
method Proved geometric convergence rate with quadratic variance bound for BBVI with STL estimator.
result BBVI with STL converges geometrically under perfect variational family specification.
In this paper, we first obtain an Lq gradient estimate for p-harmonic maps, by assuming the target manifold supporting a certain function, whose gradient and Hessian satisfy some analysis conditions. From this Lq gradient estimate, we get a corresponding Liouville type result for p-harmonic maps. Secondly, us…
We propose the first global accelerated gradient method for Riemannian manifolds. Toward establishing our result we revisit Nesterov's estimate sequence technique and develop an alternative analysis for it that may also be of independent interest. Then, we extend this analysis to the Riemannian setting, localizing the …
Survey on gradient Ricci solitons in 4D, focusing on geometry and classification.
problem Understanding gradient Ricci solitons in four dimensions.
method Geometric analysis and classification of solitons.
result Recent results on classification and rigidity of gradient Ricci solitons in 4D.
This work establishes properties on diffeological structures for set-valued maps and measures.
problem Establish rigorous properties on diffeological structures for set-valued maps and measures.
method Using diffeologies, the authors link various structures including set-valued maps, relations, gradients, measures, and shape analysis.
result Established rigorous properties on sample diffeologies.
We investigate learning of the differential geometric structure of a data manifold embedded in a high-dimensional Euclidean space. We first analyze kernel-based algorithms and show that under the usual regularizations, non-probabilistic methods cannot recover the differential geometric structure, but instead find mostl…
The paper explains how ReLU nets converge globally in high dimensions without strict assumptions.
problem Understanding global convergence of ReLU nets in very high dimensions.
method Fine-grained analysis of random activation matrices and detailed gradient norm and curvature analysis.
result Empirical loss function has favorable geometrical properties in the overparameterized setting.
Gradient descent converges geometrically to optimal self-attention parameters.
problem Training softmax self-attention layers for linear regression.
method Structure-aware gradient descent with preconditioner and regularizer.
result Gradient descent converges geometrically to global minima.
New method preserves convergence rates in gradient-based optimization.
problem How to discretize gradient-based optimization systems while preserving stability and convergence rates.
method Geometric framework for dissipative symplectic integration.
result Dissipative symplectic integrators preserve rates of convergence up to a controlled error.
Langevin MCMC samples efficiently from Riemannian manifolds with geometric Euler-Murayama analysis.
problem Efficient sampling from Gibbs distributions on Riemannian manifolds.
method Geometric Langevin MCMC, discretization error bound, contraction guarantee for Langevin Diffusion.
result Langevin MCMC iterates converge to the target distribution after a number of steps proportional to the inverse square of the desired accuracy.
Develops methods to analyze feature-outcome associations in subpopulations.
problem Challenges in understanding feature-outcome associations in high-dimensional data.
method Geometric decomposition framework using gradient flow and co-monotonicity decomposition.
result Identifies context-dependent patterns and improves statistical power and interpretability.
The article derives gradient estimations for semilinear equations on geometric flows.
problem Gradient estimation for semilinear equations on geometric flows.
method Derives both Hamilton and Souplet-Zhang type gradient estimations.
result Gradient estimations for semilinear equations on geometric flows.
We prove a geometrically meaningful stochastic representation of the derivative of the heat semigroup on sub-Riemannian manifolds with tranverse symmetries. This representation is obtained from the study of Bochner-Weitzenbock type formulas for sub-Laplacians on 1-forms. As a consequence, we prove new hypoelliptic heat…
In this paper we introduce, in the Riemannian setting, the notion of conformal Ricci soliton, which includes as particular cases Einstein manifolds, conformal Einstein manifolds and (generic and gradient) Ricci solitons. We provide here some necessary integrability conditions for the existence of these structures that …
Second-order optimizers retain residual information after data deletion, affecting machine unlearning.
problem Residual information in second-order optimizers after data deletion.
method Comparison of first-order and second-order learners, eigendecomposition analysis.
result Second-order optimizers retain residual information, not detectable by first-order analysis.
Study geometric and analytical properties of ρ-Einstein solitons.
problem Characterize geometric and analytical features of ρ-Einstein solitons. method Analyze the spectrum of the drifted Laplacian operator and prove volume growth estimates.
result Establish new volume growth estimates for geodesic balls of complete noncompact ρ-Einstein solitons. This paper uses a geometric approach to understand how normalization layers affect neural network optimization.
problem Understanding the effect of normalization layers on optimization in neural networks.
method Introduces a spherical framework to study optimization dynamics of neural networks with normalization layers from a geometric perspective.
result Derives the first effective learning rate expression of Adam and shows that SGD with NLs is equivalent to a constrained variant of Adam.
GeoAdaLer enhances geometric understanding of Adam for stochastic optimization.
problem Understanding geometric principles behind Adam's success in stochastic optimization.
method Introduces GeoAdaLer, an adaptive learning method based on geometric properties.
result Extends interpretability and effectiveness in complex optimization scenarios.
Extends gradient estimates for heat equation under Finsler geometric flows.
problem Global gradient estimates for positive solutions to heat equation.
method General compact Finsler CD(−K,N) geometric flow. result Derives Harnack inequality for positive solutions.
Score matching errors are not sufficient for measuring diffusion model quality.
problem The L2 score matching error is not a reliable measure of diffusion model performance. method Decomposed score errors into gradient and solenoidal components and analyzed their geometric properties.
result Only the gradient component of the score error affects the marginal distributional quality.
The paper estimates gradients for a weighted parabolic equation under geometric flow.
problem Estimating gradients for a specific parabolic equation on a weighted manifold.
method Obtained space-time gradient estimates through integrating the equation.
result Found corresponding Harnack inequalities through gradient estimates.
This paper provides a geometrical derivation of the Hybrid Minimum Principle (HMP) for autonomous hybrid systems whose state manifolds constitute Lie groups (G,⋆) which are left invariant under the controlled dynamics of the system, and whose switching manifolds are defined as smooth embedded time invariant subma…
Extends geometric structures to manifolds with new operators.
problem No specific problem stated; extending geometric structures.
method Defines new gradient and Laplace operators on manifolds with geometric structures.
result Provides properties of the new operators.
A recent algorithmic family for distributed optimization, DIGing's, have been shown to have geometric convergence over time-varying undirected/directed graphs. Nevertheless, an identical step-size for all agents is needed. In this paper, we study the convergence rates of the Adapt-Then-Combine (ATC) variation of the DI…
Optimal algorithms for Riemannian optimization with reduced complexity.
problem Stochastic optimization on Riemannian manifolds with limited data.
method Zeroth-order Riemannian Averaging Stochastic Approximation algorithms using Riemannian moving-average estimators and novel geometric conditions.
result Achieves optimal sample complexities for generating approximate first-order stationary solutions.
Although stochastic gradient descent (SGD) is a driving force behind the recent success of deep learning, our understanding of its dynamics in a high-dimensional parameter space is limited. In recent years, some researchers have used the stochasticity of minibatch gradients, or the signal-to-noise ratio, to better char…
The paper studies Stein-Weiss operators on symmetric tensors, extending previous work.
problem Understanding Stein-Weiss operators on symmetric tensors of arbitrary rank.
method Analyzing the decomposition of tensor spaces into irreducible components and computing Weitzenbock formulas.
result Unified framework for second-order Stein-Weiss operators and tools for geometric analysis.
In this work we propose a differential geometric motivation for Nesterov's accelerated gradient method (AGM) for strongly-convex problems. By considering the optimization procedure as occurring on a Riemannian manifold with a natural structure, The AGM method can be seen as the proximal point method applied in this cur…
Last SGD iterate bounds for overparameterized linear regression.
problem Analyzing the last iterate risk bounds of SGD with decaying stepsize for overparameterized linear regression.
method Problem-dependent analysis of last iterate risk bounds of SGD with geometrically decaying stepsize.
result Proved nearly matching upper and lower bounds on the excess risk for last iterate SGD with geometrically decaying stepsize.
SGD and stochastic gradient descent converge at optimal rates for certain non-convex functions.
problem Optimal convergence rates for non-convex functions under gradient noise.
method Geometric interpretation of the PL-condition to analyze convergence rates.
result Convergence rates of SGD and stochastic gradient descent match those of strongly convex quadratics.
Study on geometric flows and rigidity of solitons.
problem Understanding rigidity and properties of gradient solitons.
method Identifying Hamilton's identity for geometric flows and proving its utility.
result Recovery of results about rigidity and properties for arbitrary geometric flows.
The paper provides a geometric framework for understanding non-equilibrium thermodynamics.
problem Unclear geometric structure of GENERIC in non-equilibrium thermodynamics.
method Cotangent lifts of dynamics, splitting into holonomic and vertical representatives, and formulation within contact geometry.
result Physical meaning and explicit formulation of the second law of thermodynamics within evolution equations.
Bayesian inference problems require sampling or approximating high-dimensional probability distributions. The focus of this paper is on the recently introduced Stein variational gradient descent methodology, a class of algorithms that rely on iterated steepest descent steps with respect to a reproducing kernel Hilbert …
Sharp gradient estimate for scalar curvature on 3-manifolds.
problem Control the rate of change of scalar curvature on 3-manifolds.
method Using a regularized distance function and Green's function, derive a sharp gradient estimate.
result Average of gradient of regularized distance is ≤ 1 on 3-manifolds with nonnegative scalar curvature.