New proof for stability estimates in complex equations without pluripotential theory.
problem Stability estimates for complex Monge-Ampère and Hessian equations.
method New proof using general degenerations of background metrics.
result Uniform stability estimates for both equations under various degenerations.
New stability conditions for ZO methods reveal unique regularization effects.
problem Understanding optimization dynamics of ZO methods in deep learning.
method Explicit step size conditions and stability bounds derived for ZO methods.
result ZO methods operate near the edge of stability, with regularization effects specific to Hessian trace vs. eigenvalue.
New quasimetric spaces improve stability in complex Hessian equations.
problem Improving stability results for complex Hessian equations.
method Constructing a family of quasimetric spaces in generalized potential theory.
result Convergence of quasimetric spaces leads to improved stability results.
The paper analyzes stability and convergence rates of entropic and Sinkhorn potentials.
problem Stability and convergence rates of entropic and Sinkhorn potentials.
method Semiconcavity properties of entropic potentials and Schrödinger bridges.
result Exponential convergence rates for gradient and Hessian of Sinkhorn iterates.
Author presents the second variational formula for statistical biharmonic maps.
problem Developing a formula for statistical biharmonic maps.
method Introduced the second variational formula for the statistical bi-energy functional.
result The second variational formula can be represented using Hessian curvature in Hessian manifolds.
Gradient descent forces neural network eigenvalues to a specific threshold.
problem Understanding why gradient descent drives eigenvalues to a specific threshold.
method Introduced edge coupling, a functional on consecutive iterate pairs, to explain the trajectory towards the eigenvalue threshold.
result Gradient descent forces the Hessian eigenvalue to the threshold 2/η from arbitrary initialization. New criterion for solving inverse Hessian equations, including J-equation.
problem Existence of solutions to inverse Hessian equations, including J-equation.
method Stability of pairs in the sense of Paul, formulated in terms of GIT criterion.
result New numerical criterion for existence of solutions to inverse Hessian equations.
Training avoids edge of stability by aligning Jacobian matrices.
problem Training neural networks on the edge of stability causes inaccuracies.
method Used an exponential Euler solver to prevent entering the edge of stability.
result Alignment of Jacobian matrices causes sharpness increase in Hessian.
New Einstein metrics identified on a specific full flag manifold.
problem Investigating Einstein metrics on a full flag manifold.
method Analyzing G-stability of Einstein metrics on M=G/K. result Identified four new Einstein metrics on F(5), confirming pairwise non-homotheticity. On a constraint manifold we give an explicit formula for the Hessian matrix of a cost function that involves the Hessian matrix of a prolonged function and the Hessian matrices of the constraint functions. We give an explicit formula for the case of the orthogonal group O(n) by using only Euclidean coordinates …
New method accelerates neural network training by focusing on flat directions.
problem Improving neural network training speed and stability.
method Bulk-SGD, interpolated gradient methods.
result Updates along the Dominant subspace can accelerate convergence but compromise stability.
Studied how SGD's stability regularization affects generalization in neural networks.
problem Understanding why SGD often generalizes better than GD in neural networks.
method Analyzed stability of SGD and GD through Frobenius norm and trace of Hessian, and compared their generalization properties.
result Stable minima of SGD generalize well, while GD's stability-induced regularization is too weak.
Study on existence and properties of continuous solutions to complex Hessian equations.
problem Existence and properties of continuous solutions to complex Hessian equations.
method Established new capacity estimates and weak stability estimates for the m-Hessian measure. result Existence of continuous solutions to the complex Hessian equation under certain conditions.
This paper proves a Nakai-Moishezon criterion for complex Hessian equations.
problem The solvability of complex Hessian equations on Kähler manifolds.
method Establishing a Nakai-Moishezon criterion for Kähler classes on analytic Kähler varieties.
result Proves Lejmi-Szekelyhidi's conjecture for the J-equation. Solves complex Hessian equations in unstable cases, proving unique canonical solutions with singularities.
problem Existence of smooth solutions to complex Hessian equations in unstable cases.
method Parabolic flows and moment-map energy functionals, focusing on J-equation and deformed Hermitian Yang-Mills equation.
result Proves existence of unique canonical solutions with singularities on Kahler surfaces.
New findings show mini-batch SGD operates in a 'Edge of Stochastic Stability' regime.
problem Understanding the stability and convergence of mini-batch SGD.
method Analyzing the mini-batch Hessian and its directional curvature.
result Mini-batch SGD operates in a different stability regime (Edge of Stochastic Stability) compared to full-batch GD.
We prove the existence of weak solutions of complex m−Hessian equations on compact Hermitian manifolds for the nonnegative right hand side belonging to Lp,p>n/m (n is the dimension of the manifold). For smooth, positive data the equation has been recently solved by Szekelyhidi and Zhang. We also give a stabilit…
Solves open problems for fully nonlinear elliptic equations on manifolds.
problem Solving fully nonlinear elliptic equations on manifolds.
method Analytic slope invariant and Nakai-Moishezon criterion.
result Solves open problems including hessian and hessian quotient equations.
Improved graph neural network bounds using graph diffusion matrix.
problem Empirical performance of graph neural networks on real-world graphs.
method Unified model of graph neural networks, focusing on feature diffusion matrix stability.
result Generalization bounds scale with largest singular value of feature diffusion matrix, smaller than prior bounds.
Stability results for geometric equations in warped product spaces.
problem Geometric partial differential equations in warped product spaces.
method Stability theorem development for level sets of functions.
result Quantitative stability theorems for Serrin's problem and Alexandroff's theorem.
We analyze the Hessian spectra of large models up to 100B parameters.
problem Accurate Hessian spectra of large foundation models are difficult to obtain.
method We use shard-local finite-difference Hessian vector products and stochastic Lanczos quadrature.
result We produce the first large-scale spectral density estimates of foundation models.
Our work connects parameter magnitudes and Hessian eigenspaces in deep neural nets.
problem Understanding the relationship between parameter magnitudes and Hessian curvature in deep learning models.
method Developed a matrix-free algorithm based on sketched SVDs to measure similarity between parameter masks and Hessian eigenspaces.
result Top Hessian eigenvectors tend to be concentrated around larger parameters, indicating a connection between parameter magnitudes and loss curvature.
Gradient descent at edge of stability stabilizes implicitly, following projected gradient descent.
problem Gradient descent's stability and sharpness behavior at the edge of instability.
method Cubic Taylor expansion analysis of gradient descent dynamics.
result Gradient descent at edge of stability implicitly follows projected gradient descent.
Paper controls shape stability in infinite Riemannian manifolds.
problem Characterizing optimal shapes in infinite-dimensional Riemannian manifolds.
method Uses Riemannian manifold framework and mean curvature analysis.
result Control on shape stability depends only on mean curvature.
We study stability and local minimizing properties of Lp- norms of Riemannian curvature tensor denoted by Rp by variational methods. We compute the Hessian of Rp at compact rank 1 symmetric spaces and prove that they are stable for Rp for certain values of p > 2. A similar resu…
The paper introduces a Hessian-based method to improve generalization in fine-tuned deep neural networks.
problem Improving generalization in fine-tuned deep neural networks, especially in noisy conditions.
method PAC-Bayesian analysis to identify a Hessian-based distance measure, proving generalization bounds, and developing an algorithm with a generalization error guarantee.
result Hessian-based distance measure correlates well with observed generalization gaps and can match the scale of these gaps in practice.
The paper examines stability of subelliptic harmonic maps with potential.
problem Stability of subelliptic harmonic maps with potential.
method Derived first and second variation formulas, proved stability conditions, and gave instability results.
result Subelliptic harmonic maps with potential are stable under certain curvature and potential conditions.
We introduce a scalable measure of curvature for analyzing training dynamics of large language models.
problem Analyzing the training dynamics of large language models due to high computational cost of measuring Hessian sharpness.
method We introduce critical sharpness and relative critical sharpness as computationally efficient measures capturing Hessian sharpness phenomena.
result We provide the first demonstration of sharpness phenomena at scale up to 7B parameters.
Unified treatment of stability problems in geometry and analysis.
problem Spherical closeness of hypersurfaces under geometric constraints.
method Estimate relating distance to geodesic spheres with norms of traceless Hessian operator.
result Unified treatment of stability problems in geometry and analysis.
Noise injection regularizes Hessian, improving neural network training and generalization.
problem Regularizing over-parameterized neural networks with nonconvex and nonlinear geometry.
method Injecting isotropic Gaussian noise into weight matrices and designing a two-point estimate of the Hessian penalty.
result Effective regularization of Hessian improves generalization, achieving up to 2.4% test accuracy increase.
SGD noise helps select flat minima by concentrating in sharp directions and being proportional to loss value.
problem Understanding the implicit regularization of SGD and selecting flat minima in over-parameterized models.
method Relating SGD's linear stability to the Frobenius norm of the Hessian and analyzing the alignment property of SGD noise.
result Flat minima are linearly stable for SGD, and their sharpness is bounded independently of model size and sample size.
The paper proves stability of Wulff shapes using anisotropic curvature functionals.
problem Stability of Wulff shapes under anisotropic curvature.
method Estimates distance to Wulff shape using Lp-norm of traceless F-Hessian of a foliating function. result Quantitative stability results for anisotropic inequalities and problems.
Large learning rates cause parameter instability, leading to better generalization.
problem Understanding why deep neural networks perform well despite operating outside the traditional stability regime.
method Analyzing the effect of large learning rates on the orientation of Hessian eigenvectors and parameter exploration.
result Large learning rates induce parameter instability, leading to better generalization through exploration of flatter regions of the loss landscape.
SAM improves generalization by operating near the edge of stability.
problem Improving generalization in neural networks.
method Sharpness-Aware Minimization (SAM) approach to training neural networks.
result SAM operates near the 'edge of stability' identified by the analysis.
Gradient descent on neural nets often operates at the Edge of Stability, where loss behavior is complex but loss decreases over time.
problem Understanding the optimization dynamics of neural networks at the Edge of Stability.
method Empirical demonstration of gradient descent behavior in neural network training.
result Gradient descent on neural networks typically occurs at the Edge of Stability, where loss behavior is non-monotonic but loss decreases over time.
New stability bounds for Sinkhorn's algorithm in entropic optimal transport.
problem Stability and convergence of Sinkhorn's algorithm for entropic optimal transport.
method Semiconcavity approach to analyze stability and convergence.
result Exponential convergence of Sinkhorn's algorithm under semiconcavity conditions.
Proposes a new method to initialize neural networks by estimating global curvature of weights.
problem Improving the initialization of neural networks for better training and convergence.
method Estimates the global curvature of weights across layers using the Hessian matrix norm.
result The proposed method helps in more rigorously initializing weights, leading to better performance.
Lipschitz RNNs improve stability and performance in various tasks.
problem Improving stability and performance of RNNs.
method Introduced a Lipschitz recurrent unit with a linear and Lipschitz nonlinear component for stability analysis.
result Lipschitz RNNs outperform existing units on benchmark tasks.
Paper confirms conjecture for projective manifolds in supercritical phase.
problem Stability condition for deformed Hermitian-Yang-Mills equation.
method Establishes stability result not involving uniform constants.
result Confirms conjecture for projective manifolds in supercritical phase.
The paper analyzes stability and generalization of shallow neural networks using gradient methods.
problem Understanding the generalization of overparameterized shallow neural networks.
method The paper uses gradient descent and stochastic gradient descent to study shallow neural networks, developing consistent excess risk bounds.
result The analysis improves on existing methods by providing a refined estimation of iterates and Hessian eigenvalues, leading to better excess risk bounds.
Gradient descent performs well on weakly convex losses, offering generalization guarantees.
problem Learning with weakly convex losses using gradient descent.
method Analyzing the stability of gradient descent through the smallest eigenvalue of the Hessian.
result Generalization error bounds hold under a wider range of step sizes.
The study proves stability of the positive mass theorem for Kähler manifolds.
problem Stability of the positive mass theorem for Kähler manifolds.
method Integral inequality and stability results for ADM mass on AE Kähler manifolds.
result Stability of the positive mass theorem for Kähler manifolds under certain conditions.
EoS selectively shapes learning, affecting some groups more than others.
problem EoS affects learning differently across the data distribution.
method Branching intervention to enter or exit EoS regime, controlled perturbation to isolate mechanisms.
result EoS redistributes learning, amplifying progress on some groups and suppressing others.
Differentiable architecture search (DARTS) is a prevailing NAS solution to identify architectures. Based on the continuous relaxation of the architecture space, DARTS learns a differentiable architecture weight and largely reduces the search cost. However, its stability has been challenged for yielding deteriorating ar…
Gradient descent near stability threshold exhibits sharpness oscillations.
problem Understanding sharpness behavior near stability threshold in non-Euclidean norms.
method Interpreted EoS through Directional Smoothness and generalized sharpness under arbitrary norms.
result Non-Euclidean GD with generalized sharpness shows sharpness oscillations near 2/η. Variance reduction techniques like SVRG provide simple and fast algorithms for optimizing a convex finite-sum objective. For nonconvex objectives, these techniques can also find a first-order stationary point (with small gradient). However, in nonconvex optimization it is often crucial to find a second-order stationary…
Decision trees and logistic regression are one of the most popular and well-known machine learning algorithms, frequently used to solve a variety of real-world problems. Stability of learning algorithms is a powerful tool to analyze their performance and sensitivity and subsequently allow researchers to draw reliable c…
Gradient descent near stability threshold shows sharpness oscillations.
problem Understanding sharpness and stability in non-Euclidean norms during gradient descent.
method Interpreted EoS through Directional Smoothness, defined generalized sharpness for arbitrary norms.
result Non-Euclidean GD exhibits sharpness oscillations around the stability threshold.