The study analyzes sharpness dynamics in neural networks, revealing mechanisms and conditions.
problem Understanding sharpness in neural network training.
method Fixed point analysis and edge of stability analysis in a simplified 2-layer linear network.
result Reveals mechanisms behind sharpness trends, conditions for edge of stability, and a period-doubling route to chaos.
Sharp stability in Almgren problem solved in any dimension.
problem Quantitative stability in the radial isotropic Almgren problem.
method Developed a theory for estimating the sharp modulus under minimal assumptions.
result Sharp ε 2 ε^2 ε 2 in any dimension, solving the critical mass problem. We prove a sharp stability estimate for the problem of reconstructing a symmetric 2-tensor from its integrals along all maximal geodesics on a simple manifold.
Gradient descent near stability threshold exhibits sharpness oscillations.
problem Understanding sharpness behavior near stability threshold in non-Euclidean norms.
method Interpreted EoS through Directional Smoothness and generalized sharpness under arbitrary norms.
result Non-Euclidean GD with generalized sharpness shows sharpness oscillations near 2 / η 2/η 2/ η . Weight decay stabilizes training dynamics by slowing progressive sharpening.
problem Understanding how weight decay affects training stability in deep learning models.
method Analyzing weight decay effects at the Edge of Stability, developing a mathematical framework.
result Weight decay dampens oscillations and stabilizes sharpness in CNNs, causing a phase transition in MLPs.
A simple function shows how neural nets can converge despite high sharpness.
problem Understanding why neural nets converge with high sharpness.
method Constructed a minimal example function and analyzed its training dynamics rigorously.
result Final converging point has sharpness close to 2 / η 2/η 2/ η . Gradient descent near stability threshold shows sharpness oscillations.
problem Understanding sharpness and stability in non-Euclidean norms during gradient descent.
method Interpreted EoS through Directional Smoothness, defined generalized sharpness for arbitrary norms.
result Non-Euclidean GD exhibits sharpness oscillations around the stability threshold.
Sharp stability results for reverse isoperimetric inequalities in 2D.
problem Reverse isoperimetric inequalities in the plane.
method Stability analysis of λ \lambda λ -convex bodies and convex bodies with smooth boundaries. result Sharp stability results for reverse isoperimetric inequalities, including inradius and Cheeger inequalities.
Gradient descent at edge of stability stabilizes implicitly, following projected gradient descent.
problem Gradient descent's stability and sharpness behavior at the edge of instability.
method Cubic Taylor expansion analysis of gradient descent dynamics.
result Gradient descent at edge of stability implicitly follows projected gradient descent.
Sharp stability of isometries on Heisenberg group proven.
problem Quantitative stability of isometries on the Heisenberg group.
method Proving quasi-isometries close to isometries with specific closeness orders.
result Quasi-isometries of Heisenberg group are close to isometries with specific closeness orders.
New estimates show spectral gap stability in RCD spaces, close to Beta distribution.
problem Stability of spectral gap bounds in metric-measure spaces.
method Combines L 1 L^1 L 1 -functional inequality and Stein's method. result Sharp quantitative estimate for spectral gap stability.
New stability framework relaxes boundedness assumptions for generalization bounds.
problem Overly restrictive assumptions for modern learning settings with heavy-tailed or unbounded losses.
method Develops a stability-based framework requiring only finite L p L_p L p moment conditions. result Sharp generalization bounds derived for various learning paradigms.
SAM improves generalization by operating near the edge of stability.
problem Improving generalization in neural networks.
method Sharpness-Aware Minimization (SAM) approach to training neural networks.
result SAM operates near the 'edge of stability' identified by the analysis.
The study examines rigidity and stability of gradient estimates on surfaces and manifolds.
problem Rigidity and stability of gradient estimates for positive harmonic functions and solutions to heat equations.
method Sharp gradient estimates for positive harmonic functions and solutions to heat equations on surfaces and manifolds with nonnegative curvature.
result Obtained rigidity and stability results for gradient estimates.
Study on curve diffusion flows with scale-critical curvature term.
problem Analyzing stability of curve diffusion flows with scale-critical curvature.
method Introduced and studied a one-parameter family of curve diffusion flows with a scale-critical cubic curvature term. Analyzed dynamical stability of homothetic circles using variational methods.
result Established that any small perturbation of an ω ω ω -fold circle monotonically approaches the unit ω ω ω -circle after rescaling, translation, and reparametrisation. Momentum affects optimization differently at small vs large batch sizes near instability.
problem Understanding how momentum impacts optimization near the edge of stability.
method Demonstrated through batch-size dependent behavior of SGD with momentum.
result Momentum operates in two distinct regimes: amplifying stochastic fluctuations at small batch sizes and stabilizing at large batch sizes.
New findings show mini-batch SGD operates in a 'Edge of Stochastic Stability' regime.
problem Understanding the stability and convergence of mini-batch SGD.
method Analyzing the mini-batch Hessian and its directional curvature.
result Mini-batch SGD operates in a different stability regime (Edge of Stochastic Stability) compared to full-batch GD.
Explains optimal functional inequalities, focusing on Sobolev and fractional Sobolev.
problem Optimal functional inequalities and their stability.
method Compactness theorems, characterization of optimizers, and quantitative stability analysis.
result Characterization and stability of optimizers for Sobolev inequalities and their fractional generalizations.
Sharp stability of Möbius group among sphere-valued maps proved in arbitrary dimensions.
problem Proving a sharp quantitative form of Liouville's theorem for sphere-valued maps.
method New arguments and an inequality from Sobolev inequality proof.
result Sharp stability estimate for weakly conformal maps of arbitrary dimensions.
Sharp stability of Alexandrov's theorem for C 1 C^1 C 1 domains in the small-excess regime
problem Stability of Alexandrov's theorem for C 1 C^1 C 1 domains in the small-excess regime method Combines a B V BV B V version of Fuglede's spectral-gap argument, a star-shaped rearrangement for sets of finite perimeter, quantitative estimates for the part of the boundary contained in the tentacles, and a polyhedral approximation argument for the non-graphical region result Sharp stability estimate in a genuinely non-parametric regime
Sharp upper bounds found for Steklov eigenvalues of a specific hypersurface.
problem Finding upper bounds for Steklov eigenvalues of a specific type of hypersurface.
method Analytical approach to compute upper bounds and prove stability properties.
result Sharp upper bounds B n ( L ) B_n(L) B n ( L ) and B n B_n B n for Steklov eigenvalues are derived. Tian's criterion for K-stability states that a Fano variety of dimension n n n whose alpha invariant is greater than n n + 1 \frac{n}{n+1} n + 1 n is K-stable. We show that this criterion is sharp by constructing singular Fano varieties with alpha invariants n n + 1 \frac{n}{n+1} n + 1 n that are not K-polystable for sufficiently large n n n . We also…
New insights into network generalization show learning rate affects both norm and sharpness.
problem Understanding the generalization of overparameterized networks.
method Empirical analysis and theoretical proof of the trade-off between norm and sharpness.
result Learning rate influences both norm and sharpness, neither alone minimizes generalization error.
We prove that the boundary of a (not necessarily connected) bounded smooth set with constant nonlocal mean curvature is a sphere. More generally, and in contrast with what happens in the classical case, we show that the Lipschitz constant of the nonlocal mean curvature of such a boundary controls its C 2 C^2 C 2 -distance fro…
GD monotonically decreases GFS sharpness in neural networks and scalar models.
problem Oscillatory behavior of loss in GD training.
method Analysis of GFS sharpness and empirical validation.
result GFS sharpness decreases monotonically during GD training.
The paper examines stability of the Sobolev inequality in metric spaces with curvature dimension conditions.
problem Investigating stability of the Sobolev inequality in metric spaces with curvature dimension conditions.
method Assuming almost the same optimal constant, the paper shows that the cumulative distribution of almost extremal functions is close to that of an Aubin-Talenti bubble on the round sphere.
result Quantitative stability with sharp exponent for the Sobolev inequality in various curvature and dimension assumptions.
We prove a sharp L 2 → H 1 / 2 L^2\to H^{1/2} L 2 → H 1/2 stability estimate for the geodesic X-ray transform of tensor fields of order 0 0 0 , 1 1 1 and 2 2 2 on a simple Riemannian manifold with a suitable chosen H 1 / 2 H^{1/2} H 1/2 norm. We show that such an estimate holds for a family of such H 1 / 2 H^{1/2} H 1/2 norms, not topologically equivalent, but equivalent o…
Sharp Talenti-type comparison theorem for p-Laplacian on RCD(K,N) spaces.
problem Understanding the p-Laplacian on RCD(K,N) spaces.
method Proving a Talenti-type comparison theorem.
result Sharp, rigid and stable Talenti-type comparison theorem.
Cohen et al. (2021) show GD trajectories align on a bifurcation diagram.
problem Understanding the Edge of Stability (EoS) phenomenon in gradient descent.
method Empirical studies and rigorous mathematical proofs for two-layer networks and single-neuron networks.
result GD trajectories align on a specific bifurcation diagram independent of initialization.
SAM optimizer benefits from normalization, stabilizing and guiding optimization.
problem Improving deep neural network performance with SAM optimizer.
method Theoretical and empirical study of normalization in SAM for convex and non-convex functions.
result Normalization helps SAM in stabilizing and guiding optimization along a continuum of minima.
We introduce a scalable measure of curvature for analyzing training dynamics of large language models.
problem Analyzing the training dynamics of large language models due to high computational cost of measuring Hessian sharpness.
method We introduce critical sharpness and relative critical sharpness as computationally efficient measures capturing Hessian sharpness phenomena.
result We provide the first demonstration of sharpness phenomena at scale up to 7B parameters.
Training avoids edge of stability by aligning Jacobian matrices.
problem Training neural networks on the edge of stability causes inaccuracies.
method Used an exponential Euler solver to prevent entering the edge of stability.
result Alignment of Jacobian matrices causes sharpness increase in Hessian.
Optimizes sharp curvature inequality on spheres, proving near-minimizers are close to standard metric.
problem Optimizing total σ 2 σ_2 σ 2 -curvature on spheres with positive scalar curvature. method Analyzes metrics conformal to the standard sphere, uses Sobolev norms to measure closeness.
result Near-minimizers of total σ 2 σ_2 σ 2 -curvature are almost the standard metric (up to Möbius transformations). The study bounds the stability of Gaussian mixtures under small perturbations.
problem Stability of Gaussian mixtures under small changes in distribution.
method Deriving an explicit bound on parameter stability of spherical Gaussian Mixture Models (sGMM) in a pre-defined model class.
result Upper bound on parameter distance of close sGMMs to the original sGMM, dependent only on the original model.
For a convex domain D D D bounded by the hypersurface ∂ D \partial D ∂ D in a space of constant curvature we give sharp bounds on the width R − r R-r R − r of a spherical shell with radii R R R and r r r that can enclose ∂ D \partial D ∂ D , provided that normal curvatures of ∂ D \partial D ∂ D are pinched by two positive constants. Furthermore, in the …
We study the stability and convergence of training deep ResNets with gradient descent. Specifically, we show that the parametric branch in the residual block should be scaled down by a factor τ = O ( 1 / L ) τ=O(1/\sqrt{L}) τ = O ( 1/ L ) to guarantee stable forward/backward process, where L L L is the number of residual blocks. Moreover, we establi…
Study shows how a curve shortens to a half-circle under specific flow.
problem Stability of a semi-circle under curve shortening flow.
method Sharp rate of convergence for a free-boundary curve shortening flow in a convex domain.
result Established a sharp rate of convergence to a round half-point.
The paper studies rigidity and stability of minimal submanifolds in hyperbolic space.
problem Conditions for a minimal submanifold to be totally geodesic.
method Analyzes the length of the second fundamental form and eigenvalues of the super stability operator.
result Sharp upper bounds for the first eigenvalue of the super stability operator for surfaces in hyperbolic 4-space.
Sharp stability threshold found for deep residual architectures.
problem Ensuring stable training and inference in deep residual networks.
method Sublinear-growth principle and optimal-control analysis.
result Stable training condition: input-magnitude exponent q ≤ 1.
The paper proves and analyzes Minkowski inequalities for nearly spherical domains.
problem Validating and stabilizing Minkowski inequalities for perturbed balls.
method Analyzing C 1 C^1 C 1 -perturbations of the ball, proving sharp and almost sharp inequalities. result Sharp geometric and almost sharp Minkowski inequalities for nearly spherical domains.
Sharp stability result for maps near infinitely concentrated minimisers.
problem Stability of maps near minimisers with infinite concentration.
method Dynamic approach to deform maps into harmonic maps, controlling topology changes.
result Sharp quantitative estimates on map distance to infinitely concentrated minimisers.
New bounds improve generalization in learning scenarios.
problem Limitations of existing information-theoretic bounds in SCO problems.
method Sample-conditioned hypothesis stability and neighboring-hypothesis matrix.
result Sharper generalization guarantees in various learning scenarios.
Accelerated gradient method's stability deteriorates exponentially with steps.
problem Algorithmic stability of Nesterov's accelerated gradient method.
method Analysis of two notions of algorithmic stability for Nesterov's accelerated gradient method.
result Stability of Nesterov's accelerated method deteriorates exponentially with the number of gradient steps.
Sharp upper bounds on inscribed radius for metric spaces with convex boundary.
problem Bounding inscribed radius in metric measure spaces with convex boundary.
method Proves sharp upper bounds on inscribed radius for subsets with convex boundary.
result Sharp upper bounds on inscribed radius for subsets with convex boundary.
Study on stability of quaternion-Kähler manifolds using eigenvalue estimates.
problem Stability problem for positive quaternion-Kähler manifolds.
method Description of infinitesimal Einstein deformations and destabilising directions in terms of Laplace eigenfunctions and symmetric 2-tensors. Improved eigenvalue estimates for the Hodge-Laplacian on 2-forms.
result Sharp lower bound for the first non-zero eigenvalue on the parallel subbundle Sym^2 E of the 2-form bundle.
New isoperimetric inequality for clamped plates in RCD(0,N) spaces, sharp and stable.
problem Fine properties of the principal frequency of clamped plates in RCD(0,N) spaces.
method Analyzing the RCD(0,N) spaces and applying isoperimetric inequalities.
result Sharp isoperimetric inequality for the principal frequency of clamped plates in RCD(0,N) spaces.
Flow preserves curvature sharpness on weighted graphs.
problem Curvature flow on weighted graphs.
method Adapting Bakry-Émery calculus for Markovian preservation and analyzing limits.
result Flow limits to curvature sharp weighted graphs.
Sharp Gaussian isoperimetry proven along Ricci flow.
problem Proving sharp Gaussian isoperimetric inequality for Ricci flow.
method Using monotonicity formula to prove inequality.
result Exact Gaussian enlargement theorem and concentration estimates.