Newton's method converges linearly for stable Hessians, even with approximations.
problem Finding global linear convergence for functions without strong convexity or Lipschitz gradients.
method Global linear convergence of Newton's method for stable Hessians, using approximate Hessians and subproblems.
result Global linear convergence rate for a broad class of functions, superior to first-order methods.
Our work connects parameter magnitudes and Hessian eigenspaces in deep neural nets.
problem Understanding the relationship between parameter magnitudes and Hessian curvature in deep learning models.
method Developed a matrix-free algorithm based on sketched SVDs to measure similarity between parameter masks and Hessian eigenspaces.
result Top Hessian eigenvectors tend to be concentrated around larger parameters, indicating a connection between parameter magnitudes and loss curvature.
Proves solvability of general inverse σ_k equations with constant coefficients.
problem Solvability of general inverse σ_k equations with constant coefficients.
method Proves existence of unique solution if a C-subsolution exists.
result Confirms analytical conjecture for deformed Hermitian--Yang--Mills equation.
Studied how SGD's stability regularization affects generalization in neural networks.
problem Understanding why SGD often generalizes better than GD in neural networks.
method Analyzed stability of SGD and GD through Frobenius norm and trace of Hessian, and compared their generalization properties.
result Stable minima of SGD generalize well, while GD's stability-induced regularization is too weak.
Large learning rates cause parameter instability, leading to better generalization.
problem Understanding why deep neural networks perform well despite operating outside the traditional stability regime.
method Analyzing the effect of large learning rates on the orientation of Hessian eigenvectors and parameter exploration.
result Large learning rates induce parameter instability, leading to better generalization through exploration of flatter regions of the loss landscape.
Meta-learning is a promising method to achieve efficient training method towards deep neural net and has been attracting increases interests in recent years. But most of the current methods are still not capable to train complex neuron net model with long-time training process. In this paper, a novel second-order meta-…
We study stability and local minimizing properties of Lp- norms of Riemannian curvature tensor denoted by Rp by variational methods. We compute the Hessian of Rp at compact rank 1 symmetric spaces and prove that they are stable for Rp for certain values of p > 2. A similar resu…
The study proves that certain noncompact Hessian manifolds are diffeomorphic to R^n.
problem Characterizing complete noncompact Hessian manifolds with nonnegative Hessian sectional curvature.
method Using a geometric flow on noncompact affine Riemannian manifolds, constructing Hessian metrics, and proving diffeomorphism.
result Complete noncompact Hessian manifolds with nonnegative Hessian sectional curvature are diffeomorphic to R^n if their tangent bundle has maximal volume growth.
New metric for probability measures connects physics and geometry.
problem Developing a new metric for probability measures.
method Transport Hessian metric, formulated dynamical systems.
result Connections to physics equations and mathematical models.
Investigates regularity of solutions to complex Hessian equation.
problem Regularity of solutions to complex Hessian equation.
method Analyzes solutions to Dirichlet problem with specific density condition.
result Establishes conditions for regularity of solutions.
Abstract: Survey on quadratic Hessian equations, their properties, and open problems.
problem Understanding quadratic Hessian equations and their solutions.
method Survey and review of existing research.
result Survey of entire solutions, viscosity solutions, and Hessian estimates.
Paper solves Hessian equations on Kähler manifolds.
problem Solving Hessian equations on Kähler manifolds.
method Combines elementary symmetric functions; provides sufficient and necessary condition.
result Generalizes results for Hessian and Hessian quotient equations.
New Hessian estimates for heat equations on manifolds.
problem Estimating Hessian matrices for heat-type equations on Riemannian manifolds.
method Using Bismut-Stroock Hessian formula, with explicit coefficients and delay/growth rate functions.
result Novel backward weak Harnack inequality and precise pointwise Hessian estimates for eigenfunctions.
Characterizes and describes selfsimilar Hessian manifolds with homothetic vector fields.
problem Understanding the structure and properties of selfsimilar Hessian manifolds.
method Characterization and description of selfsimilar manifolds with homothetic vector fields.
result Any selfsimilar Hessian manifold with a potential homothetic vector field is locally isomorphic to a product of radiant Hessian manifolds.
This paper analyzes stability of decision trees and logistic regression.
problem Stability of decision trees and logistic regression is analyzed to understand their performance and sensitivity.
method Two stability notions (hypothesis and pointwise hypothesis stability) are derived for decision trees and logistic regression. The stability of decision trees depends on the number of leaves, while for logistic regression, it depends on the smallest eigenvalue of the Hessian matrix. Upper bounds on generalization error are constructed.
result Logistic regression is not a stable learning algorithm.
We establish sufficient conditions for existence of curves minimizing length as measured with respect to a degenerate metric on the plane while enclosing a specified amount of Euclidean area. Non-existence of minimizers can occur and examples are provided. This continues the investigation begun in [ABCDS] where the met…
SGD noise helps select flat minima by concentrating in sharp directions and being proportional to loss value.
problem Understanding the implicit regularization of SGD and selecting flat minima in over-parameterized models.
method Relating SGD's linear stability to the Frobenius norm of the Hessian and analyzing the alignment property of SGD noise.
result Flat minima are linearly stable for SGD, and their sharpness is bounded independently of model size and sample size.
Accelerates optimal transport computation by 10x with spectral insights.
problem Exponential slow-down of convergence in Entropic Optimal Transport as regularization weakens.
method Spectral insights and spectral warm-start strategy to mitigate convergence issues.
result Faster convergence compared to the reference method Sinkhorn algorithm.
Study calculates Hessian of Busemann function on Damek-Ricci spaces.
problem Calculating Hessian of Busemann function on Damek-Ricci spaces.
method Calculates eigenvalues of Hessian and proves positive definiteness.
result Hessian of Busemann function is positive definite.
The paper describes flat Hessian metrics on surfaces and their potentials.
problem Understanding Hessian metrics on surfaces.
method Theoretical description and explicit construction using integrable systems.
result Explicit construction of potentials for flat Hessian metrics on surfaces.
The paper shows infinitely many components in Floer Hessians space.
problem Understanding the structure of Floer Hessians.
method Proving the existence of infinitely many connected components.
result Proves infinitely many connected components in Floer Hessians space.
The paper examines stability of subelliptic harmonic maps with potential.
problem Stability of subelliptic harmonic maps with potential.
method Derived first and second variation formulas, proved stability conditions, and gave instability results.
result Subelliptic harmonic maps with potential are stable under certain curvature and potential conditions.
The paper quantizes Hessian structures on R^2 using KV-algebras.
problem Quantizing Hessian structures on a 2D space.
method Deformation quantization within Koszul-Vinberg algebras.
result Established links between deformation theory and Hessian geometry.
We prove that, in dimensions greater than 2, the generic metric is not a Hessian metric and find a curvature condition on Hessian metrics in dimensions greater than 3. In particular we prove that the forms used to define the Pontryagin classes in terms of the curvature vanish on a Hessian manifold. By contrast all anal…
Curved Frobenius manifolds link to Hessian metrics in geometry.
problem Understanding curved Frobenius manifolds and their relation to Hessian metrics.
method Analyzing the relationship between curved Frobenius structures and Hessian metrics on spaces with non-vanishing curvature.
result Consistent curved Frobenius structures on constant curvature spaces are linked to Hessian metrics.
Superintegrable systems on curved manifolds found to have Hessian structures.
problem Characterizing superintegrable systems on curved manifolds.
method Identifying and computing Hessian coordinates for superintegrable systems.
result Examples of superintegrable systems in 2D and 3D have natural Hessian coordinates.
Study classifies 3D Hessian manifolds, proving their topology.
problem Global topology of 3D Hessian manifolds.
method Proved structure and analyzed Betti numbers.
result Complete topological classification of 3D Hessian manifolds.
New rigidity results for a generalized Ricci-Hessian equation on manifolds.
problem Understanding rigidity in generalized Ricci-Hessian equations on manifolds.
method Proving new rigidity results related to a generalized Ricci-Hessian equation.
result New rigidity results for the generalized Ricci-Hessian equation on Riemannian manifolds.
Criterion for solvability of complex 2-Hessian equation on compact Kähler manifolds.
problem Solvability of complex 2-Hessian equation on compact Kähler manifolds.
method Nakai--Moishezon-type criterion associated with the complex 2-Hessian equation.
result Criterion equivalent to existence of a smooth 2-admissible representative in complex dimension three.
Establishes a concavity property for positive Hessian quotient operators.
problem Analyzing positive Hessian quotient operators on Riemannian manifolds.
method Proves a special concavity property and a Jacobi inequality.
result Proves a Jacobi inequality for symmetric tensors.
Estimates for complex Hessian equations on Hermitian manifolds.
problem Establishing estimates for solutions to complex Hessian equations.
method Using concavity inequality for complex sum-of-Hessian operators.
result Second-order estimates for admissible solutions on Hermitian manifolds.
Derives Hessian estimates for Lagrangian mean curvature equation.
problem Lagrangian mean curvature equation with supercritical phase and bounded second derivatives.
method Derives a priori interior Hessian estimates.
result Hessian estimates for Lagrangian mean curvature equation.
Constructs homogeneous Kähler structures on tangent bundles of Hessian manifolds.
problem Creating Kähler structures on tangent bundles of Hessian manifolds.
method Endowing Hessian manifolds with Kähler structures using group actions and homothetic vector fields.
result Homogeneous conformally Kähler structures on tangent bundles of selfsimilar Hessian manifolds.
A new unbiased Hessian estimator for expectation-based objectives.
problem Estimating Hessian for objectives with non-reparameterizable nodes.
method GO Hessian estimator for expectation-based objectives.
result GO Hessian provides unbiased and low-variance estimation of Hessian.
Paper proves inequalities on Hermitian manifolds with applications to bounded solutions.
problem Establishing mixed Hessian inequalities on Hermitian manifolds.
method Weak convergence theorem of complex Hessian operators and general mixed Hessian inequality.
result Existence of bounded solutions of complex Hessian equations.
Study of contravariant pseudo-Hessian manifolds and their Poisson structures.
problem Understanding properties of contravariant pseudo-Hessian manifolds.
method Investigation of flat connections and symmetric bivector fields satisfying a contravariant Codazzi equation.
result Association of a Poisson tensor to contravariant pseudo-Hessian manifolds.
In this paper, we implement the Stochastic Damped LBFGS (SdLBFGS) for stochastic non-convex optimization. We make two important modifications to the original SdLBFGS algorithm. First, by initializing the Hessian at each step using an identity matrix, the algorithm converges better than original algorithm. Second, by pe…
This paper uncovers the low-rank structure of neural network Hessians.
problem Understanding the structure of Hessians in neural networks.
method Proposes a decoupling conjecture to decompose layer-wise Hessians into Kronecker products of smaller matrices.
result Proves the structure of top eigenspaces in 2-layer networks and shows high overlap in top eigenvectors across different models.
The paper studies efficient Hessian fitting methods for stochastic optimization.
problem Efficient Hessian fitting for stochastic optimization.
method Preconditioned Stochastic Gradient Descent (PSGD) method and Lie groups.
result Hessian fitting problem is strongly convex in certain Lie groups.
Study solves complex Hessian equation on Hermitian manifolds.
problem Solving Hessian equations on Hermitian manifolds with mixed structure.
method Derive a priori estimates and solve Dirichlet problem under conditions.
result Solvability of the Dirichlet problem for mixed Hessian equations.
Study confirms Chern's conjecture on compact Hessian manifolds and classifies their topologies.
problem Global topological constraints and structural properties of compact Hessian manifolds.
method Novel fibration and splitting theorems, Chern's conjecture, Hitchin systems, Cheng-Yau solution.
result Topological classification of complete Hessian surfaces and closed orientable Hessian 3-manifolds.
Derives concavity inequality and estimates for k-Hessian equations.
problem Interior estimates and curvature estimates for k-Hessian equations. method Concavity inequality and semi-convexity condition.
result Interior estimates and Liouville-type result for semi-convex solutions.
Paper discusses solving generalized Hessian inequalities with various operators.
problem Finding global solutions to generalized Hessian inequalities.
method Analyzes various Hessian operators and provides conditions for global solvability.
result Provides necessary and sufficient conditions for global solvability of generalized Hessian inequalities.
Hessian alignment improves OOD generalization in deep learning.
problem Improving deep learning models' ability to generalize to out-of-distribution data.
method Analyzed Hessian and gradient alignment for domain generalization using recent OOD theory.
result Hessian alignment methods achieve promising performance on various OOD benchmarks.
New perspective on CNNs using Hessian maps reveals their structure.
problem Understanding the nature of Convolutional Neural Networks (CNNs).
method Developed a framework using Toeplitz representation of CNNs to reveal Hessian structure and prove rank bounds.
result Proved that the Hessian rank of CNNs grows as the square root of the number of parameters.
Study deepnet Hessians, revealing spiked spectrum with SGD and sample size effects.
problem Understanding the spectrum of deepnet Hessians at scale.
method Applied modern numerical linear algebra tools to approximate Hessian spectrum of large deepnets.
result Deepnet Hessians exhibit spiked behavior with outliers isolated from a continuous bulk.
Bounds on Hessian of heat equation coupled with Ricci flow.
problem Estimating the Hessian of a solution to the conjugate heat equation coupled with Ricci flow.
method Obtained upper bounds for the Hessian.
result Local and global upper bounds for the Hessian of a positive solution.
The study classifies Riemannian manifolds with specific Hessian properties.
problem Classifying Riemannian manifolds with a special Hessian structure.
method Analyzing manifolds with a non-identically vanishing function f whose Hessian is minus f times the Ricci tensor.
result Partial classification of manifolds with this Hessian structure.