arXiv research

Investigates regularity of solutions to complex Hessian equation.

New algorithm adds Hessian regularization to improve neural network robustness.

Noise injection regularizes Hessian, improving neural network training and generalization.

Abstract: Survey on quadratic Hessian equations, their properties, and open problems.

The paper studies the smoothness of critical points of variational integrals on Hessian spaces.

Hessian alignment improves OOD generalization in deep learning.

Stochastic Variance-Reduced Cubic regularization (SVRC) algorithms have received increasing attention due to its improved gradient/Hessian complexities (i.e., number of queries to stochastic gradient/Hessian oracles) to find local minima for nonconvex finite-sum optimization. However, it is unclear whether existing SVR…

Study on eigenvalues of complex Hessian operator on pseudoconvex manifolds.

New stability conditions for ZO methods reveal unique regularization effects.

We consider variants of trust-region and cubic regularization methods for non-convex optimization, in which the Hessian matrix is approximated. Under mild conditions on the inexact Hessian, and using approximate solution of the corresponding sub-problems, we provide iteration complexity to achieve $ε$-approximate seco…

We study Hessian fully nonlinear uniformly elliptic equations and show that the second derivatives of viscosity solutions of those equations (in 12 or more dimensions) can blow up in an interior point of the domain. We prove that the optimal interior regularity of such solutions is no more than C^{1+ε}, showing the opt…

We propose a sample efficient stochastic variance-reduced cubic regularization (Lite-SVRC) algorithm for finding the local minimum efficiently in nonconvex optimization. The proposed algorithm achieves a lower sample complexity of Hessian matrix computation than existing cubic regularization based methods. At the heart…

Unified framework for understanding and optimizing training acceleration.

Adapts PDE method to prove $L^\infty$ estimates for complex Hessian equations.

Established in the 30's, Schauder {\it a priori} estimates are among the most classical and powerful tools in the analysis of problems ruled by 2nd order elliptic PDEs. Since then, a central problem in regularity theory has been to understand Schauder type estimates fashioning particular borderline scenarios. In such c…

With the rapid development of social media sharing, people often need to manage the growing volume of multimedia data such as large scale video classification and annotation, especially to organize those videos containing human activities. Recently, manifold regularized semi-supervised learning (SSL), which explores th…

Develops new strategy for Hessian estimates in Lagrangian mean curvature equation.

In this paper, complex Hessian equation over Kähler manifold was studied. Under the condition that the underline Kähler manifold has non-negative holomorphic bisectional curvature, the existence and regularity of the solution was proved.

The paper describes a fine representation of the Ricci tensor and Hessian on RCD spaces.

We use the octonion algebra to construct singular solutions of Hessian fully nonlinear uniformly elliptic equations in 21 or more dimensions. The regularity of these solutions is the least possible one. The same is proven for Isaacs equtions.

We consider the Dirichlet problem for positively homogeneous, degenerate elliptic, concave (or convex) Hessian equations. Under natural and necessary conditions on the geometry of the domain, with the $C^{1,1}$ boundary data, we establish the interior $C^{1,1}$-regularity of the unique (admissible) solution, which is o…

We show vanishing theorems of $L^2$-cohomology groups of Kodaira-Nakano type on complete Hessian manifolds. We obtain further vanishing theorems of $L^2$-cohomology groups $L^2H^{p,q}(Ω)$ on a regular convex cone $Ω$ with the Cheng-Yau metric for $p>q$.

The paper studies the loss landscape of regularized deep matrix factorization, revealing unique and sharp minimizers.

Trust region and cubic regularization methods have demonstrated good performance in small scale non-convex optimization, showing the ability to escape from saddle points. Each iteration of these methods involves computation of gradient, Hessian and function value in order to obtain the search direction and adjust the r…

A new quasi-Newton method uses cubic regularization to avoid saddle points in deep learning.

The Hessian geometry is the real analogue of the Kähler one. Sasakian geometry is an odd-dimensional counterpart of the Kähler geometry. In the paper, we study the connection between projective Hessian and Sasakian manifolds analogous to the one between Hessian and Kähler manifolds. In particular, we construct a Sasaki…

New algorithm improves convergence of gradient boosting trees.

Derives estimates for geometric elliptic equations on complex manifolds.

Study shows how to balance memory and learning efficiency in continual learning.

The nonzero level sets of a homogeneous, logarithmically homogeneous, or translationally homogeneous function are affine spheres if and only if the Hessian determinant of the function is a multiple of a power or an exponential of the function. In particular, the nonzero level sets of a homogeneous polynomial are proper…

RES, a regularized stochastic version of the Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton method is proposed to solve convex optimization problems with stochastic objectives. The use of stochastic gradient descent algorithms is widespread, but the number of iterations required to approximate optimal arguments c…

New technique debiases distributed optimization, improving convergence rate.

The rapid development of computer hardware and Internet technology makes large scale data dependent models computationally tractable, and opens a bright avenue for annotating images through innovative machine learning algorithms. Semi-supervised learning (SSL) has consequently received intensive attention in recent yea…

Studied how SGD's stability regularization affects generalization in neural networks.

Large scale optimization problems are ubiquitous in machine learning and data analysis and there is a plethora of algorithms for solving such problems. Many of these algorithms employ sub-sampling, as a way to either speed up the computations and/or to implicitly implement a form of statistical regularization. In this …

We reparametrize ReLU NNs as splines to understand their learning dynamics.

New proof for convex solutions of Monge-Ampère equation.

The purpose of this article is to present a new regularization technique of quasi-plurisubharmoinc functions on a compact Kaehler manifold. The idea is to regularize the function on local coordinate balls first, and then glue each piece together. Therefore, all the higher order terms in the complex Hessian of this regu…

Subsampled Newton methods approximate Hessian matrices through subsampling techniques, alleviating the cost of forming Hessian matrices but using sufficient curvature information. However, previous results require $Ω(d)$ samples to approximate Hessians, where $d$ is the dimension of data points, making it less practica…

Introduces HTV to measure function complexity in learning schemes.

Constructs flows on manifolds with small curvature, proving Euclidean topology.

A new screening rule improves lasso solving speed.

This paper tackles catastrophic forgetting in neural networks by providing a unified framework for regularization-based continual learning.

The paper analyzes stability and convergence rates of entropic and Sinkhorn potentials.

We prove that any two Kahler potentials on a compact Kahler manifold can be connected by a geodesic segment of C^{1,1} regularity. This follows from an a priori interior real Hessian bound for solutions of the nondegenerate complex Monge-Ampere equation, which is independent of a positive lower bound for the right hand…

Develops calculus for tamed Dirichlet spaces using measure theory.

Hamiltonian Monte Carlo (HMC) is a widely deployed method to sample from high-dimensional distributions in Statistics and Machine learning. HMC is known to run very efficiently in practice and its popular second-order "leapfrog" implementation has long been conjectured to run in $d^{1/4}$ gradient evaluations. Here we …

SGD without replacement decouples into curvature-following and flatness-regularizing steps.