New Bethe-Hessian method improves community detection in sparse networks.
problem Detect communities in sparse networks efficiently.
method Spectral clustering using the Bethe-Hessian matrix.
result Bethe-Hessian consistently estimates block number above Kesten-Stigum threshold.
SNS accelerates Sinkhorn algorithm with sparse Newton iterations.
problem Slow runtime of Sinkhorn algorithm for optimal transport.
method Early stopping and Newton-type subroutine for matrix scaling steps.
result SNS converges orders of magnitude faster than Sinkhorn.
A fast spectral algorithm detects community structure in evolving graphs.
problem Detecting community structure in time-evolving sparse graphs.
method Extension of the Bethe-Hessian matrix for spectral community detection.
result The algorithm reaches the optimal detectability threshold and outperforms other methods.
EiGLasso speeds up sparse Kronecker-sum covariance estimation.
problem Sparse Kronecker-sum inverse covariance estimation challenges in scalability and parameter identification.
method Newton's method combined with eigendecomposition of sample and feature graphs, approximating Hessian for speed.
result Two to three orders-of-magnitude speed-up on simulated and real-world data.
A new screening rule improves lasso solving speed.
problem Efficiently solving lasso problems with high correlation.
method Uses second-order information from the Hessian to screen predictors.
result Outperforms alternatives on simulated and real data.
New method computes affine normal directions efficiently for sparse polynomials.
problem Computing affine normal directions is computationally expensive in high dimensions.
method Reduces third-order tensor contraction to matrix-free formulation using log-determinant gradient.
result Scalable implementations with near-linear scaling in dimension and sparsity.
The paper studies efficient Hessian fitting methods for stochastic optimization.
problem Efficient Hessian fitting for stochastic optimization.
method Preconditioned Stochastic Gradient Descent (PSGD) method and Lie groups.
result Hessian fitting problem is strongly convex in certain Lie groups.
Scalable algorithms to solve optimization and regression tasks even approximately, are needed to work with large datasets. In this paper we study efficient techniques from matrix sketching to solve a variety of convex constrained regression problems. We adopt "Iterative Hessian Sketching" (IHS) and show that the fast C…
Spectral clustering is one of the most popular, yet still incompletely understood, methods for community detection on graphs. This article studies spectral clustering based on the Bethe-Hessian matrix Hr=(r2−1)In+D−rA for sparse heterogeneous graphs (following the degree-corrected stochastic block model) in a …
New method uses sparse deep neural networks for high-dimensional regression with improved parameter estimation.
problem Improving parameter estimation in high-dimensional sparse regression models.
method Proposes nonparametric estimation of partial derivatives in sparse deep neural networks.
result Established convergence rate of nonparametric estimation of partial derivatives as O(n−1/4). We propose a DC proximal Newton algorithm for solving nonconvex regularized sparse learning problems in high dimensions. Our proposed algorithm integrates the proximal Newton algorithm with multi-stage convex relaxation based on the difference of convex (DC) programming, and enjoys both strong computational and statist…
Recently several methods were proposed for sparse optimization which make careful use of second-order information [10, 28, 16, 3] to improve local convergence rates. These methods construct a composite quadratic approximation using Hessian information, optimize this approximation using a first-order method, such as coo…
Simplifies NL models by approximating them as LPV systems and identifying NL subterms.
problem Complex NL models are hard to interpret and impractical.
method Linear approximation around operating points, sparse estimation in RKHS, LPV model reduction.
result Identifies NL subterms and their input spaces in sparse additive NL models.
Unified spectral clustering for sparse networks with heterogeneous degrees.
problem Efficiently detecting communities in sparse networks with varying degrees.
method Developed a parametrized regularized Laplacian matrix for spectral clustering.
result Improved parametrization accounts for network heterogeneity and community hardness.
Independent Component Analysis (ICA) is a technique for unsupervised exploration of multi-channel data that is widely used in observational sciences. In its classic form, ICA relies on modeling the data as linear mixtures of non-Gaussian independent sources. The maximization of the corresponding likelihood is a challen…
Algorithm learns non-Gaussian graphical models via Hessian scores and triangular transport.
problem Learning graph structure from non-Gaussian data.
method Score based on integrated Hessian information, coupled with triangular transport map.
result Algorithm successfully recovers graph structure for non-Gaussian data.
Recent work has established an empirically successful framework for adapting learning rates for stochastic gradient descent (SGD). This effectively removes all needs for tuning, while automatically reducing learning rates over time on stationary problems, and permitting learning rates to grow appropriately in non-stati…
Sparsity-constrained optimization has wide applicability in machine learning, statistics, and signal processing problems such as feature selection and compressive Sensing. A vast body of work has studied the sparsity-constrained optimization from theoretical, algorithmic, and application aspects in the context of spars…
New method generates continuous Pareto sets for multi-task learning.
problem Challenges in finding optimal solutions for correlated multi-task learning problems.
method Efficiently generates locally continuous Pareto sets and fronts in multi-objective optimization problems.
result Demonstrates continuous analysis of Pareto optimal solutions in machine learning problems.
The Bethe free energy approximation is reliable when convex on a submanifold, the 'Bethe box'.
problem Accuracy of the Bethe free energy approximation in probabilistic inference.
method Analysis of convexity and verification conditions based on the Bethe Hessian matrix.
result The Bethe approximation is mostly accurate if it is convex on a submanifold, the 'Bethe box'.
New method achieves superlinear convergence rate with limited memory.
problem Achieving superlinear convergence rate in quasi-Newton methods with limited memory.
method Limited-memory Greedy BFGS (LG-BFGS) method with displacement aggregation and basis vector selection.
result Explicit non-asymptotic superlinear convergence rate demonstrated.
We propose a novel general algorithm LHAC that efficiently uses second-order information to train a class of large-scale l1-regularized problems. Our method executes cheap iterations while achieving fast local convergence rate by exploiting the special structure of a low-rank matrix, constructed via quasi-Newton approx…
Several convex formulation methods have been proposed previously for statistical estimation with structured sparsity as the prior. These methods often require a carefully tuned regularization parameter, often a cumbersome or heuristic exercise. Furthermore, the estimate that these methods produce might not belong to th…
The study proves that certain noncompact Hessian manifolds are diffeomorphic to R^n.
problem Characterizing complete noncompact Hessian manifolds with nonnegative Hessian sectional curvature.
method Using a geometric flow on noncompact affine Riemannian manifolds, constructing Hessian metrics, and proving diffeomorphism.
result Complete noncompact Hessian manifolds with nonnegative Hessian sectional curvature are diffeomorphic to R^n if their tangent bundle has maximal volume growth.
New metric for probability measures connects physics and geometry.
problem Developing a new metric for probability measures.
method Transport Hessian metric, formulated dynamical systems.
result Connections to physics equations and mathematical models.
Investigates regularity of solutions to complex Hessian equation.
problem Regularity of solutions to complex Hessian equation.
method Analyzes solutions to Dirichlet problem with specific density condition.
result Establishes conditions for regularity of solutions.
Abstract: Survey on quadratic Hessian equations, their properties, and open problems.
problem Understanding quadratic Hessian equations and their solutions.
method Survey and review of existing research.
result Survey of entire solutions, viscosity solutions, and Hessian estimates.
Paper solves Hessian equations on Kähler manifolds.
problem Solving Hessian equations on Kähler manifolds.
method Combines elementary symmetric functions; provides sufficient and necessary condition.
result Generalizes results for Hessian and Hessian quotient equations.
New Hessian estimates for heat equations on manifolds.
problem Estimating Hessian matrices for heat-type equations on Riemannian manifolds.
method Using Bismut-Stroock Hessian formula, with explicit coefficients and delay/growth rate functions.
result Novel backward weak Harnack inequality and precise pointwise Hessian estimates for eigenfunctions.
Training neural networks involves finding minima of a high-dimensional non-convex loss function. Knowledge of the structure of this energy landscape is sparse. Relaxing from linear interpolations, we construct continuous paths between minima of recent neural network architectures on CIFAR10 and CIFAR100. Surprisingly, …
New test detects sparse alternatives in Gaussian random fields.
problem Detecting sparse alternatives in Gaussian random fields.
method Ad-hoc Kac Rice formula for second maximum distribution, exact spacing test.
result Exact t-spacing test for high power in detecting sparse alternatives. Study calculates Hessian of Busemann function on Damek-Ricci spaces.
problem Calculating Hessian of Busemann function on Damek-Ricci spaces.
method Calculates eigenvalues of Hessian and proves positive definiteness.
result Hessian of Busemann function is positive definite.
The paper shows infinitely many components in Floer Hessians space.
problem Understanding the structure of Floer Hessians.
method Proving the existence of infinitely many connected components.
result Proves infinitely many connected components in Floer Hessians space.
The paper describes flat Hessian metrics on surfaces and their potentials.
problem Understanding Hessian metrics on surfaces.
method Theoretical description and explicit construction using integrable systems.
result Explicit construction of potentials for flat Hessian metrics on surfaces.
Paper develops a distributed debiased estimator for sparse statistical inference.
problem High computational costs in debiased estimator construction for high-dimensional models.
method Develops a multi-round distributed debiased estimator using both labeled and unlabelled data.
result Unlabeled data improves statistical rate of each iteration in distributed setup.
The paper quantizes Hessian structures on R^2 using KV-algebras.
problem Quantizing Hessian structures on a 2D space.
method Deformation quantization within Koszul-Vinberg algebras.
result Established links between deformation theory and Hessian geometry.
We prove that, in dimensions greater than 2, the generic metric is not a Hessian metric and find a curvature condition on Hessian metrics in dimensions greater than 3. In particular we prove that the forms used to define the Pontryagin classes in terms of the curvature vanish on a Hessian manifold. By contrast all anal…
New method estimates Nishimori temperature for node classification in weighted graphs.
problem Estimating Nishimori temperature for Bayesian inference.
method Spectral method using eigenvalues of Bethe Hessian matrix.
result Spectral method outperforms existing approaches in node classification.
Sparse codes improve optimal control tasks with correlated inputs.
problem Optimal control tasks with correlated feature inputs.
method Used a sparse code to represent natural images in an optimal control task solved with neuro-dynamic programming.
result An over-complete sparse code increases memory capacity and learning speed beyond a complete code.
Superintegrable systems on curved manifolds found to have Hessian structures.
problem Characterizing superintegrable systems on curved manifolds.
method Identifying and computing Hessian coordinates for superintegrable systems.
result Examples of superintegrable systems in 2D and 3D have natural Hessian coordinates.
Curved Frobenius manifolds link to Hessian metrics in geometry.
problem Understanding curved Frobenius manifolds and their relation to Hessian metrics.
method Analyzing the relationship between curved Frobenius structures and Hessian metrics on spaces with non-vanishing curvature.
result Consistent curved Frobenius structures on constant curvature spaces are linked to Hessian metrics.
New rigidity results for a generalized Ricci-Hessian equation on manifolds.
problem Understanding rigidity in generalized Ricci-Hessian equations on manifolds.
method Proving new rigidity results related to a generalized Ricci-Hessian equation.
result New rigidity results for the generalized Ricci-Hessian equation on Riemannian manifolds.
Study classifies 3D Hessian manifolds, proving their topology.
problem Global topology of 3D Hessian manifolds.
method Proved structure and analyzed Betti numbers.
result Complete topological classification of 3D Hessian manifolds.
Sparse connectivity improves generalization in neural networks below the Edge of Stability.
problem Generalization guarantees for fully-connected networks fail at the Edge of Stability.
method Analyzed sparse connectivity's impact on generalization in two-layer ReLU networks.
result Sparse connectivity changes the effective constraint, leading to non-vacuous generalization bounds.
Criterion for solvability of complex 2-Hessian equation on compact Kähler manifolds.
problem Solvability of complex 2-Hessian equation on compact Kähler manifolds.
method Nakai--Moishezon-type criterion associated with the complex 2-Hessian equation.
result Criterion equivalent to existence of a smooth 2-admissible representative in complex dimension three.
Establishes a concavity property for positive Hessian quotient operators.
problem Analyzing positive Hessian quotient operators on Riemannian manifolds.
method Proves a special concavity property and a Jacobi inequality.
result Proves a Jacobi inequality for symmetric tensors.
Estimates for complex Hessian equations on Hermitian manifolds.
problem Establishing estimates for solutions to complex Hessian equations.
method Using concavity inequality for complex sum-of-Hessian operators.
result Second-order estimates for admissible solutions on Hermitian manifolds.
Derives Hessian estimates for Lagrangian mean curvature equation.
problem Lagrangian mean curvature equation with supercritical phase and bounded second derivatives.
method Derives a priori interior Hessian estimates.
result Hessian estimates for Lagrangian mean curvature equation.