arXiv research

SGD's training dynamics align with Hessian and gradient spectra in high-dimensional classification tasks.

FedPower improves eigenspace estimation privacy in federated learning.

Our work connects parameter magnitudes and Hessian eigenspaces in deep neural nets.

This paper presents a novel time series clustering method, the self-organising eigenspace map (SOEM), based on a generalisation of the well-known self-organising feature map (SOFM). The SOEM operates on the eigenspaces of the embedded covariance structures of time series which are related directly to modes in those tim…

Proposes BONMI for integrating noisy matrices from multi-source data.

New algorithm reduces communication in distributed eigenspace estimation.

SGD updates align with a low-rank subspace but do not lead to further loss reduction.

This paper tackles distributed estimation of the top-L eigenspace in PCA for large data sets.

We investigate numerically efficient approximations of eigenspaces associated to symmetric and general matrices. The eigenspaces are factored into a fixed number of fundamental components that can be efficiently manipulated (we consider extended orthogonal Givens or scaling and shear transformations). The number of the…

In this note we explore a connection between finite covers of surfaces and the Teichmüller polynomial of a fibered face of a hyperbolic 3--manifold. We consider the action of a homological pseudo-Anosov homeomorphism $ψ$ on the homology groups of a class of finite abelian covers of a surface $Σ_{g,n}$. Eigenspaces of t…

New algorithm estimates eigenspace with faulty nodes, matching performance of existing methods.

A method to analyze neural network performance by measuring layer saturation.

Physics-informed GP regression solves eigenvalue problems by identifying non-trivial eigenspaces.

Compressing word embeddings is important for deploying NLP models in memory-constrained settings. However, understanding what makes compressed embeddings perform well on downstream tasks is challenging---existing measures of compression quality often fail to distinguish between embeddings that perform well and those th…

Paper identifies key function spaces for ReLU networks based on Fisher information.

For a compact homogeneous space $G/K$, we study the problem of existence of $G$-invariant Riemannian metrics such that each eigenspace of the Laplacian is a real irreducible representation of $G$. We prove that the normal metric of a compact irreducible symmetric space has this property only in rank one. Furthermore, w…

New methods compare Steklov eigenspaces of free boundary minimal surfaces in balls.

If $G$ is a compact Lie group endowed with a left invariant metric $g$, then $G$ acts via pullback by isometries on each eigenspace of the associated Laplace operator $Δ_g$. We establish algebraic criteria for the existence of left invariant metrics $g$ on $G$ such that each eigenspace of $Δ_g$, regarded as the real ve…

In this paper, we present an online adaptive PCA algorithm that is able to compute the full dimensional eigenspace per new time-step of sequential data. The algorithm is based on a one-step update rule that considers all second order correlations between previous samples and the new time-step. Our algorithm has O(n) co…

In this paper, we aim at recovering an undirected weighted graph of $N$ vertices from the knowledge of a perturbed version of the eigenspaces of its adjacency matrix $W$. For instance, this situation arises for stationary signals on graphs or for Markov chains observed at random times. Our approach is based on minimizi…

The instanton Floer homology of a knot in the three-sphere is a vector space with a canonical mod 2 grading. It carries a distinguished endomorphism of even degree,arising from the 2-dimensional homology class represented by a Seifert surface. The Floer homology decomposes as a direct sum of the generalized eigenspaces…

Analysis of DPPs and k-DPPs via spectral decomposition reveals identifiable parameters and non-identifiability gaps.

A local classification of locally conformal flat Riemannian Einstein-like four-manifolds as well as a local classification of all locally conformal flat Riemannian four-manifolds for which all Jacobi operators have parallel eigenspaces along every geodesic is given. Non-trivial explicit examples are presented. The prob…

TransNet improves community detection on target networks using privacy-preserved source networks.

We focus in this work on the estimation of the first $k$ eigenvectors of any graph Laplacian using filtering of Gaussian random signals. We prove that we only need $k$ such signals to be able to exactly recover as many of the smallest eigenvectors, regardless of the number of nodes in the graph. In addition, we address…

Essential self-adjointness and spectrum of CR GJMS operator proved.

A theory of feature geometry using spectral analysis of weight matrices.

Principal components analysis (PCA) is a widely used dimension reduction technique with an extensive range of applications. In this paper, an online distributed algorithm is proposed for recovering the principal eigenspaces. We further establish its rate of convergence and show how it relates to the number of nodes emp…

Study of eigenvalues in nonlinear kernels for classification of separable data.

New method accelerates neural network training by focusing on flat directions.

This paper addresses a gap in the classifcation of Codazzi tensors with exactly two eigenfunctions on a Riemannian manifold of dimension three or higher. Derdzinski proved that if the trace of such a tensor is constant and the dimension of one of the the eigenspaces is $n-1$, then the metric is a warped product where t…

For generic torus-invariant metrics, eigenspaces are 2D and nodal sets are connected hypersurfaces.

Study spectral properties of graph Laplacian for manifold data.

SU(3) instanton homology counts Tait colorings for webs and foams.

A fundamental operation in many vision tasks, including motion understanding, stereopsis, visual odometry, or invariant recognition, is establishing correspondences between images or between images and data from other modalities. We present an analysis of the role that multiplicative interactions play in learning such …

Researchers create surfaces with exceptionally high Steklov eigenvalues.

In this undergraduate thesis, we present an analytical proof of the Morse inequalities for closed smooth $n$-manifolds following Witten's approach. Using techniques from PDE theory, the proof is reduced to study the eigenspaces and eigenvalues of harmonic oscillators on $\mathbb{R}^n$.

New principle reveals how neural networks learn complex interactions.

We show that the eigenspaces of the Laplacian $Δ_k$ on $k$-forms on a compact Kähler manifold carry Hodge and Lefschetz decompositions. Among other consequences, we show that the positive part of the spectrum of $Δ_k$ lies in the spectrum of $Δ_{k+1}$.

This paper uncovers the low-rank structure of neural network Hessians.

The paper calibrates shrinkage covariance estimators for spectral functionals in high dimensions.

Let $(E,h)$ be a holomorphic Hermitian vector bundle over a polarized manifold. We provide a canonical quantization of the Laplacian operator acting on sections of the bundle of Hermitian endomorphisms of $E$. If $E$ is simple we obtain an approximation of the eigenvalues and eigenspaces of the Laplacian.

The resonant band is a useful notion for the computation of the nontrivial monodromy eigenspaces of the Milnor fiber of a real line arrangement. In this article, we develop the resonant band description for the cohomology of the Aomoto complex. As an application, we prove that real 4-nets do not exist.

Let $(M,g)$ be a compact Riemannian manifold and $P_g$ an elliptic, formally self-adjoint, conformally covariant operator of order $m$ acting on smooth sections of a bundle over $M$. We prove that if $P_g$ has no rigid eigenspaces (see Definition 2.2), the set of functions $f \in C^\infty(M, R)$ for which $P_{e^fg}$ ha…

In this note we prove that toric Kähler metrics on complex projective space which are also $U(n)$-invariant are determined by their equivariant spectrum i.e. the list of eigenvalues of the Laplacian together with weights of the torus representation on the eigenspaces.

Resolution of a compact group action in the sense described by Albin and Melrose is applied to the conjugation action by the unitary group on self-adjoint matrices. It is shown that the eigenvalues are smooth on the resolved space and that the trivial bundle smoothly decomposes into the direct sum of global one-dimensi…

This work studies low-rank approximation of a positive semidefinite matrix from partial entries via nonconvex optimization. We characterized how well local-minimum based low-rank factorization approximates a fixed positive semidefinite matrix without any assumptions on the rank-matching, the condition number or eigensp…

Spectral clustering for geometric graphs achieves strong consistency in community recovery.