The paper explores how word embeddings affect the stability of downstream NLP models.
problem Small changes in training data can cause significant changes in model predictions.
method Empirical and theoretical analysis of embedding instability, including the introduction of eigenspace instability measure.
result Increasing embedding memory can reduce the disagreement in predictions by 5% to 37%.
Efficiently approximates eigenspaces for symmetric and general matrices.
problem Fast computation of eigenspaces for large matrices.
method Factor eigenspaces into fundamental components using transformations, solve minimization problems, and iteratively update.
result Improved computational efficiency for eigenspace approximation.
This paper tackles distributed estimation of the top-L eigenspace in PCA for large data sets.
problem Challenges in estimating the top-L eigenspace in principal component analysis for large data sets.
method Proposes a novel multi-round algorithm using shift-and-invert preconditioning and convex optimization.
result Achieves a fast convergence rate and covers the targeted top-L eigenspace without explicit eigengap assumption.
In this note we explore a connection between finite covers of surfaces and the Teichmüller polynomial of a fibered face of a hyperbolic 3--manifold. We consider the action of a homological pseudo-Anosov homeomorphism ψ on the homology groups of a class of finite abelian covers of a surface Σg,n. Eigenspaces of t…
Our work connects parameter magnitudes and Hessian eigenspaces in deep neural nets.
problem Understanding the relationship between parameter magnitudes and Hessian curvature in deep learning models.
method Developed a matrix-free algorithm based on sketched SVDs to measure similarity between parameter masks and Hessian eigenspaces.
result Top Hessian eigenvectors tend to be concentrated around larger parameters, indicating a connection between parameter magnitudes and loss curvature.
New algorithm estimates eigenspace with faulty nodes, matching performance of existing methods.
problem Estimating eigenspace in distributed systems with node failures.
method Develops an eigenspace estimation algorithm for distributed environments with arbitrary node failures.
result Matches performance of existing non-robust estimator up to an additive error.
FedPower improves eigenspace estimation privacy in federated learning.
problem Privacy breaches and communication challenges in federated eigenspace estimation.
method FedPower uses a power method with local power iterations and global aggregation, weighted by OPT, and adds Gaussian noise for privacy.
result FedPower provides convergence bounds and demonstrates effectiveness in experiments.
A method to analyze neural network performance by measuring layer saturation.
problem Understanding which layers contribute to network performance.
method Layer saturation method: restricts layer output to eigenspace of variance matrix.
result Layer saturation indicates which layers contribute to network performance.
Physics-informed GP regression solves eigenvalue problems by identifying non-trivial eigenspaces.
problem Solving eigenvalue problems of linear operators with trivial solutions.
method Constructing a transfer function-type indicator using physics-informed Gaussian Process posterior.
result The posterior covariance is non-trivial only for eigenvalues of the operator, indicating non-trivial eigenspaces.
Compressing word embeddings is important for deploying NLP models in memory-constrained settings. However, understanding what makes compressed embeddings perform well on downstream tasks is challenging---existing measures of compression quality often fail to distinguish between embeddings that perform well and those th…
A new GNN architecture called coVariance neural network (VNN) improves stability and transferability of covariance matrix analysis.
problem Stability and transferability issues in covariance matrix analysis.
method Developed coVariance neural network (VNN) that operates on sample covariance matrices.
result VNN is more stable and transferable than PCA-based approaches.
Paper identifies key function spaces for ReLU networks based on Fisher information.
problem Understanding the structure of Fisher information matrices in ReLU networks.
method Spectral decomposition of Fisher information matrices, focusing on the first three eigenspaces.
result The first three eigenspaces account for 97.7% of the trace of the Fisher information matrix, corresponding to spherical harmonic functions of order ≤2.
For a compact homogeneous space G/K, we study the problem of existence of G-invariant Riemannian metrics such that each eigenspace of the Laplacian is a real irreducible representation of G. We prove that the normal metric of a compact irreducible symmetric space has this property only in rank one. Furthermore, w…
New methods compare Steklov eigenspaces of free boundary minimal surfaces in balls.
problem Comparing Steklov eigenspaces of free boundary minimal surfaces.
method Developed new methods to compare span of coordinate functions with Steklov eigenspace.
result Proved congruence of free boundary minimal annuli in 3D unit ball.
If G is a compact Lie group endowed with a left invariant metric g, then G acts via pullback by isometries on each eigenspace of the associated Laplace operator Δg. We establish algebraic criteria for the existence of left invariant metrics g on G such that each eigenspace of Δg, regarded as the real ve…
In this paper, we present an online adaptive PCA algorithm that is able to compute the full dimensional eigenspace per new time-step of sequential data. The algorithm is based on a one-step update rule that considers all second order correlations between previous samples and the new time-step. Our algorithm has O(n) co…
In this paper, we aim at recovering an undirected weighted graph of N vertices from the knowledge of a perturbed version of the eigenspaces of its adjacency matrix W. For instance, this situation arises for stationary signals on graphs or for Markov chains observed at random times. Our approach is based on minimizi…
The instanton Floer homology of a knot in the three-sphere is a vector space with a canonical mod 2 grading. It carries a distinguished endomorphism of even degree,arising from the 2-dimensional homology class represented by a Seifert surface. The Floer homology decomposes as a direct sum of the generalized eigenspaces…
Analysis of DPPs and k-DPPs via spectral decomposition reveals identifiable parameters and non-identifiability gaps.
problem Identifying parameters of DPPs and k-DPPs through spectral decomposition.
method Spectral decomposition of the covariance matrix, analysis of invariances, and counting arguments.
result Identifiability of parameters changes fundamentally for k-DPPs, with specific invariances and non-identifiability gaps.
A local classification of locally conformal flat Riemannian Einstein-like four-manifolds as well as a local classification of all locally conformal flat Riemannian four-manifolds for which all Jacobi operators have parallel eigenspaces along every geodesic is given. Non-trivial explicit examples are presented. The prob…
This paper presents a novel time series clustering method, the self-organising eigenspace map (SOEM), based on a generalisation of the well-known self-organising feature map (SOFM). The SOEM operates on the eigenspaces of the embedded covariance structures of time series which are related directly to modes in those tim…
TransNet improves community detection on target networks using privacy-preserved source networks.
problem Community detection on sensitive network data with privacy constraints.
method Spectral clustering framework leveraging locally distributed privacy-preserved auxiliary networks via randomized response and adaptive weighting.
result TransNet delivers strong gains in community detection across various privacy levels and heterogeneity patterns.
We focus in this work on the estimation of the first k eigenvectors of any graph Laplacian using filtering of Gaussian random signals. We prove that we only need k such signals to be able to exactly recover as many of the smallest eigenvectors, regardless of the number of nodes in the graph. In addition, we address…
Essential self-adjointness and spectrum of CR GJMS operator proved.
problem Characterizing the spectrum of CR GJMS operator.
method Proving essential self-adjointness and closed range, analyzing spectrum.
result CR GJMS operator has discrete spectrum with finite-dimensional eigenspaces.
We improve current instability-based methods for the selection of the number of clusters k in cluster analysis by developing a normalized cluster instability measure that corrects for the distribution of cluster sizes, a previously unaccounted driver of cluster instability. We show that our normalized instability mea…
A theory of feature geometry using spectral analysis of weight matrices.
problem Current methods decompose neural network activations into sparse linear features, losing geometric structure.
method Develops a theory by analyzing the spectra of weight-derived matrices, introducing the frame operator.
result Features collapse onto single eigenspaces, organizing into tight frames, and admit discrete classification.
Principal components analysis (PCA) is a widely used dimension reduction technique with an extensive range of applications. In this paper, an online distributed algorithm is proposed for recovering the principal eigenspaces. We further establish its rate of convergence and show how it relates to the number of nodes emp…
Modeling HFT interactions reveals market instability.
problem Market instability caused by HFT dynamic coupling.
method Developed a recurrence relations framework to model HFT interactions.
result Unexpected latency and feedback can trigger market instability.
New proof of instability for certain Einstein metrics.
problem Einstein metrics on specific 4-manifolds.
method Proving instability of conformally Kähler, Einstein metrics.
result Proven instability of certain Einstein metrics.
This paper addresses a gap in the classifcation of Codazzi tensors with exactly two eigenfunctions on a Riemannian manifold of dimension three or higher. Derdzinski proved that if the trace of such a tensor is constant and the dimension of one of the the eigenspaces is n−1, then the metric is a warped product where t…
For generic torus-invariant metrics, eigenspaces are 2D and nodal sets are connected hypersurfaces.
problem Understanding spectral multiplicity and nodal sets for generic torus-invariant metrics.
method Analyzing real Δg-eigenspaces and nodal sets for generic T-invariant metrics. result For generic T-invariant metrics, real Δg-eigenspaces are irreducible and have dimension at most 2, and nodal sets are connected hypersurfaces with specific properties. Study stability and instability of Ricci-flat metrics under generalized Ricci flow.
problem Stability and instability of Ricci-flat metrics under Ricci flow.
method Analysis of generalized Ricci flow for Ricci-flat metrics and vanishing 3-forms.
result Dynamical stability and instability results for Ricci-flat metrics and vanishing 3-forms.
Some exotic compact objects possess evanescent ergosurfaces: timelike submanifolds on which a Killing vector field, which is timelike everywhere else, becomes null. We show that any manifold possessing an evanescent ergosurface but no event horizon exhibits a linear instability of a peculiar kind: either there are solu…
Interval Neural Networks detect instabilities in image reconstructions.
problem Detecting instabilities in deep learning image reconstructions.
method Employed uncertainty quantification methods with Interval Neural Networks.
result Interval Neural Networks effectively reveal image reconstruction instabilities.
Note on instabilities in super-time-stepping methods for Heston model.
problem Instabilities in super-time-stepping methods applied to Heston model.
method Exploration of explicit super-time-stepping schemes (RK-Chebyshev, RK-Legendre) for Heston model.
result Relevance of stability remarks beyond super-time-stepping schemes.
Study spectral properties of graph Laplacian for manifold data.
problem Understanding spectral properties of graph Laplacian for manifold data.
method Non-asymptotic error bounds on spectral properties of empirical graph Laplacian.
result Eigenvalues and eigenspaces of empirical graph Laplacian are close to Laplace-Beltrami operator of manifold.
Binary perceptron's instability linked to replica symmetry breaking.
problem Understanding the relationship between algorithmic instability and replica symmetry breaking in binary perceptron learning.
method Established the connection between algorithmic instability and replica symmetry breaking by comparing the instability condition around the fixed point to the instability for breaking the replica symmetric solution of the free energy function.
result The instability condition around the algorithmic fixed point is identical to the instability for breaking the replica symmetric saddle point solution of the free energy function.
A central area of research in nonlinear science is the study of instabilities that drive the emergence of extreme events. Unfortunately, experimental techniques for measuring such phenomena often provide only partial characterization. For example, real-time studies of instabilities in nonlinear fibre optics frequently …
Proposes BONMI for integrating noisy matrices from multi-source data.
problem Integrating noisy matrices from multi-source data with block-wise missingness.
method Exploits orthogonal Procrustes problem to align eigenspaces and completes missing blocks.
result Statistical rate for eigenspace of underlying matrix comparable to independently missing assumption.
SU(3) instanton homology counts Tait colorings for webs and foams.
problem Counting Tait colorings for webs and foams.
method SU(3) gauge theory with structure group, eigenspace decomposition, skein exact triangles.
result SU(3) homology counts Tait colorings, Euler characteristic interpretable as polynomial invariant value.
A fundamental operation in many vision tasks, including motion understanding, stereopsis, visual odometry, or invariant recognition, is establishing correspondences between images or between images and data from other modalities. We present an analysis of the role that multiplicative interactions play in learning such …
The study examines stability and instability of Poincaré-Einstein metrics using Ricci flow.
problem Stability and instability of Poincaré-Einstein metrics.
method Variant of expander entropy for asymptotically hyperbolic manifolds, local positive mass theorem, volume comparison.
result Characterization of stability and instability in terms of local positive mass theorem and volume comparison.
New algorithm reduces communication in distributed eigenspace estimation.
problem Efficiently estimating eigenspaces in distributed settings without excessive communication.
method Communication-efficient distributed algorithm using Procrustean alignment.
result Achieves similar error rate to centralized estimator for PCA.
Researchers create surfaces with exceptionally high Steklov eigenvalues.
problem Creating surfaces with first non-zero Steklov eigenvalue of large multiplicity.
method Constructing surfaces with specific isometry groups and gluing them based on Cayley graph structures, then analyzing the eigenspace properties.
result Surfaces with arbitrarily large multiplicity for their first non-zero Steklov eigenvalue are constructed.
Study shows instability of naked singularities in perfect fluid models.
problem Instability of naked singularities in Einstein equations coupled with isothermal perfect fluid.
method Investigated spherically symmetric self-similar naked singularities under C1,α perturbations of an external massless scalar field. result Spherically symmetric self-similar naked singularities are unstable to trapped surface formation.
Study on spectral stability of Riemannian coverings.
problem Stability of eigenvalues in Riemannian coverings.
method Analysis of Laplacian eigenvalues under finite coverings.
result Necessary conditions for spectral stability or instability.
Clinical models can be unstable, leading to unreliable predictions.
problem Stability of clinical prediction models developed using statistical or machine learning methods.
method Simulation and case studies of statistical and machine learning approaches to show instability in model predictions.
result Model instability often leads to miscalibration of predictions in new data.
In this undergraduate thesis, we present an analytical proof of the Morse inequalities for closed smooth n-manifolds following Witten's approach. Using techniques from PDE theory, the proof is reduced to study the eigenspaces and eigenvalues of harmonic oscillators on Rn.