The paper embeds manifolds into finite Euclidean spaces using eigenvector fields of the connection Laplacian.
problem Embedding manifolds into finite-dimensional Euclidean spaces using eigenvector fields of the connection Laplacian.
method Constructing local coordinate charts with low distortion using eigenvector fields and proving estimates for eigenvector fields and the heat kernel.
result The distortion constants depend only on geometric properties of manifolds in the little Hölder space c2,α, allowing for embedding into a finite-dimensional Euclidean space. New method estimates sparse eigenvectors without sacrificing orthogonality.
problem Estimating sparse eigenvectors of a symmetric matrix.
method Developed a new method using MM framework and Procrustes reformulation.
result Improves support recovery and explained variance compared to existing methods.
The paper explores how kernel eigenalignments affect generalization in KRR.
problem Achieving robust generalization in kernel methods.
method Direct connection between generalization and matrix eigenvectors/eigenvalues, focusing on finite-sample settings.
result Strong generalization requires increasing eigenvector alignment, eigenvalue magnitude, or gaps between eigenvalues.
The paper presents efficient algorithms for large-scale CCA and generalized eigenvector problems.
problem Canonical-correlation analysis and generalized eigenvector problems for symmetric matrices.
method Simple iterative algorithms with global linear convergence, reducing CCA to the top-k generalized eigenvector problem. result The proposed algorithms have improved runtimes and global linear convergence, suitable for large-scale matrices.
New method detects global structures in sparse or noisy data.
problem Localization of eigenvectors in sparse or noisy data.
method Learn a regularization matrix from localized eigenvectors.
result Suppresses eigenvalues associated with localized eigenvectors.
The paper tackles learning symmetries in data without expert knowledge.
problem Learning symmetries in data from raw data without prior knowledge.
method Develops methods to select eigenvectors for orthogonal symmetries and compares their effectiveness.
result The problem of learning symmetries is as hard as the graph automorphism problem in the worst case, but can be simplified with certain restrictions.
New algorithm updates eigenvectors of evolving graphs efficiently.
problem Updating eigenvectors of dynamic graphs.
method Subspace projection based on Rayleigh-Ritz projections.
result Strong performance in eigenvector approximation and downstream tasks.
A new algorithm reduces online eigenvector computation time while maintaining optimal performance.
problem Online learning of top eigenvectors in both adversarial and stochastic settings.
method Follow the Compressed Leader (FTCL) framework, compressing the matrix strategy to dimensions 3 (adversarial) and 1 (stochastic).
result Achieves optimal regret without sacrificing running time, resolving open questions.
Paper addresses eigenvector perturbation in small eigen-gap scenarios.
problem Fine-grained behavior of eigenvectors in the presence of small eigen-gaps.
method Develops de-biased estimators for linear functions of an unknown eigenvector.
result Achieves minimax lower bounds for a family of scenarios, even with small eigen-gaps.
Study eigenvalues and eigenvectors in neural networks, focusing on signal propagation.
problem Characterize signal eigenvalues and eigenvectors in neural networks.
method Characterizes signal eigenvalues and eigenvectors for a nonlinear spiked covariance model.
result Provides precise quantitative characterizations of signal eigenvalues and eigenvectors in neural networks.
New method improves subspace iteration for eigenvectors in machine learning.
problem Computing eigenvectors for large-scale problems in machine learning.
method Subspace iteration with ℓ2o∞ norm convergence analysis. result Deterministic bounds and practical stopping criterion for improved performance.
Exact formulas for eigenvector overlaps in correlated random matrices.
problem Understanding overlaps between eigenvectors of correlated random matrices.
method Exact formulas derived for overlaps between eigenvectors of large correlated random matrices with additive or multiplicative noise.
result Overlaps only depend on measurable quantities and do not require knowledge of the noiseless matrices.
In many applications, one has side information, e.g., labels that are provided in a semi-supervised manner, about a specific target region of a large data set, and one wants to perform machine learning and data analysis tasks "nearby" that prespecified target region. For example, one might be interested in the clusteri…
We calculate eigenvector overlaps between intersecting time periods of covariance matrices.
problem Analyzing overlapping time periods in covariance matrices.
method Girko linearisation and extended local laws.
result Computed eigenvector overlaps for intersecting time intervals.
SCI-PI solves scale invariant problems efficiently.
problem Solving scale invariant problems in optimization.
method Introduces SCI-PI and proves its convergence.
result SCI-PI achieves local linear convergence.
Machine learning models perform better with location coordinates alone, not Moran Eigenvectors.
problem Improving machine learning models for spatial data.
method Examined Moran Eigenvectors as additional spatial features in machine learning models using synthetic datasets.
result Machine learning models using only location coordinates achieve better accuracies than eigenvector-based approaches.
Spectral clustering performance depends on eigenvector fluctuations, shown to be Gaussian.
problem Predicting the performance of spectral clustering.
method General spike random matrix model and rotational invariance of noise.
result Fluctuations of eigenvector entries are Gaussian in large-dimensional regime.
New theory for eigenvectors of generalized Laplacian matrices, addressing dependency issues.
problem Dependency in random matrix theory hinders eigenvector analysis for latent embeddings.
method Introduces generalized Laplacian matrices and a new asymptotic theory framework.
result Established asymptotic normalities for spiked eigenvectors and eigenvalues.
We quantify uncertainty in Oja's algorithm's leading eigenvector estimation.
problem Estimating the error of Oja's algorithm's leading eigenvector from streaming data.
method Combining U-statistics, high-dimensional central limit theorems, and multiplier bootstrap.
result Established a weighted χ² approximation for the error between the eigenvector and algorithm output.
New insights into spectral clustering reveal strong connections within eigenvectors.
problem Clustering on graphs when there are two underlying clusters.
method Analyzes the eigenvector corresponding to the second largest eigenvalue of the adjacency matrix.
result Vertices with extreme values in the eigenvector are more reliably classified.
This paper proves the convergence rate of Krasulina's estimator for least eigenvalue and eigenvector.
problem Finding the least eigenvalue and eigenvector of an unknown covariance matrix.
method Developed a convergence proof for Krasulina's estimator.
result Established the convergence rate of Krasulina's estimator for the least eigenvalue and eigenvector.
We propose a general framework to study the stability of the subspace spanned by P consecutive eigenvectors of a generic symmetric matrix H0, when a small perturbation is added. This problem is relevant in various contexts, including quantum dissipation (H0 is then the Hamiltonian) and financial ris…
We propose a general framework to study the stability of the subspace spanned by P consecutive eigenvectors of a generic symmetric matrix H0, when a small perturbation is added. This problem is relevant in various contexts, including quantum dissipation (H0 is then the Hamiltonian) and risk control …
Kernel methods are successful approaches for different machine learning problems. This success is mainly rooted in using feature maps and kernel matrices. Some methods rely on the eigenvalues/eigenvectors of the kernel matrix, while for other methods the spectral information can be used to estimate the excess risk. An …
New algorithms solve k-GenEV and k-CCA problems faster.
problem Finding top k generalized eigenvectors and canonical-correlation vectors.
method Proposed LazyEV and LazyCCA algorithms with linear dependence on input size and k. result First doubly-accelerated algorithms with running times independent of matrix condition number and eigengap.
New method learns high-quality Laplacian representations for reinforcement learning.
problem Lack of accurate Laplacian representations in large or continuous state spaces.
method Reformulated spectral graph drawing objective to have eigenvectors as unique global minimizer.
result Learned Laplacian representations more faithfully approximate the ground truth.
A new method approximates Laplacian eigenvectors for RL efficiently.
problem Efficiently learning state representations in RL.
method General and scalable approach to approximating Laplacian eigenvectors.
result Empirically shows improved performance in RL tasks.
New neural architectures invariant to sign flips and basis symmetries for graph representation learning.
problem Learning invariant graph representations from eigenvectors.
method SignNet and BasisNet neural architectures that are invariant to sign flips and basis symmetries.
result Proven to be universal, approximating any continuous function of eigenvectors with desired invariances.
Improved portfolio optimization using Kendall-like correlation coefficients.
problem Accurate estimation of eigenvectors in data-poor regimes for portfolio optimization.
method Developed generalized correlation coefficients based on Kendall's rank correlation.
result Markowitz portfolios with lower out-of-sample risk using these coefficients.
Study eigenvector overlaps in large Gaussian matrices, simplifying for GOE.
problem Investigate eigenvector overlaps in large Gaussian matrices.
method Analysis of eigenvector flow under Dyson Brownian motion.
result Explicit computation of limiting rescaled mean squared overlaps.
Describes maps with prescribed eigenvectors of Jacobian matrices.
problem Maps with specific eigenvectors of Jacobian matrices.
method Coordinate-independent definition of Jacobian, Frobenius integrability theorems, rich partial frames.
result Complete analysis for rich partial frames, partial results for non-rich and non-involutive cases.
Many problems in machine learning and statistics can be formulated as (generalized) eigenproblems. In terms of the associated optimization problem, computing linear eigenvectors amounts to finding critical points of a quadratic function subject to quadratic constraints. In this paper we show that a certain class of con…
Recently, Mahoney and Orecchia demonstrated that popular diffusion-based procedures to compute a quick \emph{approximation} to the first nontrivial eigenvector of a data graph Laplacian \emph{exactly} solve certain regularized Semi-Definite Programs (SDPs). In this paper, we extend that result by providing a statistica…
Study shows how many samples are needed for eigenvector/eigenvalue accuracy.
problem Guaranteeing eigenvector and eigenvalue accuracy of sample vs actual covariance matrices.
method Proves inner product decrease proportional to eigenvalue distance for various distributions.
result Non-asymptotic concentration bounds and conditions for distinguishing principal components.
Matrix factorization simplifies user-item co-occurrence analysis.
problem Understanding the meaning of low-dimensional matrices in matrix factorization.
method Showed matrix factorization equals calculating eigenvectors of co-occurrence matrices, using RMT insights.
result Low-dimension matrices represent a reduced noise user and item co-occurrence space.
In this paper, we consider an ℓ0-norm penalized formulation of the generalized eigenvalue problem (GEP), aimed at extracting the leading sparse generalized eigenvector of a matrix pair. The formulation involves maximization of a discontinuous nonconcave objective function over a nonconvex constraint set, and is…
Graph convolutional networks fail to use eigenvectors beyond the first, unlike spectral embedding.
problem Understanding when graph convolutional networks fail compared to spectral embedding.
method Presented a simple generative model to illustrate failure.
result Graph convolutional networks fail to use eigenvectors beyond the first in certain graphs.
A simple sketch improves online eigenvector and SDP problems.
problem Online eigenvector and semidefinite programming problems.
method Randomized mirror projection and mirror descent analysis.
result Regret bounds similar to MMW with reduced complexity.
Paper develops IFTRR to solve sparse generalized eigenvalue problems efficiently.
problem Finding the leading eigenvector with at most k nonzero entries in sparse generalized eigenvalue problems.
method Inverse-free truncated Rayleigh-Ritz method (IFTRR) with a new truncation strategy.
result IFTRR efficiently finds the support set of the leading eigenvector for large scale problems.
Eigenvalue and eigenvector estimation improves with asymmetric data.
problem Estimating eigenvalues and eigenvectors from asymmetrically perturbed symmetric matrices.
method Eigenvalue and eigenvector analyses of asymmetrically perturbed low-rank matrices.
result The leading eigenvalue of the data matrix can be significantly more accurate than its singular value.
LEGO estimates tangent spaces more robustly than LPCA in noisy data.
problem Estimating tangent spaces in high-noise settings.
method Spectral method using graph Laplacian eigenvectors and gradient orthogonization.
result LEGO yields more robust tangent space estimates than LPCA.
Paper tackles small eigen-gap estimation and inference for noisy symmetric matrices.
problem Estimating eigenvectors with small eigen-gap and fine-grained statistical reasoning.
method Eigen-decomposition of asymmetric data matrix, distribution-free procedures, adaptive to heteroscedastic noise.
result Minimax optimal under Gaussian noise, confidence intervals for eigenvalues, small eigen-gap handling.
Biological neurons learn tensor decompositions of higher-order correlations using nonlinear Hebbian plasticity.
problem Learning higher-order correlations in biological neurons.
method Introduce and study generalized nonlinear Hebbian learning rules.
result Neurons can learn tensor eigenvectors of higher-order input correlation tensors.
New methods find eigenvectors faster than Lanczos's method.
problem Finding the leading eigenvector efficiently.
method Coordinate-wise methods combining shift-and-invert with linear regression.
result Global convergence with runtime guarantees better than Lanczos's method.
In spectral clustering, one defines a similarity matrix for a collection of data points, transforms the matrix to get the Laplacian matrix, finds the eigenvectors of the Laplacian matrix, and obtains a partition of the data using the leading eigenvectors. The last step is sometimes referred to as rounding, where one ne…
Sep-SpectralNet improves SE for broader applicability and scalability.
problem Three main drawbacks of current SE implementations: generalizability, scalability, and eigenvectors separation.
method Sep-SpectralNet extends SpectralNet with an eigenvector separation post-processing step.
result Sep-SpectralNet achieves consistent SE approximation and generalization, enhancing scalability and applicability.
New metric tensor field on symmetric matrices simplifies eigenvector computation.
problem Complex eigenvector computation for 2x2 symmetric matrices.
method Introducing a metric tensor field on the space of symmetric matrices, resulting in a curved manifold.
result Parallel transport simplifies eigenvector computation for one-parameter families of matrices.
Paper proposes a distributed method to estimate principal eigenvector from high-rate streaming data.
problem Estimating principal eigenvector from high streaming data rate.
method Distributed Krasulina (D-Krasulina) and mini-batch extension (DM-Krasulina) methods.
result Achieves optimal estimation error rates under high streaming conditions.