Paper addresses eigenvector perturbation in small eigen-gap scenarios.
problem Fine-grained behavior of eigenvectors in the presence of small eigen-gaps.
method Develops de-biased estimators for linear functions of an unknown eigenvector.
result Achieves minimax lower bounds for a family of scenarios, even with small eigen-gaps.
Improved spectral clustering with fewer eigenvectors performs better.
problem Improving spectral clustering performance under weaker conditions.
method Tighter analysis and using fewer eigenvectors for embedding.
result Spectral clustering can produce better results with fewer eigenvectors.
In many applications, one has side information, e.g., labels that are provided in a semi-supervised manner, about a specific target region of a large data set, and one wants to perform machine learning and data analysis tasks "nearby" that prespecified target region. For example, one might be interested in the clusteri…
New method improves subspace iteration for eigenvectors in machine learning.
problem Computing eigenvectors for large-scale problems in machine learning.
method Subspace iteration with ℓ2o∞ norm convergence analysis. result Deterministic bounds and practical stopping criterion for improved performance.
LEGO estimates tangent spaces more robustly than LPCA in noisy data.
problem Estimating tangent spaces in high-noise settings.
method Spectral method using graph Laplacian eigenvectors and gradient orthogonization.
result LEGO yields more robust tangent space estimates than LPCA.
The paper explores how kernel eigenalignments affect generalization in KRR.
problem Achieving robust generalization in kernel methods.
method Direct connection between generalization and matrix eigenvectors/eigenvalues, focusing on finite-sample settings.
result Strong generalization requires increasing eigenvector alignment, eigenvalue magnitude, or gaps between eigenvalues.
Paper proposes a distributed method to estimate principal eigenvector from high-rate streaming data.
problem Estimating principal eigenvector from high streaming data rate.
method Distributed Krasulina (D-Krasulina) and mini-batch extension (DM-Krasulina) methods.
result Achieves optimal estimation error rates under high streaming conditions.
We study the problem asking if one can embed manifolds into finite dimensional Euclidean spaces by taking finite number of eigenvector fields of the connection Laplacian. This problem is essential for the dimension reduction problem in massive data analysis. Singer-Wu proposed the vector diffusion map which embeds mani…
Algorithm estimates principal eigenvector with adaptive sensing, improving over non-adaptive methods.
problem Estimating principal eigenvector with limited scalar measurements.
method Compressed variant of Oja's algorithm using two adaptive measurements per sample.
result Convergence rate of O(λ1λ2d2/(Δ2t)) after t iterations, matching information-theoretic lower bound. New insights into spectral clustering reveal strong connections within eigenvectors.
problem Clustering on graphs when there are two underlying clusters.
method Analyzes the eigenvector corresponding to the second largest eigenvalue of the adjacency matrix.
result Vertices with extreme values in the eigenvector are more reliably classified.
New algorithm updates eigenvectors of evolving graphs efficiently.
problem Updating eigenvectors of dynamic graphs.
method Subspace projection based on Rayleigh-Ritz projections.
result Strong performance in eigenvector approximation and downstream tasks.
The paper proves Lipschitz regularity of graph Laplacian eigenvectors on random data clouds.
problem Analyzing the regularity of solutions to graph Laplacian equations on random data points.
method Probabilistic coupling of random walks and interpolation method for point clouds to continuum.
result Graph Laplacian eigenvectors are essentially Lipschitz with constants depending on eigenvalues.
Fast algorithm recovers principal eigenvector from noisy matrices.
problem Recovering the first principal eigenvector from noisy positive semidefinite matrices.
method Cone projected power iteration algorithm.
result Achieves polynomial time complexity and small error for certain convex cones.
Unified framework for multi-view learning with orthogonal projections.
problem Learning individual orthogonal projections for multiple views.
method Successive approximations via eigenvectors, iterative Krylov subspace method.
result Consistently competitive and often better than existing methods.
We apply random matrix theory to compare correlation matrix estimators C obtained from emerging market data. The correlation matrices are constructed from 10 years of daily data for stocks listed on the Johannesburg Stock Exchange (JSE) from January 1993 to December 2002. We test the spectral properties of C against ra…
Novel spectral embedding considers node weights for graph analysis.
problem Graph node importance quantification.
method Normalized Laplacian eigenvectors for low-energy configurations.
result Weighted embeddings improve graph configurations.
Proposes a new method for high-dimensional data analysis.
problem Sparse PCA limitations in high-dimensional data analysis.
method Low-rank principal eigenmatrix analysis, matricized rank-truncated power method.
result Competitive empirical performance in synthetic data sets.
Paper tackles small eigen-gap estimation and inference for noisy symmetric matrices.
problem Estimating eigenvectors with small eigen-gap and fine-grained statistical reasoning.
method Eigen-decomposition of asymmetric data matrix, distribution-free procedures, adaptive to heteroscedastic noise.
result Minimax optimal under Gaussian noise, confidence intervals for eigenvalues, small eigen-gap handling.
A new algorithm improves Wasserstein discriminant analysis for better data classification.
problem Improving data classification in machine learning.
method Bi-level nonlinear eigenvector algorithm (WDA-nepv) for optimal transport and trace ratio optimizations.
result WDA-nepv enhances classification accuracy and scalability.
Random matrix analysis reveals that neural network weights are mostly random, with some indicating learned information.
problem Understanding how neural networks store information needed for tasks.
method Random matrix theory (RMT) applied to weight matrices of trained deep neural networks.
result Most singular values and eigenvectors of trained neural networks follow universal RMT predictions, suggesting they are random and do not contain system-specific information.
We quantify uncertainty in Oja's algorithm's leading eigenvector estimation.
problem Estimating the error of Oja's algorithm's leading eigenvector from streaming data.
method Combining U-statistics, high-dimensional central limit theorems, and multiplier bootstrap.
result Established a weighted χ² approximation for the error between the eigenvector and algorithm output.
Eigenvalue and eigenvector estimation improves with asymmetric data.
problem Estimating eigenvalues and eigenvectors from asymmetrically perturbed symmetric matrices.
method Eigenvalue and eigenvector analyses of asymmetrically perturbed low-rank matrices.
result The leading eigenvalue of the data matrix can be significantly more accurate than its singular value.
Spectral methods are popular in detecting global structures in the given data that can be represented as a matrix. However when the data matrix is sparse or noisy, classic spectral methods usually fail to work, due to localization of eigenvectors (or singular vectors) induced by the sparsity or noise. In this work, we …
A new algorithm reduces data dimensionality and decorrelation in a distributed setting.
problem Distributed PCA for decorrelated features in big data.
method Feedforward neural network-based one time-scale algorithm for estimating eigenvectors of distributed data covariance matrix.
result DSA converges linearly to the true solution.
Complex network analysis reveals dominant stocks in financial stock returns correlations.
problem Inferring financial stock returns correlations from complex network analysis.
method Simulated geometric Brownian motion for stocks, complex network analysis, eigenvector centrality, clustering.
result Returns correlation matrix is dominated by stocks with high eigenvector centrality and clustering.
This paper focuses on obtaining clustering information about a distribution from its i.i.d. samples. We develop theoretical results to understand and use clustering information contained in the eigenvectors of data adjacency matrices based on a radial kernel function with a sufficiently fast tail decay. In particular, …
We perform a comparative analysis of the Chinese stock market around the occurrence of the 2008 crisis based on the random matrix analysis of high-frequency stock returns of 1228 stocks listed on the Shanghai and Shenzhen stock exchanges. Both raw correlation matrix and partial correlation matrix with respect to the ma…
A simple sketch improves online eigenvector and SDP problems.
problem Online eigenvector and semidefinite programming problems.
method Randomized mirror projection and mirror descent analysis.
result Regret bounds similar to MMW with reduced complexity.
SCI-PI solves scale invariant problems efficiently.
problem Solving scale invariant problems in optimization.
method Introduces SCI-PI and proves its convergence.
result SCI-PI achieves local linear convergence.
Kernel method is a very powerful tool in machine learning. The trick of kernel has been effectively and extensively applied in many areas of machine learning, such as support vector machine (SVM) and kernel principal component analysis (kernel PCA). Kernel trick is to define a kernel function which relies on the inner-…
A new method for streaming PCA provides confidence intervals for eigenvector entries.
problem Uncertainty quantification for individual entries in streaming PCA.
method Oja's algorithm, Bernstein-type concentration bound, Central Limit Theorem, subsampling algorithm.
result Sharp concentration bound and Central Limit Theorem for streaming PCA entries.
Study eigenvector overlaps in large Gaussian matrices, simplifying for GOE.
problem Investigate eigenvector overlaps in large Gaussian matrices.
method Analysis of eigenvector flow under Dyson Brownian motion.
result Explicit computation of limiting rescaled mean squared overlaps.
We examine volatility of an Indian stock market in terms of aspects like participation, synchronization of stocks and quantification of volatility using the random matrix approach. Volatility pattern of the market is found using the BSE index for the three-year period 2000-2002. Random matrix analysis is carried out us…
This paper considers the problem of canonical-correlation analysis (CCA) (Hotelling, 1936) and, more broadly, the generalized eigenvector problem for a pair of symmetric matrices. These are two fundamental problems in data analysis and scientific computing with numerous applications in machine learning and statistics (…
In this paper the exact linear relation between the leading eigenvectors of the modularity matrix and the singular vectors of an uncentered data matrix is developed. Based on this analysis the concept of a modularity component is defined, and its properties are developed. It is shown that modularity component analysis …
Machine learning models perform better with location coordinates alone, not Moran Eigenvectors.
problem Improving machine learning models for spatial data.
method Examined Moran Eigenvectors as additional spatial features in machine learning models using synthetic datasets.
result Machine learning models using only location coordinates achieve better accuracies than eigenvector-based approaches.
Harmonic analysis on directed graphs for signal modeling and semi-supervised learning.
problem Signal analysis on directed graphs.
method Introduced a Fourier-type basis using eigenvectors of the random walk operator, developed wavelet transforms for multi-scale analysis.
result Efficiency of the proposed framework for semi-supervised learning and signal modeling on directed graphs.
In this on-going work, I explore certain theoretical and empirical implications of data transformations under the PCA. In particular, I state and prove three theorems about PCA, which I paraphrase as follows: 1). PCA without discarding eigenvector rows is injective, but looses this injectivity when eigenvector rows are…
Principal component analysis (PCA) is one of the most commonly used statistical procedures with a wide range of applications. Consider the points X1,X2,...,Xn are vectors drawn i.i.d. from a distribution with mean zero and covariance Σ, where Σ is unknown. Let An=XnXnT, then E[An]=Σ. This paper …
SEDA improves RLDA for high-dimensional data.
problem Inconsistent performance of RLDA in high-dimensional scenarios.
method Developed a non-asymptotic approximation of misclassification rate, derived new theoretical results on eigenvectors, and proposed SEDA algorithm.
result SEDA achieves higher classification accuracy and dimensionality reduction compared to existing LDA methods.
We analyze cross-correlations between price fluctuations of different stocks using methods of random matrix theory (RMT). Using two large databases, we calculate cross-correlation matrices C of returns constructed from (i) 30-min returns of 1000 US stocks for the 2-yr period 1994--95 (ii) 30-min returns of 881 US stock…
We calculate eigenvector overlaps between intersecting time periods of covariance matrices.
problem Analyzing overlapping time periods in covariance matrices.
method Girko linearisation and extended local laws.
result Computed eigenvector overlaps for intersecting time intervals.
In this article, we study spectral methods for community detection based on α-parametrized normalized modularity matrix hereafter called Lα in heterogeneous graph models. We show, in a regime where community detection is not asymptotically trivial, that Lα can be well approximated by a more tract…
Algorithm improves online canonical correlation analysis.
problem Online canonical correlation analysis.
method Stochastic Scaled-Gradient Descent (SSGD) for minimizing expectation over Riemannian manifolds.
result Achieved optimal one-time-scale algorithm with explicit rate of local asymptotic convergence.
Paper develops IFTRR to solve sparse generalized eigenvalue problems efficiently.
problem Finding the leading eigenvector with at most k nonzero entries in sparse generalized eigenvalue problems.
method Inverse-free truncated Rayleigh-Ritz method (IFTRR) with a new truncation strategy.
result IFTRR efficiently finds the support set of the leading eigenvector for large scale problems.
This work provides improved guarantees for streaming principle component analysis (PCA). Given A1,…,An∈Rd×d sampled independently from distributions satisfying E[Ai]=Σ for Σ⪰0, this work provides an O(d)-space linear-time single-pass streaming algorithm …
In spectral clustering, one defines a similarity matrix for a collection of data points, transforms the matrix to get the Laplacian matrix, finds the eigenvectors of the Laplacian matrix, and obtains a partition of the data using the leading eigenvectors. The last step is sometimes referred to as rounding, where one ne…
Two new algorithms improve robust PCA and Schatten packing.
problem Robustly estimating the top eigenvector of corrupted sub-Gaussian data.
method Two iterative filtering and nearly-linear time algorithms.
result First polynomial-time algorithms for non-trivial covariance estimation.