LEGO estimates tangent spaces more robustly than LPCA in noisy data.
problem Estimating tangent spaces in high-noise settings.
method Spectral method using graph Laplacian eigenvectors and gradient orthogonization.
result LEGO yields more robust tangent space estimates than LPCA.
The paper tackles learning symmetries in data without expert knowledge.
problem Learning symmetries in data from raw data without prior knowledge.
method Develops methods to select eigenvectors for orthogonal symmetries and compares their effectiveness.
result The problem of learning symmetries is as hard as the graph automorphism problem in the worst case, but can be simplified with certain restrictions.
The problem of estimating sparse eigenvectors of a symmetric matrix attracts a lot of attention in many applications, especially those with high dimensional data set. While classical eigenvectors can be obtained as the solution of a maximization problem, existing approaches formulate this problem by adding a penalty te…
We provide new examples of diffusion operators in dimension 2 and 3 which have orthogonal polynomials as eigenvectors. Their construction rely on the finite subgroups of O(3) and their invariant polynomials.
Unified framework for multi-view learning with orthogonal projections.
problem Learning individual orthogonal projections for multiple views.
method Successive approximations via eigenvectors, iterative Krylov subspace method.
result Consistently competitive and often better than existing methods.
SpecNet2 improves spectral embedding without orthogonalization, achieving better performance and efficiency.
problem Improving spectral embedding methods for better performance and efficiency.
method Optimizes an equivalent objective of the eigen-problem without orthogonalization, allowing separate row and column sampling.
result Local and global convergence of the new objective using batch-based gradient descent is proven, and improved performance and efficiency are demonstrated on simulated and image datasets.
The study extends Jacobi-orthogonality to indefinite scalar product spaces.
problem Generalizing Jacobi-orthogonality to indefinite scalar product spaces.
method Comparing principles, investigating tensor relations, proving properties.
result Every quasi-Clifford tensor is Jacobi-orthogonal; certain tensors are Jacobi-dual or Osserman.
EigenGame reinterprets PCA as a game to find eigenvectors.
problem Finding principal components efficiently and accurately.
method EigenGame treats PCA as a Nash equilibrium game, using gradient-based updates.
result EigenGame algorithm combines Oja's rule and Gram-Schmidt orthogonalization.
Spectral methods achieve near-optimal performance in orthogonal and permutation group synchronization.
problem Recovering group elements from pairwise measurements in computer vision.
method Spectral methods applied with the leave-one-out technique.
result Near-optimal performance bounds for orthogonal and permutation group synchronization established.
Sign equivariant networks improve model expressiveness for spectral geometric learning.
problem Limited expressiveness of sign invariant models for tasks like graph link prediction.
method Developed sign equivariant neural network architectures based on new analytic sign equivariant polynomials.
result Sign equivariant models achieve theoretical benefits in spectral geometric learning tasks.
We propose a general framework to study the stability of the subspace spanned by P consecutive eigenvectors of a generic symmetric matrix H0, when a small perturbation is added. This problem is relevant in various contexts, including quantum dissipation (H0 is then the Hamiltonian) and risk control …
New mathematical foundations for stable RKHSs improve system identification.
problem Improving stability tests and modeling of impulse responses.
method Providing new structural properties and stability conditions for stable RKHSs.
result Any stable kernel admits feature maps induced by orthogonal eigenvectors in l2.
In this paper, we apply tools from the random matrix theory (RMT) to estimates of correlations across volatility of various assets in the S&P 500. The volatility inputs are estimated by modeling price fluctuations as GARCH(1,1) process. The corresponding correlation matrix is constructed. It is found that the distribut…
In many applications, one has side information, e.g., labels that are provided in a semi-supervised manner, about a specific target region of a large data set, and one wants to perform machine learning and data analysis tasks "nearby" that prespecified target region. For example, one might be interested in the clusteri…
Random matrix ensembles yield uniform distributions on manifolds.
problem Understanding distributions of vectors in random matrix ensembles.
method Analyzing eigenvalues, singular values, and Autonne-Takagi vectors of various random matrix ensembles.
result Uniform distributions on specific manifolds for different types of random matrix ensembles.
We use methods of random matrix theory to analyze the cross-correlation matrix C of price changes of the largest 1000 US stocks for the 2-year period 1994-95. We find that the statistics of most of the eigenvalues in the spectrum of C agree with the predictions of random matrix theory, but there are deviations for a fe…
Algorithm estimates principal eigenvector with adaptive sensing, improving over non-adaptive methods.
problem Estimating principal eigenvector with limited scalar measurements.
method Compressed variant of Oja's algorithm using two adaptive measurements per sample.
result Convergence rate of O(λ1λ2d2/(Δ2t)) after t iterations, matching information-theoretic lower bound. We propose a general framework to study the stability of the subspace spanned by P consecutive eigenvectors of a generic symmetric matrix H0, when a small perturbation is added. This problem is relevant in various contexts, including quantum dissipation (H0 is then the Hamiltonian) and financial ris…
We analyze cross-correlations between price fluctuations of different stocks using methods of random matrix theory (RMT). Using two large databases, we calculate cross-correlation matrices C of returns constructed from (i) 30-min returns of 1000 US stocks for the 2-yr period 1994--95 (ii) 30-min returns of 881 US stock…
Study rigid classes on hyperkahler manifolds, showing general ones are rigid.
problem Characterize rigid classes on compact hyperkahler manifolds.
method Analyze eigenvectors of hyperbolic automorphisms and use BBF form.
result General parabolic classes on hyperkahler manifolds are rigid.
New formulas for flag manifolds simplify eigenvector perturbation.
problem Eigenvector perturbation problem
method Explicit formulas derived from normal homogeneous spaces
result Effective formulas for flag manifolds simplify eigenvector perturbation
We introduce a novel harmonic analysis for functions defined on the vertices of a strongly connected directed graph of which the random walk operator is the cornerstone. As a first step, we consider the set of eigenvectors of the random walk operator as a non-orthogonal Fourier-type basis for functions over directed gr…
A new classifier uses weighted orthogonal regression for robust classification with limited data.
problem Challenges in classification with insufficient training data.
method Exploits intrinsic structure of data through Eigen components with specific weights determined by eigenvalues.
result Robust learning in classification problems with limited data.
I introduce Forecastable Component Analysis (ForeCA), a novel dimension reduction technique for temporally dependent signals. Based on a new forecastability measure, ForeCA finds an optimal transformation to separate a multivariate time series into a forecastable and an orthogonal white noise space. I present a converg…
New method trains neural networks in spectral domain for improved performance.
problem Training deep neural networks in the space of nodes.
method Trains neural networks in the spectral domain, modifying eigenvalues and eigenvectors of transfer operators.
result Superior performance compared to standard methods, especially when adjusting eigenvalues.
Introduces tunable basis functions for Gaussian processes.
problem Reduces computational complexity in Gaussian process approximations.
method Introduces tunable, local, and bounded basis functions for kernel approximation.
result Demonstrates superior performance compared to state-of-the-art methods, especially with poorly chosen kernel functions.
A new method for sparse PCA using orthogonal rotations and soft-thresholding.
problem Sparse PCA with a new basis using orthogonal rotations.
method Initialize with leading principal components, apply kimesk orthogonal rotation, and soft-threshold the rotated components. result The proposed method is more stable and explains more variance compared to alternatives.
Study eigenvector overlaps in large Gaussian matrices, simplifying for GOE.
problem Investigate eigenvector overlaps in large Gaussian matrices.
method Analysis of eigenvector flow under Dyson Brownian motion.
result Explicit computation of limiting rescaled mean squared overlaps.
Latent variable models with hidden binary units appear in various applications. Learning such models, in particular in the presence of noise, is a challenging computational problem. In this paper we propose a novel spectral approach to this problem, based on the eigenvectors of both the second order moment matrix and t…
New method identifies structural parameters without assuming uncorrelated errors.
problem Identifying structural parameters in simultaneous equation models.
method Exploits higher-order cumulant restrictions, not requiring uncorrelated errors.
result Simple diagonality condition on hth-order cumulants identifies structural parameter matrix. Machine learning models perform better with location coordinates alone, not Moran Eigenvectors.
problem Improving machine learning models for spatial data.
method Examined Moran Eigenvectors as additional spatial features in machine learning models using synthetic datasets.
result Machine learning models using only location coordinates achieve better accuracies than eigenvector-based approaches.
Autoencoder performance is predicted by eigenvalues of weight matrices.
problem Predicting an autoencoder's generalization ability without dataset knowledge.
method Analyze Jacobian matrices' eigenvalues to bound mean squared errors.
result Eigenvalues are good predictors of MSE on test points.
Paper addresses eigenvector perturbation in small eigen-gap scenarios.
problem Fine-grained behavior of eigenvectors in the presence of small eigen-gaps.
method Develops de-biased estimators for linear functions of an unknown eigenvector.
result Achieves minimax lower bounds for a family of scenarios, even with small eigen-gaps.
We calculate eigenvector overlaps between intersecting time periods of covariance matrices.
problem Analyzing overlapping time periods in covariance matrices.
method Girko linearisation and extended local laws.
result Computed eigenvector overlaps for intersecting time intervals.
This work improves disentanglement in latent space models without sacrificing generation quality.
problem Trade-off between disentanglement and generation quality in latent space models.
method Manifold optimization with a sum of autoencoder and PCA reconstruction errors, on the Stiefel manifold.
result Improves disentanglement without sacrificing generation quality.
In spectral clustering, one defines a similarity matrix for a collection of data points, transforms the matrix to get the Laplacian matrix, finds the eigenvectors of the Laplacian matrix, and obtains a partition of the data using the leading eigenvectors. The last step is sometimes referred to as rounding, where one ne…
We study the problem asking if one can embed manifolds into finite dimensional Euclidean spaces by taking finite number of eigenvector fields of the connection Laplacian. This problem is essential for the dimension reduction problem in massive data analysis. Singer-Wu proposed the vector diffusion map which embeds mani…
This paper introduces a novel framework for generative models based on Restricted Kernel Machines (RKMs) with joint multi-view generation and uncorrelated feature learning, called Gen-RKM. To enable joint multi-view generation, this mechanism uses a shared representation of data from various views. Furthermore, the mod…
New metric tensor field on symmetric matrices simplifies eigenvector computation.
problem Complex eigenvector computation for 2x2 symmetric matrices.
method Introducing a metric tensor field on the space of symmetric matrices, resulting in a curved manifold.
result Parallel transport simplifies eigenvector computation for one-parameter families of matrices.
Study the spectrum of Poincaré operator in triaxial ellipsoids.
problem Spectrum of the Poincaré operator in triaxial ellipsoids.
method Microlocal analysis of partial differential equations and polynomial vector fields.
result Polynomial eigenvectors and large-degree asymptotics of the operator.
Spectral clustering performance depends on eigenvector fluctuations, shown to be Gaussian.
problem Predicting the performance of spectral clustering.
method General spike random matrix model and rotational invariance of noise.
result Fluctuations of eigenvector entries are Gaussian in large-dimensional regime.
New neural architectures invariant to sign flips and basis symmetries for graph representation learning.
problem Learning invariant graph representations from eigenvectors.
method SignNet and BasisNet neural architectures that are invariant to sign flips and basis symmetries.
result Proven to be universal, approximating any continuous function of eigenvectors with desired invariances.
New algorithm updates eigenvectors of evolving graphs efficiently.
problem Updating eigenvectors of dynamic graphs.
method Subspace projection based on Rayleigh-Ritz projections.
result Strong performance in eigenvector approximation and downstream tasks.
How many samples are sufficient to guarantee that the eigenvectors and eigenvalues of the sample covariance matrix are close to those of the actual covariance matrix? For a wide family of distributions, including distributions with finite second moment and distributions supported in a centered Euclidean ball, we prove …
A special Kähler-Ricci potential on a Kähler manifold is any nonconstant C∞ function τ such that J(∇τ) is a Killing vector field and, at every point with dτ=0, all nonzero tangent vectors orthogonal to ∇τ and J(∇τ) are eigenvectors of both ∇dτ and the Ricci tensor. For instan…
A new method cleans and analyzes stock return correlation matrices.
problem Improving the accuracy of covariance/correlation matrices in financial data.
method Constrained principal component analysis using financial data and optimal portfolios.
result Identified stylized patterns in correlation matrix eigenvalues and weights.
New algorithm consistently orients eigenvectors for machine learning.
problem Inconsistent eigenvector orientation in machine learning.
method Postprocesses well-established eigen calls to create consistently oriented eigenvectors.
result Interpretable time series of training weights in machine learning models.
New theory for eigenvectors of generalized Laplacian matrices, addressing dependency issues.
problem Dependency in random matrix theory hinders eigenvector analysis for latent embeddings.
method Introduces generalized Laplacian matrices and a new asymptotic theory framework.
result Established asymptotic normalities for spiked eigenvectors and eigenvalues.