arXiv research

Sep-SpectralNet improves SE for broader applicability and scalability.

New method detects global factors near BBP phase transition in high-dimensional data.

Algorithm estimates principal eigenvector with adaptive sensing, improving over non-adaptive methods.

In this technical report, we discuss several sampling algorithms for Determinantal Point Processes (DPP). DPPs have recently gained a broad interest in the machine learning and statistics literature as random point processes with negative correlation, i.e., ones that can generate a "diverse" sample from a set of items.…

The extraction of clusters from a dataset which includes multiple clusters and a significant background component is a non-trivial task of practical importance. In image analysis this manifests for example in anomaly detection and target detection. The traditional spectral clustering algorithm, which relies on the lead…

Paper characterizes optimal graph clustering limits under a new model.

In this paper, we consider an $\ell_{0}$-norm penalized formulation of the generalized eigenvalue problem (GEP), aimed at extracting the leading sparse generalized eigenvector of a matrix pair. The formulation involves maximization of a discontinuous nonconcave objective function over a nonconvex constraint set, and is…

The paper extends hypothesis testing to non-diagonalizable matrices, improving network statistics inference.

Graph matching aims at finding the vertex correspondence between two unlabeled graphs that maximizes the total edge weight correlation. This amounts to solving a computationally intractable quadratic assignment problem. In this paper we propose a new spectral method, GRAph Matching by Pairwise eigen-Alignments (GRAMPA)…

Study eigenvector overlaps in large Gaussian matrices, simplifying for GOE.

The proprietary nature of Hedge Fund investing means that it is common practise for managers to release minimal information about their returns. The construction of a Fund of Hedge Funds portfolio requires a correlation matrix which often has to be estimated using a relatively small sample of monthly returns data which…

Machine learning models perform better with location coordinates alone, not Moran Eigenvectors.

I introduce Forecastable Component Analysis (ForeCA), a novel dimension reduction technique for temporally dependent signals. Based on a new forecastability measure, ForeCA finds an optimal transformation to separate a multivariate time series into a forecastable and an orthogonal white noise space. I present a converg…

Paper addresses eigenvector perturbation in small eigen-gap scenarios.

We calculate eigenvector overlaps between intersecting time periods of covariance matrices.

In many applications, one has side information, e.g., labels that are provided in a semi-supervised manner, about a specific target region of a large data set, and one wants to perform machine learning and data analysis tasks "nearby" that prespecified target region. For example, one might be interested in the clusteri…

In spectral clustering, one defines a similarity matrix for a collection of data points, transforms the matrix to get the Laplacian matrix, finds the eigenvectors of the Laplacian matrix, and obtains a partition of the data using the leading eigenvectors. The last step is sometimes referred to as rounding, where one ne…

We study the problem asking if one can embed manifolds into finite dimensional Euclidean spaces by taking finite number of eigenvector fields of the connection Laplacian. This problem is essential for the dimension reduction problem in massive data analysis. Singer-Wu proposed the vector diffusion map which embeds mani…

New metric tensor field on symmetric matrices simplifies eigenvector computation.

For Machine Learning (ML) classification problem, where a vector of $\mathbf{x}$--observations (values of attributes) is mapped to a single $y$ value (class label), a generalized Radon--Nikodym type of solution is proposed. Quantum--mechanics --like probability states $ψ^2(\mathbf{x})$ are considered and "Cluster Cente…

Spectral clustering performance depends on eigenvector fluctuations, shown to be Gaussian.

New neural architectures invariant to sign flips and basis symmetries for graph representation learning.

New algorithm updates eigenvectors of evolving graphs efficiently.

How many samples are sufficient to guarantee that the eigenvectors and eigenvalues of the sample covariance matrix are close to those of the actual covariance matrix? For a wide family of distributions, including distributions with finite second moment and distributions supported in a centered Euclidean ball, we prove …

Extended study improves covariance matrix estimation for portfolio managers.

New algorithm consistently orients eigenvectors for machine learning.

Study of eigenvalues in nonlinear kernels for classification of separable data.

Automates PDE model reduction with time-scale separation.

SpecNet2 improves spectral embedding without orthogonalization, achieving better performance and efficiency.

New theory for eigenvectors of generalized Laplacian matrices, addressing dependency issues.

Fast algorithm recovers principal eigenvector from noisy matrices.

New method scales features for better clustering.

This paper develops the exact linear relationship between the leading eigenvector of the unnormalized modularity matrix and the eigenvectors of the adjacency matrix. We propose a method for approximating the leading eigenvector of the modularity matrix, and we derive the error of the approximation. There is also a comp…

New method improves subspace iteration for eigenvectors in machine learning.

The paper explores how kernel eigenalignments affect generalization in KRR.

The problem of estimating sparse eigenvectors of a symmetric matrix attracts a lot of attention in many applications, especially those with high dimensional data set. While classical eigenvectors can be obtained as the solution of a maximization problem, existing approaches formulate this problem by adding a penalty te…

Improved spectral clustering with fewer eigenvectors performs better.

Paper tackles small eigen-gap estimation and inference for noisy symmetric matrices.

New insights into spectral clustering reveal strong connections within eigenvectors.

The original contributions of this paper are twofold: a new understanding of the influence of noise on the eigenvectors of the graph Laplacian of a set of image patches, and an algorithm to estimate a denoised set of patches from a noisy image. The algorithm relies on the following two observations: (1) the low-index e…

We characterize the contractions that are similar to the backward shift in the Hardy space $H^2$. This characterization is given in terms of the geometry of the eigenvector bundles of the operators.

Graph convolutional networks fail to use eigenvectors beyond the first, unlike spectral embedding.

We provide new examples of diffusion operators in dimension 2 and 3 which have orthogonal polynomials as eigenvectors. Their construction rely on the finite subgroups of O(3) and their invariant polynomials.

Practically, we are often in the dilemma that the labeled data at hand are inadequate to train a reliable classifier, and more seriously, some of these labeled data may be mistakenly labeled due to the various human factors. Therefore, this paper proposes a novel semi-supervised learning paradigm that can handle both l…

Clustering is the problem of separating a set of objects into groups (called clusters) so that objects within the same cluster are more similar to each other than to those in different clusters. Spectral clustering is a now well-known method for clustering which utilizes the spectrum of the data similarity matrix to pe…

Study eigenvalues and eigenvectors in neural networks, focusing on signal propagation.

The paper tackles learning symmetries in data without expert knowledge.

In this paper, we study the spectrum and the eigenvectors of radial kernels for mixtures of distributions in $\mathbb{R}^n$. Our approach focuses on high dimensions and relies solely on the concentration properties of the components in the mixture. We give several results describing of the structure of kernel matrices …