arXiv research

Self-attention networks localize when eigenspectrum variance is small.

We present an original and novel method based on random matrix approach that enables to distinguish the respective role of temporal autocorrelations inside given time series and cross correlations between various time series. The proposed algorithm is based on properties of Wigner eigenspectrum of random matrices inste…

ImageNet pre-training has been regarded as essential for training accurate object detectors for a long time. Recently, it has been shown that object detectors trained from randomly initialized weights can be on par with those fine-tuned from ImageNet pre-trained models. However, the effects of pre-training and the diff…

This article investigates the eigenspectrum of the inner product-type kernel matrix $\sqrt{p} \mathbf{K}=\{f( \mathbf{x}_i^{\sf T} \mathbf{x}_j/\sqrt{p})\}_{i,j=1}^n$ under a binary mixture model in the high dimensional regime where the number of data $n$ and their dimension $p$ are both large and comparable. Based on…

New method preserves spectral clustering performance under aggressive sparsification and quantization.

Lower bounds on eigenspectrum show rich action spaces force polynomial regret in linear bandits.

Analyzes Hessian spectrum for neural networks near optimal learning.

A new mutual information optimization method using self-supervised binary contrastive learning.

SGD in linear regression overfits but performs well due to bias-variance trade-off.

The matrix-based Renyi's α-entropy functional and its multivariate extension were recently developed in terms of the normalized eigenspectrum of a Hermitian matrix of the projected data in a reproducing kernel Hilbert space (RKHS). However, the utility and possible applications of these new estimators are rather new an…

Linear dynamical systems are a fundamental and powerful parametric model class. However, identifying the parameters of a linear dynamical system is a venerable task, permitting provably efficient solutions only in special cases. This work shows that the eigenspectrum of unknown linear dynamics can be identified without…

New bounds for KRR condition number reveal overfitting phenomena.

This paper proposes a new Nystrom-based clustering algorithm for large-scale data.

Improves regression models' performance on covariate shift.

The matrix-based Renyi's α-order entropy functional was recently introduced using the normalized eigenspectrum of a Hermitian matrix of the projected data in a reproducing kernel Hilbert space (RKHS). However, the current theory in the matrix-based Renyi's α-order entropy functional only defines the entropy of a single…

The von Neumann graph entropy (VNGE) facilitates measurement of information divergence and distance between graphs in a graph sequence. It has been successfully applied to various learning tasks driven by network-based data. While effective, VNGE is computationally demanding as it requires the full eigenspectrum of the…

The smallest eigenvalues and the associated eigenvectors (i.e., eigenpairs) of a graph Laplacian matrix have been widely used for spectral clustering and community detection. However, in real-life applications the number of clusters or communities (say, $K$) is generally unknown a-priori. Consequently, the majority of …

The smallest eigenvalues and the associated eigenvectors (i.e., eigenpairs) of a graph Laplacian matrix have been widely used in spectral clustering and community detection. However, in real-life applications the number of clusters or communities (say, $K$) is generally unknown a-priori. Consequently, the majority of t…

Data structure affects deep learning performance, study finds.

Financial markets are highly correlated systems that reveal both the inter-market dependencies and the correlations among their different components. Standard analyzing techniques include correlation coefficients for pairs of signals and correlation matrices for rich multivariate data. In the latter case one constructs…

Characterizes Hessian eigenspectra for realistic nonlinear models.

The paper analyzes optimal implicit bias in linear regression for over-parameterized models.

We present in this work a new family of kernels to compare positive measures on arbitrary spaces $\Xcal$ endowed with a positive kernel $κ$, which translates naturally into kernels between histograms or clouds of points. We first cover the case where $\Xcal$ is Euclidian, and focus on kernels which take into account th…

A-DOGE embeds attributed graphs efficiently using density of states.

This paper speeds up spectral clustering for large graphs by dilating their eigenspectrum.

SAT improves adversarial training by smoothing the loss landscape through curriculum learning.

RNNs struggle with chaotic dynamics due to exploding gradients, but we found a way to optimize training.

This paper optimizes matrix-based Renyi's entropy computation for large datasets.

New study finds many neural networks are not benignly overfitting.

Efficient approximations reduce computation of matrix-based Renyi's entropy.

Spectro-Riemannian Graph Neural Networks integrate spectral and curvature signals for better graph representation learning.

Geometric stability measures neural network robustness, distinguishing from similarity metrics.

The paper analyzes diffusion condensation for data geometry and topology.

New method estimates large matrices' spectra from small sub-matrices.

The paper investigates the convergence of Vendi scores under finite samples and introduces a truncated version for better performance.

TRF uses ternary random features to improve ML performance without extra computation.