arXiv research

Study of Langevin dynamics for tensor PCA recovery in high dimensions.

SGD recovers multiple signal vectors in noisy tensor PCA.

A central problem of random matrix theory is to understand the eigenvalues of spiked random matrix models, introduced by Johnstone, in which a prominent eigenvector (or "spike") is planted into a random matrix. These distributions form natural statistical models for principal component analysis (PCA) problems throughou…

SMPI recovers tensor spikes from noisy data with improved performance.

New algorithms recover sparse tensor principal components efficiently.

Study efficient power iteration for tensor models, proving convergence under specific conditions.

New combinatorial method for sparse PCA works beyond spiked identity model.

A central problem of random matrix theory is to understand the eigenvalues of spiked random matrix models, in which a prominent eigenvector is planted into a random matrix. These distributions form natural statistical models for principal component analysis (PCA) problems throughout the sciences. Baik, Ben Arous and Pé…

New algorithms improve tensor CP decomposition under mild conditions.

New insights into SGD and SGD-M in high dimensions.

Developing electrophysiological recordings of brain neuronal activity and their analysis provide a basis for exploring the structure of brain function and nervous system investigation. The recorded signals are typically a combination of spikes and noise. High amounts of background noise and possibility of electric sign…

Non-asymptotic tail bounds for Kostlan-Shub-Smale field on sphere

Develops path integral for spiked tensor model dynamics.

Attention learns PCA on Gaussian data, proving its connection to principal component analysis.

Study recovers spike order in noisy tensor estimation without SNR assumptions.

The paper analyzes deflation for estimating a low-rank spike in large tensors with noise.

Paper develops methods for PCA inference with missing data and heteroskedastic noise.

With the development of high-throughput technologies, principal component analysis (PCA) in the high-dimensional regime is of great interest. Most of the existing theoretical and methodological results for high-dimensional PCA are based on the spiked population model in which all the population eigenvalues are equal ex…

Study reveals efficient recovery of multi-modal signals via Bayesian methods and sequential learning.

We consider streaming, one-pass principal component analysis (PCA), in the high-dimensional regime, with limited memory. Here, $p$-dimensional samples are presented sequentially, and the goal is to produce the $k$-dimensional subspace that best approximates these points. Standard algorithms require $O(p^2)$ memory; mea…

Paper optimizes private PCA for covariance estimation in statistics.

QAOA matches classical tensor power iteration in spiked tensor model recovery.

Paper studies tensor models using random matrix theory.

Paper studies statistical-computational trade-offs in tensor PCA and related problems.

Principal component analysis (PCA) is an unsupervised method for learning low-dimensional features with orthogonal projections. Multilinear PCA methods extend PCA to deal with multidimensional data (tensors) directly via tensor-to-tensor projection or tensor-to-vector projection (TVP). However, under the TVP setting, i…

Optimal spectral method found for inhomogeneous spiked Wigner model.

Simple AMP algorithm robust to adversarial corruption.

New framework predicts AMP behavior in spiked models for finite iterations.

Paper optimizes tensor deflation for non-orthogonal signals.

Study analyzes Hotelling-type tensor deflation for spiked tensors, providing insights into signal and noise.

Efficient algorithm for tensor PCA with improved time complexity.

Bayes-optimal limits in PCA with structured noise are determined.

In this paper, we propose an acceleration scheme for online memory-limited PCA methods. Our scheme converges to the first $k>1$ eigenvectors in a single data pass. We provide empirical convergence results of our scheme based on the spiked covariance model. Our scheme does not require any predefined parameters such as t…

We consider the Principal Component Analysis problem for large tensors of arbitrary order $k$ under a single-spike (or rank-one plus noise) model. On the one hand, we use information theory, and recent results in probability theory, to establish necessary and sufficient conditions under which the principal component ca…

EB-PCA reduces noise in high-dimensional PCA by estimating a joint prior distribution.

A method to remove mean-shift noise from PCA using knockoffs.

Estimating the leading principal components of data, assuming they are sparse, is a central task in modern high-dimensional statistics. Many algorithms were developed for this sparse PCA problem, from simple diagonal thresholding to sophisticated semidefinite programming (SDP) methods. A key theoretical question is und…

Face recognition is the important field in machine learning and pattern recognition research area. It has a lot of applications in military, finance, public security, to name a few. In this paper, the combination of the tensor sparse PCA with the nearest-neighbor method (and with the kernel ridge regression method) wil…

PCA++ improves robustness to background noise in contrastive learning.

A general framework for principal component analysis (PCA) in the presence of heteroskedastic noise is introduced. We propose an algorithm called HeteroPCA, which involves iteratively imputing the diagonal entries of the sample covariance matrix to remove estimation bias due to heteroskedasticity. This procedure is com…

Paper proposes a new method for PCA using generative models.

Tensor PCA problem analyzed with statistical query lower bounds.

Study detects signals in spiked Wigner models using log likelihood ratio.

Sharp analysis of power iteration for tensor PCA, improving convergence and stopping criteria.

Paper develops inference methods for low-rank tensors without debiasing.

Study of asymmetric rank-one tensor models with non-Gaussian noise.

In the past decade, sparse principal component analysis has emerged as an archetypal problem for illustrating statistical-computational tradeoffs. This trend has largely been driven by a line of research aiming to characterize the average-case complexity of sparse PCA through reductions from the planted clique (PC) con…

Fast algorithm recovers principal eigenvector from noisy matrices.