arXiv research

In this paper, we present a novel approach for initializing deep neural networks, i.e., by turning PCA into neural layers. Usually, the initialization of the weights of a deep neural network is done in one of the three following ways: 1) with random values, 2) layer-wise, usually as Deep Belief Network or as auto-encod…

Combines PCA and AMP for better signal estimation in noisy data.

pPCA speeds up PCA by priming initial estimates for faster, more accurate results.

Time delay estimation (TDE) is a critical and challenging step in all ultrasound elastography methods. A growing number of TDE techniques require an approximate but robust and fast method to initialize solving for TDE. Herein, we present a fast method for calculating an approximate TDE between two radio frequency (RF) …

We study the convergence properties of the VR-PCA algorithm introduced by \cite{shamir2015stochastic} for fast computation of leading singular vectors. We prove several new results, including a formal analysis of a block version of the algorithm, and convergence from random initialization. We also make a few observatio…

Sharp analysis of power iteration for tensor PCA, improving convergence and stopping criteria.

Sparse principal component analysis (PCA) involves nonconvex optimization for which the global solution is hard to obtain. To address this issue, one popular approach is convex relaxation. However, such an approach may produce suboptimal estimators due to the relaxation effect. To optimally estimate sparse principal su…

A new single-pass algorithm improves sparse PCA under limited computational resources.

GCN and GPCA are mathematically connected, leading to improved node classification performance.

Robust PCA is a widely used statistical procedure to recover a underlying low-rank matrix with grossly corrupted observations. This work considers the problem of robust PCA as a nonconvex optimization problem on the manifold of low-rank matrices, and proposes two algorithms (for two versions of retractions) based on ma…

A new robust PCA method uses Innovation Search and Leverage Scores.

Lower bound for VAE training objective for binary data.

A new method for sparse PCA using orthogonal rotations and soft-thresholding.

Paper proposes GPM for heteroscedastic PCA estimation.

SGD recovers multiple signal vectors in noisy tensor PCA.

Optimization problems with rank constraints arise in many applications, including matrix regression, structured PCA, matrix completion and matrix decomposition problems. An attractive heuristic for solving such problems is to factorize the low-rank matrix, and to run projected gradient descent on the nonconvex factoriz…

We study a cutting-plane method for semidefinite optimization problems (SDOs), and supply a proof of the method's convergence, under a boundedness assumption. By relating the method's rate of convergence to an initial outer approximation's diameter, we argue that the method performs well when initialized with a second-…

Developing efficient and guaranteed nonconvex algorithms has been an important challenge in modern machine learning. Algorithms with good empirical performance such as stochastic gradient descent often lack theoretical guarantees. In this paper, we analyze the class of homotopy or continuation methods for global optimi…

Dynamic robust PCA refers to the dynamic (time-varying) extension of robust PCA (RPCA). It assumes that the true (uncorrupted) data lies in a low-dimensional subspace that can change with time, albeit slowly. The goal is to track this changing subspace over time in the presence of sparse outliers. We develop and study …

New framework predicts AMP behavior in spiked models for finite iterations.

This work provides improved guarantees for streaming principle component analysis (PCA). Given $A_1, \ldots, A_n\in \mathbb{R}^{d\times d}$ sampled independently from distributions satisfying $\mathbb{E}[A_i] = Σ$ for $Σ\succeq \mathbf{0}$, this work provides an $O(d)$-space linear-time single-pass streaming algorithm …

SMPI recovers tensor spikes from noisy data with improved performance.

Simple AMP algorithm robust to adversarial corruption.

New algorithms improve tensor CP decomposition under mild conditions.

Simplifies fair PCA with fast, efficient solution.

In this paper, we consider the sparse eigenvalue problem wherein the goal is to obtain a sparse solution to the generalized eigenvalue problem. We achieve this by constraining the cardinality of the solution to the generalized eigenvalue problem and obtain sparse principal component analysis (PCA), sparse canonical cor…

KAN-PCA improves asset return analysis by capturing more variance than classical PCA during market crises.

New algorithm solves fair PCA, robust PCA, and sparse PCA problems efficiently.

Unified framework improves PCA for outliers and distributed data.

TL-PCA uses transfer learning to improve PCA performance with limited target data.

Bucketed PCA-NN outperforms DNNs by 96% on MNIST.

A new low-dimensional parameterization based on principal component analysis (PCA) and convolutional neural networks (CNN) is developed to represent complex geological models. The CNN-PCA method is inspired by recent developments in computer vision using deep learning. CNN-PCA can be viewed as a generalization of an ex…

Anchor PCA improves robustness in multi-domain PCA.

A new method for fair PCA ensures balanced error across groups.

This is a detailed tutorial paper which explains the Principal Component Analysis (PCA), Supervised PCA (SPCA), kernel PCA, and kernel SPCA. We start with projection, PCA with eigen-decomposition, PCA with one and multiple projection directions, properties of the projection matrix, reconstruction error minimization, an…

Revisits PCA with new formulations and insights.

Proposes $σ$-PCA to learn identifiable linear transformations without whitening.

Solution to sparse PCA tuning problem using Empirical Bayes.

PCA++ improves robustness to background noise in contrastive learning.

EB-PCA reduces noise in high-dimensional PCA by estimating a joint prior distribution.

New analysis improves black-box $k$-PCA algorithms, reducing parameter loss.

Over the past years Robust PCA has been established as a standard tool for reliable low-rank approximation of matrices in the presence of outliers. Recently, the Robust PCA approach via nuclear norm minimization has been extended to matrices with linear structures which appear in applications such as system identificat…

Derives kernel PCA with Nyström method for scalability.

We develop efficient algorithms for robust PCA that handle outliers.

Principal Component Analysis (PCA) has wide applications in machine learning, text mining and computer vision. Classical PCA based on a Gaussian noise model is fragile to noise of large magnitude. Laplace noise assumption based PCA methods cannot deal with dense noise effectively. In this paper, we propose Cauchy Princ…

A new PCA method using T$\ell_1$-norm outperforms existing methods.

Study efficient power iteration for tensor models, proving convergence under specific conditions.

Singular Value Decomposition (and Principal Component Analysis) is one of the most widely used techniques for dimensionality reduction: successful and efficiently computable, it is nevertheless plagued by a well-known, well-documented sensitivity to outliers. Recent work has considered the setting where each point has …