arXiv research

One technique to visualize the training of neural networks is to perform PCA on the parameters over the course of training and to project to the subspace spanned by the first few PCA components. In this paper we compare this technique to the PCA of a high dimensional random walk. We compute the eigenvalues and eigenvec…

A novel unsupervised outlier detection method using Randomized PCA Forest.

A new low-dimensional parameterization based on principal component analysis (PCA) and convolutional neural networks (CNN) is developed to represent complex geological models. The CNN-PCA method is inspired by recent developments in computer vision using deep learning. CNN-PCA can be viewed as a generalization of an ex…

Unified methodology for statistical inference in least squares and PCA via randomized sketching.

A method to remove mean-shift noise from PCA using knockoffs.

We analyzed SVD and variants for eigenpair computation, comparing their time and space complexities.

A central problem of random matrix theory is to understand the eigenvalues of spiked random matrix models, introduced by Johnstone, in which a prominent eigenvector (or "spike") is planted into a random matrix. These distributions form natural statistical models for principal component analysis (PCA) problems throughou…

The paper analyzes sparse PCA for incomplete data and proves support recovery conditions.

Sparse PCA selects variables with FDR control for improved performance.

PCA-based MTL improves performance with reduced computation.

We present and analyze a simple, two-step algorithm to approximate the optimal solution of the sparse PCA problem. Our approach first solves a L1 penalized version of the NP-hard sparse PCA optimization problem and then uses a randomized rounding strategy to sparsify the resulting dense solution. Our main theoretical r…

In this paper, we present a novel approach for initializing deep neural networks, i.e., by turning PCA into neural layers. Usually, the initialization of the weights of a deep neural network is done in one of the three following ways: 1) with random values, 2) layer-wise, usually as Deep Belief Network or as auto-encod…

Efficient CF approach using fast adaptive PCA for recommender systems.

A central problem of random matrix theory is to understand the eigenvalues of spiked random matrix models, in which a prominent eigenvector is planted into a random matrix. These distributions form natural statistical models for principal component analysis (PCA) problems throughout the sciences. Baik, Ben Arous and Pé…

Classical methods such as Principal Component Analysis (PCA) and Canonical Correlation Analysis (CCA) are ubiquitous in statistics. However, these techniques are only able to reveal linear relationships in data. Although nonlinear variants of PCA and CCA have been proposed, these are computationally prohibitive in the …

New local-search methods close the gap in sparse tensor PCA.

Bucketed PCA-NN outperforms DNNs by 96% on MNIST.

EB-PCA reduces noise in high-dimensional PCA by estimating a joint prior distribution.

A new dynamical formulation of log-PCA captures local principal modes of geodesic variations.

We solve Bayesian PCA's rotational symmetry issue by rotation-invariant parameterization.

PCA-Guided Quantile Sampling preserves data structure in large datasets.

Improved SVD for shifted matrices without explicit matrix construction.

Algorithm estimates top k eigenvectors of shared covariance matrices while preserving privacy.

Improved PCA accuracy using maximum entropy method.

Principal component analysis (PCA) is widely used for dimension reduction and embedding of real data in social network analysis, information retrieval, and natural language processing, etc. In this work we propose a fast randomized PCA algorithm for processing large sparse data. The algorithm has similar accuracy to th…

New method estimates Gaussian vector functions more efficiently.

Novel PCA method for high-dimensional inverse problems.

Algorithm recovers sparse PCA support from incomplete data.

Paper improves robust PCA for noisy, outlier, and missing data.

Kernel PCA helps analyze multivariate extremes and clusters them effectively.

A new PCA method for analyzing point processes.

PCA outperforms random projections in retaining second order signals from latent groups.

The CUR decomposition provides an approximation of a matrix $X$ that has low reconstruction error and that is sparse in the sense that the resulting approximation lies in the span of only a few columns of $X$. In this regard, it appears to be similar to many sparse PCA methods. However, CUR takes a randomized algorithm…

A new single-pass algorithm improves sparse PCA under limited computational resources.

DP-PCA improves privacy in PCA computations with optimal statistical error.

Paper develops methods for PCA inference with missing data and heteroskedastic noise.

Sequential or online dimensional reduction is of interests due to the explosion of streaming data based applications and the requirement of adaptive statistical modeling, in many emerging fields, such as the modeling of energy end-use profile. Principal Component Analysis (PCA), is the classical way of dimensional redu…

Study shows resampling can drastically alter PCA results.

We study the convergence properties of the VR-PCA algorithm introduced by \cite{shamir2015stochastic} for fast computation of leading singular vectors. We prove several new results, including a formal analysis of a block version of the algorithm, and convergence from random initialization. We also make a few observatio…

We study the statistical and computational aspects of kernel principal component analysis using random Fourier features and show that under mild assumptions, $O(\sqrt{n} \log n)$ features suffices to achieve $O(1/ε^2)$ sample complexity. Furthermore, we give a memory efficient streaming algorithm based on classical Oja…

We consider principal component analysis for contaminated data-set in the high dimensional regime, where the dimensionality of each observation is comparable or even more than the number of observations. We propose a deterministic high-dimensional robust PCA algorithm which inherits all theoretical properties of its ra…

Principal component analysis (PCA) is a widely used technique for data analysis and dimension reduction with numerous applications in science and engineering. However, the standard PCA suffers from the fact that the principal components (PCs) are usually linear combinations of all the original variables, and it is thus…

New Gaussian DPP model reveals directionality in data.

Motivated by the Bagging Partial Least Squares (PLS) and Principal Component Analysis (PCA) algorithms, we propose a Principal Model Analysis (PMA) method in this paper. In the proposed PMA algorithm, the PCA and the PLS are combined. In the method, multiple PLS models are trained on sub-training sets, derived from the…

Paper presents a randomized algorithm for SPCA with high probability approximation.

Efficiently estimates covariance for sparse functional data.

Federated PCA algorithm uses limited memory and privacy-preserving techniques.

Consider a two-class clustering problem where we observe $X_i = \ell_i μ+ Z_i$, $Z_i \stackrel{iid}{\sim} N(0, I_p)$, $1 \leq i \leq n$. The feature vector $μ\in R^p$ is unknown but is presumably sparse. The class labels $\ell_i\in\{-1, 1\}$ are also unknown and the main interest is to estimate them. We are interested …