Proposes SPCR-glm for generalized linear models combining PCA and regression losses.
problem Lack of response variable information in traditional PCA.
method Combines PCA and regression losses with a sparse penalty for parameter estimation.
result Improves interpretability and classification of principal components.
Principal component regression (PCR) is a two-stage procedure that selects some principal components and then constructs a regression model regarding them as new explanatory variables. Note that the principal components are obtained from only explanatory variables and not considered with the response variable. To addre…
A new PCR method using SVD with sparse regularization.
problem Lack of response variable information in traditional PCR.
method One-stage SVD approach with two loss functions and sparse regularization.
result Obtains principal component loadings with response variable information.
A new method for sparse regression using principal components.
problem Wide data with many features and few observations.
method Combines lasso (ℓ1) sparsity with quadratic penalty towards principal components. result Powerful feature selection and group-wise shrinkage.
Efficient private matrix analysis algorithms for recent variants.
problem Private analysis of recent matrix updates.
method Identifying sufficient conditions on positive semidefinite matrices.
result First efficient differentially private algorithms for various matrix analysis tasks.
New method connects Sparse PCA and Sparse Linear Regression.
problem Sparse Principal Component Analysis and Sparse Linear Regression.
method Transforming a solver for Sparse Linear Regression into an algorithm for Sparse Principal Component Analysis.
result The derived SPCA algorithm achieves near state-of-the-art guarantees for testing and support recovery.
We propose a penalized orthogonal-components regression (POCRE) for large p small n data. Orthogonal components are sequentially constructed to maximize, upon standardization, their correlation to the response residuals. A new penalization framework, implemented via empirical Bayes thresholding, is presented to effecti…
VC-PCR improves prediction by clustering correlated variables.
problem Decreased prediction accuracy due to cluster structure in predictor variables.
method Supervised variable selection and clustering to integrate cluster information into a sparse modeling process.
result VC-PCR achieves better prediction, variable selection, and clustering performance.
The paper introduces a method for interpretable principal component analysis of high-dimensional time series.
problem Inconsistent and difficult-to-interpret principal component estimates in high-dimensional regimes.
method Localized sparse principal component analysis of spectral density matrices in frequency domain.
result Efficient algorithm for sparse-localized estimates of principal subspaces.
We study sparse principal component analysis for high dimensional vector autoregressive time series under a doubly asymptotic framework, which allows the dimension d to scale with the series length T. We treat the transition matrix of time series as a nuisance parameter and directly apply sparse principal component…
The computation of the sparse principal component of a matrix is equivalent to the identification of its principal submatrix with the largest maximum eigenvalue. Finding this optimal submatrix is what renders the problem NP-hard. In this work, we prove that, if the matrix is positive semidefinite and its …
A new method for sparse PCA using orthogonal rotations and soft-thresholding.
problem Sparse PCA with a new basis using orthogonal rotations.
method Initialize with leading principal components, apply kimesk orthogonal rotation, and soft-threshold the rotated components. result The proposed method is more stable and explains more variance compared to alternatives.
We study sparse principal components analysis in high dimensions, where p (the number of variables) can be much larger than n (the number of observations), and analyze the problem of estimating the subspace spanned by the principal eigenvectors of the population covariance matrix. We introduce two complementary not…
Efficiently projects vectors onto top PCA components without explicit PCA.
problem Efficiently project vectors onto top principal components of a matrix.
method Iterative algorithm using ridge regression and polynomial approximation.
result First runtime improvement for principal component regression.
msPCA solves sparse PCA for multiple components efficiently.
problem Sparse principal component analysis with multiple components.
method Alternating maximization algorithm for sparse loading vectors, with orthogonality or zero correlation constraints.
result Achieves high variance explained with sparse components and controlled feasibility violations.
PCHAL and PCHAR use principal components to speed up HAL and HAR methods.
problem Computational infeasibility in high dimensions for HAL and HAR.
method Outcome-blind principal component reduction of HAL basis.
result Empirical performance comparable to HAL and HAR, with computational gains.
Improved estimation of multiple principal components using manifold optimization and iterative deflation techniques.
problem Estimating multiple principal components efficiently and orthogonally.
method Extended SFPCA using manifold optimization and iterative deflation techniques.
result Alternative deflation schemes improve signal extraction and component estimation.
AdvPCA uses robust optimization to achieve sparse PCA without tuning.
problem Sparse PCA for high-dimensional data with implicit sparsity.
method Adversarial PCA (AdvPCA) using robust optimization.
result AdvPCA achieves effective sparse PCA with a closed-form solution.
R package spca computes sparse principal components efficiently.
problem Sparse principal components analysis (SPCA) for interpretable data.
method Least squares sparse principal component analysis (LS-SPCA) with efficient C++ backend.
result Computes sparse principal components that maximize variance and maintain strong correlations with PCs.
SP-SPCA improves sparse PCA by adaptively adjusting variable penalties, enhancing interpretability and stability.
problem Poor interpretability and variable redundancy in PCA for high-dimensional data.
method Introduces a single equilibrium parameter to adaptively adjust variable penalties in the L2 regularization framework.
result Consistently outperforms standard sparse PCA methods in identifying sparse loading patterns and preserving cumulative variance.
CPCR mitigates bias in PCR for overparameterized models.
problem Bias in Principal Component Regression (PCR) for overparameterized models.
method Calibrated Principal Component Regression (CPCR) learns a low-variance prior in the PC subspace and calibrates the model in the original feature space.
result CPCR outperforms standard PCR in overparameterized settings, improving prediction across multiple problems.
We perform a finite sample analysis of the detection levels for sparse principal components of a high-dimensional covariance matrix. Our minimax optimal test is based on a sparse eigenvalue statistic. Alas, computing this test is known to be NP-complete in general, and we describe a computationally efficient alternativ…
Improved data analysis with robust SPCA algorithm.
problem Identifying localized spatial structures and disambiguating time scales in low-rank data.
method Formulated as a value-function optimization problem, then extended with randomized linear algebra methods for scalability.
result Robust and efficient sparse principal components in corrupted data.
Paper presents a faster classical algorithm for principal component regression.
problem Efficiently solving principal component regression problems.
method Uses quantum-inspired linear algebra techniques.
result Achieves polylogarithmic runtime, significantly faster than state-of-the-art.
This paper compares and analyzes random projections and column sub-sampling for dimension reduction in regression.
problem Computational efficiency in dimension reduction for large datasets.
method Analysis of random projections and column sub-sampling methods for regression.
result Random projections and column sub-sampling can achieve similar prediction error to Principal Components Regression (PCR) but with less computational cost.
Solves PCR with fewer calls to ridge regression.
problem Principal component regression (PCR) with high accuracy.
method Reduces PCR to ridge regression calls and develops stable recurrence for matrix Chebyshev polynomials.
result Achieves PCR with multiplicative accuracy up to 1+γ using fewer calls. XCAN uses cross-product penalization for sparse matrix factorization.
problem Understanding complex data structures.
method Sparse matrix factorization with a loss function balancing variance and structural preservation.
result Flexible modeling approach for diverse applications.
New method for sparse PCA using random projections, non-iterative and fast.
problem Sparse principal component analysis (PCA)
method Axis-aligned random projections of sample covariance matrix
result Non-iterative method achieves optimal convergence rate in polynomial time
The paper improves RPCA for separating sparse and manifold components on noisy data.
problem Separating sparse and manifold components from noisy data.
method Nonlinear Robust Principal Component Analysis (RPCA) framework.
result The method successfully separates sparse and manifold components under noisy data.
Paper proposes Sp-GD for sparse max-affine regression with theoretical guarantees.
problem Sparse max-affine regression model selection and estimation.
method Sparse Gradient Descent (Sp-GD) initialization using sparse PCA and covering search.
result Sp-GD provides ε-accurate estimates with optimal number of observations.
Survey of SDR methods for high-dimensional regression and embedding.
problem Reducing dimensionality in high-dimensional data.
method Involves both statistical and machine learning approaches, covering inverse and forward regression methods.
result Supervised Kernel Dimension Reduction is equivalent to supervised PCA.
Unified study of principal component analysis under various structured signal models.
problem Principal component analysis with structured signals.
method Unified analysis using the spiked Wishart model and projected power method.
result Established fundamental limits and demonstrated local convergence for structured signal models.
The paper analyzes the effectiveness of principal component regression with varying numbers of features.
problem The study examines the performance of principal component regression with different numbers of selected features.
method The analysis considers the least squares linear regression over uncorrelated Gaussian features selected in decreasing variance order. It also analyzes the prediction error in the average-case setting as the number of features and samples grow.
result The prediction error shows a 'double descent' shape as the number of features increases, and conditions are established for achieving minimum risk in the interpolating regime.
Regularized variants of Principal Components Analysis, especially Sparse PCA and Functional PCA, are among the most useful tools for the analysis of complex high-dimensional data. Many examples of massive data, have both sparse and functional (smooth) aspects and may benefit from a regularization scheme that can captur…
SOFAR learns large-scale association networks efficiently.
problem Efficiently understanding large-scale response-predictor association networks.
method Sparse Orthogonal Factor Regression (SOFAR) via sparse singular value decomposition with orthogonality constraints.
result SOFAR achieves statistical efficiency and scientific insights.
Spofe bridges statistical rigor and interpretability in feature extraction from tabular data.
problem Ensuring statistical rigor and interpretability in feature extraction from complex tabular data.
method Spofe combines kernel principal components and sparse polynomial functions with a multi-objective knockoff selection procedure.
result Spofe consistently outperforms other methods in feature selection for regression and classification tasks.
We propose a new two stage algorithm LING for large scale regression problems. LING has the same risk as the well known Ridge Regression under the fixed design setting and can be computed much faster. Our experiments have shown that LING performs well in terms of both prediction accuracy and computational efficiency co…
Study on reducing dimensionality in high-dimensional regression with kernel methods and stability analysis.
problem Analyzing errors in high-dimensional regression with dimensionality reduction and kernel regression.
method Derive a stability result for kernel regression with Wasserstein distance and apply it to PCA to deduce convergence rates.
result Two-step procedure yields useful convergence rates in semi-supervised settings.
This paper provides theoretical guarantees for SPCA using the Elastic Net.
problem Lack of theoretical guarantees for the SPCA algorithm.
method Revisited and improved the SPCA algorithm of Zou et al. (2006) using the Elastic Net.
result Both algorithms can recover the principal subspace consistently under mild conditions.
Paper proposes a new method for exact recovery in robust tensor principal component analysis.
problem Exact recovery of low-rank and sparse components in tensors.
method Proposes a new method based on tensor-tensor product and t-SVD to solve a convex optimization problem.
result Exact recovery achieved in a deterministic fashion without randomness assumptions.
Federated learning improves SPCA for sparse components.
problem Data privacy and sharing constraints in machine learning.
method Federated learning framework applied to SPCA with L1 regularization and smoothing.
result Federated SPCA achieves sparse component loadings with improved interpretability.
sPCA models may not have orthogonal scores and loadings, complicating interpretation.
problem sPCA scores and loadings may not be orthogonal.
method Illustrated and numerically demonstrated the implications of sPCA on scores, residuals, and variance explained.
result sPCA approaches perform poorly on noise-free, sparse data.
Advances robust principal component analysis with transformed ℓ1 regularization.
problem Recovering low-rank structures from noisy, partially observed data corrupted by sparse outliers.
method Proposes transformed ℓ1 (TL1) regularization to improve approximations of rank and ℓ0 functional.
result Achieves higher accuracy in estimating low-rank and sparse components compared to classical convex models, especially under non-uniform sampling schemes.
RKPCA improves robustness of PCA for high-rank matrices.
problem Robust recovery of high-rank matrices corrupted by sparse noises.
method RKPCA decomposes matrices into sparse and low-rank components.
result RKPCA provides high recovery accuracy with theoretical guarantees.
A fast method for sparse PCA reduces computation time.
problem Time-consuming implementation of SPCA on high-dimensional data.
method Subspace projections using Household QR factorization for efficient deflation.
result Developed SPCA-SP method maintains good tradeoffs between various criteria.
ReFACTor improves low-rank matrix recovery from noisy data.
problem Recovering low-rank matrices from noisy column-sparse data.
method A simple variation of TSVD, leveraging column-sparsity.
result ReFACTor outperforms TSVD and PCA in various scenarios.
Principal component analysis (PCA) is a statistical technique commonly used in multivariate data analysis. However, PCA can be difficult to interpret and explain since the principal components (PCs) are linear combinations of the original variables. Sparse PCA (SPCA) aims to balance statistical fidelity and interpretab…
New FGSPCA method captures grouping and sparse structures in PCA without prior info.
problem Capture grouping and sparse structures in PCA without prior info.
method Truncated regularization with alternating algorithm.
result FGSPCA method reduces model complexity and increases interpretability.