SDSPCA improves PCA for disease diagnosis using sparse components and discriminative information.
problem Class ambiguity and low interpretability in traditional PCA.
method Incorporates discriminative information and sparsity into PCA, focusing on sparse components.
result SDSPCA outperforms other methods in gene selection and tumor classification on multi-view biological data.
SDSPCAAN combines supervised and local data structures for better dimensionality reduction.
problem Preserving both global and local data structures for noisy high-dimensional data.
method Supervised discriminative sparse PCA with adaptive neighbors (SDSPCAAN).
result SDSPCAAN improves classification accuracy on high-dimensional datasets.
In this paper, we consider the sparse eigenvalue problem wherein the goal is to obtain a sparse solution to the generalized eigenvalue problem. We achieve this by constraining the cardinality of the solution to the generalized eigenvalue problem and obtain sparse principal component analysis (PCA), sparse canonical cor…
Solution to sparse PCA tuning problem using Empirical Bayes.
problem Sparse PCA multiple tuning problem (MTP).
method Empirical Bayes covariance decomposition for penalized PCA.
result Empirical Bayes approach efficiently solves MTP in sparse PCA.
A new method generalizing subspace learning for improved classification.
problem Improving classification accuracy using subspace learning methods.
method Roweis Discriminant Analysis (RDA) which generalizes PCA, SPCA, and FDA.
result RDA and kernel RDA improve classification accuracy on benchmark datasets.
Paper proposes tensor sparse PCA for improved face recognition accuracy.
problem Face recognition accuracy improvement using novel methods.
method Combines tensor sparse PCA with nearest-neighbor and kernel ridge regression methods.
result Tensor sparse PCA method yields better accuracy than PCA method alone.
New combinatorial method for sparse PCA works beyond spiked identity model.
problem Sparse PCA under general covariance matrices.
method Combinatorial truncated power method with global convergence guarantee.
result First combinatorial sparse PCA method provably successful for general covariance matrices.
Efficiently approximates Sparse PCA with significant speedups and minor error.
problem Sparse Principal Component Analysis (Sparse PCA) is NP-hard and computationally expensive.
method Approximates the covariance matrix with block-diagonal form, solves sub-problems in each block, and reconstructs the solution.
result Significant computational speedups with minor additive error.
Sparse PCA selects variables with FDR control for improved performance.
problem Sparse PCA selects irrelevant variables when maximizing explained variance.
method Proposes FDR-controlled selection using T-Rex selector.
result Significant performance improvement over traditional sparse PCA.
We give a reduction from {\sc clique} to establish that sparse PCA is NP-hard. The reduction has a gap which we use to exclude an FPTAS for sparse PCA (unless P=NP). Under weaker complexity assumptions, we also exclude polynomial constant-factor approximation algorithms.
New algorithm solves fair PCA, robust PCA, and sparse PCA problems efficiently.
problem Fair Principal Component Analysis (FPCA) to ensure fairness in PCA solutions.
method Iterative MM algorithm with SDP reformulation to quadratic program.
result Algorithm monotonically improves fairness objectives at each iteration.
Sparse principal component analysis (sparse PCA) aims at finding a sparse basis to improve the interpretability over the dense basis of PCA, meanwhile the sparse basis should cover the data subspace as much as possible. In contrast to most of existing work which deal with the problem by adding some sparsity penalties o…
A new method for sparse PCA using orthogonal rotations and soft-thresholding.
problem Sparse PCA with a new basis using orthogonal rotations.
method Initialize with leading principal components, apply kimesk orthogonal rotation, and soft-threshold the rotated components. result The proposed method is more stable and explains more variance compared to alternatives.
Efficient algorithm for sparse PCA reduces data complexity.
problem Sparse PCA for high-dimensional data with non-convex optimization issues.
method Convex FPS formulation, gradient-based optimization, online learning extension.
result Explicit bounds on optimization error and statistical accuracy.
The paper provides entrywise bounds for Sparse PCA, improving upon previous results.
problem Sparse Principal Component Analysis (PCA) recovery error characterization in spectral or Frobenius norms.
method Entrywise ℓ2,∞ bounds for Sparse PCA under general high-dimensional subgaussian design, using sparsistent algorithms. result Improved entrywise bounds for Sparse PCA, finer characterization of estimation error.
msPCA solves sparse PCA for multiple components efficiently.
problem Sparse principal component analysis with multiple components.
method Alternating maximization algorithm for sparse loading vectors, with orthogonality or zero correlation constraints.
result Achieves high variance explained with sparse components and controlled feasibility violations.
The CUR decomposition provides an approximation of a matrix X that has low reconstruction error and that is sparse in the sense that the resulting approximation lies in the span of only a few columns of X. In this regard, it appears to be similar to many sparse PCA methods. However, CUR takes a randomized algorithm…
Explains Fisher and Kernel Fisher Discriminant Analysis with examples and comparisons.
problem Classifying data with different features and dimensions.
method Projection and reconstruction, scatters analysis, PCA comparison, Fisher forest.
result Equivalence of Fisher and Linear Discriminant Analysis, effectiveness of Fisher forest.
New method solves sparse PCA and CCA with guaranteed convergence.
problem Sparse PCA and CCA for large-scale data analysis.
method Alternating manifold proximal gradient method.
result Unified convergence analysis for the proposed method.
New PCA method handles multiple datasets and detects sparse patterns robustly.
problem Handling multi-source data with sparse and outlier-robust PCA.
method Developed a regularization problem with a penalty for structured sparsity and outlier resistance.
result The method detects global and local patterns across multiple data sources robustly.
Algorithm recovers sparse PCA support from incomplete data.
problem Sparse PCA with incomplete and noisy data.
method Semidefinite program (SDP) relaxation of non-convex l1-regularized PCA. result SDP enables exact recovery of true support of sparse leading eigenvector.
Sparse PCA provides a linear combination of small number of features that maximizes variance across data. Although Sparse PCA has apparent advantages compared to PCA, such as better interpretability, it is generally thought to be computationally much more expensive. In this paper, we demonstrate the surprising fact tha…
In this paper, a new method is proposed for sparse PCA based on the recursive divide-and-conquer methodology. The main idea is to separate the original sparse PCA problem into a series of much simpler sub-problems, each having a closed-form solution. By recursively solving these sub-problems in an analytical way, an ef…
Sparse PCA algorithm improves upon existing methods with better guarantees.
problem Recovering sparse vectors from Gaussian samples with adversarial perturbations.
method New algorithm running in polynomial time with improved β threshold.
result Better guarantees than Covariance Thresholding for large t.
The paper analyzes sparse PCA for incomplete data and proves support recovery conditions.
problem Support recovery in sparse PCA with non-random missing data.
method Semidefinite relaxation of the ℓ1-regularized PCA problem. result Support of the sparse leading eigenvector can be recovered with high probability.
New algorithms recover sparse tensor principal components efficiently.
problem Recovering sparse tensor principal components from noisy data.
method Family of algorithms interpolating between polynomial-time and exhaustive search, tailored for sparse and highly sparse regimes.
result Our algorithms recover sparse vectors for signal-to-noise ratios beyond previous limits, with time complexity ildeO(np+t). Sparse versions of principal component analysis (PCA) have imposed themselves as simple, yet powerful ways of selecting relevant features of high-dimensional data in an unsupervised manner. However, when several sparse principal components are computed, the interpretation of the selected variables is difficult since ea…
Paper proposes estimators for sparse PCA with oracle property.
problem Estimating sparse principal subspace in high-dimensional settings.
method Semidefinite relaxation with novel regularizations.
result One estimator achieves exact support recovery and statistical rate.
The presence of a sparse "truth" has been a constant assumption in the theoretical analysis of sparse PCA and is often implicit in its methodological development. This naturally raises questions about the properties of sparse PCA methods and how they depend on the assumption of sparsity. Under what conditions can the r…
Principal Component Analysis (PCA) is a dimension reduction technique. It produces inconsistent estimators when the dimensionality is moderate to high, which is often the problem in modern large-scale applications where algorithm scalability and model interpretability are difficult to achieve, not to mention the preval…
It is well known that Sparse PCA (Sparse Principal Component Analysis) is NP-hard to solve exactly on worst-case instances. What is the complexity of solving Sparse PCA approximately? Our contributions include: 1) a simple and efficient algorithm that achieves an n−1/3-approximation; 2) NP-hardness of approximatio…
Reduces average-case complexity of sparse PCA from weak PC conjectures.
problem Characterizing the average-case complexity of sparse PCA.
method Reduction from planted clique conjecture to spiked covariance model.
result First full characterization of computational barrier in spiked covariance model, providing tight lower bounds at all sparsities.
Regularized variants of Principal Components Analysis, especially Sparse PCA and Functional PCA, are among the most useful tools for the analysis of complex high-dimensional data. Many examples of massive data, have both sparse and functional (smooth) aspects and may benefit from a regularization scheme that can captur…
Principal component analysis (PCA) is a widely used technique for data analysis and dimension reduction with numerous applications in science and engineering. However, the standard PCA suffers from the fact that the principal components (PCs) are usually linear combinations of all the original variables, and it is thus…
We present and analyze a simple, two-step algorithm to approximate the optimal solution of the sparse PCA problem. Our approach first solves a L1 penalized version of the NP-hard sparse PCA optimization problem and then uses a randomized rounding strategy to sparsify the resulting dense solution. Our main theoretical r…
Principal components analysis (PCA) is the optimal linear auto-encoder of data, and it is often used to construct features. Enforcing sparsity on the principal components can promote better generalization, while improving the interpretability of the features. We study the problem of constructing optimal sparse linear a…
Unified framework for structured principal subspace estimation with bounds and rates.
problem Structured principal subspace estimation problems.
method Unified framework, minimax lower and upper bounds, information-geometric complexity.
result Minimax rates of convergence for specific settings, including optimal rates for non-negative PCA/SVD.
The paper studies PCA of probability measures with varying sample sizes and finds optimal convergence rates.
problem PCA of multiple probability measures with varying sample sizes.
method Double asymptotic regime analysis with convergence rates n−1/2+m−α for empirical covariance and PCA risk. result Optimal convergence rates for empirical covariance and PCA risk in the dense regime are proven.
Principal component analysis (PCA) is widely used for feature extraction and dimensionality reduction, with documented merits in diverse tasks involving high-dimensional data. Standard PCA copes with one dataset at a time, but it is challenged when it comes to analyzing multiple datasets jointly. In certain data scienc…
Sparse APCA identifies sparse factors in financial returns over time.
problem Analyzing co-movements of high-dimensional panel data over time.
method Sparse asymptotic PCA with truncated power method for sparse factors and sequential deflation for multi-factor cases.
result Identification of nine risk factors influencing the S&P 500 stock market.
We discuss a clustering method for Gaussian mixture model based on the sparse principal component analysis (SPCA) method and compare it with the IF-PCA method. We also discuss the dependent case where the covariance matrix Σ is not necessarily diagonal.
A new method detects sparse changes in high-dimensional data streams using tailored PCA projections.
problem Detecting sparse changes in high-dimensional data streams.
method Tailored PCA projections for online change detection.
result High efficiency in detecting even very sparse changes in mean, variance, and correlation.
sPCA models may not have orthogonal scores and loadings, complicating interpretation.
problem sPCA scores and loadings may not be orthogonal.
method Illustrated and numerically demonstrated the implications of sPCA on scores, residuals, and variance explained.
result sPCA approaches perform poorly on noise-free, sparse data.
AdvPCA uses robust optimization to achieve sparse PCA without tuning.
problem Sparse PCA for high-dimensional data with implicit sparsity.
method Adversarial PCA (AdvPCA) using robust optimization.
result AdvPCA achieves effective sparse PCA with a closed-form solution.
New method solves sparse PCA for multiple components efficiently.
problem Sparse PCA for multiple orthogonal components.
method Reformulates orthogonality as rank constraints, uses semidefinite relaxations and bounds.
result Exact solutions with near-optimal variance explained and orthogonality.
PCA minor projection is most sensitive to distributional changes in bivariate data.
problem Detecting sparse distributional changes in high-dimensional data.
method Proved that the minor projection of PCA-rotated data is most sensitive to distributional changes defined by Hellinger distance.
result The minor projection is the most sensitive to sparse distributional changes in high-dimensional data.
Paper proposes distributed sparse multicategory discriminant analysis for classification.
problem Sparse multicategory classification with distributed data.
method Convex formulation, distributed setting, invariant discriminant subspace recovery.
result Distributed sparse multicategory linear discriminant analysis performs as good as centralized version after a few rounds of communications.
Unified analysis for robust PCA decomposition with sparse components in known dictionaries.
problem Robust PCA decomposition with sparse components in known dictionaries.
method Convex demixing method for undercomplete and overcomplete dictionary cases.
result Successful recovery of constituent components up to a certain global sparsity level.