Explains various PCA and SPCA methods with theory and applications.
problem No specific problem stated; focuses on explaining methods.
method Explains PCA, SPCA, kernel PCA, and kernel SPCA methods with theory and applications.
result Comprehensive coverage of PCA and SPCA methods with theory and applications.
A new method generalizing subspace learning for improved classification.
problem Improving classification accuracy using subspace learning methods.
method Roweis Discriminant Analysis (RDA) which generalizes PCA, SPCA, and FDA.
result RDA and kernel RDA improve classification accuracy on benchmark datasets.
R package spca computes sparse principal components efficiently.
problem Sparse principal components analysis (SPCA) for interpretable data.
method Least squares sparse principal component analysis (LS-SPCA) with efficient C++ backend.
result Computes sparse principal components that maximize variance and maintain strong correlations with PCs.
Stochastic principal component analysis (SPCA) has become a popular dimensionality reduction strategy for large, high-dimensional datasets. We derive a simplified algorithm, called Lazy SPCA, which has reduced computational complexity and is better suited for large-scale distributed computation. We prove that SPCA and …
Principal component analysis (PCA) is a statistical technique commonly used in multivariate data analysis. However, PCA can be difficult to interpret and explain since the principal components (PCs) are linear combinations of the original variables. Sparse PCA (SPCA) aims to balance statistical fidelity and interpretab…
sPCA models may not have orthogonal scores and loadings, complicating interpretation.
problem sPCA scores and loadings may not be orthogonal.
method Illustrated and numerically demonstrated the implications of sPCA on scores, residuals, and variance explained.
result sPCA approaches perform poorly on noise-free, sparse data.
Bayesian SPCA method tackles orthogonality constraint with spike and slab prior.
problem Bayesian SPCA method for high-dimensional data with orthogonality constraint.
method Parameter-expanded coordinate ascent variational inference (PX-CAVI) with spike and slab prior.
result PX-CAVI algorithm outperforms existing SPCA approaches in performance.
Sparse Principal Component Analysis (SPCA) and Sparse Linear Regression (SLR) have a wide range of applications and have attracted a tremendous amount of attention in the last two decades as canonical examples of statistical problems in high dimension. A variety of algorithms have been proposed for both SPCA and SLR, b…
Federated learning improves SPCA for sparse components.
problem Data privacy and sharing constraints in machine learning.
method Federated learning framework applied to SPCA with L1 regularization and smoothing.
result Federated SPCA achieves sparse component loadings with improved interpretability.
This paper provides theoretical guarantees for SPCA using the Elastic Net.
problem Lack of theoretical guarantees for the SPCA algorithm.
method Revisited and improved the SPCA algorithm of Zou et al. (2006) using the Elastic Net.
result Both algorithms can recover the principal subspace consistently under mild conditions.
This work includes all the technical details of the Sequential Principal Curves Analysis (SPCA) in a single document. SPCA is an unsupervised nonlinear and invertible feature extraction technique. The identified curvilinear features can be interpreted as a set of nonlinear sensors: the response of each sensor is the pr…
A fast method for sparse PCA reduces computation time.
problem Time-consuming implementation of SPCA on high-dimensional data.
method Subspace projections using Household QR factorization for efficient deflation.
result Developed SPCA-SP method maintains good tradeoffs between various criteria.
Sparse principal component analysis (SPCA) has emerged as a powerful technique for modern data analysis, providing improved interpretation of low-rank structures by identifying localized spatial structures in the data and disambiguating between distinct time scales. We demonstrate a robust and scalable SPCA algorithm b…
New method optimizes PCA for better prediction and variance.
problem Improve PCA for better prediction and variance.
method Jointly optimize prediction error and variance explained.
result Our method outperforms existing approaches in both prediction and variance.
SPCA improves PCA by learning from simple to complex samples.
problem Noise and outliers in complex data.
method Self-paced Principal Component Analysis (SPCA) that integrates samples from simple to more complex.
result SPCA improves state-of-the-art results on popular datasets.
SP-SPCA improves sparse PCA by adaptively adjusting variable penalties, enhancing interpretability and stability.
problem Poor interpretability and variable redundancy in PCA for high-dimensional data.
method Introduces a single equilibrium parameter to adaptively adjust variable penalties in the L2 regularization framework.
result Consistently outperforms standard sparse PCA methods in identifying sparse loading patterns and preserving cumulative variance.
Principal component analysis (PCA) is an exploratory tool widely used in data analysis to uncover dominant patterns of variability within a population. Despite its ability to represent a data set in a low-dimensional space, the interpretability of PCA remains limited. However, in neuroimaging, it is essential to uncove…
Paper presents a randomized algorithm for SPCA with high probability approximation.
problem Sparse Principal Component Analysis (SPCA) is NP-hard.
method Based on basic SDP relaxation, the algorithm constructs deterministic and randomized solutions.
result The algorithm achieves an approximation ratio of at most the sparsity constant with high probability.
We discuss a clustering method for Gaussian mixture model based on the sparse principal component analysis (SPCA) method and compare it with the IF-PCA method. We also discuss the dependent case where the covariance matrix Σ is not necessarily diagonal.
XCAN uses cross-product penalization for sparse matrix factorization.
problem Understanding complex data structures.
method Sparse matrix factorization with a loss function balancing variance and structural preservation.
result Flexible modeling approach for diverse applications.
Proposes SPCA to incorporate structural constraints in model identification.
problem Model identification with partial structural knowledge.
method Structural Principal Component Analysis (SPCA) that leverages structural information.
result Demonstrates improved model estimates using synthetic and industrial data.
Mechanisms of human color vision are characterized by two phenomenological aspects: the system is nonlinear and adaptive to changing environments. Conventional attempts to derive these features from statistics use separate arguments for each aspect. The few statistical approaches that do consider both phenomena simulta…
In statistical dimensionality reduction, it is common to rely on the assumption that high dimensional data tend to concentrate near a lower dimensional manifold. There is a rich literature on approximating the unknown manifold, and on exploiting such approximations in clustering, data compression, and prediction. Most …
This paper proposes exact and approximation algorithms for Sparse PCA, improving interpretability and scalability.
problem Selecting a prespecified-size principal submatrix from a covariance matrix to maximize its largest eigenvalue.
method Proposes two exact mixed-integer SDPs and a mixed-integer linear program (MILP) for SPCA, analyzes theoretical optimality gaps, and develops approximation algorithms.
result The proposed algorithms achieve strong theoretical optimality and effective scalability, with continuous relaxations close to optimality and MILP solving small to medium-size instances.
Given a multivariate data set, sparse principal component analysis (SPCA) aims to extract several linear combinations of the variables that together explain the variance in the data as much as possible, while controlling the number of nonzero loadings in these combinations. In this paper we consider 8 different optimiz…
Supervised dimensionality reduction strategies have been of great interest. However, current supervised dimensionality reduction approaches are difficult to scale for situations characterized by large datasets given the high computational complexities associated with such methods. While stochastic approximation strateg…
A new PCA method using Tℓ1-norm outperforms existing methods.
problem Outliers and noise sensitivity in classical PCA.
method PCA based on Tℓ1-norm maximization. result The method outperforms PCA-ℓp, ℓpSPCA, and PCA in numerical experiments. Paper develops IFTRR to solve sparse generalized eigenvalue problems efficiently.
problem Finding the leading eigenvector with at most k nonzero entries in sparse generalized eigenvalue problems.
method Inverse-free truncated Rayleigh-Ritz method (IFTRR) with a new truncation strategy.
result IFTRR efficiently finds the support set of the leading eigenvector for large scale problems.
Sparse matrix decomposition identifies key design variables for ICF experiments.
problem Improving predictive capability of ICF simulation codes through better understanding of design inputs and outcomes.
method Sparse Principal Component Analysis (SPCA) and Random Forest (RF) surrogate model.
result Identified clusters of design variables related to physical processes, revealing important variables not previously considered.
Many unsupervised kernel methods rely on the estimation of the kernel covariance operator (kernel CO) or kernel cross-covariance operator (kernel CCO). Both kernel CO and kernel CCO are sensitive to contaminated data, even when bounded positive definite kernels are used. To the best of our knowledge, there are few well…
Survey of kernels, RKHS, and their applications in machine learning.
problem Understanding kernels and their applications in machine learning.
method Review of historical context, mathematical definitions, and practical applications of kernels.
result Comprehensive overview of kernels, RKHS, and their applications.
To the best of our knowledge, there are no general well-founded robust methods for statistical unsupervised learning. Most of the unsupervised methods explicitly or implicitly depend on the kernel covariance operator (kernel CO) or kernel cross-covariance operator (kernel CCO). They are sensitive to contaminated data, …
Deep neural kernels and Laplace kernel have equivalent RKHS on spheres.
problem Comparing RKHS of deep neural tangent and Laplace kernels.
method Proof of RKHS equivalence using sphere restrictions and kernel properties.
result RKHS of deep neural tangent kernel and Laplace kernel are the same on Sd−1. Kernel methods linked to feature subspaces and maximal correlation kernels.
problem Understanding kernel methods and their relationship to feature extraction.
method Established a correspondence between feature subspaces and kernels, introduced maximal correlation kernels, and demonstrated their optimality.
result Kernel SVM on maximal correlation kernel achieves minimum prediction error.
Adapts manifold structure for better clustering performance.
problem Lack of consideration for local manifold structure in existing multiple kernel k-means methods.
method Adopts manifold adaptive kernel to integrate local manifold structure of kernels.
result Proposed method outperforms state-of-the-art methods.
PGF kernels analyze spherical data using generalized RBF kernels.
problem Analysis of spherical data.
method Introduced PGF kernels and a semi-parametric learning algorithm.
result PGF kernels generalize RBF kernels for spherical data.
Optimal kernel in KR can be data-dependent, improving model performance.
problem Fixed kernel in KR limits model performance.
method Considered data-dependent kernels for KR, using posterior covariance.
result Data-dependent kernel choice leads to optimal performance.
Quantum kernels can be efficiently embedded into classical feature spaces.
problem Can all quantum kernels be efficiently embedded into classical feature spaces?
method Invoking computational universality and using techniques like random Fourier features, the authors show that certain classes of quantum kernels can be efficiently embedded.
result For shift-invariant and composition kernels, embedding quantum kernels are universal and efficient.
New random feature maps for Laplacian and related kernels.
problem Challenges in approximating the Laplacian kernel and its generalizations.
method Developed random feature maps for Laplacian and related kernels, providing efficient sampling schemes.
result Demonstrated the efficacy of these random feature maps on real datasets.
New method for learning with non-Euclidean data using decomposable kernels.
problem Difficulty in using classical kernels for non-Euclidean data.
method Reproducing kernel Krein space (RKKS) methods for kernels that admit a positive decomposition.
result Invariant kernels can be used for learning in non-Euclidean spaces.
New estimator reduces kernel mean estimation error.
problem Kernel mean estimation in reproducing kernel Hilbert spaces.
method Corrupt data with known distributions and estimate kernel mean under the corrupted distribution.
result The marginalized kernel mean estimator achieves lower estimation error.
Estimates kernel eigenvalues for compositional dot-product kernels.
problem Improving estimates for kernel eigenvalues.
method Eigenvalue decay estimates of integral operators associated with dot-product kernels.
result Improved estimates for kernel volumes in reproducing kernel Hilbert spaces.
A new kernel, Isolation Kernel, simplifies large scale online kernel learning without sacrificing accuracy.
problem Building efficient and scalable kernel-based models from large datasets with high accuracy.
method Introducing Isolation Kernel, which creates an exact, sparse, and finite-dimensional feature map of a kernel, allowing for efficient large scale online kernel learning without accuracy loss.
result Large scale online kernel learning can be achieved efficiently and accurately using Isolation Kernel.
Optimal Biweight kernel and computationally efficient Epanechnikov kernel for modal linear regression.
problem Finding the best kernel for modal linear regression.
method Refined analysis of asymptotic statistical behavior and IRLS algorithm convergence.
result Biweight kernel minimizes asymptotic mean squared error, Epanechnikov kernel guarantees IRLS convergence.
We present Random Partition Kernels, a new class of kernels derived by demonstrating a natural connection between random partitions of objects and kernels between those objects. We show how the construction can be used to create kernels from methods that would not normally be viewed as random partitions, such as Random…
The NNGP kernel's predictions closely match those of the Matern kernel under certain conditions.
problem Comparing NNGP kernels to Matern kernels in practical applications.
method Demonstrated the necessity of normalization for NNGP kernels, explored numerical challenges, and compared predictions and performance.
result NNGP kernel predictions closely match Matern kernel predictions under specific circumstances.
In this paper, we compare 5 different nonlinear kernels: min-max, RBF, fRBF (folded RBF), acos, and acos-χ2, on a wide range of publicly available datasets. The proposed fRBF kernel performs very similarly to the RBF kernel. Both RBF and fRBF kernels require an important tuning parameter (γ). Interestingly, for a …
New kernels allow learning from non-separable data.
problem Learning from non-separable data.
method Introducing entangled kernels and a two-step algorithm.
result Efficient algorithm for learning entangled kernels.