Local PCA detects intrinsic parameterization of complex thermo-chemical state-spaces.
problem Detecting intrinsic parameterization of complex thermo-chemical state-spaces.
method Local PCA applied to local clusters of data.
result Local PCA finds meaningful parameterization linked to local stoichiometry, reaction progress, and soot formation processes.
Modified PCA algorithm with continual learning preserves features of previous modes for multimode process monitoring.
problem Catastrophic forgetting of previous modes in monitoring models for successive modes.
method Modified PCA algorithm with elastic weight consolidation (EWC) to preserve features of previous modes.
result PCA-EWC algorithm effectively monitors multimode processes without performance decrease.
CA-PCA improves manifold dimension estimation by accounting for curvature.
problem Estimating the dimension of manifolds in high-dimensional data.
method Develops CA-PCA, a local PCA method calibrated with a quadratic embedding to account for curvature.
result Improves manifold dimension estimation in various settings.
A new dynamical formulation of log-PCA captures local principal modes of geodesic variations.
problem Learning principal variations of random probability measures under Wasserstein geometry.
method Introducing a new dynamical formulation of log-PCA as a variational approach.
result Deriving a general statistical convergence rate for empirical WT-PCA.
We propose a spectral clustering method based on local principal components analysis (PCA). After performing local PCA in selected neighborhoods, the algorithm builds a nearest neighbor graph weighted according to a discrepancy between the principal subspaces in the neighborhoods, and then applies spectral clustering. …
Estimates manifold dimension using local graph structure.
problem Estimating the intrinsic dimension of manifolds from data.
method Regression on local PCA coordinates, focusing on local graph structure.
result Proposed QE and TLS estimators outperform existing methods.
This work connects LLE, factor analysis, and probabilistic PCA through a stochastic perspective.
problem Exploring the theoretical connection between LLE, factor analysis, and probabilistic PCA.
method Solving the stochastic linear reconstruction of LLE using expectation maximization.
result LLE, factor analysis, and probabilistic PCA are shown to be connected through a stochastic perspective.
Dimensionality reduction is an important operation in information visualization, feature extraction, clustering, regression, and classification, especially for processing noisy high dimensional data. However, most existing approaches preserve either the global or the local structure of the data, but not both. Approache…
New PCA method handles multiple datasets and detects sparse patterns robustly.
problem Handling multi-source data with sparse and outlier-robust PCA.
method Developed a regularization problem with a penalty for structured sparsity and outlier resistance.
result The method detects global and local patterns across multiple data sources robustly.
New approach combines PCA and t-sne for better data analysis.
problem Multiscale complexity in high-dimensional data.
method Multiscale joint characterization using PCA and t-sne.
result Joint characterization detects signals not seen by PCA or t-sne alone.
Unified framework improves PCA for outliers and distributed data.
problem Outliers and limitations in PCA for large-scale applications.
method φ-PCA framework that retains PCA efficiency and adds robustness.
result HM-PCA achieves optimal robustness and efficiency.
R-PCA extends PCA to Riemannian manifolds for structured data.
problem Applying PCA to data on Riemannian manifolds without vector space operations.
method Adapting PCA to Riemannian manifolds by equipping data with local metrics.
result Unified approach for dimensionality reduction and statistical analysis on manifolds.
New local-search methods close the gap in sparse tensor PCA.
problem Sparse tensor PCA underperforms compared to other methods.
method Proposes new local-search methods including greedy and random-threshold variants.
result Proves local-search methods close the gap to best known polynomial-time procedures.
Paper optimizes PCA for fairness using MMD and Stiefel manifold optimization.
problem Fair principal component analysis (PCA) to minimize MMD between protected classes.
method Formulates fair PCA as non-convex optimization over Stiefel manifold, solves using REPMS with theoretical guarantees.
result Our approach outperforms prior work in fairness, explained variance, and runtime.
A new method for distributed PCA using matrix β-mean.
problem Efficiently aggregating PCA results across multiple machines with reduced computational overhead.
method Proposes a novel DPCA method that incorporates eigenvalue information using the matrix β-mean.
result The matrix β-mean method improves robustness and stability of eigenvector ordering.
New methods explain NE embeddings by identifying key variables.
problem Lack of interpretability in NE techniques.
method Combining PCA, Q-residuals, Hotelling's T2, and visualization.
result Identifies discriminatory features not seen in standard approaches.
Constructing an efficient parameterization of a large, noisy data set of points lying close to a smooth manifold in high dimension remains a fundamental problem. One approach consists in recovering a local parameterization using the local tangent plane. Principal component analysis (PCA) is often the tool of choice, as…
Lie PCA improves density estimation on symmetric manifolds.
problem Density estimation for symmetric manifolds.
method Spectral method to approximate Lie algebra of symmetry group.
result Improved sample complexity and density estimation on various data sets.
In this paper we develop a new framework that captures the common landscape underlying the common non-convex low-rank matrix problems including matrix sensing, matrix completion and robust PCA. In particular, we show for all above problems (including asymmetric cases): 1) all local minima are also globally optimal; 2) …
This paper investigates the generalization of Principal Component Analysis (PCA) to Riemannian manifolds. We first propose a new and general type of family of subspaces in manifolds that we call barycentric subspaces. They are implicitly defined as the locus of points which are weighted means of k+1 reference points.…
The Variational Autoencoder (VAE) is a powerful architecture capable of representation learning and generative modeling. When it comes to learning interpretable (disentangled) representations, VAE and its variants show unparalleled performance. However, the reasons for this are unclear, since a very particular alignmen…
Principal component analysis (PCA) is a widely used technique for data analysis and dimension reduction with numerous applications in science and engineering. However, the standard PCA suffers from the fact that the principal components (PCs) are usually linear combinations of all the original variables, and it is thus…
We present an algorithm for L1-norm kernel PCA and provide a convergence analysis for it. While an optimal solution of L2-norm kernel PCA can be obtained through matrix decomposition, finding that of L1-norm kernel PCA is not trivial due to its non-convexity and non-smoothness. We provide a novel reformulation through …
New algorithm reduces communication in distributed eigenspace estimation.
problem Efficiently estimating eigenspaces in distributed settings without excessive communication.
method Communication-efficient distributed algorithm using Procrustean alignment.
result Achieves similar error rate to centralized estimator for PCA.
Paper proposes GPM for heteroscedastic PCA estimation.
problem Estimating ground truth from heterogeneous data.
method Generalized power method (GPM) for HQPOC.
result GPM achieves geometrically decreasing distances to ground truth.
Paper optimizes federated PCA for covariance estimation under privacy constraints.
problem Privacy-preserving covariance estimation in federated learning.
method Federated PCA, matrix version of van Trees' inequality, three-layer spectral decomposition.
result Optimal rates of convergence for central server's estimation, robust to inconsistent local estimators.
We consider principal component analysis (PCA) in decomposable Gaussian graphical models. We exploit the prior information in these models in order to distribute its computation. For this purpose, we reformulate the problem in the sparse inverse covariance (concentration) domain and solve the global eigenvalue problem …
UMAP compared to other methods for dimensionality reduction.
problem Comparing UMAP to other dimensionality reduction techniques.
method Comprehensive evaluation of UMAP and other methods.
result Supervised UMAP performs well for classification but not for regression.
Study shows simple vector quantization measures correlate with deep learning generalization.
problem Understanding and predicting generalization in deep learning models.
method Applying complexity measures from approximation and information theory to deep learning features.
result Simple vector quantization measures correlate well with generalization performance in deep learning.
This work studies low-rank approximation of a positive semidefinite matrix from partial entries via nonconvex optimization. We characterized how well local-minimum based low-rank factorization approximates a fixed positive semidefinite matrix without any assumptions on the rank-matching, the condition number or eigensp…
Sig-PCA integrates model outputs and observations to correct model biases.
problem Improving model accuracy and reliability by correcting biases and numerical approximations.
method Sig-PCA framework that combines summary statistics from model outputs with localized observations via a neural network.
result Corrects model outputs to align closely with observational data, preserving essential statistical information.
Max-Cut decision tree improves classification accuracy and reduces computation time.
problem Improving decision tree accuracy and efficiency for complex classification tasks.
method Alternative splitting metric (max cut) and PCA-based feature selection at each node.
result 49% improvement in accuracy with 94% reduction in CPU time on CIFAR-100 data.
Simplifies fair PCA with fast, efficient solution.
problem Learning fair low-rank approximations of data.
method Conceptually simple approach with analytic solution.
result Faster and similar results to existing fair PCA methods.
When confronted with massive data streams, summarizing data with dimension reduction methods such as PCA raises theoretical and algorithmic pitfalls. Principal curves act as a nonlinear generalization of PCA and the present paper proposes a novel algorithm to automatically and sequentially learn principal curves from d…
KAN-PCA improves asset return analysis by capturing more variance than classical PCA during market crises.
problem Inefficient classical PCA during market crises when correlations between assets change dramatically.
method KAN-PCA uses KAN (Kolmogorov-Arnold Networks) with B-spline functions to learn nonlinear projections.
result KAN-PCA achieves a higher reconstruction R^2 (66.57%) compared to classical PCA (62.99%) on 20 S&P 500 stocks.
New algorithm solves fair PCA, robust PCA, and sparse PCA problems efficiently.
problem Fair Principal Component Analysis (FPCA) to ensure fairness in PCA solutions.
method Iterative MM algorithm with SDP reformulation to quadratic program.
result Algorithm monotonically improves fairness objectives at each iteration.
PerPCA separates unique and shared features from heterogeneous data.
problem Extracting shared and unique features from data collected from different sources with varying trends.
method Personalized PCA (PerPCA) uses orthogonal global and local principal components to encode both unique and shared features.
result PerPCA can identify and recover both unique and shared features under mild conditions.
TL-PCA uses transfer learning to improve PCA performance with limited target data.
problem PCA performance is limited with scarce target data.
method Transfer learning approach to PCA (TL-PCA) that combines source task knowledge with target task data.
result Improved PCA representation for dimensionality reduction with limited target data.
Bucketed PCA-NN outperforms DNNs by 96% on MNIST.
problem Benchmarking deep neural networks for supervised classification.
method Applies PCA to individual buckets constructed in two phases, retains neural network architecture, and uses neurons that mirror input signals.
result Bucketed PCA-NN achieves 96% accuracy on MNIST, similar to DNNs.
A new low-dimensional parameterization based on principal component analysis (PCA) and convolutional neural networks (CNN) is developed to represent complex geological models. The CNN-PCA method is inspired by recent developments in computer vision using deep learning. CNN-PCA can be viewed as a generalization of an ex…
Anchor PCA improves robustness in multi-domain PCA.
problem PCA on pooled data can focus on spurious directions.
method Anchor PCA focuses on shared directions of variation.
result Anchor PCA outperforms pooling and worst-case alternatives.
A new method for fair PCA ensures balanced error across groups.
problem Balancing approximation error across different groups in multi-group data.
method Iterative method to compute fair principal components minimizing max group-wise reconstruction error.
result Preserves the containment property of standard PCA and reduces to standard PCA for single-group data.
This is a detailed tutorial paper which explains the Principal Component Analysis (PCA), Supervised PCA (SPCA), kernel PCA, and kernel SPCA. We start with projection, PCA with eigen-decomposition, PCA with one and multiple projection directions, properties of the projection matrix, reconstruction error minimization, an…
Developing efficient and guaranteed nonconvex algorithms has been an important challenge in modern machine learning. Algorithms with good empirical performance such as stochastic gradient descent often lack theoretical guarantees. In this paper, we analyze the class of homotopy or continuation methods for global optimi…
With the development of high-throughput technologies, principal component analysis (PCA) in the high-dimensional regime is of great interest. Most of the existing theoretical and methodological results for high-dimensional PCA are based on the spiked population model in which all the population eigenvalues are equal ex…
Revisits PCA with new formulations and insights.
problem Improving PCA formulations and understanding.
method Difference-of-convex (DC) framework, kernelizability, out-of-sample applicability, simultaneous iteration, DCA perspective.
result PCA-like problems are kernelizable and have new optimization perspectives.
Proposes σ-PCA to learn identifiable linear transformations without whitening.
problem Cannot identify axes with equal variances in PCA.
method Unified model for linear and nonlinear PCA, introducing a missing piece to eliminate rotational indeterminacy.
result Eliminates subspace rotational indeterminacy in PCA.
Solution to sparse PCA tuning problem using Empirical Bayes.
problem Sparse PCA multiple tuning problem (MTP).
method Empirical Bayes covariance decomposition for penalized PCA.
result Empirical Bayes approach efficiently solves MTP in sparse PCA.