arXiv research

This work connects LLE, factor analysis, and probabilistic PCA through a stochastic perspective.

Probabilistic Autoencoder learns latent space weights' distribution.

Principal Component Analysis (PCA) is a popular tool for dimensionality reduction and feature extraction in data analysis. There is a probabilistic version of PCA, known as Probabilistic PCA (PPCA). However, standard PCA and PPCA are not robust, as they are sensitive to outliers. To alleviate this problem, this paper i…

HeMPPCAT improves PCA for data with varying noise.

Paper develops a dual formulation for PCA in Hilbert spaces.

Sparse versions of principal component analysis (PCA) have imposed themselves as simple, yet powerful ways of selecting relevant features of high-dimensional data in an unsupervised manner. However, when several sparse principal components are computed, the interpretation of the selected variables is difficult since ea…

Paper revisits PCA for anomaly detection in network security.

Principal component analysis (PCA) is one of the most widely used dimension reduction and multivariate statistical techniques. From a probabilistic perspective, PCA seeks a low-dimensional representation of data in the presence of independent identical Gaussian noise. Probabilistic PCA (PPCA) and its variants have been…

Auto-Associative models cover a large class of methods used in data analysis. In this paper, we describe the generals properties of these models when the projection component is linear and we propose and test an easy to implement Probabilistic Semi-Linear Auto- Associative model in a Gaussian setting. We show it is a g…

We present a method to compute the Shapley values of reconstruction errors of principal component analysis (PCA), which is particularly useful in explaining the results of anomaly detection based on PCA. Because features are usually correlated when PCA-based anomaly detection is applied, care must be taken in computing…

We consider probabilistic PCA and related factor models from a Bayesian perspective. These models are in general not identifiable as the likelihood has a rotational symmetry. This gives rise to complicated posterior distributions with continuous subspaces of equal density and thus hinders efficiency of inference as wel…

Survey of factor analysis, PCA, variational inference, and VAE.

CAVI converges exponentially fast for Bayesian PCA models.

Paper proposes GPM for heteroscedastic PCA estimation.

PGPCA improves PCA for nonlinear data in neuroscience.

We shed new insights on the two commonly used updates for the online $k$-PCA problem, namely, Krasulina's and Oja's updates. We show that Krasulina's update corresponds to a projected gradient descent step on the Stiefel manifold of the orthonormal $k$-frames, while Oja's update amounts to a gradient descent step using…

Proposes MPCA for robust PCA using mode estimation.

This paper improves PPCA robustness using $t$-distributions.

We present a unifying framework which reduces the construction of probabilistic component analysis techniques to a mere selection of the latent neighbourhood, thus providing an elegant and principled framework for creating novel component analysis models as well as constructing probabilistic equivalents of deterministi…

Principal component analysis (PCA) is arguably the most popular tool in multivariate exploratory data analysis. In this paper, we consider the question of how to handle heterogeneous variables that include continuous, binary, and ordinal. In the probabilistic interpretation of low-rank PCA, the data has a normal multiv…

Autoencoders are a deep learning model for representation learning. When trained to minimize the distance between the data and its reconstruction, linear autoencoders (LAEs) learn the subspace spanned by the top principal directions but cannot learn the principal directions themselves. In this paper, we prove that $L_2…

RFPCA improves robustness of FPCA for matrix data.

A novel 3D shape registration method using spectral graph embedding and probabilistic matching.

Efficient and high-fidelity prior sampling and inversion for complex geological media is still a largely unsolved challenge. Here, we use a deep neural network of the variational autoencoder type to construct a parametric low-dimensional base model parameterization of complex binary geological media. For inversion purp…

In the era of big data, reducing data dimensionality is critical in many areas of science. Widely used Principal Component Analysis (PCA) addresses this problem by computing a low dimensional data embedding that maximally explain variance of the data. However, PCA has two major weaknesses. Firstly, it only considers li…

Normalizing flows optimize Jacobian determinant for unique likelihood objective.

Using the linear Gaussian latent variable model as a starting point we relax some of the constraints it imposes by deriving a nonparametric latent feature Gaussian variable model. This model introduces additional discrete latent variables to the original structure. The Bayesian nonparametric nature of this new model al…

Dimensionality reduction on Riemannian manifolds is challenging due to the complex nonlinear data structures. While probabilistic principal geodesic analysis~(PPGA) has been proposed to generalize conventional principal component analysis (PCA) onto manifolds, its effectiveness is limited to data with a single modality…

PCA whitening weighted by Zipfian word frequencies improves task performance.

Introduces a probabilistic framework for dimension reduction methods.

ProbDR framework interprets DR algorithms as probabilistic inference.

This paper proposes a probabilistic imputation method with uncertainty quantification.

An important preprocessing step in most data analysis pipelines aims to extract a small set of sources that explain most of the data. Currently used algorithms for blind source separation (BSS), however, often fail to extract the desired sources and need extensive cross-validation. In contrast, their rarely used probab…

Simplifies fair PCA with fast, efficient solution.

Generative neural networks model multivariate time series data.

KAN-PCA improves asset return analysis by capturing more variance than classical PCA during market crises.

New algorithm solves fair PCA, robust PCA, and sparse PCA problems efficiently.

Unified framework improves PCA for outliers and distributed data.

HPPCA improves imputation of longitudinal data with missing values.

TL-PCA uses transfer learning to improve PCA performance with limited target data.

Bucketed PCA-NN outperforms DNNs by 96% on MNIST.

A new low-dimensional parameterization based on principal component analysis (PCA) and convolutional neural networks (CNN) is developed to represent complex geological models. The CNN-PCA method is inspired by recent developments in computer vision using deep learning. CNN-PCA can be viewed as a generalization of an ex…

New method improves calibration in multi-output probabilistic models.

Anchor PCA improves robustness in multi-domain PCA.

A new method for fair PCA ensures balanced error across groups.

This is a detailed tutorial paper which explains the Principal Component Analysis (PCA), Supervised PCA (SPCA), kernel PCA, and kernel SPCA. We start with projection, PCA with eigen-decomposition, PCA with one and multiple projection directions, properties of the projection matrix, reconstruction error minimization, an…

Revisits PCA with new formulations and insights.

Proposes $σ$-PCA to learn identifiable linear transformations without whitening.