We show that the skip-gram formulation of word2vec trained with negative sampling is equivalent to a weighted logistic PCA. This connection allows us to better understand the objective, compare it to other word embedding methods, and extend it to higher dimensional models.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “logistic PCA”
Principal component analysis (PCA) for binary data, known as logistic PCA, has become a popular alternative to dimensionality reduction of binary data. It is motivated as an extension of ordinary PCA by means of a matrix factorization, akin to the singular value decomposition, that maximizes the Bernoulli log-likelihoo…
Multivariate binary data is becoming abundant in current biological research. Logistic principal component analysis (PCA) is one of the commonly used tools to explore the relationships inside a multivariate binary data set by exploiting the underlying low rank structure. We re-expressed the logistic PCA model based on …
Sequential or online dimensional reduction is of interests due to the explosion of streaming data based applications and the requirement of adaptive statistical modeling, in many emerging fields, such as the modeling of energy end-use profile. Principal Component Analysis (PCA), is the classical way of dimensional redu…
Anomaly detection in multivariate time series is improved using ensemble techniques.
problem Anomaly detection in multivariate time series data is challenging due to sparse anomalies and feature subset anomalies.
method Feature-bagging technique, nested rotation PCA, ensemble of base models, semi-supervised Logistic Regressor.
result The proposed ensemble technique outperforms basic algorithms by 2% unsupervised and at least 10% semi-supervised.
Estimates mean of distributed vectors with sparsification and spatial/temporal correlations.
problem Estimating mean of high-dimensional vectors distributed across nodes with low communication cost.
method Modifies decoding method to leverage spatial and temporal correlations in sparsified vectors.
result Estimators consistently outperform more sophisticated sparsification methods.
Class prediction is an important application of microarray gene expression data analysis. The high-dimensionality of microarray data, where number of genes (variables) is very large compared to the number of samples (obser- vations), makes the application of many prediction techniques (e.g., logistic regression, discri…
New embeddings capture local structure in complex networks.
problem Embeddings cannot capture local structure in complex networks.
method Logistic Principal Component Analysis (LPCA) algorithm for exact low-rank representations.
result Exact low-rank representations of real-world networks are possible.
Simplifies fair PCA with fast, efficient solution.
problem Learning fair low-rank approximations of data.
method Conceptually simple approach with analytic solution.
result Faster and similar results to existing fair PCA methods.
Study enhances neural network interpretability through statistical methods.
problem Complexity and interpretability challenges in neural networks.
method Theoretical framework, statistical tests, dimensionality reduction algorithms.
result Developed bootstrapping technique and statistical tests for ANN performance.
Many classical algorithms are found until several years later to outlive the confines in which they were conceived, and continue to be relevant in unforeseen settings. In this paper, we show that SVRG is one such method: being originally designed for strongly convex objectives, it is also very robust in non-strongly co…
Local Linear Convergence of Gradient Methods for Subspace Optimization via Strict Complementaritymath.OC
New method solves subspace optimization problems efficiently.
problem Finding a k-dimensional subspace in high dimensions.
method Local linear convergence of gradient methods under strict complementarity.
result Gradient method converges linearly in high dimensions.
KAN-PCA improves asset return analysis by capturing more variance than classical PCA during market crises.
problem Inefficient classical PCA during market crises when correlations between assets change dramatically.
method KAN-PCA uses KAN (Kolmogorov-Arnold Networks) with B-spline functions to learn nonlinear projections.
result KAN-PCA achieves a higher reconstruction R^2 (66.57%) compared to classical PCA (62.99%) on 20 S&P 500 stocks.
New algorithm solves fair PCA, robust PCA, and sparse PCA problems efficiently.
problem Fair Principal Component Analysis (FPCA) to ensure fairness in PCA solutions.
method Iterative MM algorithm with SDP reformulation to quadratic program.
result Algorithm monotonically improves fairness objectives at each iteration.
Unified framework improves PCA for outliers and distributed data.
problem Outliers and limitations in PCA for large-scale applications.
method φ-PCA framework that retains PCA efficiency and adds robustness.
result HM-PCA achieves optimal robustness and efficiency.
TL-PCA uses transfer learning to improve PCA performance with limited target data.
problem PCA performance is limited with scarce target data.
method Transfer learning approach to PCA (TL-PCA) that combines source task knowledge with target task data.
result Improved PCA representation for dimensionality reduction with limited target data.
Bucketed PCA-NN outperforms DNNs by 96% on MNIST.
problem Benchmarking deep neural networks for supervised classification.
method Applies PCA to individual buckets constructed in two phases, retains neural network architecture, and uses neurons that mirror input signals.
result Bucketed PCA-NN achieves 96% accuracy on MNIST, similar to DNNs.
A new low-dimensional parameterization based on principal component analysis (PCA) and convolutional neural networks (CNN) is developed to represent complex geological models. The CNN-PCA method is inspired by recent developments in computer vision using deep learning. CNN-PCA can be viewed as a generalization of an ex…
Anchor PCAstat.ML
Anchor PCA improves robustness in multi-domain PCA.
problem PCA on pooled data can focus on spurious directions.
method Anchor PCA focuses on shared directions of variation.
result Anchor PCA outperforms pooling and worst-case alternatives.
A new method for fair PCA ensures balanced error across groups.
problem Balancing approximation error across different groups in multi-group data.
method Iterative method to compute fair principal components minimizing max group-wise reconstruction error.
result Preserves the containment property of standard PCA and reduces to standard PCA for single-group data.
This is a detailed tutorial paper which explains the Principal Component Analysis (PCA), Supervised PCA (SPCA), kernel PCA, and kernel SPCA. We start with projection, PCA with eigen-decomposition, PCA with one and multiple projection directions, properties of the projection matrix, reconstruction error minimization, an…
Revisits PCA with new formulations and insights.
problem Improving PCA formulations and understanding.
method Difference-of-convex (DC) framework, kernelizability, out-of-sample applicability, simultaneous iteration, DCA perspective.
result PCA-like problems are kernelizable and have new optimization perspectives.
Proposes -PCA to learn identifiable linear transformations without whitening.
problem Cannot identify axes with equal variances in PCA.
method Unified model for linear and nonlinear PCA, introducing a missing piece to eliminate rotational indeterminacy.
result Eliminates subspace rotational indeterminacy in PCA.
Solution to sparse PCA tuning problem using Empirical Bayes.
problem Sparse PCA multiple tuning problem (MTP).
method Empirical Bayes covariance decomposition for penalized PCA.
result Empirical Bayes approach efficiently solves MTP in sparse PCA.
PCA++ improves robustness to background noise in contrastive learning.
problem Recovering shared signal subspaces from positive pairs in high-dimensional data with structured background noise.
method PCA++ uses hard uniformity-constrained contrastive learning to enforce identity covariance on projected features.
result PCA++ outperforms standard PCA and alignment-only PCA+ in simulations and real-world datasets.
EB-PCA reduces noise in high-dimensional PCA by estimating a joint prior distribution.
problem High-dimensional PCA noise in samples comparable to or larger than data.
method Empirical Bayes PCA using Kiefer-Wolfowitz MLE, random matrix theory, and AMP algorithm.
result EB-PCA achieves Bayes-optimal accuracy in spiked models and significantly improves over PCA in simulations and real data.
New analysis improves black-box -PCA algorithms, reducing parameter loss.
problem Designing efficient -PCA algorithms with black-box access to a 1-PCA oracle.
method Black-box deflation methods, analyzing ePCA and cPCA approximations.
result Deflation methods suffer no asymptotic parameter loss for -cPCA in feasible regimes.
Derives kernel PCA with Nyström method for scalability.
problem Scalability of kernel PCA.
method Nyström method for kernel PCA.
result Provides scalable alternative to full kernel PCA.
We develop efficient algorithms for robust PCA that handle outliers.
problem Finding principal components in datasets with outliers.
method Nearly-linear time and streaming algorithms for robust PCA.
result Near-optimal error guarantees for robust PCA with nearly-linear time and memory usage.
Principal Component Analysis (PCA) has wide applications in machine learning, text mining and computer vision. Classical PCA based on a Gaussian noise model is fragile to noise of large magnitude. Laplace noise assumption based PCA methods cannot deal with dense noise effectively. In this paper, we propose Cauchy Princ…
A new PCA method using T-norm outperforms existing methods.
problem Outliers and noise sensitivity in classical PCA.
method PCA based on T-norm maximization.
result The method outperforms PCA-, SPCA, and PCA in numerical experiments.
New fair PCA method using streaming algorithms with statistical guarantees.
problem Perform PCA while ensuring fair projected distributions.
method Formulated new notion PAFO learnability, proposed fair noisy power method (FNPM) for memory efficiency.
result First statistical guarantee for fair PCA in streaming setting.
PCA-Guided Quantile Sampling preserves data structure in large datasets.
problem Preserving data structure in large-scale subsampling.
method PCA QS uses leading principal components to guide stratified sampling.
result PCA QS maintains statistical and geometric structure without distorting semantics.
A method to remove mean-shift noise from PCA using knockoffs.
problem High sensitivity of PCA to mean-shift contamination in high-dimensional data.
method Introducing knockoff mean-shift perturbation to separate and remove mean-shift components from PCA.
result The mean-shift spikes are spectrally separable from stable eigenvalues, allowing for robust PCA.
AgFlow: Fast Model Selection of Penalized PCA via Implicit Regularization Effects of Gradient Flowcs.LG
AgFlow speeds up model selection in penalized PCA.
problem Efficient model selection in penalized PCA for HDLSS settings.
method Implicit regularization effect of gradient flow to reduce computation complexity.
result AgFlow achieves the complete solution path of L2-penalized PCA.
One technique to visualize the training of neural networks is to perform PCA on the parameters over the course of training and to project to the subspace spanned by the first few PCA components. In this paper we compare this technique to the PCA of a high dimensional random walk. We compute the eigenvalues and eigenvec…
GT-PCA improves PCA for image and time series data.
problem Lack of robustness to transformations in PCA.
method GT-PCA is a neural network that estimates components invariant to specific transformations.
result GT-PCA outperforms alternative methods in synthetic and real data experiments.
Modified PCA algorithm with continual learning preserves features of previous modes for multimode process monitoring.
problem Catastrophic forgetting of previous modes in monitoring models for successive modes.
method Modified PCA algorithm with elastic weight consolidation (EWC) to preserve features of previous modes.
result PCA-EWC algorithm effectively monitors multimode processes without performance decrease.
PCA-Net combines PCA and neural networks for operator approximation, with new bounds on complexity.
problem Developing approximation theory for PCA-Net architecture.
method Combines PCA and neural networks, derives universal approximation results and lower bounds on complexity.
result PCA-Net can overcome the curse of parametric complexity for specific operators.
A new dynamical formulation of log-PCA captures local principal modes of geodesic variations.
problem Learning principal variations of random probability measures under Wasserstein geometry.
method Introducing a new dynamical formulation of log-PCA as a variational approach.
result Deriving a general statistical convergence rate for empirical WT-PCA.
We performed an empirical comparison of ICA and PCA algorithms by applying them on two simulated noisy time series with varying distribution parameters and level of noise. In general, ICA shows better results than PCA because it takes into account higher moments of data distribution. On the other hand, PCA remains quit…
Generative Kernel PCA explores latent spaces for data interpretation and novelty detection.
problem Exploring latent spaces of datasets for better data interpretation.
method Generative Kernel PCA using hidden and visible units similar to Restricted Boltzmann Machines.
result Gradually moving in the latent space allows for interpretation of components and detection of novel patterns.
Principal Component Analysis (PCA) has been used to study the pathogenesis of diseases. To enhance the interpretability of classical PCA, various improved PCA methods have been proposed to date. Among these, a typical method is the so-called sparse PCA, which focuses on seeking sparse loadings. However, the performance…
A fair PCA method using JEVD ensures balanced data representation.
problem PCA's bias in data with demographic characteristics.
method Joint Eigenvalue Decomposition (JEVD) for fair PCA.
result JEVD optimally balances fairness and PCA's data structure.
Two new PCA variants improve financial data analysis.
problem Numerical instability and nonstationarity in PCA for finance.
method Iterated and exponentially weighted moving PCA variants using Ogita-Aishima iteration.
result Improved stability and adaptability in financial data analysis.
One-shot algorithm for feature-distributed kernel PCA reduces communication costs.
problem Efficiently perform kernel PCA in distributed computing environments.
method Inspired by dual relationship between sample-distributed and feature-distributed scenarios, proposes a one-shot algorithm for feature-distributed kernel PCA.
result The algorithm provides high-quality results with low communication costs, especially when eigenvalues decay fast.
EigenGame reinterprets PCA as a game to find eigenvectors.
problem Finding principal components efficiently and accurately.
method EigenGame treats PCA as a Nash equilibrium game, using gradient-based updates.
result EigenGame algorithm combines Oja's rule and Gram-Schmidt orthogonalization.
CA-PCA improves manifold dimension estimation by accounting for curvature.
problem Estimating the dimension of manifolds in high-dimensional data.
method Develops CA-PCA, a local PCA method calibrated with a quadratic embedding to account for curvature.
result Improves manifold dimension estimation in various settings.