GCN and GPCA are mathematically connected, leading to improved node classification performance.
problem Improving node classification performance in semi-supervised settings.
method Established a mathematical connection between GCN and GPCA, demonstrating their equivalence and using this to design an effective initialization strategy.
result GPCA paired with a simple MLP achieves similar or better performance than GCN on semi-supervised node classification tasks.
Proposes a method for multi-view clustering that integrates consistent and complementary graph regularizers.
problem Multi-view clustering where views have both consistent and complementary information.
method Consistent and complementary graph-regularized multi-view subspace clustering (GRMSC).
result The proposed method outperforms state-of-the-art methods on benchmark datasets.
Learning the "blocking" structure is a central challenge for high dimensional data (e.g., gene expression data). Recently, a sparse singular value decomposition (SVD) has been used as a biclustering tool to achieve this goal. However, this model ignores the structural information between variables (e.g., gene interacti…
Sparse Tucker decomposition with graph regularization improves time series forecasting accuracy.
problem High-dimensional time series forecasting with over-parameterization issue.
method Sparse Tucker decomposition and graph regularization for tensor-based model.
result Non-asymptotic error bound and superior performance in numerical experiments.
Ranked data appear in many different applications, including voting and consumer surveys. There often exhibits a situation in which data are partially ranked. Partially ranked data is thought of as missing data. This paper addresses parameter estimation for partially ranked data under a (possibly) non-ignorable missing…
Graph regularized autoencoder improves anomaly detection performance.
problem Unsupervised anomaly detection in high-dimensional data.
method Developed a graph regularized autoencoder using MST-based distances.
result Outperforms alternative methods on 20 benchmark anomaly detection datasets.
GRTR framework uses graph regularization to improve financial forecasting.
problem High computational costs and economic domain knowledge loss in tensor models.
method Graph-Regularized Tensor Regression (GRTR) framework incorporating economic domain knowledge.
result Improved performance in multi-way financial forecasting with reduced computational costs.
This paper explores the recently proposed Graph Convolutional Network architecture proposed in (Kipf & Welling, 2016) The key points of their work is summarized and their results are reproduced. Graph regularization and alternative graph convolution approaches are explored. I find that explicit graph regularization was…
A new method improves graph-based learning for high-dimensional data.
problem Inconsistent high-dimensional learning efficiency of semi-supervised graph regularization.
method Introducing a novel regularization approach involving centering operation.
result Empirical results show improved performance over spectral clustering.
Computer-aided breast cancer diagnosis in mammography is limited by inadequate data and the similarity between benign and cancerous masses. To address this, we propose a signed graph regularized deep neural network with adversarial augmentation, named \textsc{DiagNet}. Firstly, we use adversarial learning to generate p…
Learning image representations to capture fine-grained semantics has been a challenging and important task enabling many applications such as image search and clustering. In this paper, we present Graph-Regularized Image Semantic Embedding (Graph-RISE), a large-scale neural graph learning framework that allows us to tr…
New method clusters multi-view data by squeezing hybrid knowledge.
problem Removal of redundant information and fusion of multi-view features.
method Low-rank subspace multi-view clustering with adaptive graph regularization.
result Our method outperforms state-of-the-art algorithms on multi-view benchmarks.
New view: Deep GCNs learn to anti-oversmooth during training.
problem Performance drop in deep GCNs due to oversmoothing.
method Interpreted GCN as MLP + graph regularization, analyzed training process.
result Deep GCNs learn to anti-oversmooth during training, not over-smooth.
Unified taxonomy for graph representation learning.
problem Lack of unified understanding and integration of graph representation learning methods.
method Proposes a Graph Encoder Decoder Model (GRAPHEDM) to unify graph neural networks, network embedding, and graph regularization.
result Unified taxonomy and Graph Encoder Decoder Model (GRAPHEDM) for graph representation learning.
Multi-task learning is a type of transfer learning that trains multiple tasks simultaneously and leverages the shared information between related tasks to improve the generalization performance. However, missing features in the input matrix is a much more difficult problem which needs to be carefully addressed. Removin…
We propose methods for distributed graph-based multi-task learning that are based on weighted averaging of messages from other machines. Uniform averaging or diminishing stepsize in these methods would yield consensus (single task) learning. We show how simply skewing the averaging weights or controlling the stepsize a…
DeepVir uses deep matrix factorization to predict antivirals for COVID-19.
problem Predicting effective antivirals for COVID-19 using known drug-virus associations.
method Graphical deep matrix factorization with HyPALM optimization.
result DeepVir outperforms state-of-the-art techniques in predicting antivirals for COVID-19.
Simplifies fair PCA with fast, efficient solution.
problem Learning fair low-rank approximations of data.
method Conceptually simple approach with analytic solution.
result Faster and similar results to existing fair PCA methods.
KAN-PCA improves asset return analysis by capturing more variance than classical PCA during market crises.
problem Inefficient classical PCA during market crises when correlations between assets change dramatically.
method KAN-PCA uses KAN (Kolmogorov-Arnold Networks) with B-spline functions to learn nonlinear projections.
result KAN-PCA achieves a higher reconstruction R^2 (66.57%) compared to classical PCA (62.99%) on 20 S&P 500 stocks.
New algorithm solves fair PCA, robust PCA, and sparse PCA problems efficiently.
problem Fair Principal Component Analysis (FPCA) to ensure fairness in PCA solutions.
method Iterative MM algorithm with SDP reformulation to quadratic program.
result Algorithm monotonically improves fairness objectives at each iteration.
Unified framework improves PCA for outliers and distributed data.
problem Outliers and limitations in PCA for large-scale applications.
method φ-PCA framework that retains PCA efficiency and adds robustness.
result HM-PCA achieves optimal robustness and efficiency.
TL-PCA uses transfer learning to improve PCA performance with limited target data.
problem PCA performance is limited with scarce target data.
method Transfer learning approach to PCA (TL-PCA) that combines source task knowledge with target task data.
result Improved PCA representation for dimensionality reduction with limited target data.
Bucketed PCA-NN outperforms DNNs by 96% on MNIST.
problem Benchmarking deep neural networks for supervised classification.
method Applies PCA to individual buckets constructed in two phases, retains neural network architecture, and uses neurons that mirror input signals.
result Bucketed PCA-NN achieves 96% accuracy on MNIST, similar to DNNs.
A new low-dimensional parameterization based on principal component analysis (PCA) and convolutional neural networks (CNN) is developed to represent complex geological models. The CNN-PCA method is inspired by recent developments in computer vision using deep learning. CNN-PCA can be viewed as a generalization of an ex…
Anchor PCA improves robustness in multi-domain PCA.
problem PCA on pooled data can focus on spurious directions.
method Anchor PCA focuses on shared directions of variation.
result Anchor PCA outperforms pooling and worst-case alternatives.
A new method for fair PCA ensures balanced error across groups.
problem Balancing approximation error across different groups in multi-group data.
method Iterative method to compute fair principal components minimizing max group-wise reconstruction error.
result Preserves the containment property of standard PCA and reduces to standard PCA for single-group data.
This is a detailed tutorial paper which explains the Principal Component Analysis (PCA), Supervised PCA (SPCA), kernel PCA, and kernel SPCA. We start with projection, PCA with eigen-decomposition, PCA with one and multiple projection directions, properties of the projection matrix, reconstruction error minimization, an…
Revisits PCA with new formulations and insights.
problem Improving PCA formulations and understanding.
method Difference-of-convex (DC) framework, kernelizability, out-of-sample applicability, simultaneous iteration, DCA perspective.
result PCA-like problems are kernelizable and have new optimization perspectives.
Proposes σ-PCA to learn identifiable linear transformations without whitening.
problem Cannot identify axes with equal variances in PCA.
method Unified model for linear and nonlinear PCA, introducing a missing piece to eliminate rotational indeterminacy.
result Eliminates subspace rotational indeterminacy in PCA.
Canonical correlation analysis (CCA) is a powerful technique for discovering whether or not hidden sources are commonly present in two (or more) datasets. Its well-appreciated merits include dimensionality reduction, clustering, classification, feature selection, and data fusion. The standard CCA however, does not expl…
Solution to sparse PCA tuning problem using Empirical Bayes.
problem Sparse PCA multiple tuning problem (MTP).
method Empirical Bayes covariance decomposition for penalized PCA.
result Empirical Bayes approach efficiently solves MTP in sparse PCA.
New method for efficient graph learning on large graphs.
problem High memory complexity for graph learning on large, non-sparse graphs.
method Approximate large graphs as intersecting communities, using a new graph regularity lemma.
result Efficient graph learning algorithm with linear memory and time complexity.
PCA++ improves robustness to background noise in contrastive learning.
problem Recovering shared signal subspaces from positive pairs in high-dimensional data with structured background noise.
method PCA++ uses hard uniformity-constrained contrastive learning to enforce identity covariance on projected features.
result PCA++ outperforms standard PCA and alignment-only PCA+ in simulations and real-world datasets.
EB-PCA reduces noise in high-dimensional PCA by estimating a joint prior distribution.
problem High-dimensional PCA noise in samples comparable to or larger than data.
method Empirical Bayes PCA using Kiefer-Wolfowitz MLE, random matrix theory, and AMP algorithm.
result EB-PCA achieves Bayes-optimal accuracy in spiked models and significantly improves over PCA in simulations and real data.
New analysis improves black-box k-PCA algorithms, reducing parameter loss.
problem Designing efficient k-PCA algorithms with black-box access to a 1-PCA oracle. method Black-box deflation methods, analyzing ePCA and cPCA approximations.
result Deflation methods suffer no asymptotic parameter loss for k-cPCA in feasible regimes. Derives kernel PCA with Nyström method for scalability.
problem Scalability of kernel PCA.
method Nyström method for kernel PCA.
result Provides scalable alternative to full kernel PCA.
We develop efficient algorithms for robust PCA that handle outliers.
problem Finding principal components in datasets with outliers.
method Nearly-linear time and streaming algorithms for robust PCA.
result Near-optimal error guarantees for robust PCA with nearly-linear time and memory usage.
Principal Component Analysis (PCA) has wide applications in machine learning, text mining and computer vision. Classical PCA based on a Gaussian noise model is fragile to noise of large magnitude. Laplace noise assumption based PCA methods cannot deal with dense noise effectively. In this paper, we propose Cauchy Princ…
A new PCA method using Tℓ1-norm outperforms existing methods.
problem Outliers and noise sensitivity in classical PCA.
method PCA based on Tℓ1-norm maximization. result The method outperforms PCA-ℓp, ℓpSPCA, and PCA in numerical experiments. New fair PCA method using streaming algorithms with statistical guarantees.
problem Perform PCA while ensuring fair projected distributions.
method Formulated new notion PAFO learnability, proposed fair noisy power method (FNPM) for memory efficiency.
result First statistical guarantee for fair PCA in streaming setting.
PCA-Guided Quantile Sampling preserves data structure in large datasets.
problem Preserving data structure in large-scale subsampling.
method PCA QS uses leading principal components to guide stratified sampling.
result PCA QS maintains statistical and geometric structure without distorting semantics.
A method to remove mean-shift noise from PCA using knockoffs.
problem High sensitivity of PCA to mean-shift contamination in high-dimensional data.
method Introducing knockoff mean-shift perturbation to separate and remove mean-shift components from PCA.
result The mean-shift spikes are spectrally separable from stable eigenvalues, allowing for robust PCA.
AgFlow speeds up model selection in penalized PCA.
problem Efficient model selection in penalized PCA for HDLSS settings.
method Implicit regularization effect of gradient flow to reduce computation complexity.
result AgFlow achieves the complete solution path of L2-penalized PCA.
One technique to visualize the training of neural networks is to perform PCA on the parameters over the course of training and to project to the subspace spanned by the first few PCA components. In this paper we compare this technique to the PCA of a high dimensional random walk. We compute the eigenvalues and eigenvec…
GT-PCA improves PCA for image and time series data.
problem Lack of robustness to transformations in PCA.
method GT-PCA is a neural network that estimates components invariant to specific transformations.
result GT-PCA outperforms alternative methods in synthetic and real data experiments.
Modified PCA algorithm with continual learning preserves features of previous modes for multimode process monitoring.
problem Catastrophic forgetting of previous modes in monitoring models for successive modes.
method Modified PCA algorithm with elastic weight consolidation (EWC) to preserve features of previous modes.
result PCA-EWC algorithm effectively monitors multimode processes without performance decrease.
PCA-Net combines PCA and neural networks for operator approximation, with new bounds on complexity.
problem Developing approximation theory for PCA-Net architecture.
method Combines PCA and neural networks, derives universal approximation results and lower bounds on complexity.
result PCA-Net can overcome the curse of parametric complexity for specific operators.
A new dynamical formulation of log-PCA captures local principal modes of geodesic variations.
problem Learning principal variations of random probability measures under Wasserstein geometry.
method Introducing a new dynamical formulation of log-PCA as a variational approach.
result Deriving a general statistical convergence rate for empirical WT-PCA.