arXiv research

A fast algorithm for L1-norm kernel PCA with convergence analysis.

Principal component analysis (PCA) is often used to reduce the dimension of data by selecting a few orthonormal vectors that explain most of the variance structure of the data. L1 PCA uses the L1 norm to measure error, whereas the conventional PCA uses the L2 norm. For the L1 PCA problem minimizing the fitting error of…

New method solves rank-1 L1-norm TUCKER2 decomposition efficiently.

A typical approach in estimating the learning rate of a regularized learning scheme is to bound the approximation error by the sum of the sampling error, the hypothesis error and the regularization error. Using a reproducing kernel space that satisfies the linear representer theorem brings the advantage of discarding t…

Targeting at sparse learning, we construct Banach spaces B of functions on an input space X with the properties that (1) B possesses an l1 norm in the sense that it is isometrically isomorphic to the Banach space of integrable functions on X with respect to the counting measure; (2) point evaluations are continuous lin…

For supervised and unsupervised learning, positive definite kernels allow to use large and potentially infinite dimensional feature spaces with a computational cost that only depends on the number of observations. This is usually done through the penalization of predictor functions by Euclidean or Hilbertian norms. In …

Derives kernel PCA with Nyström method for scalability.

Explains various PCA and SPCA methods with theory and applications.

QAPCA uses quantum annealing for robust PCA.

Generative Kernel PCA explores latent spaces for data interpretation and novelty detection.

One-shot algorithm for feature-distributed kernel PCA reduces communication costs.

Simplifies fair PCA with fast, efficient solution.

SPCA improves PCA by learning from simple to complex samples.

A new method generalizing subspace learning for improved classification.

Efficiently performs PCA on large datasets using Nyström sampling.

Kernel method is a very powerful tool in machine learning. The trick of kernel has been effectively and extensively applied in many areas of machine learning, such as support vector machine (SVM) and kernel principal component analysis (kernel PCA). Kernel trick is to define a kernel function which relies on the inner-…

Kernel PCA helps analyze multivariate extremes and clusters them effectively.

We describe how cross-kernel matrices, that is, kernel matrices between the data and a custom chosen set of `feature spanning points' can be used for learning. The main potential of cross-kernels lies in the fact that (a) only one side of the matrix scales with the number of data points, and (b) cross-kernels, as oppos…

A new algorithm for faster incremental kernel PCA and Nyström approximation.

New bounds on KPCA efficiency reveal conditions for fast convergence.

New algorithm reduces kernel PCA to $O(\sqrt{n})$ features for streaming data.

In this on-going work, I explore certain theoretical and empirical implications of data transformations under the PCA. In particular, I state and prove three theorems about PCA, which I paraphrase as follows: 1). PCA without discarding eigenvector rows is injective, but looses this injectivity when eigenvector rows are…

New PCA method detects faults using occupation kernels.

DKPCA improves WSD accuracy with scarce labeled data.

Revisits PCA with new formulations and insights.

KPCA improves OoD detection by separating InD and OoD data.

Paper proposes tensor sparse PCA for improved face recognition accuracy.

We propose a data aggregation-based algorithm with monotonic convergence to a global optimum for a generalized version of the L1-norm error fitting model with an assumption of the fitting function. The proposed algorithm generalizes the recent algorithm in the literature, aggregate and iterative disaggregate (AID), whi…

Generalizes PCA to maximize any convex function of components.

This work analyzes when contrastive models are close to PCA or kernel methods.

Enhances KLR for indefinite kernels with $L_1$-norm regularization.

Kernel PCA explains self-attention mechanisms in deep learning models.

Paper develops a dual formulation for PCA in Hilbert spaces.

New method identifies key features from unlabeled data, improving classification.

This paper improves fairness in PCA and kernel PCA.

New method uses sparse kernel PCA for better outlier detection.

Explains Fisher and Kernel Fisher Discriminant Analysis with examples and comparisons.

Proposes a new method for hyperspectral image dimensionality reduction.

NCA improves fault detection in nonlinear processes.

Correntropy is a second order statistical measure in kernel space, which has been successfully applied in robust learning and signal processing. In this paper, we define a nonsecond order statistical measure in kernel space, called the kernel mean-p power error (KMPE), including the correntropic loss (CLoss) as a speci…

Study spectral properties of radial kernels for high-dimensional mixtures.

The paper studies convergence of kernel autocovariance operators for stationary processes.

The problem of estimating the kernel mean in a reproducing kernel Hilbert space (RKHS) is central to kernel methods in that it is used by classical approaches (e.g., when centering a kernel PCA matrix), and it also forms the core inference step of modern kernel methods (e.g., kernel-based non-parametric tests) that rel…

We propose and analyze a novel framework for learning sparse representations, based on two statistical techniques: kernel smoothing and marginal regression. The proposed approach provides a flexible framework for incorporating feature similarity or temporal information present in data sets, via non-parametric kernel sm…

In this paper, Kernel PCA is reinterpreted as the solution to a convex optimization problem. Actually, there is a constrained convex problem for each principal component, so that the constraints guarantee that the principal component is indeed a solution, and not a mere saddle point. Although these insights do not impl…

The paper presents a method to reduce computational and storage costs in PCA and spectral clustering.

Novel L1-norm and L2-norm LDA methods improve discriminant analysis.

BALSON optimizes parameters with Bayesian approach and Dirichlet distribution.