arXiv research

Paper develops efficient AltMin algorithm for SRPCP robust matrix recovery.

Recently, the principal component pursuit has received increasing attention in signal processing research ranging from source separation to video surveillance. So far, all existing formulations are real-valued and lack the concept of phase, which is inherent in inputs such as complex spectrograms or color images. Thus,…

Informed by recent work on tensor singular value decomposition and circulant algebra matrices, this paper presents a new theoretical bridge that unifies the hypercomplex and tensor-based approaches to singular value decomposition and robust principal component analysis. We begin our work by extending the principal comp…

We introduce a new convex formulation for stable principal component pursuit (SPCP) to decompose noisy signals into low-rank and sparse representations. For numerical solutions of our SPCP formulation, we first develop a convex variational framework and then accelerate it with quasi-Newton methods. We show, via synthet…

In this paper we study the shape space of curves with values in a homogeneous space $M = G/K$, where $G$ is a Lie group and $K$ is a compact Lie subgroup. We generalize the square root velocity framework to obtain a reparametrization invariant metric on the space of curves in $M$. By identifying curves in $M$ with thei…

In this paper, auto-associative models are proposed as candidates to the generalization of Principal Component Analysis. We show that these models are dedicated to the approximation of the dataset by a manifold. Here, the word "manifold" refers to the topology properties of the structure. The approximating manifold is …

We extend the theoretical analysis of a recently proposed single subspace learning algorithm, called Dual Principal Component Pursuit (DPCP), to the case where the data are drawn from of a union of hyperplanes. To gain insight into the properties of the $\ell_1$ non-convex problem associated with DPCP, we develop a geo…

Deep learning is a form of machine learning for nonlinear high dimensional pattern matching and prediction. By taking a Bayesian probabilistic perspective, we provide a number of insights into more efficient algorithms for optimisation and hyper-parameter tuning. Traditional high-dimensional data reduction techniques, …

Non-Gaussian component analysis (NGCA) is an unsupervised linear dimension reduction method that extracts low-dimensional non-Gaussian "signals" from high-dimensional data contaminated with Gaussian noise. NGCA can be regarded as a generalization of projection pursuit (PP) and independent component analysis (ICA) to mu…

Clustering of data sets is a standard problem in many areas of science and engineering. The method of spectral clustering is based on embedding the data set using a kernel function, and using the top eigenvectors of the normalized Laplacian to recover the connected components. We study the performance of spectral clust…

R package spca computes sparse principal components efficiently.

This paper presents a remarkably simple, yet powerful, algorithm termed Coherence Pursuit (CoP) to robust Principal Component Analysis (PCA). As inliers lie in a low dimensional subspace and are mostly correlated, an inlier is likely to have strong mutual coherence with a large number of data points. By contrast, outli…

A new robust PCA estimator combining M-estimators and minimum divergence estimators.

Principal Component Analysis (PCA) is a very successful dimensionality reduction technique, widely used in predictive modeling. A key factor in its widespread use in this domain is the fact that the projection of a dataset onto its first $K$ principal components minimizes the sum of squared errors between the original …

The autoencoder is an effective unsupervised learning model which is widely used in deep learning. It is well known that an autoencoder with a single fully-connected hidden layer, a linear activation function and a squared error cost function trains weights that span the same subspace as the one spanned by the principa…

Principal component analysis (PCA) is often used to reduce the dimension of data by selecting a few orthonormal vectors that explain most of the variance structure of the data. L1 PCA uses the L1 norm to measure error, whereas the conventional PCA uses the L2 norm. For the L1 PCA problem minimizing the fitting error of…

Decomposing market impact into diffusive components

Study differential properties of matrix square roots in specific cases.

New method differentiates square-root Kalman filters robustly.

This study improves estimation of the first principal component in multivariate functional data.

A new method simulates square-root processes efficiently.

We compare the risk of ridge regression to a simple variant of ordinary least squares, in which one simply projects the data onto a finite dimensional subspace (as specified by a Principal Component Analysis) and then performs an ordinary (un-regularized) least squares regression in this subspace. This note shows that …

In many physical, statistical, biological and other investigations it is desirable to approximate a system of points by objects of lower dimension and/or complexity. For this purpose, Karl Pearson invented principal component analysis in 1901 and found 'lines and planes of closest fit to system of points'. The famous k…

Develops a new tensor PCA method for analyzing multiple network data.

Guarantees uniform convergence for square-root Lipschitz losses.

Federated learning improves SPCA for sparse components.

Paper forecasts recession indicators using yield spread models.

It is well known that Principal Component Analysis (PCA) is strongly affected by outliers and a lot of effort has been put into robustification of PCA. In this paper we present a new algorithm for robust PCA minimizing the trimmed reconstruction error. By directly minimizing over the Stiefel manifold, we avoid deflatio…

Study finds price impact follows a 'double' square-root law, suggesting mechanical origin.

Denise learns a function to quickly decompose covariance matrices robustly.

Model identifies order splitting and liquidity replenishment as necessary for the square-root law of market impact.

Heterogeneity is often natural in many contemporary applications involving massive data. While posing new challenges to effective learning, it can play a crucial role in powering meaningful scientific discoveries through the understanding of important differences among subpopulations of interest. In this paper, we expl…

Through the direct study of the analysis estimator we derive oracle inequalities with fast and slow rates by adapting the arguments involving projections by Dalalyan, Hebiri and Lederer (2017). We then extend the theory to the square root analysis estimator. Finally, we focus on (square root) total variation regularize…

The literature provides strong evidence that stock prices can be predicted from past price data. Principal component analysis (PCA) is a widely used mathematical technique for dimensionality reduction and analysis of data by identifying a small number of principal components to explain the variation found in a data set…

Unified geometric interpretation of statistical estimation inequalities.

We prove that in Riemannian manifolds the $k$-th Steklov eigenvalue on a domain and the square root of the $k$-th Laplacian eigenvalue on its boundary can be mutually controlled in terms of the maximum principal curvature of the boundary under sectional curvature conditions. As an application, we derive a Weyl-type upp…

Recovering matrices from compressive and grossly corrupted observations is a fundamental problem in robust statistics, with rich applications in computer vision and machine learning. In theory, under certain conditions, this problem can be solved in polynomial time via a natural convex relaxation, known as Compressive …

Root Laplacian Eigenmaps help in spectral embedding of graphs.

Motivated by the Bagging Partial Least Squares (PLS) and Principal Component Analysis (PCA) algorithms, we propose a Principal Model Analysis (PMA) method in this paper. In the proposed PMA algorithm, the PCA and the PLS are combined. In the method, multiple PLS models are trained on sub-training sets, derived from the…

A new method solves large-scale sparse group square-root Lasso problems efficiently.

Singular Value Decomposition (and Principal Component Analysis) is one of the most widely used techniques for dimensionality reduction: successful and efficiently computable, it is nevertheless plagued by a well-known, well-documented sensitivity to outliers. Recent work has considered the setting where each point has …

New algorithm rSVDdpd improves robustness and scalability for video surveillance background modeling.

Square-root natural-gradient improves variational inference convergence.

New tools in nonlinear random matrices improve understanding of the Sum of Squares hierarchy.

Many independent studies on stocks and futures contracts have established that market impact is proportional to the square-root of the executed volume. Is market impact quantitatively similar for option markets as well? In order to answer this question, we have analyzed the impact of a large proprietary data set of opt…

The notion of market impact is subtle and sometimes misinterpreted. Here we argue that impact should not be misconstrued as volatility. In particular, the so-called ``square-root impact law'', which states that impact grows as the square-root of traded volume, has nothing to do with price diffusion, i.e. that typical p…

Characterizes algebraic squares of irreducible complex spinors in various dimensions.

In this paper, we provide explicit formulas, in terms of the covariances of sample covariances or sample correlations, for the asymptotic covariances of unrotated factor loading estimates and unique variance estimates. These estimates are extracted from least square, principal, iterative principal component, alpha or i…