arXiv research

Paper bounds subspace estimator error from noisy projections.

Bayesian methods reduce variance in subspace identification for small data sets.

In this letter, we consider two sets of observations defined as subspace signals embedded in noise and we wish to analyze the distance between these two subspaces. The latter entails evaluating the angles between the subspaces, an issue reminiscent of the well-known Procrustes problem. A Bayesian approach is investigat…

This paper considers the problem of robust subspace recovery: given a set of $N$ points in $\mathbb{R}^D$, if many lie in a $d$-dimensional subspace, then can we recover the underlying subspace? We show that Tyler's M-estimator can be used to recover the underlying subspace, if the percentage of the inliers is larger t…

Unified framework for structured principal subspace estimation with bounds and rates.

Theoretical guarantees for STE, a robust subspace recovery method.

New method estimates active subspaces for jump-discontinuous functions.

We study sparse principal components analysis in high dimensions, where $p$ (the number of variables) can be much larger than $n$ (the number of observations), and analyze the problem of estimating the subspace spanned by the principal eigenvectors of the population covariance matrix. We introduce two complementary not…

A method for identifying joint and individual subspaces from multi-view data.

Proposes a new algorithm to estimate invariant subspaces across multilayer networks.

Study optimizes shared singular subspace estimation from noisy matrices.

Paper proves IRLS converges to subspace from any start, with practical benefits.

This paper analyzes AJIVE for estimating shared subspace across multiple datasets, revealing its strengths and limitations.

We study the problem of robust subspace recovery (RSR) in the presence of adversarial outliers. That is, we seek a subspace that contains a large portion of a dataset when some fraction of the data points are arbitrarily corrupted. We first examine a theoretical estimator that is intractable to calculate and use it to …

ALPCAHUS clusters data from multiple subspaces with varying noise.

In this paper, we exhibit the tradeoffs between the (training) sample, computation and storage complexity for the problem of supervised classification using signal subspace estimation. Our main tool is the use of tensor subspaces, i.e. subspaces with a Kronecker structure, for embedding the data into lower dimensions. …

Paper studies efficient function approximation in high-dimensional spaces with low-dimensional structures.

Non-Gaussian component analysis (NGCA) is aimed at identifying a linear subspace such that the projected data follows a non-Gaussian distribution. In this paper, we propose a novel NGCA algorithm based on log-density gradient estimation. Unlike existing methods, the proposed NGCA algorithm identifies the linear subspac…

We investigate a Gaussian mixture model (GMM) with component means constrained in a pre-selected subspace. Applications to classification and clustering are explored. An EM-type estimation algorithm is derived. We prove that the subspace containing the component means of a GMM with a common covariance matrix also conta…

POTD estimates SDR subspace using optimal transport for binary response.

This work presents GROUSE (Grassmanian Rank-One Update Subspace Estimation), an efficient online algorithm for tracking subspaces from highly incomplete observations. GROUSE requires only basic linear algebraic manipulations at each iteration, and each subspace update can be performed in linear time in the dimension of…

A large number of algorithms in machine learning, from principal component analysis (PCA), and its non-linear (kernel) extensions, to more recent spectral embedding and support estimation methods, rely on estimating a linear subspace from samples. In this paper we introduce a general formulation of this problem and der…

Paper connects contrastive learning to MI maximization and establishes robust methods for nonlinear ICA and subspace estimation.

ALPCAH improves PCA for noisy data samples.

Tyler's M-estimator's phase transition at DS-SNR = 1 is resolved.

New algorithm reduces complexity for SPD manifold optimization.

Paper analyzes singular subspace estimation in noisy matrix models.

This paper introduces a simple and efficient density estimator that enables fast systematic search. To show its advantage over commonly used kernel density estimator, we apply it to outlying aspects mining. Outlying aspects mining discovers feature subsets (or subspaces) that describe how a query stand out from a given…

Proposes a method to enforce nestedness in subspace learning methods.

Estimates shared linear subspace from noisy data with multiple users.

GAME improves matrix completion by considering subgroup-specific latent structures.

We consider the problem of subspace estimation in a Bayesian setting. Since we are operating in the Grassmann manifold, the usual approach which consists of minimizing the mean square error (MSE) between the true subspace $U$ and its estimate $\hat{U}$ may not be adequate as the MSE is not the natural metric in the Gra…

PCA is one of the most widely used dimension reduction techniques. A related easier problem is "subspace learning" or "subspace estimation". Given relatively clean data, both are easily solved via singular value decomposition (SVD). The problem of subspace learning or PCA in the presence of outliers is called robust su…

Extracting the underlying low-dimensional space where high-dimensional signals often reside has long been at the center of numerous algorithms in the signal processing and machine learning literature during the past few decades. At the same time, working with incomplete (partly observed) large scale datasets has recent…

Model improves covariance estimation from shared and distinct datasets.

We study the accuracy of estimating the covariance and the precision matrix of a $D$-variate sub-Gaussian distribution along a prescribed subspace or direction using the finite sample covariance. Our results show that the estimation accuracy depends almost exclusively on the components of the distribution that correspo…

Proposes an optimization framework for sparse robust subspace estimation.

We study the basic problem of robust subspace recovery. That is, we assume a data set that some of its points are sampled around a fixed subspace and the rest of them are spread in the whole ambient space, and we aim to recover the fixed underlying subspace. We first estimate "robust inverse sample covariance" by solvi…

Memory-efficient optimizers fail to track a subspace, leading to unpredictable model performance.

A new method for anomaly detection using random subspaces and Gaussian mixture models.

We consider the problem of subspace clustering: given points that lie on or near the union of many low-dimensional linear subspaces, recover the subspaces. To this end, one first identifies sets of points close to the same subspace and uses the sets to estimate the subspaces. As the geometric structure of the clusters …

Investigates projections onto explicit subspaces and their variance effects.

Paper proposes estimators for sparse PCA with oracle property.

In recent years, active subspace methods (ASMs) have become a popular means of performing subspace sensitivity analysis on black-box functions. Naively applied, however, ASMs require gradient evaluations of the target function. In the event of noisy, expensive, or stochastic simulators, evaluating gradients via finite …

This paper deals with robust regression and subspace estimation and more precisely with the problem of minimizing a saturated loss function. In particular, we focus on computational complexity issues and show that an exact algorithm with polynomial time-complexity with respect to the number of data can be devised for r…

The Nearest subspace classifier (NSS) finds an estimation of the underlying subspace within each class and assigns data points to the class that corresponds to its nearest subspace. This paper mainly studies how well NSS can be generalized to new samples. It is proved that NSS is strongly consistent under certain assum…

Networked sensing, where the goal is to perform complex inference using a large number of inexpensive and decentralized sensors, has become an increasingly attractive research topic due to its applications in wireless sensor networks and internet-of-things. To reduce the communication, sensing and storage complexity, t…

When data is sampled from an unknown subspace, principal component analysis (PCA) provides an effective way to estimate the subspace and hence reduce the dimension of the data. At the heart of PCA is the Eckart-Young-Mirsky theorem, which characterizes the best rank k approximation of a matrix. In this paper, we prove …