arXiv research

Study optimizes shared singular subspace estimation from noisy matrices.

Paper analyzes singular subspace estimation in noisy matrix models.

In this letter, we consider two sets of observations defined as subspace signals embedded in noise and we wish to analyze the distance between these two subspaces. The latter entails evaluating the angles between the subspaces, an issue reminiscent of the well-known Procrustes problem. A Bayesian approach is investigat…

Low-rank matrix regression refers to the instances of recovering a low-rank matrix based on specially designed measurements and the corresponding noisy outcomes. In the last decade, numerous statistical methodologies have been developed for efficiently recovering the unknown low-rank matrices. However, in some applicat…

The higher order singular value decomposition (HOSVD) of tensors is a generalization of matrix SVD. The perturbation analysis of HOSVD under random noise is more delicate than its matrix counterpart. Recently, polynomial time algorithms have been proposed where statistically optimal estimates of the singular subspaces …

Study analyzes perturbations in singular subspaces under random noise.

New method estimates high-dimensional GoM models efficiently.

The Davis-Kahan-Wedin $\sin Θ$ theorem describes how the singular subspaces of a matrix change when subjected to a small perturbation. This classic result is sharp in the worst case scenario. In this paper, we prove a stochastic version of the Davis-Kahan-Wedin $\sin Θ$ theorem when the perturbation is a Gaussian rando…

A new method for anomaly detection using random subspaces and Gaussian mixture models.

PCA is one of the most widely used dimension reduction techniques. A related easier problem is "subspace learning" or "subspace estimation". Given relatively clean data, both are easily solved via singular value decomposition (SVD). The problem of subspace learning or PCA in the presence of outliers is called robust su…

Novel tensor perturbation bounds for orthogonal iteration methods.

Efficient algorithms for low-rank bandits using subspace recovery.

GROUSE (Grassmannian Rank-One Update Subspace Estimation) is an incremental algorithm for identifying a subspace of Rn from a sequence of vectors in this subspace, where only a subset of components of each vector is revealed at each iteration. Recent analysis has shown that GROUSE converges locally at an expected linea…

We show that all closed $2$-dimensional singularities for higher codimension mean curvature flow that cannot be perturbed away have uniform entropy bounds and lie in a linear subspace of small dimension. The entropy and dimension of the subspace are both $\leq C\,(1+γ)$ for some universal constant $C$ and genus $γ$. Th…

The paper extends hypothesis testing to non-diagonalizable matrices, improving network statistics inference.

New spectral methods improve matrix estimation in RL with low-rank structure.

Transfer knowledge from multiple sources to improve matrix completion.

New method corrects Laplace/BIC errors in singular models, revealing effective dimension.

This paper is on the normal approximation of singular subspaces when the noise matrix has i.i.d. entries. Our contributions are three-fold. First, we derive an explicit representation formula of the empirical spectral projectors. The formula is neat and holds for deterministic matrix perturbations. Second, we calculate…

Paper bounds subspace estimator error from noisy projections.

A general framework for principal component analysis (PCA) in the presence of heteroskedastic noise is introduced. We propose an algorithm called HeteroPCA, which involves iteratively imputing the diagonal entries of the sample covariance matrix to remove estimation bias due to heteroskedasticity. This procedure is com…

Fast and accurate methods for low-rank learning problems.

Bayesian methods reduce variance in subspace identification for small data sets.

The paper analyzes PLS-SVD in high-dimensional data integration, revealing its strengths and limitations.

The paper updates SVD of evolving matrices using projection techniques.

This paper considers the problem of robust subspace recovery: given a set of $N$ points in $\mathbb{R}^D$, if many lie in a $d$-dimensional subspace, then can we recover the underlying subspace? We show that Tyler's M-estimator can be used to recover the underlying subspace, if the percentage of the inliers is larger t…

Unified framework for structured principal subspace estimation with bounds and rates.

We investigate Riemannian (non-Kahler) Ricci flow solutions that develop finite-time Type-I singularities and present evidence in favor of a conjecture that parabolic rescalings at the singularities converge to singularity models that are shrinking Kahler-Ricci solitons. Specifically, the singularity model for these so…

Theoretical guarantees for STE, a robust subspace recovery method.

A new method learns outcome-aware spectral features for causal effect estimation.

New method estimates active subspaces for jump-discontinuous functions.

In this paper, we study the adversarial robustness of subspace learning problems. Different from the assumptions made in existing work on robust subspace learning where data samples are contaminated by gross sparse outliers or small dense noises, we consider a more powerful adversary who can first observe the data matr…

Matrix rank minimization problem is in general NP-hard. The nuclear norm is used to substitute the rank function in many recent studies. Nevertheless, the nuclear norm approximation adds all singular values together and the approximation error may depend heavily on the magnitudes of singular values. This might restrict…

We study sparse principal components analysis in high dimensions, where $p$ (the number of variables) can be much larger than $n$ (the number of observations), and analyze the problem of estimating the subspace spanned by the principal eigenvectors of the population covariance matrix. We introduce two complementary not…

A method for identifying joint and individual subspaces from multi-view data.

Proposes a new algorithm to estimate invariant subspaces across multilayer networks.

Study flat manifolds' collapsed limits as flat orbifolds.

Matrix completion is a widely used technique for image inpainting and personalized recommender system, etc. In this work, we focus on accelerating the matrix completion using faster randomized singular value decomposition (rSVD). Firstly, two fast randomized algorithms (rSVD-PI and rSVD- BKI) are proposed for handling …

SMART transfers knowledge across related studies for multi-task learning.

Study vector fields and flows on singular spaces like submanifolds.

Paper proves IRLS converges to subspace from any start, with practical benefits.

Kaczmarz++ accelerates convergence for ill-conditioned systems.

This paper analyzes AJIVE for estimating shared subspace across multiple datasets, revealing its strengths and limitations.

The paper generalizes curvature bounds for submanifolds with singularities.

Paper develops inference methods for low-rank tensors without debiasing.

Autoencoders are a deep learning model for representation learning. When trained to minimize the distance between the data and its reconstruction, linear autoencoders (LAEs) learn the subspace spanned by the top principal directions but cannot learn the principal directions themselves. In this paper, we prove that $L_2…

This is a detailed tutorial paper which explains the Fisher discriminant Analysis (FDA) and kernel FDA. We start with projection and reconstruction. Then, one- and multi-dimensional FDA subspaces are covered. Scatters in two- and then multi-classes are explained in FDA. Then, we discuss on the rank of the scatters and …

Subspace segmentation assumes that data comes from the union of different subspaces and the purpose of segmentation is to partition the data into the corresponding subspace. Low-rank representation (LRR) is a classic spectral-type method for solving subspace segmentation problems, that is, one first obtains an affinity…