Study optimizes shared singular subspace estimation from noisy matrices.
problem Estimating shared singular subspaces across multiple noisy matrices.
method Low-rank matrix denoising framework with Stack-SVD and novel estimators.
result Stack-SVD achieves minimax rate-optimality for identical shared subspaces, and novel estimators for partial sharing.
Paper analyzes singular subspace estimation in noisy matrix models.
problem Estimating low-rank signals in noisy matrix data.
method Asymptotic distributional theory, extreme value theory, saddle point approximation, random matrix theory.
result Plug-in test statistic based on two-to-infinity norm has higher power for detecting structured alternatives.
In this letter, we consider two sets of observations defined as subspace signals embedded in noise and we wish to analyze the distance between these two subspaces. The latter entails evaluating the angles between the subspaces, an issue reminiscent of the well-known Procrustes problem. A Bayesian approach is investigat…
Low-rank matrix regression refers to the instances of recovering a low-rank matrix based on specially designed measurements and the corresponding noisy outcomes. In the last decade, numerous statistical methodologies have been developed for efficiently recovering the unknown low-rank matrices. However, in some applicat…
The higher order singular value decomposition (HOSVD) of tensors is a generalization of matrix SVD. The perturbation analysis of HOSVD under random noise is more delicate than its matrix counterpart. Recently, polynomial time algorithms have been proposed where statistically optimal estimates of the singular subspaces …
Study analyzes perturbations in singular subspaces under random noise.
problem Understanding singular vector and subspace changes in signal-plus-noise models.
method Generalized Davis-Kahan-Wedin theorem for any unitarily invariant norm, considering ℓ∞ and ℓ2,∞ bounds. result Fine-grained insights into singular vector and subspace perturbations, including ℓ∞ and ℓ2,∞ bounds. New method estimates high-dimensional GoM models efficiently.
problem Estimating GoM models for high-dimensional polytomous data.
method Flattening three-way quasi-tensor into a matrix, performing singular value decomposition.
result Established finite-sample error bounds for estimated parameters.
The Davis-Kahan-Wedin sinΘ theorem describes how the singular subspaces of a matrix change when subjected to a small perturbation. This classic result is sharp in the worst case scenario. In this paper, we prove a stochastic version of the Davis-Kahan-Wedin sinΘ theorem when the perturbation is a Gaussian rando…
A new method for anomaly detection using random subspaces and Gaussian mixture models.
problem Anomaly detection in high-dimensional data.
method Statistical estimation of probability density using random subspaces combined with geometric averaging.
result The method achieves competitive AUC scores and is interpretable.
PCA is one of the most widely used dimension reduction techniques. A related easier problem is "subspace learning" or "subspace estimation". Given relatively clean data, both are easily solved via singular value decomposition (SVD). The problem of subspace learning or PCA in the presence of outliers is called robust su…
Novel tensor perturbation bounds for orthogonal iteration methods.
problem Developing robust bounds for tensor reconstruction and subspace estimation.
method Blockwise tensor perturbation bounds for high-order orthogonal iteration (HOOI).
result Upper bounds for singular subspace estimation converge linearly and tensor reconstruction error bound is characterized by a simple quantity.
Efficient algorithms for low-rank bandits using subspace recovery.
problem Contextual bandits with low-rank reward matrices.
method Spectral methods for subspace recovery, reformulating as linear bandits.
result Nearly optimal policy evaluation and best policy identification, minimax guarantees for regret minimization.
GROUSE (Grassmannian Rank-One Update Subspace Estimation) is an incremental algorithm for identifying a subspace of Rn from a sequence of vectors in this subspace, where only a subset of components of each vector is revealed at each iteration. Recent analysis has shown that GROUSE converges locally at an expected linea…
We show that all closed 2-dimensional singularities for higher codimension mean curvature flow that cannot be perturbed away have uniform entropy bounds and lie in a linear subspace of small dimension. The entropy and dimension of the subspace are both ≤C(1+γ) for some universal constant C and genus γ. Th…
The paper extends hypothesis testing to non-diagonalizable matrices, improving network statistics inference.
problem Testing on non-diagonalizable matrices for network statistics.
method Generalizes Wald and t-tests to non-symmetric matrices, controlling convergence rates.
result Improved inference on network statistics from directed networks.
New spectral methods improve matrix estimation in RL with low-rank structure.
problem Estimating matrices with low-rank structure in reinforcement learning.
method Spectral-based matrix estimation approaches.
result Spectral methods efficiently recover singular subspaces and minimize entry-wise error.
Transfer knowledge from multiple sources to improve matrix completion.
problem Matrix completion with noisy data.
method Aggregating singular subspaces information from multiple sources to solve a two-way PCA problem and transform into a low-dimensional linear regression.
result Guaranteed statistical efficiency in transforming the high-dimensional target matrix completion problem.
New method corrects Laplace/BIC errors in singular models, revealing effective dimension.
problem Laplace/BIC errors in singular models due to incorrect effective dimension assumption.
method RLCT (real log canonical threshold) to correct effective dimension in linear models.
result Correct evidence slope and effective dimension estimation in linear settings.
This paper is on the normal approximation of singular subspaces when the noise matrix has i.i.d. entries. Our contributions are three-fold. First, we derive an explicit representation formula of the empirical spectral projectors. The formula is neat and holds for deterministic matrix perturbations. Second, we calculate…
Paper bounds subspace estimator error from noisy projections.
problem Estimating subspaces from noisy data.
method Derives perturbation bound on optimal subspace estimator.
result Fundamental result with implications in matrix completion and clustering.
A general framework for principal component analysis (PCA) in the presence of heteroskedastic noise is introduced. We propose an algorithm called HeteroPCA, which involves iteratively imputing the diagonal entries of the sample covariance matrix to remove estimation bias due to heteroskedasticity. This procedure is com…
Fast and accurate methods for low-rank learning problems.
problem Partial singular value decomposition and numerical rank estimation of huge matrices.
method Krylov subspaces and Ritz vectors for fast and accurate solutions.
result Advantages over traditional methods in accuracy and speed.
Bayesian methods reduce variance in subspace identification for small data sets.
problem High variance in traditional subspace identification methods for large models or small sample sizes.
method Investigation of Bayesian estimation solutions (regularized and shrinkage estimators) for subspace identification.
result Bayesian estimators reduce estimation risk by up to 40% compared to traditional methods.
The paper analyzes PLS-SVD in high-dimensional data integration, revealing its strengths and limitations.
problem Understanding the behavior of PLS-SVD in high-dimensional data integration.
method Analysis using random matrix theory and singular value decomposition.
result PLS-SVD exhibits counter-intuitive or limiting behavior in certain regimes and outperforms PCA when detecting common latent subspace.
The paper updates SVD of evolving matrices using projection techniques.
problem Updating the rank-k truncated SVD of evolving matrices.
method Projection viewpoint, building subspaces to approximate singular vectors.
result The proposed algorithm leads to higher accuracy, especially for large singular values.
This paper considers the problem of robust subspace recovery: given a set of N points in RD, if many lie in a d-dimensional subspace, then can we recover the underlying subspace? We show that Tyler's M-estimator can be used to recover the underlying subspace, if the percentage of the inliers is larger t…
Unified framework for structured principal subspace estimation with bounds and rates.
problem Structured principal subspace estimation problems.
method Unified framework, minimax lower and upper bounds, information-geometric complexity.
result Minimax rates of convergence for specific settings, including optimal rates for non-negative PCA/SVD.
We investigate Riemannian (non-Kahler) Ricci flow solutions that develop finite-time Type-I singularities and present evidence in favor of a conjecture that parabolic rescalings at the singularities converge to singularity models that are shrinking Kahler-Ricci solitons. Specifically, the singularity model for these so…
Theoretical guarantees for STE, a robust subspace recovery method.
problem Recovering a low-dimensional subspace from corrupted data.
method Subspace-constrained Tyler's estimator (STE) with initialization conditions.
result STE can effectively recover the subspace under certain conditions.
A new method learns outcome-aware spectral features for causal effect estimation.
problem Estimation of causal effects in the presence of hidden confounders.
method Augmented Spectral Feature Learning framework that minimizes a contrastive loss derived from an augmented operator incorporating outcome information.
result Our method remains effective even under spectral misalignment.
New method estimates active subspaces for jump-discontinuous functions.
problem Estimating active subspaces for discontinuous functions like ABMs.
method Extending active subspaces to discontinuous functions, using Gaussian process.
result Identifies important parameters in ABM simulations of refugee movement.
In this paper, we study the adversarial robustness of subspace learning problems. Different from the assumptions made in existing work on robust subspace learning where data samples are contaminated by gross sparse outliers or small dense noises, we consider a more powerful adversary who can first observe the data matr…
Matrix rank minimization problem is in general NP-hard. The nuclear norm is used to substitute the rank function in many recent studies. Nevertheless, the nuclear norm approximation adds all singular values together and the approximation error may depend heavily on the magnitudes of singular values. This might restrict…
We study sparse principal components analysis in high dimensions, where p (the number of variables) can be much larger than n (the number of observations), and analyze the problem of estimating the subspace spanned by the principal eigenvectors of the population covariance matrix. We introduce two complementary not…
A method for identifying joint and individual subspaces from multi-view data.
problem Unclear conditions for reliably identifying joint and individual subspaces from noisy, high-dimensional measurements.
method Rigorously quantifies conditions based on signal rank, principal angles, and noise levels. Characterizes spectrum perturbations of product of projection matrices.
result Estimates joint and individual subspaces more accurately than existing approaches in simulations and real-world applications.
Proposes a new algorithm to estimate invariant subspaces across multilayer networks.
problem Estimating invariant subspaces across heterogeneous multiple networks.
method Bias-corrected joint spectral embedding algorithm that recursively calibrates diagonal bias and iteratively updates the subspace estimator.
result Established entrywise subspace perturbation bound and entrywise eigenvector central limit theorem for the algorithm.
Study flat manifolds' collapsed limits as flat orbifolds.
problem Understanding collapsed limits of flat manifolds.
method Analyzing totally geodesic foliations and Gromov-Hausdorff limits.
result Identify collapsed limits as flat orbifolds and provide criteria for singularity.
Matrix completion is a widely used technique for image inpainting and personalized recommender system, etc. In this work, we focus on accelerating the matrix completion using faster randomized singular value decomposition (rSVD). Firstly, two fast randomized algorithms (rSVD-PI and rSVD- BKI) are proposed for handling …
SMART transfers knowledge across related studies for multi-task learning.
problem Deterioration of multi-task learning performance with small target sample size.
method SMART assumes spectral similarity between source and target models, estimating target coefficients through structured regularization.
result SMART achieves near-minimax error rates, improving estimation accuracy and robustness to negative transfer.
A new model reduces noise and speeds up subspace segmentation.
problem Subspace segmentation from noisy data.
method Group norm regularized factorization model (GNRFM) with AALM algorithm.
result The method is faster and more robust to noise.
Study vector fields and flows on singular spaces like submanifolds.
problem Understanding vector fields and flows on singular spaces.
method Integrate derivations of the C∞-ring of global smooth functions into flows. result Derivations integrate to smooth flows on subcartesian spaces.
Paper proves IRLS converges to subspace from any start, with practical benefits.
problem Robust subspace estimation in machine learning.
method Iteratively Reweighted Least Squares (IRLS) with dynamic smoothing regularization.
result IRLS converges linearly to the underlying subspace from any initialization under deterministic conditions.
Kaczmarz++ accelerates convergence for ill-conditioned systems.
problem Solving ill-conditioned linear systems efficiently.
method Adaptive momentum acceleration, Tikhonov-regularized projections, and memoization.
result Kaczmarz++ converges faster than Krylov methods on ill-conditioned systems.
This paper analyzes AJIVE for estimating shared subspace across multiple datasets, revealing its strengths and limitations.
problem Estimating shared subspace across multiple datasets with varying degrees of misalignment.
method Angle-based Joint and Individual Variation Explained (AJIVE) method, a two-stage spectral approach.
result AJIVE's performance in high signal-to-noise ratio (SNR) regimes and its non-diminishing error in low-SNR settings.
The paper generalizes curvature bounds for submanifolds with singularities.
problem Bounding the total absolute curvature of submanifolds with singularities.
method Generalization of Chern-Lashof theorem for frontals with singularities.
result Total absolute curvature is at least the sum of Betti numbers.
Paper develops inference methods for low-rank tensors without debiasing.
problem Statistical inference for low-rank tensor models.
method Two-iteration alternating minimization for asymptotic distribution.
result Asymptotic distributions and confidence regions for singular subspaces.
Autoencoders are a deep learning model for representation learning. When trained to minimize the distance between the data and its reconstruction, linear autoencoders (LAEs) learn the subspace spanned by the top principal directions but cannot learn the principal directions themselves. In this paper, we prove that $L_2…
This is a detailed tutorial paper which explains the Fisher discriminant Analysis (FDA) and kernel FDA. We start with projection and reconstruction. Then, one- and multi-dimensional FDA subspaces are covered. Scatters in two- and then multi-classes are explained in FDA. Then, we discuss on the rank of the scatters and …