arXiv research

Paper bounds subspace estimator error from noisy projections.

Study analyzes perturbations in singular subspaces under random noise.

P-OCS detects OOD samples in a low-dimensional subspace, outperforming existing methods.

Study on low-dimensional adversarial perturbations in classification models.

The Davis-Kahan-Wedin $\sin Θ$ theorem describes how the singular subspaces of a matrix change when subjected to a small perturbation. This classic result is sharp in the worst case scenario. In this paper, we prove a stochastic version of the Davis-Kahan-Wedin $\sin Θ$ theorem when the perturbation is a Gaussian rando…

The higher order singular value decomposition (HOSVD) of tensors is a generalization of matrix SVD. The perturbation analysis of HOSVD under random noise is more delicate than its matrix counterpart. Recently, polynomial time algorithms have been proposed where statistically optimal estimates of the singular subspaces …

WPCA improves subspace recovery robustness to outliers.

Study robust estimation of principal components under adversarial perturbations.

Two-layer networks trained on low-dimensional subspaces are vulnerable to adversarial examples.

Novel tensor perturbation bounds for orthogonal iteration methods.

Proposes a new algorithm to estimate invariant subspaces across multilayer networks.

This work examines how adversarial vulnerability changes with the dimensionality of the subspace of perturbations.

We show that all closed $2$-dimensional singularities for higher codimension mean curvature flow that cannot be perturbed away have uniform entropy bounds and lie in a linear subspace of small dimension. The entropy and dimension of the subspace are both $\leq C\,(1+γ)$ for some universal constant $C$ and genus $γ$. Th…

Paper proves noise-tolerant SSC using greedy methods under coherence conditions.

Paper analyzes singular subspace estimation in noisy matrix models.

Subspace clustering is the problem of clustering data points into a union of low-dimensional linear/affine subspaces. It is the mathematical abstraction of many important problems in computer vision, image processing and machine learning. A line of recent work (4, 19, 24, 20) provided strong theoretical guarantee for s…

New method improves subspace iteration for eigenvectors in machine learning.

This work improves PCA for robustness against adversarial perturbations.

A method for identifying joint and individual subspaces from multi-view data.

Subspace clustering is a growing field of unsupervised learning that has gained much popularity in the computer vision community. Applications can be found in areas such as motion segmentation and face clustering. It assumes that data originate from a union of subspaces, and clusters the data depending on the correspon…

Constructing an efficient parameterization of a large, noisy data set of points lying close to a smooth manifold in high dimension remains a fundamental problem. One approach consists in recovering a local parameterization using the local tangent plane. Principal component analysis (PCA) is often the tool of choice, as…

High-dimensional data often lie in low-dimensional subspaces corresponding to different classes they belong to. Finding sparse representations of data points in a dictionary built using the collection of data helps to uncover low-dimensional subspaces and address problems such as clustering, classification, subset sele…

In his celebrated paper "Generic projections", John Mather has shown that almost all linear projections from a submanifold of a vector space into a subspace are transverse with respect to a given modular submanifold. In this paper, an improvement of Mather's result is stated. Namely, we show that almost all linear pert…

New bounds for private matrix approximation using Gaussian noise and Dyson Brownian Motion.

Understanding and characterizing the subspaces of adversarial examples aid in studying the robustness of deep neural networks (DNNs) to adversarial perturbations. Very recently, Ma et al. (ICLR 2018) proposed to use local intrinsic dimensionality (LID) in layer-wise hidden representations of DNNs to study adversarial s…

Efficiently compress pretrained models using RSI for improved predictive accuracy.

Study shows how tangle moduli spaces relate to boundary surfaces.

We study the accuracy of estimating the covariance and the precision matrix of a $D$-variate sub-Gaussian distribution along a prescribed subspace or direction using the finite sample covariance. Our results show that the estimation accuracy depends almost exclusively on the components of the distribution that correspo…

Projective DP-SGD reduces privacy error by identifying low-dimensional gradient subspaces.

New framework assesses neural sensitivity to small perturbations.

The paper finds Koopman invariant subspaces using personalized PageRank.

We investigate the effect of the dimensionality of the representations learned in Deep Neural Networks (DNNs) on their robustness to input perturbations, both adversarial and random. To achieve low dimensionality of learned representations, we propose an easy-to-use, end-to-end trainable, low-rank regularizer (LR) that…

SAMS-VAE models cellular perturbations using sparse additive mechanisms.

New cyclicity measures defined in weighted Besov spaces, with stability and geometric analysis.

Adversarial examples are maliciously perturbed inputs designed to mislead machine learning (ML) models at test-time. They often transfer: the same adversarial example fools more than one model. In this work, we propose novel methods for estimating the previously unknown dimensionality of the space of adversarial inputs…

This paper is on the normal approximation of singular subspaces when the noise matrix has i.i.d. entries. Our contributions are three-fold. First, we derive an explicit representation formula of the empirical spectral projectors. The formula is neat and holds for deterministic matrix perturbations. Second, we calculate…

New algorithm updates eigenvectors of evolving graphs efficiently.

Optimizes parameters in high-dimensional spaces for practical applications.

Improved private AdaGrad achieves faster convergence rates for convex functions.

We prove generic regularity and Uhlenbeck-type compactification theorems for the moduli spaces of PU(2)-monopoles. Generic regularity is NOT obtained in the usual way (by applying Sard theorem to a smooth parameterized moduli space), since the parameterized moduli space can be a priori singular. We explain why, using t…

PCA is a classical statistical technique whose simplicity and maturity has seen it find widespread use as an anomaly detection technique. However, it is limited in this regard by being sensitive to gross perturbations of the input, and by seeking a linear subspace that captures normal behaviour. The first issue has bee…

We consider training machine learning models that are fair in the sense that their performance is invariant under certain sensitive perturbations to the inputs. For example, the performance of a resume screening system should be invariant under changes to the gender and/or ethnicity of the applicant. We formalize this …

We propose a general framework to study the stability of the subspace spanned by $P$ consecutive eigenvectors of a generic symmetric matrix ${\bf H}_0$, when a small perturbation is added. This problem is relevant in various contexts, including quantum dissipation (${\bf H}_0$ is then the Hamiltonian) and risk control …

Deep neural networks (DNNs) are powerful nonlinear architectures that are known to be robust to random perturbations of the input. However, these models are vulnerable to adversarial perturbations--small input changes crafted explicitly to fool the model. In this paper, we ask whether a DNN can distinguish adversarial …

The intent of this short note is to provide context for and an independent proof of the discovery of Klaus Kroencke that complex projective space with its canonical Fubini--Study metric is dynamically unstable under Ricci flow in all complex dimensions N>1. The unstable perturbation is not Kaehler. This provides a coun…

LSDAT reduces query efficiency for decision-based adversarial attacks.

Spanning attack improves black-box attacks with unlabeled data.

This article deals with two topics: the first, which has a general character, is a variation formula for the the determinant line bundle in non-Kählerian geometry. This formula, which is a consequence of the non-Kählerian version of the Grothendieck-Riemann Roch theorem proved recently by Bismut, gives the variation of…