arXiv research

Kernel models learn low-dimensional predictive subspaces from input data.

Investigates projections onto explicit subspaces and their variance effects.

Paper shows affine constraint is unnecessary for high-dimensional data.

A new method learns multiple subspaces from data.

This work presents GROUSE (Grassmanian Rank-One Update Subspace Estimation), an efficient online algorithm for tracking subspaces from highly incomplete observations. GROUSE requires only basic linear algebraic manipulations at each iteration, and each subspace update can be performed in linear time in the dimension of…

Study non-asymptotic bounds on correlation in high-dimensional linear systems, revealing invariant subspaces and bottlenecks.

Agents collaborate to reduce regret in a multi-agent linear bandit problem with side information.

Paper proves linear convergence of SCMS algorithm for directional data.

We study the geometry of an important class of generic curves in the Grassmannian manifolds of $n$-dimensional subspaces and Lagrangian subspaces of $R^{2n}$ under the action of the linear and linear symplectic group.

This paper offers a new algebraic perspective of GCCA using subspace intersection.

This paper investigates the generalization of Principal Component Analysis (PCA) to Riemannian manifolds. We first propose a new and general type of family of subspaces in manifolds that we call barycentric subspaces. They are implicitly defined as the locus of points which are weighted means of $k+1$ reference points.…

Subspace clustering assumes that the data is sepa-rable into separate subspaces. Such a simple as-sumption, does not always hold. We assume that, even if the raw data is not separable into subspac-es, one can learn a representation (transform coef-ficients) such that the learnt representation is sep-arable into subspac…

We present a framework for supervised subspace tracking, when there are two time series $x_t$ and $y_t$, one being the high-dimensional predictors and the other being the response variables and the subspace tracking needs to take into consideration of both sequences. It extends the classic online subspace tracking work…

We consider the problem of subspace clustering: given points that lie on or near the union of many low-dimensional linear subspaces, recover the subspaces. To this end, one first identifies sets of points close to the same subspace and uses the sets to estimate the subspaces. As the geometric structure of the clusters …

ASGD outperforms SGD in overparameterized linear regression, especially in subspaces of small eigenvalues.

The paper improves conditions for unique recovery in homomorphic sensing of subspaces.

Subspace clustering aims to cluster unlabeled data that lies in a union of low-dimensional linear subspaces. Deep subspace clustering approaches based on auto-encoders have become very popular to solve subspace clustering problems. However, the training of current deep methods converges slowly, which is much less effic…

The Grassmannian of affine subspaces is a natural generalization of both the Euclidean space, points being zero-dimensional affine subspaces, and the usual Grassmannian, linear subspaces being special cases of affine subspaces. We show that, like the Grassmannian, the affine Grassmannian has rich geometrical and topolo…

Motivated by vision tasks such as robust face and object recognition, we consider the following general problem: given a collection of low-dimensional linear subspaces in a high-dimensional ambient (image) space, and a query point (image), efficiently determine the nearest subspace to the query in $\ell^1$ distance. In…

Non-Gaussian component analysis (NGCA) is aimed at identifying a linear subspace such that the projected data follows a non-Gaussian distribution. In this paper, we propose a novel NGCA algorithm based on log-density gradient estimation. Unlike existing methods, the proposed NGCA algorithm identifies the linear subspac…

Consider a dataset of vector-valued observations that consists of noisy inliers, which are explained well by a low-dimensional subspace, along with some number of outliers. This work describes a convex optimization problem, called REAPER, that can reliably fit a low-dimensional model to this type of data. This approach…

A low-rank transformation learning framework for subspace clustering and classification is here proposed. Many high-dimensional data, such as face images and motion sequences, approximately lie in a union of low-dimensional subspaces. The corresponding subspace clustering problem has been extensively studied in the lit…

Over the past decade a wide spectrum of machine learning models have been developed to model the neurodegenerative diseases, associating biomarkers, especially non-intrusive neuroimaging markers, with key clinical scores measuring the cognitive status of patients. Multi-task learning (MTL) has been commonly utilized by…

Algorithm finds a subspace minimizing distances to inliers with outliers.

Efficiently extracts linear dynamics from complex observations.

Subspace clustering methods based on expressing each data point as a linear combination of all other points in a dataset are popular unsupervised learning techniques. However, existing methods incur high computational complexity on large-scale datasets as they require solving an expensive optimization problem and perfo…

Analyzes word2vec-like models revealing linear subspaces learned during training.

This work takes the first steps towards solving the "phaseless subspace tracking" (PST) problem. PST involves recovering a time sequence of signals (or images) from phaseless linear projections of each signal under the following structural assumption: the signal sequence is generated from a much lower dimensional subsp…

A hierarchical approach improves classification accuracy in large datasets.

A new method for one-class classification using ellipsoidal encapsulation.

It is a key to construct a similarity graph in graph-oriented subspace learning and clustering. In a similarity graph, each vertex denotes a data point and the edge weight represents the similarity between two points. There are two popular schemes to construct a similarity graph, i.e., pairwise distance based scheme an…

Two-layer networks trained on low-dimensional subspaces are vulnerable to adversarial examples.

We study the linear subspace fitting problem in the overparameterized setting, where the estimated subspace can perfectly interpolate the training examples. Our scope includes the least-squares solutions to subspace fitting tasks with varying levels of supervision in the training data (i.e., the proportion of input-out…

In this paper we provide some stability criteria for systems of linear subspaces of $V \otimes W$ and for systems of quotient coherent sheaves, using, respectively, the Hilbert-Mumford numerical criterion and moment map. Along the way, we generalize the Gelfand-MacPherson correspondence [11] from point sets to sets of …

This paper proposes a novel deep subspace clustering approach which uses convolutional autoencoders to transform input images into new representations lying on a union of linear subspaces. The first contribution of our work is to insert multiple fully-connected linear layers between the encoder layers and their corresp…

Despite the fact that nonlinear subspace learning techniques (e.g. manifold learning) have successfully applied to data representation, there is still room for improvement in explainability (explicit mapping), generalization (out-of-samples), and cost-effectiveness (linearization). To this end, a novel linearized subsp…

New method disentangles hidden data structures using HSIC and supervision.

Sparse subspace clustering (SSC) is an elegant approach for unsupervised segmentation if the data points of each cluster are located in linear subspaces. This model applies, for instance, in motion segmentation if some restrictions on the camera model hold. SSC requires that problems based on the $l_1$-norm are solved …

Paper improves MFC algorithm for clustering linear subspaces.

We present a simple and fast geometric method for modeling data by a union of affine subspaces. The method begins by forming a collection of local best-fit affine subspaces, i.e., subspaces approximating the data in local neighborhoods. The correct sizes of the local neighborhoods are determined automatically by the Jo…

We propose Deep Closed-Form Subspace Clustering (DCFSC), a new embarrassingly simple model for subspace clustering with learning non-linear mapping. Compared with the previous deep subspace clustering (DSC) techniques, our DCFSC does not have any parameters at all for the self-expressive layer. Instead, DCFSC utilizes …

We assume i.i.d. data sampled from a mixture distribution with K components along fixed d-dimensional linear subspaces and an additional outlier component. For p>0, we study the simultaneous recovery of the K fixed subspaces by minimizing the l_p-averaged distances of the sampled data points from any K subspaces. Under…

Sliced inverse regression is a popular tool for sufficient dimension reduction, which replaces covariates with a minimal set of their linear combinations without loss of information on the conditional distribution of the response given the covariates. The estimated linear combinations include all covariates, making res…

This work improves sample efficiency in meta-learning for nonlinear tasks.

In this paper, we propose a novel method for projecting data from multiple modalities to a new subspace optimized for one-class classification. The proposed method iteratively transforms the data from the original feature space of each modality to a new common feature space along with finding a joint compact descriptio…

LOL method simplifies forming linear combinations of latent variables.

Given a complex structure $J$ on a real (finite or infinite dimensional) Hilbert space $H$, we study the geometry of the Lagrangian Grassmannian $Λ(H)$ of $H$, i.e. the set of closed linear subspaces $L\subset H$ such that $J(L)=L^\perp.$ The complex unitary group $U(H_J)$, consisting of the elements of the orthogona…

The Nearest subspace classifier (NSS) finds an estimation of the underlying subspace within each class and assigns data points to the class that corresponds to its nearest subspace. This paper mainly studies how well NSS can be generalized to new samples. It is proved that NSS is strongly consistent under certain assum…