arXiv research

Optimal subspace embedding with near-optimal sparsity for high-dimensional data.

We develop embeddings for nonlinear subspaces preserving vector norms.

In this paper new general modewise Johnson-Lindenstrauss (JL) subspace embeddings are proposed that are both considerably faster to generate and easier to store than traditional JL embeddings when working with extremely large vectors and/or tensors. Corresponding embedding results are then proven for two different type…

Sparse OSEs achieve optimal embedding dimension of O(d).

In this paper, we exhibit the tradeoffs between the (training) sample, computation and storage complexity for the problem of supervised classification using signal subspace estimation. Our main tool is the use of tensor subspaces, i.e. subspaces with a Kronecker structure, for embedding the data into lower dimensions. …

The paper constructs submanifolds with corners in Delzant polytopes from affine subspaces.

Multiple clustering aims at discovering diverse ways of organizing data into clusters. Despite the progress made, it's still a challenge for users to analyze and understand the distinctive structure of each output clustering. To ease this process, we consider diverse clusterings embedded in different subspaces, and ana…

In this paper we propose and study the novel problem of explaining node embeddings by finding embedded human interpretable subspaces in already trained unsupervised node representation embeddings. We use an external knowledge base that is organized as a taxonomy of human-understandable concepts over entities as a guide…

Proposes a new algorithm to estimate invariant subspaces across multilayer networks.

A new method compresses NLP networks by using multiple subspaces instead of a single one.

We discuss an "extrinsic" property of knots in a 3-subspace of the 3-sphere $S^3$ to characterize how the subspace is embedded in $S^3$. Specifically, we show that every knot in a subspace of the 3-sphere is transient if and only if the exterior of the subspace is a disjoint union of handlebodies, i.e. regular neighbor…

A new framework for graph representation learning.

We prove optimal subspace embedding conjecture up to sub-polylogarithmic factors.

In this letter, we consider two sets of observations defined as subspace signals embedded in noise and we wish to analyze the distance between these two subspaces. The latter entails evaluating the angles between the subspaces, an issue reminiscent of the well-known Procrustes problem. A Bayesian approach is investigat…

Analyzes word2vec-like models revealing linear subspaces learned during training.

The hyperbolic manifold is a smooth manifold of negative constant curvature. While the hyperbolic manifold is well-studied in the literature, it has gained interest in the machine learning and natural language processing communities lately due to its usefulness in modeling continuous hierarchies. Tasks with hierarchica…

We prove, using the subspace embedding guarantee in a black box way, that one can achieve the spectral norm guarantee for approximate matrix multiplication with a dimensionality-reducing map having $m = O(\tilde{r}/\varepsilon^2)$ rows. Here $\tilde{r}$ is the maximum stable rank, i.e. squared ratio of Frobenius and op…

This paper investigates the generalization of Principal Component Analysis (PCA) to Riemannian manifolds. We first propose a new and general type of family of subspaces in manifolds that we call barycentric subspaces. They are implicitly defined as the locus of points which are weighted means of $k+1$ reference points.…

Constructs equivariant embeddings of Hermitian symmetric spaces into tangent spaces.

New method finds unbranched covers with non-kernel homology.

Subspace clustering is the problem of partitioning unlabeled data points into a number of clusters so that data points within one cluster lie approximately on a low-dimensional linear subspace. In many practical scenarios, the dimensionality of data points to be clustered are compressed due to constraints of measuremen…

Paper shows affine constraint is unnecessary for high-dimensional data.

This paper improves spectral embedding for multipartite networks, revealing latent subspaces and providing consistent node representations.

Feature extraction and dimension reduction for networks is critical in a wide variety of domains. Efficiently and accurately learning features for multiple graphs has important applications in statistical inference on graphs. We propose a method to jointly embed multiple undirected graphs. Given a set of graphs, the jo…

Motivated by vision tasks such as robust face and object recognition, we consider the following general problem: given a collection of low-dimensional linear subspaces in a high-dimensional ambient (image) space and a query point (image), efficiently determine the nearest subspace to the query in $\ell^1$ distance. We …

We show local rigidity of hyperbolic triangle groups generated by reflections in pairs of $n$-dimensional subspaces of $R^{2n}$ obtained by composition of the geometric representation in $PGL(2, R)$ with the diagonal embeddings into $PGL(2n, R)$ and $PSp^\pm(2n, R)$.

EGORSE optimizes high-dimensional problems using random and supervised embeddings.

Flow Matching models help generative models stay within the subspace of real data.

BOIDS optimizes high-dimensional problems by guiding optimization with one-dimensional lines.

SA-REMBO adapts to nonstationary high-dimensional optimization.

In the machine learning field, dimensionality reduction is an important task. It mitigates the undesired properties of high-dimensional spaces to facilitate classification, compression, and visualization of high-dimensional data. During the last decade, researchers proposed many new (non-linear) techniques for dimensio…

The paper addresses data uncertainty in graph embedding by modeling data points as Gaussian distributions.

For a Veech surface (x,ω), we characterize subspaces of X^n, invariant under the diagonal action of the affine group of X. We prove that non-arithmetic Veech surfaces have only finitely many invariant subspaces of very particular shape (in any dimension). Among other consequences we find copies of (X,ω) embedded in the…

Despite the fact that nonlinear subspace learning techniques (e.g. manifold learning) have successfully applied to data representation, there is still room for improvement in explainability (explicit mapping), generalization (out-of-samples), and cost-effectiveness (linearization). To this end, a novel linearized subsp…

In this paper, we propose a Tensor Train Neighborhood Preserving Embedding (TTNPE) to embed multi-dimensional tensor data into low dimensional tensor subspace. Novel approaches to solve the optimization problem in TTNPE are proposed. For this embedding, we evaluate novel trade-off gain among classification, computation…

Robust PCA methods are typically batch algorithms which requires loading all observations into memory before processing. This makes them inefficient to process big data. In this paper, we develop an efficient online robust principal component methods, namely online moving window robust principal component analysis (OMW…

Optimal hashing embeddings reduce linear least squares solving time.

The paper analyzes side effects of learning from low-dimensional data embedded in a Euclidean space.

New algorithm updates eigenvectors of evolving graphs efficiently.

Paper perfect clusters sparse, diverse multilayer networks.

Study the embedding space of a Hopf link in 3D and 3-manifolds.

We show that any infinite order element $g$ of a virtually cyclic hyperbolically embedded subgroup of a group $G$ is Morse, that is to say any quasi-geodesic connecting points in the cyclic group $C$ generated by $g$ stays close to $C$. This answers a question of Dahmani-Guirardel-Osin. What is more, we show that hyper…

A large number of algorithms in machine learning, from principal component analysis (PCA), and its non-linear (kernel) extensions, to more recent spectral embedding and support estimation methods, rely on estimating a linear subspace from samples. In this paper we introduce a general formulation of this problem and der…

This works extends the Random Embedding Bayesian Optimization approach by integrating a warping of the high dimensional subspace within the covariance kernel. The proposed warping, that relies on elementary geometric considerations, allows mitigating the drawbacks of the high extrinsic dimensionality while avoiding the…

BSA reduces network data by interpreting feature subspaces.

This work tackles the problem of learning a set of language specific acoustic units from unlabeled speech recordings given a set of labeled recordings from other languages. Our approach may be described by the following two steps procedure: first the model learns the notion of acoustic units from the labelled data and …

Spaces of circle embeddings in curved surfaces indexed by trees.

Improved bounds for $\ell_p$ sensitivity sampling reducing the sample complexity for structured matrices.