arXiv research

Unified analysis simplifies Johnson-Lindenstrauss lemma for data reduction.

A new method for private query release using Johnson-Lindenstrauss projection.

We present a theory for Euclidean dimensionality reduction with subgaussian matrices which unifies several restricted isometry property and Johnson-Lindenstrauss type results obtained earlier for specific data sets. In particular, we recover and, in several cases, improve results for sets of sparse and structured spars…

This paper surveys various methods for dimensionality reduction and nearest neighbor search.

Optimization can learn Johnson-Lindenstrauss embeddings without randomization.

Let $Φ\in\mathbb{R}^{m\times n}$ be a sparse Johnson-Lindenstrauss transform [KN14] with $s$ non-zeroes per column. For a subset $T$ of the unit sphere, $\varepsilon\in(0,1/2)$ given, we study settings for $m,s$ required to ensure $\mathop{\mathbb{E}}_Φ\sup_{x\in T} \left|\|Φx\|_2^2 - 1 \right| < \varepsilon ,$ i.e…

Optimizes embedding accuracy for data variance and error.

We develop a probabilistic framework for sequential random projection.

Novel privatization framework for high-dimensional variable selection with differential privacy.

Random projections help in representing sparse graphs efficiently.

Principal Components Regression (PCR) is a traditional tool for dimension reduction in linear regression that has been both criticized and defended. One concern about PCR is that obtaining the leading principal components tends to be computationally demanding for large data sets. While random projections do not possess…

New algorithms cluster high-dimensional polygonal curves efficiently.

Paper uses random projection to preserve subspace structure for efficient data analysis.

The Johnson-Lindenstrauss Lemma allows for the projection of $n$ points in $p-$dimensional Euclidean space onto a $k-$dimensional Euclidean space, with $k \ge \frac{24\ln \emph{n}}{3ε^2-2ε^3}$, so that the pairwise distances are preserved within a factor of $1\pmε$. Here, working directly with the distributions of the …

The paper studies how norms of random vectors are preserved by random projections.

Tensorized Rademacher projections outperform Gaussian projections in reducing tensor dimensions.

HD-BWDM improves clustering validation in high-dimensional data.

We consider the problem of efficient randomized dimensionality reduction with norm-preservation guarantees. Specifically we prove data-dependent Johnson-Lindenstrauss-type geometry preservation guarantees for Ho's random subspace method: When data satisfy a mild regularity condition -- the extent of which can be estima…

The paper examines how well node similarities are preserved by random projections in graph embeddings.

Aggregates predictions from multiple regression models using random projections and kernel methods.

This work improves tensor decomposition methods, especially for large datasets.

Unified bounds for sketched bilinear forms in machine learning and statistics.

Derandomization reveals structure in neural networks, reducing sample complexity.

This paper deals with two related problems, namely distance-preserving binary embeddings and quantization for compressed sensing . First, we propose fast methods to replace points from a subset $\mathcal{X} \subset \mathbb{R}^n$, associated with the Euclidean metric, with points in the cube $\{\pm 1\}^m$ and we associa…

A fast binary embedding method preserves Euclidean distances in high-dimensional data.

We examine a class of embeddings based on structured random matrices with orthogonal rows which can be applied in many machine learning applications including dimensionality reduction and kernel approximation. For both the Johnson-Lindenstrauss transform and the angular kernel, we show that we can select matrices yield…

Gaussian sketching preserves kernel inner products in low dimensions.

New faster, space-saving methods for subspace embeddings in tensors.

We introduce sparse random projection, an important dimension-reduction tool from machine learning, for the estimation of discrete-choice models with high-dimensional choice sets. Initially, high-dimensional data are compressed into a lower-dimensional Euclidean space using random projections. Subsequently, estimation …

Neural networks approximate high-dimensional functions better than theory predicts.

We present a new paradigm for speeding up randomized computations of several frequently used functions in machine learning. In particular, our paradigm can be applied for improving computations of kernels based on random embeddings. Above that, the presented framework covers multivariate randomized functions. As a bypr…

In this paper, we study a fast approximation method for {\it large-scale high-dimensional} sparse least-squares regression problem by exploiting the Johnson-Lindenstrauss (JL) transforms, which embed a set of high-dimensional vectors into a low-dimensional space. In particular, we propose to apply the JL transforms to …

New algorithm speeds up regression tasks for large datasets.

Optimally sketches tensors with minimal rows for preserving norms.

For various applications, the relations between the dependent and independent variables are highly nonlinear. Consequently, for large scale complex problems, neural networks and regression trees are commonly preferred over linear models such as Lasso. This work proposes learning the feature nonlinearities by binning fe…

Let $\varepsilon\in(0,1)$ and $X\subset\mathbb R^d$ be arbitrary with $|X|$ having size $n>1$. The Johnson-Lindenstrauss lemma states there exists $f:X\rightarrow\mathbb R^m$ with $m = O(\varepsilon^{-2}\log n)$ such that $\forall x\in X\ \forall y\in X, \|x-y\|_2 \le \|f(x)-f(y)\|_2 \le (1+\varepsilon)\|x-y\|_2 .$…

Randomized matrix compression techniques, such as the Johnson-Lindenstrauss transform, have emerged as an effective and practical way for solving large-scale problems efficiently. With a focus on computational efficiency, however, forsaking solutions quality and accuracy becomes the trade-off. In this paper, we investi…

We develop a novel methodology based on the marriage between the Bhattacharyya distance, a measure of similarity across distributions of random variables, and the Johnson-Lindenstrauss Lemma, a technique for dimension reduction. The resulting technique is a simple yet powerful tool that allows comparisons between data-…

Tensorized random projections reduce high-dimensional tensor size efficiently.

In this paper we show that the computational complexity of the Iterative Thresholding and K-residual-Means (ITKrM) algorithm for dictionary learning can be significantly reduced by using dimensionality-reduction techniques based on the Johnson-Lindenstrauss lemma. The dimensionality reduction is efficiently carried out…

A new method uses matrix sketches for efficient graph clustering in dynamic environments.

New parametrization of 3-spheres using Johnson subgroups.

The Johnson kernel is the subgroup of the mapping class group of a surface generated by Dehn twists along bounding simple closed curves, and has the second Johnson homomorphism as a free abelian quotient. In terms of the representation theory of the symplectic group, we give a complete description of cup products of tw…

Corrects bias in random sampling matrices for improved ML methods.

Feature hashing and other random projection schemes are commonly used to reduce the dimensionality of feature vectors. The goal is to efficiently project a high-dimensional feature vector living in $\mathbb{R}^n$ into a much lower-dimensional space $\mathbb{R}^m$, while approximately preserving Euclidean norm. These sc…

Study 2-loop part of Johnson cokernel using trace map.

We consider two mod-p central series of the free group given by Stallings and Zassenhaus. Applying these series to definitions of Dennis Johnson's filtration of the mapping class group we obtain two mod-p Johnson filtrations. Further, we adapt the definition of the Johnson homomorphisms to obtain mod-p Johnson homomorp…

Study of handlebody group intersections with Torelli and Johnson kernels.