Unified analysis simplifies Johnson-Lindenstrauss lemma for data reduction.
problem Efficiently reducing high-dimensional data while preserving geometry.
method Unified analysis of various JL constructions using probabilistic tools.
result First rigorous proof and extension of spherical construction's effectiveness.
This paper surveys various methods for dimensionality reduction and nearest neighbor search.
problem Efficiently reducing high-dimensional data to lower dimensions while preserving essential information.
method Linear and nonlinear random projections, including sparse random projections, random Fourier Features, and Random Kitchen Sinks.
result Various methods for dimensionality reduction and nearest neighbor search are explained and compared.
A new method for private query release using Johnson-Lindenstrauss projection.
problem Private release of query answers with minimal privacy loss.
method Random projection of query answers to a lower dimension, followed by noise addition.
result Optimal worst-case sample complexity for answering a workload of k queries.
Improved Johnson-Lindenstrauss lemma for preserving distances in Euclidean space.
problem Preserving distances in Euclidean space for arbitrary sets.
method Tighter analysis of embedding recipe from [MMMR18].
result Stronger version of Johnson-Lindenstrauss lemma with improved dimensionality.
We develop a probabilistic framework for sequential random projection.
problem Challenges of sequential decision-making under uncertainty.
method Novel construction of a stopped process and method of mixtures.
result Achieved a non-asymptotic probability bound for random projection.
Sparse random projections simplify complex choice models.
problem Estimating models with large choice sets.
method Sparse random projections followed by cyclic monotonicity moment inequalities.
result The method works well in simulations and real data applications.
The Johnson-Lindenstrauss Lemma allows for the projection of n n n points in p − p- p − dimensional Euclidean space onto a k − k- k − dimensional Euclidean space, with k ≥ 24 ln n 3 ε 2 − 2 ε 3 k \ge \frac{24\ln \emph{n}}{3ε^2-2ε^3} k ≥ 3 ε 2 − 2 ε 3 24 l n n , so that the pairwise distances are preserved within a factor of 1 ± ε 1\pmε 1 ± ε . Here, working directly with the distributions of the …
Let Φ ∈ R m × n Φ\in\mathbb{R}^{m\times n} Φ ∈ R m × n be a sparse Johnson-Lindenstrauss transform [KN14] with s s s non-zeroes per column. For a subset T T T of the unit sphere, ε ∈ ( 0 , 1 / 2 ) \varepsilon\in(0,1/2) ε ∈ ( 0 , 1/2 ) given, we study settings for m , s m,s m , s required to ensure E Φ sup x ∈ T ∣ ∥ Φ x ∥ 2 2 − 1 ∣ < ε , \mathop{\mathbb{E}}_Φ\sup_{x\in T} \left|\|Φx\|_2^2 - 1 \right| < \varepsilon , E Φ sup x ∈ T ∥Φ x ∥ 2 2 − 1 < ε , i.e…
Combines distance measures and dimension reduction for comparing distributions.
problem Comparing different entities' distributions and their similarities.
method Marriage between Bhattacharyya distance and Johnson-Lindenstrauss Lemma for dimension reduction.
result Demonstrates relationship between covariance and distance measures.
The paper examines how well node similarities are preserved by random projections in graph embeddings.
problem The preservation of node similarities under random projections in graph embeddings.
method Investigation of dot product and cosine similarity preservation by random projections over graph matrix rows.
result Random projections produce unreliable embeddings for dot product, especially for high-degree nodes.
New algorithm reduces dictionary learning complexity.
problem Efficiently learning dictionaries from high-dimensional data.
method IcTKM algorithm using dimensionality reduction and fast Fourier transform.
result Locally recovers dictionary with high probability.
Unified bounds for sketched bilinear forms in machine learning and statistics.
problem Uniform bounds on sketched bilinear forms for modern analyses.
method Generic chaining and new techniques for handling suprema over pairs of sets.
result Improved convergence bounds for sketched Federated Learning and bandit algorithms.
Derandomization reveals structure in neural networks, reducing sample complexity.
problem Understanding feature learning dynamics in neural networks.
method Derandomization lemma applied to arbitrary NNs with any smooth loss function.
result Optimizing function converges to zero weight matrix, revealing structure.
Aggregates predictions from multiple regression models using random projections and kernel methods.
problem Combining predictions from multiple regression models to improve accuracy.
method Random projection of high-dimensional feature space, followed by kernel-based consensual aggregation.
result The aggregation scheme performs similarly to using the original high-dimensional features, with high probability.
Gaussian sketching preserves kernel inner products in low dimensions.
problem Preserving kernel inner products in low-dimensional spaces.
method Gaussian sketching of kernel Gram matrices and random projections in RKHS.
result Sketching yields a random projection operator that preserves weighted RKHS inner products.
We present a theory for Euclidean dimensionality reduction with subgaussian matrices which unifies several restricted isometry property and Johnson-Lindenstrauss type results obtained earlier for specific data sets. In particular, we recover and, in several cases, improve results for sets of sparse and structured spars…
Optimization can learn Johnson-Lindenstrauss embeddings without randomization.
problem Achieving compact data representations with theoretical guarantees.
method A novel optimization-based approach over the space of random solution samplers.
result The method avoids bad stationary points and converges to a deterministic solution.
This paper compares and analyzes random projections and column sub-sampling for dimension reduction in regression.
problem Computational efficiency in dimension reduction for large datasets.
method Analysis of random projections and column sub-sampling methods for regression.
result Random projections and column sub-sampling can achieve similar prediction error to Principal Components Regression (PCR) but with less computational cost.
Efficiently reduces data dimensionality with guaranteed geometry preservation.
problem Efficiently reducing high-dimensional data while preserving its geometric structure.
method Random subspace method with Johnson-Lindenstrauss guarantees, densifying preprocessing for sparse data.
result Random subspace method achieves geometry preservation with logarithmic dimensionality in data points.
Market microstructure studies aim to reduce frictions in financial markets by analyzing price, volume, and volatility.
problem Reducing frictions in financial markets to improve efficiency and reduce costs.
method Application of Bhattacharyya distance and Johnson Lindenstrauss Lemma for comparing distributions of financial variables.
result Different markets or securities have distinct measures of their distributions, aiding in diversification or finding similar assets.
Optimizes embedding accuracy for data variance and error.
problem Efficiently embedding data while minimizing distortion.
method Uses Johnson-Lindenstrauss embeddings with orthogonal matrices and singular-value latent variables.
result Achieves best accuracy in variance, mean-squared error, and length distortion.
Novel privatization framework for high-dimensional variable selection with differential privacy.
problem High-dimensional controlled variable selection with rigorous FDR control under differential privacy constraints.
method Gaussian Johnson-Lindenstrauss Transformation for privatizing the knockoff matrix.
result The proposed private variable selection procedure maintains statistical power even under strict privacy budgets.
Random projections help in representing sparse graphs efficiently.
problem Efficiently representing sparse graphs of varying sizes and vertex sets.
method Random projection of adjacency matrices to retain graph functionality and properties.
result Random projections can accurately represent graphs of different sizes and vertex sets in the same space.
New algorithms cluster high-dimensional polygonal curves efficiently.
problem Clustering high-dimensional polygonal curves with many vertices.
method Johnson-Lindenstrauss projection for polygonal curves, subsampling, probabilistic reduction of dependency on vertices.
result Achieves sublinear dependency on the number of input curves.
Paper uses random projection to preserve subspace structure for efficient data analysis.
problem Efficiently analyzing data with low-dimensional structure.
method Compressed Subspace Learning (CSL) framework based on Johnson-Lindenstrauss property.
result Random projection preserves the UoS structure of data, enabling efficient analysis.
This paper improves binary embeddings and quantized compressed sensing methods.
problem Distance-preserving binary embeddings and quantization for compressed sensing.
method Quantization of fast Johnson-Lindenstrauss embeddings and bounded orthonormal systems.
result Quantization methods yield reconstruction errors that decay polynomially and exponentially in the number of measurements.
The paper studies how norms of random vectors are preserved by random projections.
problem Understanding how random matrix affects norms of random vectors.
method Proved the distribution of the norm of random vector is preserved by random projection.
result Random matrix preserves the distribution of the norm of random vectors with i.i.d. entries.
Tensorized Rademacher projections outperform Gaussian projections in reducing tensor dimensions.
problem Reducing the dimension of high-dimensional tensors for machine learning.
method Tensorized Rademacher random projections using Tensor Train decomposition.
result Tensorized Rademacher projections can replace Gaussian projections in tensor compression.
This work proposes a method to learn nonlinear feature relations using non-convex regularized binned regression.
problem Learning feature nonlinearities in large scale complex problems.
method Binning feature values, finding the best fit in each quantile using non-convex regularized linear regression, enforcing smoothness via piecewise-constant/linear approximation, and selecting a sparse subset of features.
result The proposed algorithm achieves linear rate of convergence while requiring near-minimal number of samples, accurately learning feature nonlinearities.
HD-BWDM improves clustering validation in high-dimensional data.
problem Determining the right number of clusters in high-dimensional data.
method HD-BWDM integrates random projection, PCA, trimmed clustering, and medoid-based distances.
result HD-BWDM remains stable and interpretable under high-dimensional projections and contamination.
Optimally sketches tensors with minimal rows for preserving norms.
problem Efficiently preserving tensor norms in high dimensions.
method Constructs a matrix M to preserve tensor norms with minimal rows.
result Optimal tensor sketch with minimal rows and improved application time.
This work improves tensor decomposition methods, especially for large datasets.
problem Lack of efficient methods for estimating Tucker decompositions.
method Applies Johnson-Lindenstrauss type guarantees to Tucker decompositions with random embeddings.
result Effective dimension reduction with minimal error for large tensors.
Sparse JL with higher sparsity improves feature hashing accuracy.
problem Efficiently reducing high-dimensional feature vectors to lower dimensions.
method Sparse Johnson-Lindenstrauss transform with varying sparsity levels.
result Sparse JL with sparsity greater than 1 provides better norm preservation.
New bounds on random quadratic forms hold under dependence, useful for adaptive modeling.
problem Need for independence in bounds on random quadratic forms.
method Uniform bounds on random quadratic forms of conditionally independent and sub-Gaussian stochastic processes.
result Bounds hold under general dependencies and sequential design.
A fast binary embedding method preserves Euclidean distances in high-dimensional data.
problem Preserving Euclidean distances in high-dimensional datasets.
method Stable noise-shaping quantization of A x A x A x with A A A a sparse Gaussian random matrix, followed by a linear transformation. result Euclidean distances are approximated by the ℓ 1 \ell_1 ℓ 1 norm on binary sequences, leading to accurate binary codes. From a fresh data science perspective, this thesis discusses the prediction of coronary artery disease based on genetic variations at the DNA base pair level, called Single-Nucleotide Polymorphisms (SNPs), collected from the Ontario Heart Genomics Study (OHGS). First, the thesis explains two commonly used supervised le…
New faster, space-saving methods for subspace embeddings in tensors.
problem Efficiently embedding large tensors with fewer random bits.
method Modewise Johnson-Lindenstrauss embeddings for rank- r r r tensors. result Improved space complexity for tensor subspaces with fewer random bits.
Improved machine learning performance through structured random orthogonal embeddings.
problem Improving accuracy and speed in machine learning applications.
method Structured random orthogonal matrices for dimensionality reduction and kernel approximation.
result Significant improvement in accuracy and speed compared to existing methods.
New method for computing terminal embeddings in sublinear time.
problem Efficiently computing terminal embeddings with sublinear time complexity.
method Developed a data structure to compute terminal embeddings in sublinear time.
result Achieved sublinear time computation of terminal embeddings.
Neural networks approximate high-dimensional functions better than theory predicts.
problem Current theory struggles to explain why small neural networks work well in high-dimensional inverse problems.
method Bounding complexity required for neural networks to approximate Hölder or uniformly continuous functions on high-dimensional sets.
result A general theoretical framework explaining empirical successes of smaller networks in inverse problems.
In this paper, we study a fast approximation method for {\it large-scale high-dimensional} sparse least-squares regression problem by exploiting the Johnson-Lindenstrauss (JL) transforms, which embed a set of high-dimensional vectors into a low-dimensional space. In particular, we propose to apply the JL transforms to …
New algorithm speeds up regression tasks for large datasets.
problem Efficiently solving optimization and regression tasks with large datasets.
method Iterative Hessian Sketching (IHS) combined with matrix sketching techniques.
result Significantly faster algorithms for constrained regression tasks.
New method speeds up machine learning computations using structured matrices.
problem Improving efficiency of machine learning computations, especially for nonlinear embeddings.
method Applying structured matrices to speed up randomized computations of kernels and multivariate functions.
result Significant reduction in space complexity and improvement in quality of embeddings.
Randomized matrix compression techniques, such as the Johnson-Lindenstrauss transform, have emerged as an effective and practical way for solving large-scale problems efficiently. With a focus on computational efficiency, however, forsaking solutions quality and accuracy becomes the trade-off. In this paper, we investi…
Tensorized random projections reduce high-dimensional tensor size efficiently.
problem Efficiently reducing the dimension of very high-dimensional tensors.
method Proposes two tensorized random projection maps using TT and CP decompositions.
result TT format offers superior performance in terms of required random projection size.
A new method uses matrix sketches for efficient graph clustering in dynamic environments.
problem Efficiently clustering large, dynamic graphs in distributed memory systems.
method Inspired by spectral clustering, the approach uses random dimension-reducing projections to derive matrix sketches.
result The method produces embeddings that yield performant clustering results in a fully-dynamic stochastic block model stream.
Extends classification of invariant measures on geodesic currents.
problem Classifying invariant measures on geodesic currents.
method Decomposition of currents into measured laminations, multi-curves, and bound currents.
result Extension of classification to geodesic currents.
Corrects bias in random sampling matrices for improved ML methods.
problem Inversion bias in random sampling matrices hampers ML applications.
method Corrects inversion bias for various random sampling methods.
result Establishes local convergence rates for sub-sampled Newton methods.