New LSH methods for tensor data improve efficiency and space usage.
problem Efficiency and space usage issues in LSH for tensor data.
method Proposes new LSH methods using CP and TT decompositions for Euclidean and cosine similarity.
result Space-efficient and scalable LSH for tensor data.
We present a generic compact computational framework relying on structured random matrices that can be applied to speed up several machine learning algorithms with almost no loss of accuracy. The applications include new fast LSH-based algorithms, efficient kernel computations via random feature maps, convex optimizati…
Norm-ranging LSH improves MIPS performance by addressing 2-norm distribution issues.
problem Long tails in 2-norm distribution of real datasets affect Simple-LSH performance.
method Norm-ranging LSH partitions datasets into sub-datasets and builds independent hash indexes.
result Norm-ranging LSH achieves an order of magnitude speedup over Simple-LSH.
We consider the problem of designing locality sensitive hashes (LSH) for inner product similarity, and of the power of asymmetric hashes in this context. Shrivastava and Li argue that there is no symmetric LSH for the problem and propose an asymmetric LSH based on different mappings for query and database points. Howev…
BioHash improves similarity search performance using sparse high-dimensional hash codes.
problem Improving similarity search performance in high-dimensional data.
method BioHash produces sparse high-dimensional hash codes through a data-driven approach based on synaptic plasticity.
result BioHash outperforms previous hashing methods in various similarity search tasks.
Locality sensitive hashing (LSH) is a powerful tool for sublinear-time approximate nearest neighbor search, and a variety of hashing schemes have been proposed for different dissimilarity measures. However, hash codes significantly depend on the dissimilarity, which prohibits users from adjusting the dissimilarity at q…
The paper compares different hashing schemes for their performance on structured data.
problem The performance of hashing schemes on structured input is not well understood.
method The paper compares mixed tabulation hashing, multiply-mod-prime hashing, and MurmurHash3.
result Mixed tabulation hashing performs similarly to truly random hashing but is faster and has a proven guarantee.
LSH methods extend to function spaces for efficient similarity search.
problem Efficient similarity search in function spaces.
method Locality-sensitive hashing (LSH) extended to Lp spaces using function approximation or Monte Carlo techniques. result An LSH family for Wasserstein distance over continuous probability distributions.
Norm-range partition improves MIPS search efficiency by reducing query complexity.
problem Efficiently searching for maximum inner product in large datasets.
method Norm-range partition technique that divides datasets into sub-datasets with similar norms and builds independent hash indexes.
result Significantly reduces the number of probed buckets for LSH-based MIPS algorithms.
We present the first provably sublinear time algorithm for approximate \emph{Maximum Inner Product Search} (MIPS). Our proposal is also the first hashing algorithm for searching with (un-normalized) inner product as the underlying similarity measure. Finding hashing schemes for MIPS was considered hard. We formally sho…
Efficient clustering in high dimensions with Quick Shift and LSH.
problem Density-based clustering in high-dimensional data.
method Combines Quick Shift and LSH for efficient density estimation.
result Achieves almost linear time complexity for consistency.
New method improves nearest neighbor search using neural networks and graph partitioning.
problem Efficient nearest neighbor search in high-dimensional spaces.
method Developed a new framework for space partitioning using neural networks and graph partitioning.
result Neural LSH partitions outperform existing methods on standard benchmarks.
A new MI estimator reduces complexity to linear time, achieving optimal MSE rates.
problem High computational complexity of MI estimators.
method Ensemble Dependency Graph Estimator (EDGE) combining LSH, dependency graphs, and ensemble bias-reduction.
result EDGE achieves optimal computational complexity O(N) and parametric MSE rate O(1/N). Develops LSH schemes for f-divergences and mutual information loss.
problem Approximating nearest neighbors in high-dimensional probability distributions.
method General framework and specific LSH schemes for f-divergences and mutual information loss.
result Generalized Jensen-Shannon divergence can be approximated by Hellinger distance.
C-kNN-LSH identifies similar patient histories for causal inference in longitudinal data.
problem Estimating causal effects from longitudinal trajectories with high-dimensional confounding.
method C-kNN-LSH uses locality-sensitive hashing to find clinical twins and estimate treatment effects.
result C-kNN-LSH outperforms existing methods in capturing recovery heterogeneity and estimating policy values.
Paper proposes using LSTM for LSH-based sequence alignment.
problem Sequence alignment using deep learning models.
method Deep bidirectional LSTM for feature learning and LSH-based sequence alignment.
result Higher accuracy achieved with LSTM-based model.
Sublinear memory sketch finds nearest neighbors in streaming data.
problem Finding nearest neighbors in large datasets with limited memory.
method Combines LSH, online kernel density estimation, and compressed sensing to achieve sublinear memory.
result Achieves sublinear memory performance on stable queries, reporting nearest neighbors efficiently.
Sparse neural networks can match dense models on Lipschitz functions.
problem Sparse networks are more efficient but lack theoretical guarantees.
method Formal model of sparse networks, LSH-based routing function, Lipschitz function approximation.
result Sparse networks can approximate dense networks on Lipschitz functions.
LGD breaks the chicken-and-egg loop in adaptive SGD by using LSH sampling.
problem Challenging per-iteration cost of adaptive gradient sampling.
method Locality Sensitive Hashing (LSH) sampled Stochastic Gradient Descent (LGD).
result Superior and faster gradient estimation with similar per-iteration cost.
New LSH algorithms improve nearest-neighbor search performance.
problem Efficiently searching for similar high-dimensional data.
method High-dimensional locality-sensitive hashing (LSH) based on fruit fly olfactory circuit.
result New LSH algorithms outperform existing methods on benchmark datasets.
Sublinear LSVI via LSH reduces runtime to sublinear in actions.
problem Efficiently estimating value functions in reinforcement learning with sublinear runtime.
method Formulated as approximate maximum inner product search, used LSH to solve with sublinear time complexity.
result Sublinear runtime while maintaining LSVI's regret.
Paper finds periodic orbits for convex Lagrangian systems on noncompact manifolds.
problem Existence of periodic orbits in convex Lagrangian systems on complete Riemannian manifolds.
method Developed a modified minimax principle to prove the existence of periodic orbits.
result Proved the existence of contractible periodic orbits for almost every energy level.
New hashing methods improve entity resolution in conflict datasets.
problem Improving entity resolution in databases with unknown unique identifiers.
method Introducing and comparing new locality sensitive hashing methods.
result New hashing methods outperform existing approaches in conflict datasets.
A new model predicts network events with improved accuracy and interpretability.
problem Predicting and understanding complex dynamic relational data in networks.
method Mutually Exciting Latent Space Hawkes (LSH) model for continuous-time networks.
result The LSH model outperforms existing models in prediction accuracy and interpretability.
Fair near neighbor search ensures equal probability of reporting points in a query's radius.
problem Ensuring equal probability of reporting points within a query's radius.
method LSH-based algorithms modified to ensure uniform probability of reporting points.
result Algorithm reports points in the query's radius with almost uniform probability.
ForestDSH hashes improve nearest neighbor search in high-dimensional data.
problem High-dimensional classification and nearest neighbor search.
method Distribution-sensitive hashing using a forest of decision trees.
result ForestDSH hashes outperform LSH and state-of-the-art methods in speed and accuracy.
MinHash and SimHash are the two widely adopted Locality Sensitive Hashing (LSH) algorithms for large-scale data processing applications. Deciding which LSH to use for a particular problem at hand is an important question, which has no clear answer in the existing literature. In this study, we provide a theoretical answ…
New algorithm finds nearest neighbors with less computation.
problem Finding nearest neighbors in high-dimensional data efficiently.
method Bandit-based Monte Carlo optimization for nearest neighbors.
result Algorithm identifies exact nearest neighbors with high probability.
Existing methods for retrieving k-nearest neighbours suffer from the curse of dimensionality. We argue this is caused in part by inherent deficiencies of space partitioning, which is the underlying strategy used by most existing methods. We devise a new strategy that avoids partitioning the vector space and present a n…
Paper proposes an approximate margin method for fast multi-class classification.
problem Challenges in multi-class classification with many classes.
method Uses ANN search structures and LSH for approximate margin estimation.
result Approximate margin method is highly competitive in time, memory, and performance.
Log-linear models are arguably the most successful class of graphical models for large-scale applications because of their simplicity and tractability. Learning and inference with these models require calculating the partition function, which is a major bottleneck and intractable for large state spaces. Importance Samp…
Paper proposes efficient inner product approximation for hybrid sparse and dense vectors.
problem Efficient search in hybrid spaces with both sparse and dense components is challenging.
method Proposes a technique to approximate inner product computation in hybrid vectors.
result Achieves over 10x speedup and higher accuracy in search compared to baselines.
Two log-linear approximations speed up optimal transport for deep learning applications.
problem Computing optimal transport in high dimensions is computationally expensive.
method Locality-sensitive hashing (LSH) and Nyström approximation with LSH-based sparse corrections.
result Log-linear time algorithms for entropy-regularized OT perform well in high-dimensional spaces.
New split-merge MCMC proposals improve efficiency and speed.
problem Scaling issues in split-merge MCMC for large datasets.
method Locality Sensitive Sampling (LSS) combined with weighted MinHash.
result Significantly faster than state-of-the-art methods on large datasets.
ACE algorithm detects anomalies faster and uses less memory.
problem Anomaly detection in big data with memory and latency constraints.
method Arrays of Count Estimators (ACE) using Locality Sensitive Hashing (LSH).
result 60x faster than existing methods, requires <4MB memory.
Efficiently clusters large datasets with approximate nearest neighbors.
problem Efficiently clustering large datasets with high computational cost.
method Approximates density gradient ascent and nearest neighbors using LSH for scalable clustering.
result Improves clustering accuracy and scalability for large datasets.
Method learns radial basis function distributions from samples.
problem Learning radial basis function distributions from training samples.
method Projected particle Langevin optimization method with distributionally robust optimization.
result Empirical measure of Langevin particles converges to a reflected Itô diffusion-drift process.
New sublinear sketches improve ANN and KDE for massive data streams.
problem Efficiently approximate nearest neighbors and kernel density estimation in large datasets.
method Developed sublinear space and query time algorithms for ANN and A-KDE in streaming and sliding-window models.
result Achieved near-optimal trade-offs between memory size and approximation error for ANN.
This paper calculates data valuation for nearest neighbor models efficiently.
problem Distributing payment for training ML models based on data contributions.
method Defined 'relative value of data' via Shapley value for fairness and decentralizability. Developed algorithms for exact and approximate computation of Shapley values for nearest neighbor models.
result Exact computation of Shapley values for nearest neighbor models in O(N log N) time, significantly faster than previous methods.
We propose a quantization based approach for fast approximate Maximum Inner Product Search (MIPS). Each database vector is quantized in multiple subspaces via a set of codebooks, learned directly by minimizing the inner product quantization error. Then, the inner product of a query to a database vector is approximated …
Efficient Maximum Inner Product Search (MIPS) is an important task that has a wide applicability in recommendation systems and classification with a large number of classes. Solutions based on locality-sensitive hashing (LSH) as well as tree-based solutions have been investigated in the recent literature, to perform ap…
Most exact methods for k-nearest neighbour search suffer from the curse of dimensionality; that is, their query times exhibit exponential dependence on either the ambient or the intrinsic dimensionality. Dynamic Continuous Indexing (DCI) offers a promising way of circumventing the curse and successfully reduces the dep…
Combines fast evaluation with Bayes consistency in nearest neighbors.
problem Balancing fast evaluation time with Bayes consistency in nearest neighbors.
method Combines locality-sensitive hashing (LSH) with a missing-mass argument.
result Fast and Bayes-consistent classifier with comparable risk decay rates.
This paper improves nearest neighbor search by learning optimal routing functions.
problem Local minima issues in greedy routing on similarity graphs.
method Learn routing function that considers global graph structure.
result Significant improvement in search performance via learning.
Improved KNN data valuation method with reduced computation time.
problem Efficiently valuing individual data points in KNN models.
method Proposed a new utility function and derived its calculation for KNN classifiers/regressors, achieving similar time complexity as the original method.
result Soft-label KNN-SV outperforms the original method in mislabeled data detection.
Sketch-GNN reduces GNN training time and memory usage to sublinear scales.
problem Training GNNs on large graphs is computationally expensive and memory-intensive.
method Develops a sketch-based algorithm that trains GNNs on compact sketches of graph adjacency and node embeddings.
result Training time and memory usage grow sublinearly with respect to graph size.
Proposes MC-AE for better unsupervised clustering of unlabeled data.
problem Lack of consideration for multi-local collaborative relationships in autoencoders.
method Integrates LSH for multi-local cross blocks, mcrRBM and mcrGRBM models.
result MC-AE improves unsupervised clustering performance.
This paper surveys various methods for dimensionality reduction and nearest neighbor search.
problem Efficiently reducing high-dimensional data to lower dimensions while preserving essential information.
method Linear and nonlinear random projections, including sparse random projections, random Fourier Features, and Random Kitchen Sinks.
result Various methods for dimensionality reduction and nearest neighbor search are explained and compared.