2-bit coding scheme improves near neighbor search efficiency.
problem Efficiently estimating data similarities in near neighbor search.
method Developed 2-bit quantization and nonlinear estimators.
result 2-bit strategy significantly reduces computational complexity.
Fair near neighbor search ensures equal probability of reporting points in a query's radius.
problem Ensuring equal probability of reporting points within a query's radius.
method LSH-based algorithms modified to ensure uniform probability of reporting points.
result Algorithm reports points in the query's radius with almost uniform probability.
Sublinear memory sketch finds nearest neighbors in streaming data.
problem Finding nearest neighbors in large datasets with limited memory.
method Combines LSH, online kernel density estimation, and compressed sensing to achieve sublinear memory.
result Achieves sublinear memory performance on stable queries, reporting nearest neighbors efficiently.
ForestDSH hashes improve nearest neighbor search in high-dimensional data.
problem High-dimensional classification and nearest neighbor search.
method Distribution-sensitive hashing using a forest of decision trees.
result ForestDSH hashes outperform LSH and state-of-the-art methods in speed and accuracy.
This paper compares FAISS and FENSHSES for nearest neighbor search in Hamming space.
problem Comparing nearest neighbor search systems in Hamming space.
method Comprehensive evaluations of indexing speed, search latency, and RAM consumption.
result Better understanding of trade-offs between main memory and secondary memory systems.
Recently it was shown that the problem of Maximum Inner Product Search (MIPS) is efficient and it admits provably sub-linear hashing algorithms. Asymmetric transformations before hashing were the key in solving MIPS which was otherwise hard. In the prior work, the authors use asymmetric transformations which convert th…
The paper explains how nearest neighbor methods succeed in prediction.
problem Explaining the success of nearest neighbor methods in prediction.
method The paper covers both theoretical and practical aspects of nearest neighbor methods, including statistical guarantees and practical algorithms.
result The paper provides nonasymptotic statistical guarantees and practical algorithms for nearest neighbor methods.
New method speeds up k-means clustering for large k by improving nearest-neighbor search.
problem Efficiently clustering large datasets with high-dimensional points.
method Seeded Approximate Nearest-Neighbor Search methods to improve Lloyd's algorithm.
result Significantly faster k-means clustering for large k values.
A new multilabel classification framework improves ANN search performance.
problem Efficiently finding approximate nearest neighbors in large datasets.
method Formulated ANN search as a multilabel classification problem, using partitioning classifiers.
result Natural classifier leads to strictly improved performance in ANN search.
ProbMinHash improves Jaccard similarity hashing for big data applications.
problem Efficiently estimating set similarities in big data with weighted elements.
method Locality-sensitive hash algorithms that calculate signatures collectively.
result Significantly faster than the original approach, with improved estimation error.
New methods use vector search and nearest-neighbor matching for policy learning in causal inference.
problem Learning optimal policies in causal inference with limited data.
method RAG-based policy learning with vector search and nearest-neighbor matching.
result The methods bound the within-candidate choice regret and evaluate the one-step method directly as a policy.
C-MinHash reduces the number of permutations needed for MinHash from thousands to just two.
problem Approximating Jaccard similarity in large binary datasets using many permutations.
method Initial permutation followed by circulant shifting of a second permutation to generate hashes.
result C-MinHash achieves unbiased Jaccard similarity estimation with uniformly smaller variance.
This paper improves nearest neighbor search by learning optimal routing functions.
problem Local minima issues in greedy routing on similarity graphs.
method Learn routing function that considers global graph structure.
result Significant improvement in search performance via learning.
The paper investigates learning conditional distributions on multi-dimensional spaces using clustering and neural networks.
problem Learning conditional distributions on multi-dimensional spaces with varying dimensions.
method The approach involves clustering data near varying query points in the feature space to create empirical measures in the target space using two clustering schemes: fixed-radius ball and nearest neighbors. The convergence rates of both methods are analyzed, and the nearest neighbors method is incorporated into neural network training.
result The empirical analysis shows that the nearest neighbors method has better performance in practice and can adapt to a suitable level of Lipschitz continuity locally.
Python package reduces hubness in high-dimensional data.
problem Hubness in high-dimensional data affects machine learning tasks.
method Hubness analysis, reduction, and approximate neighbor search.
result Improves neighbor retrieval and learning tasks in high dimensions.
Fast approximate nearest neighbor (NN) search in large databases is becoming popular. Several powerful learning-based formulations have been proposed recently. However, not much attention has been paid to a more fundamental question: how difficult is (approximate) nearest neighbor search in a given data set? And which …
Automatically tunes hyperparameters for faster approximate nearest neighbor search.
problem Tuning hyperparameters for efficient approximate nearest neighbor search is slow and impractical.
method Proposes an algorithm using randomized space-partitioning trees to automatically tune hyperparameters.
result Significantly faster than existing approaches and competitive in query time.
New method improves nearest neighbor search using neural networks and graph partitioning.
problem Efficient nearest neighbor search in high-dimensional spaces.
method Developed a new framework for space partitioning using neural networks and graph partitioning.
result Neural LSH partitions outperform existing methods on standard benchmarks.
Paper improves full-text search engines for fast exact NNS in binary codes.
problem Efficient nearest neighbor search in Hamming space for full-text search engines.
method Revisits and combines three techniques from information retrieval: bit operation, subs-code filtering, and data preprocessing with permutation.
result Significant speed-ups for NNS in binary codes over state-of-the-art term match approach.
Predicts vessel destinations using AIS data and nearest neighbor search.
problem Accurately predict the destination ports and arrival times of vessel trips.
method Partitioned training routes by destination port, use nearest neighbor search, and incorporate improvements like avoiding frequent port changes and automating parameter tuning.
result Significant improvements in prediction accuracy compared to baseline methods.
The paper enhances model robustness by using confidence information from adversarial training.
problem Improving adversarial robustness of machine learning models.
method Proposes a framework called tHCNN that combines confidence information and nearest neighbor search. result Demonstrates that confidence information from adversarial training can be used to distinguish between correct and incorrect predictions.
A clustering algorithm for natural hierarchical clusters with near-linear time complexity.
problem Hierarchical clustering with near-linear time complexity.
method Nearest neighbor based clustering algorithm that defines clusters naturally.
result Near-linear time and space complexity for certain datasets.
Agent finds hidden objects in DAGs, stopping and restarting.
problem Finding hidden objects in a directed acyclic graph.
method Sequential search-and-stop strategy, learning from unknown distributions.
result Near-optimal strategy for unknown distributions, maximizing object collection.
We present the first provably sublinear time algorithm for approximate \emph{Maximum Inner Product Search} (MIPS). Our proposal is also the first hashing algorithm for searching with (un-normalized) inner product as the underlying similarity measure. Finding hashing schemes for MIPS was considered hard. We formally sho…
Efficiently selects nearest neighbors for labeling to speed up active learning.
problem Intractable active learning and search for large-scale unlabeled data.
method Restricts candidate pool to nearest neighbors of labeled set.
result Achieved similar performance to global approach but reduced computational cost by up to 3 orders of magnitude.
Tunable GMM kernels improve on original GMM in various classification tasks.
problem Improving the efficiency and performance of GMM kernels.
method Developed three tunable GMM kernels: eGMM, pGMM, and epGMM.
result Tunable GMM kernels typically improve over the original GMM kernel on 60 datasets.
SOLAR improves search efficiency and accuracy with sparse, orthogonal embeddings.
problem Bottleneck of indexing large dense vectors and NNS for query efficiency and accuracy.
method Proposes SOLAR embeddings: sparse, orthogonal, learned, and random vectors across multiple GPUs.
result Successfully trains 500K dimensional SOLAR embeddings for 1.6M books and multi-label classification.
rpForests uses random projections to speed up kNN search.
problem Efficient kNN search in high-dimensional data.
method Random projection forests aggregating results from multiple random projection trees.
result rpForests achieves fast decay in missing rate and kNN distance discrepancy.
New algorithm improves similarity graph construction for nearest neighbor search.
problem Improving nearest neighbor search performance with more effective similarity graphs.
method Probabilistic model of a similarity graph learned through reinforcement learning.
result Higher recall rates achieved for the same number of distance computations.
The paper analyzes how much data points can be altered to change their rank in nearest neighbor searches.
problem Vulnerability of nearest neighbor search in high-dimensional data.
method Statistical analysis of perturbation needed to change neighbor rank.
result Derived statistical distribution of perturbation needed to modify neighbor rank.
This paper provides fast estimates for complex option types.
problem Estimating prices for constrained multiple exercise American options.
method Lookahead search for lower estimates and nearest-neighbor martingale for upper estimates.
result Probabilistic convergence guarantees for the algorithms.
A comparison-based algorithm finds nearest neighbors in metric spaces.
problem Finding nearest neighbors without direct distance information.
method Recursive splitting using random pivot points to form a comparison tree.
result The height of the comparison tree is logarithmic in the number of points, leading to efficient search performance.
Efficiently find approximate nearest neighbors in high dimensions.
problem Finding the closest point in a high-dimensional dataset.
method Develops efficient data structures for approximate nearest neighbor search.
result Efficient solutions for approximate nearest neighbor problem.
ICQ improves high-dimensional similarity search without sacrificing precision.
problem High-dimensional similarity search is computationally expensive.
method Interleaved Composite Quantization (ICQ) reduces code length and quantization error.
result ICQ achieves fast similarity search without using shorter codes.
New LSH algorithms improve nearest-neighbor search performance.
problem Efficiently searching for similar high-dimensional data.
method High-dimensional locality-sensitive hashing (LSH) based on fruit fly olfactory circuit.
result New LSH algorithms outperform existing methods on benchmark datasets.
This paper surveys various methods for dimensionality reduction and nearest neighbor search.
problem Efficiently reducing high-dimensional data to lower dimensions while preserving essential information.
method Linear and nonlinear random projections, including sparse random projections, random Fourier Features, and Random Kitchen Sinks.
result Various methods for dimensionality reduction and nearest neighbor search are explained and compared.
KL nearest neighbor estimator achieves near-minimax rates for differential entropy.
problem Estimating differential entropy with Hölder smoothness.
method Uniform upper and lower bounds on KL estimator performance.
result KL estimator is near-minimax rate-optimal without knowing smoothness.
Neighbor-encoder learns representations by reconstructing neighbors, outperforming autoencoders.
problem Learning effective representations for various data types.
method Reconstructs neighbors instead of inputs, incorporating domain knowledge through similarity definitions.
result Neighbor-encoder outperforms autoencoders in diverse domains and tasks.
ADSH uses asymmetric deep hashing for faster training and better performance.
problem Efficiently utilizing supervised information for large-scale database in deep supervised hashing.
method ADSH learns a deep hash function only for query points and directly learns hash codes for database points.
result ADSH achieves state-of-the-art performance in real applications.
Study nearest-neighbor radii under dependent sampling, finding they remain informative.
problem Analyzing nearest-neighbor radii under dependent sampling.
method Consider strong mixing dependent observations, establish distribution-free almost sure convergence and sharp non-asymptotic moment bounds.
result Nearest-neighbor geometry remains informative under dependence sampling.
Efficiently find near-optimal medical treatments with less trial and error.
problem Finding effective medical treatments through trial and error.
method Formalizes the problem, uses a causal inference framework, and proposes model-based dynamic programming and greedy algorithms.
result Our methods compare favorably to model-free reinforcement learning, offering a more transparent trade-off between search time and treatment efficacy.
Efficiently approximates time series correlation using Fourier transform and neural networks.
problem Efficiently approximating correlation in time series data.
method Embeds time series into a low-dimensional Euclidean space using Fourier transform and neural networks, ensuring accurate correlation approximation from Euclidean distance.
result Our method reduces approximation loss by half and improves top-k correlation search precision from 5% to 20%. Adaptive algorithm speeds up k-nearest-neighbor searches.
problem Finding k nearest neighbors with varying efficiency.
method Adaptive estimation of distances to optimize search.
result The algorithm achieves significant speedups compared to naive methods.
Unsupervised space partitioning improves ANNS performance without pre-processing.
problem Efficient nearest neighbor search in high-dimensional spaces.
method Custom unsupervised learning framework for space partitioning and learning-to-search.
result Our method outperforms state-of-the-art approaches on ANNS benchmarks.
BOCK optimizes Bayesian Optimization by transforming the search space to reduce boundary evaluations.
problem Bayesian Optimization struggles with boundary issues, wasting evaluations near the search space boundary.
method BOCK uses a cylindrical transformation to redirect Gaussian Process efforts away from the boundary and towards the center of the search space.
result BOCK achieves better accuracy and efficiency, scaling to high-dimensional problems and optimizing neural network layers and hyperparameters.
SANNS secures k-NNS queries and results without revealing them.
problem Securely searching for nearest neighbors without exposing sensitive data.
method Optimized linear scan and clustering-based algorithm with cryptographic primitives.
result SANNS achieves up to 31x faster response times and scales to 10 million entries.
Algorithm finds adversarial examples for k-NN classifiers using Voronoi diagrams.
problem Ensuring robustness of k-NN classifiers against adversarial attacks.
method Geometric approach expanding outwards from input points to find minimum-norm adversarial examples.
result Our method outperforms existing approaches on various datasets.
The paper analyzes how modern machine learning models can achieve zero training error and robust generalization.
problem Understanding why modern machine learning models achieve strong generalization despite achieving zero training error.
method The paper analyzes local interpolating schemes including geometric simplicial interpolation and singularly weighted k-nearest neighbor methods.
result The nearest neighbor schemes exhibit optimal rates under standard statistical assumptions and provide insights into adversarial examples.