This paper improves nearest neighbor search by learning optimal routing functions.
problem Local minima issues in greedy routing on similarity graphs.
method Learn routing function that considers global graph structure.
result Significant improvement in search performance via learning.
Study shows SNN graph Laplacians converge to k-NN graph Laplacians under large scale asymptotics.
problem Understanding the convergence of SNN graph Laplacians to k-NN graph Laplacians.
method Analyzing the asymptotic behavior of SNN and k-NN graph Laplacians.
result The graph Laplacians of SNN and k-NN graphs converge to the same limit under large scale asymptotics.
Method reconstructs missing wind farm data using graph theory and nearest neighbors.
problem Missing data in wind farm records due to sensor failures.
method Combines spectral graph theory and k-Nearest Neighbors to estimate missing data.
result Significant improvement in data reconstruction over existing methods.
We study clustering algorithms based on neighborhood graphs on a random sample of data points. The question we ask is how such a graph should be constructed in order to obtain optimal clustering results. Which type of neighborhood graph should one choose, mutual k-nearest neighbor or symmetric k-nearest neighbor? What …
Algorithm learns nearest neighbor graph from noisy distance queries.
problem Learning nearest neighbor graph from noisy distance samples.
method Active algorithm to find graph with high probability, analyzing query complexity.
result Empirically and theoretically efficient, needing only O(n log(n)Delta^-2) queries.
If we pick n random points uniformly in [0,1]d and connect each point to its k−nearest neighbors, then it is well known that there exists a giant connected component with high probability. We prove that in [0,1]d it suffices to connect every point to cd,1loglogn points chosen randomly among its $…
Consider a weighted or unweighted k-nearest neighbor graph that has been built on n data points drawn randomly according to some density p on R^d. We study the convergence of the shortest path distance in such graphs as the sample size tends to infinity. We prove that for unweighted kNN graphs, this distance converges …
New method constructs graphs from data efficiently, suitable for large datasets.
problem Memory and runtime limitations of traditional TMFG for large datasets.
method Uses k-Nearest Neighbors Graphs and memory management for scalable graph construction.
result Provides a parsimonious way to construct graphs for learning tasks.
A new graph-based clustering method for moderate-dimensional data.
problem Performance degradation of existing graph-based clustering methods in high dimensions.
method Introduces UN-CCDs using NND-based MC-SRT for covering radii determination.
result UN-CCDs provide stable and competitive performance in moderate-sized datasets.
k Nearest Neighbors (kNN) is one of the most widely used supervised learning algorithms to classify Gaussian distributed data, but it does not achieve good results when it is applied to nonlinear manifold distributed data, especially when a very limited amount of labeled samples are available. In this paper, we pro…
New method speeds up k-means clustering for large k by improving nearest-neighbor search.
problem Efficiently clustering large datasets with high-dimensional points.
method Seeded Approximate Nearest-Neighbor Search methods to improve Lloyd's algorithm.
result Significantly faster k-means clustering for large k values.
ASK-NN detects distribution drifts in LLM-generated text.
problem Hallucinations and artificial text in LLM-generated outputs.
method Asymmetric two-sample test based on directed k-nearest-neighbor graph.
result ASK-NN is competitive with baselines on various benchmarks.
New algorithms detect outliers in high-dimensional data with arbitrary shapes.
problem Challenges of high dimensionality and varying cluster shapes in traditional outlier detection methods.
method Cluster Catch Digraphs (CCDs) and their variants (U-MCCD, UN-MCCD, SU-MCCD, SUN-MCCD).
result U-MCCD efficiently identifies outliers with high true negative rates, and SU-MCCD improves handling of non-uniform clusters.
New algorithm improves similarity graph construction for nearest neighbor search.
problem Improving nearest neighbor search performance with more effective similarity graphs.
method Probabilistic model of a similarity graph learned through reinforcement learning.
result Higher recall rates achieved for the same number of distance computations.
The paper studies recovering hidden nearest neighbor graphs in large networks.
problem Discovering strong ties in social networks and assembling genome subsequences.
method Maximum likelihood estimator for recovering hidden 2k-nearest neighbor graphs. result The maximum likelihood estimator achieves asymptotic recovery guarantees under specific conditions.
We present a simple, yet effective, approach to Semi-Supervised Learning. Our approach is based on estimating density-based distances (DBD) using a shortest path calculation on a graph. These Graph-DBD estimates can then be used in any distance-based supervised learning method, such as Nearest Neighbor methods and SVMs…
UMAP's true loss function differs from what was previously thought, focusing on nearest neighbor graph similarities.
problem Understanding the true effectiveness and mechanism of UMAP for high-dimensional data visualization.
method Deriving UMAP's effective loss function in closed form and analyzing its optimization scheme.
result UMAP aims to reproduce similarities encoded in the nearest neighbor graph, not the full high-dimensional similarities.
New clustering algorithm for mixed data improves applicability and efficiency.
problem Clustering large, mixed data with improved accuracy and efficiency.
method Developed a new clustering algorithm using peak-finding technique, reducing computational complexity.
result Algorithm detects outliers, clusters of lower density, and determines correct number of clusters.
Study nearest-neighbor radii under dependent sampling, finding they remain informative.
problem Analyzing nearest-neighbor radii under dependent sampling.
method Consider strong mixing dependent observations, establish distribution-free almost sure convergence and sharp non-asymptotic moment bounds.
result Nearest-neighbor geometry remains informative under dependence sampling.
Deep nearest neighbors outperform self-supervised methods in anomaly detection.
problem Anomaly detection using self-supervised deep methods.
method Simple nearest-neighbor approach on Imagenet pretrained features.
result Nearest-neighbor method outperforms self-supervised methods in accuracy, few shot generalization, training time, and noise robustness.
Characterizes Lebesgue points using nearest neighbor methods.
problem Consistency of classification algorithms based on nearest neighbors.
method Characterization of Lebesgue points via 1-Nearest Neighbor regression.
result Proves convergence of 1-Nearest Neighbor classification algorithms in metric spaces.
The paper explains how nearest neighbor methods succeed in prediction.
problem Explaining the success of nearest neighbor methods in prediction.
method The paper covers both theoretical and practical aspects of nearest neighbor methods, including statistical guarantees and practical algorithms.
result The paper provides nonasymptotic statistical guarantees and practical algorithms for nearest neighbor methods.
We present simple and computationally efficient nonparametric estimators of Rényi entropy and mutual information based on an i.i.d. sample drawn from an unknown, absolutely continuous distribution over Rd. The estimators are calculated as the sum of p-th powers of the Euclidean lengths of the edges of the `genera…
In the k-nearest neighborhood model (k-NN), we are given a set of points P, and we shall answer queries q by returning the k nearest neighbors of q in P according to some metric. This concept is crucial in many areas of data analysis and data processing, e.g., computer vision, document retrieval and machi…
AWNN improves matrix completion by adaptively weighting nearest neighbors.
problem Matrix completion with optimal nearest neighbor weights and radii selection.
method Adaptively weighted nearest neighbor method for matrix completion.
result Theoretical guarantees and synthetic experiments support the effectiveness of AWNN.
A new method uses nearest neighbors for importance weighting.
problem Data covariate shift problems in machine learning.
method Nearest neighbor classification scheme for determining importance weights.
result Demonstrated effectiveness through comparative experiments on various classification tasks.
Adaptive algorithm speeds up k-nearest-neighbor searches.
problem Finding k nearest neighbors with varying efficiency.
method Adaptive estimation of distances to optimize search.
result The algorithm achieves significant speedups compared to naive methods.
This paper compares FAISS and FENSHSES for nearest neighbor search in Hamming space.
problem Comparing nearest neighbor search systems in Hamming space.
method Comprehensive evaluations of indexing speed, search latency, and RAM consumption.
result Better understanding of trade-offs between main memory and secondary memory systems.
Space partitions of Rd underlie a vast and important class of fast nearest neighbor search (NNS) algorithms. Inspired by recent theoretical work on NNS for general metric spaces [Andoni, Naor, Nikolov, Razenshteyn, Waingarten STOC 2018, FOCS 2018], we develop a new framework for building space partitions re…
Nearest neighbor methods are a popular class of nonparametric estimators with several desirable properties, such as adaptivity to different distance scales in different regions of space. Prior work on convergence rates for nearest neighbor classification has not fully reflected these subtle properties. We analyze the b…
The estimation of optimal treatment regimes is of considerable interest to precision medicine. In this work, we propose a causal k-nearest neighbor method to estimate the optimal treatment regime. The method roots in the framework of causal inference, and estimates the causal treatment effects within the nearest neig…
A defense against adversarial examples using k-Nearest Neighbor and deep learning.
problem Evaluating robustness of k-Nearest Neighbor and its deep learning combination.
method Proposed heuristic attack to find adversarial examples for kNN and DkNN.
result Our attack significantly outperforms other attacks on DkNN.
VNNGP uses nearest neighbors to approximate GPs, improving scalability and performance.
problem Scalability issues in Gaussian process approximations.
method Sparse precision structure via nearest neighbors, variational framework.
result VNNGP outperforms low-rank methods and is less prone to overfitting.
Enhanced KNN-TSVM reduces noise and overfitting with stable learning rules.
problem High computational cost and overfitting in KNN-based TSVM classifiers.
method Weighted KNN with stabilizer term and LDMDBA for efficient computation.
result Effective in both classification accuracy and computational time, with up to 14x speedup.
BigNN classifier improves nearest neighbor classification for large datasets.
problem Classification of large datasets that cannot fit into a single machine's memory.
method Divide and conquer scheme with majority voting for final decision; pre-training acceleration technique.
result Rates of convergence for bigNN classifier under minimal assumptions, proving it as optimal.
Introduces MFVDM for high-dimensional data analysis.
problem Non-linear dimensionality reduction of high-dimensional datasets.
method Combines multiple unitary irreducible representations for nonlinear embeddings.
result Achieves better nearest neighbor search and alignment estimation on noisy data.
Java implementation improves nearest neighbor algorithm complexity.
problem Improving efficiency of nearest neighbor descent algorithm.
method Parallel streams implementation with statistical termination criterion.
result Complexity up to O(nK2logK(n)) for K-nearest neighbors. The paper tackles time series data by applying conformal prediction with nearest neighbors.
problem Time series data violates the exchangeability assumption required for conformal prediction.
method The approach uses the nearest neighbors method with fast parameter tuning and weighted nearest neighbors (FPTO-WNN) to construct reliable prediction intervals.
result Data analysis shows the effectiveness of the proposed approach.
PPC learns binary codes from data similarities and dissimilarities.
problem Creating efficient binary codes from data similarities and dissimilarities.
method PPC learns binary codes by modeling attractive and repulsive forces in a signed graph.
result PPC achieves superior results in nearest-neighbor searches compared to spectral methods.
A new multilabel classification framework improves ANN search performance.
problem Efficiently finding approximate nearest neighbors in large datasets.
method Formulated ANN search as a multilabel classification problem, using partitioning classifiers.
result Natural classifier leads to strictly improved performance in ANN search.
K-Nearest neighbor classifier (k-NNC) is simple to use and has little design time like finding k values in k-nearest neighbor classifier, hence these are suitable to work with dynamically varying data-sets. There exists some fundamental improvements over the basic k-NNC, like weighted k-nearest neighbors classifier (wh…
The paper studies empirical processes from nearest neighbors in regression.
problem Estimating conditional cumulative distribution functions and local linear regression.
method Uniform central limit theorem and non-asymptotic bound under local bracketing entropy and uniform entropy numbers.
result Gaussian limit of empirical process with simple covariance.
This paper proposes a new hashing-based KNN technique for faster nearest neighbor selection.
problem Slowness of KNN in big datasets due to searching entire dataset.
method Divide data space into subcells, use hashing to map data points, and select nearest neighbors layer by layer.
result The proposed technique offers competitive performance with KNN and KDtree while significantly improving time efficiency.
New methods use vector search and nearest-neighbor matching for policy learning in causal inference.
problem Learning optimal policies in causal inference with limited data.
method RAG-based policy learning with vector search and nearest-neighbor matching.
result The methods bound the within-candidate choice regret and evaluate the one-step method directly as a policy.
Sublinear memory sketch finds nearest neighbors in streaming data.
problem Finding nearest neighbors in large datasets with limited memory.
method Combines LSH, online kernel density estimation, and compressed sensing to achieve sublinear memory.
result Achieves sublinear memory performance on stable queries, reporting nearest neighbors efficiently.
kNN-MTS improves MTS forecasting by using nearest neighbor retrieval over a large dataset.
problem Limited ability of current MTS forecasting methods to identify similar patterns and handle sparsely distributed correlations.
method kNN-MTS framework using nearest neighbor retrieval over a large datastore of cached series, with representations from MTS model for similarity search.
result Significant improvement in forecasting performance on real-world datasets.
The nearest neighbor problem is defined as follows: Given a set P of n points in some metric space (X,D), build a data structure that, given any point q, returns a point in P that is closest to q (its "nearest neighbor" in P). The data structure stores additional information about the set P, which is th…
In our previous works, we proposed a physically-inspired rule to organize the data points into an in-tree (IT) structure, in which some undesired edges are allowed to occur. By removing those undesired or redundant edges, this IT structure is divided into several separate parts, each representing one cluster. In this w…