New method accelerates large margin metric learning for nearest neighbor classification.
problem Efficiently learning metrics for nearest neighbor classification.
method Triplet mining and stratified sampling for large margin metric learning.
result Improved efficiency and scalability of optimization.
SqueezeFit reduces high-dimensional data to lower dimensions while preserving label distances.
problem Label-aware dimensionality reduction in high-dimensional spaces.
method Semidefinite programming relaxation of nearest neighbor classification.
result Provable recovery of a planted projection operator from labeled data.
Novel active learning algorithm with improved convergence rate under local smoothness condition.
problem Improving convergence rates in active learning under specific smoothness assumptions.
method Developed a novel active learning algorithm with a rate of convergence better than in passive learning, using a local smoothness assumption for k-nearest neighbors.
result The algorithm achieves a better convergence rate than passive learning algorithms, avoiding strong density assumptions.
kNN-MTS improves MTS forecasting by using nearest neighbor retrieval over a large dataset.
problem Limited ability of current MTS forecasting methods to identify similar patterns and handle sparsely distributed correlations.
method kNN-MTS framework using nearest neighbor retrieval over a large datastore of cached series, with representations from MTS model for similarity search.
result Significant improvement in forecasting performance on real-world datasets.
Enhances motion data analysis using metric learning for DTW.
problem Improving classification accuracy in motion capture data analysis.
method Extends LMNN principle to DTW, treating component-wise dissimilarity values as features.
result Significantly enhances classification accuracy in motion capture data analysis.
Characterizes Lebesgue points using nearest neighbor methods.
problem Consistency of classification algorithms based on nearest neighbors.
method Characterization of Lebesgue points via 1-Nearest Neighbor regression.
result Proves convergence of 1-Nearest Neighbor classification algorithms in metric spaces.
A defense against adversarial examples using k-Nearest Neighbor and deep learning.
problem Evaluating robustness of k-Nearest Neighbor and its deep learning combination.
method Proposed heuristic attack to find adversarial examples for kNN and DkNN.
result Our attack significantly outperforms other attacks on DkNN.
Given an n-sample of random vectors (Xi,Yi)1≤i≤n whose joint law is unknown, the long-standing problem of supervised classification aims to \textit{optimally} predict the label Y of a given a new observation X. In this context, the nearest neighbor rule is a popular flexible and intuitive method …
BigNN classifier improves nearest neighbor classification for large datasets.
problem Classification of large datasets that cannot fit into a single machine's memory.
method Divide and conquer scheme with majority voting for final decision; pre-training acceleration technique.
result Rates of convergence for bigNN classifier under minimal assumptions, proving it as optimal.
This paper compares FAISS and FENSHSES for nearest neighbor search in Hamming space.
problem Comparing nearest neighbor search systems in Hamming space.
method Comprehensive evaluations of indexing speed, search latency, and RAM consumption.
result Better understanding of trade-offs between main memory and secondary memory systems.
New method speeds up k-means clustering for large k by improving nearest-neighbor search.
problem Efficiently clustering large datasets with high-dimensional points.
method Seeded Approximate Nearest-Neighbor Search methods to improve Lloyd's algorithm.
result Significantly faster k-means clustering for large k values.
Improved nearest neighbor classification for time series data.
problem High storage and computation requirements for large training sets in DTW space.
method Extends LVQ to DTW spaces using asymmetric weighted averaging.
result Asymmetric GLVQ outperforms other methods in nearest neighbor classification.
This study observes accuracy variations of KNN, SVM, LMNN, and ENN on 11 UCI datasets.
problem To observe the variation of accuracies of KNN, SVM, LMNN, and ENN algorithms on different datasets.
method Implemented KNN, SVM, LMNN, and ENN on 11 UCI datasets and analyzed their accuracy.
result Comparison of algorithm performances on each dataset.
A novel distributed adaptive NN classifier for large data sets.
problem Handling large and distributed data for efficient classification.
method Distributed adaptive nearest neighbor classifier with stochastic tuning parameter selection and early stopping rule.
result Achieves nearly optimal convergence rate under large sub-sample sizes.
We show that a simple modification of the 1-nearest neighbor classifier yields a strongly Bayes consistent learner. Prior to this work, the only strongly Bayes consistent proximity-based method was the k-nearest neighbor classifier, for k growing appropriately with sample size. We will argue that a margin-regularized 1…
A new method for learning distance metrics for K-NN classification.
problem Improving the performance of K-NN classifier by learning an appropriate distance metric.
method Designing a continuous decision function for K-NN and minimizing its continuous empirical risk function.
result The proposed ANN algorithm outperforms existing methods like LMNN, NCA, and pairwise constraints.
Exemplar VAEs link generative models with nearest neighbor retrieval and data augmentation.
problem Improving generative model performance and data augmentation effectiveness.
method Exemplar VAEs with Parzen window prior, retrieval augmented training, exemplar leave-one-out, and subsampling.
result Generative data augmentation reduces classification error on MNIST and Fashion MNIST.
New estimator uses k-nearest neighbor distances for density functionals.
problem Estimating general density functionals from data.
method Asymptotically unbiased estimator using inverse Laplace transform.
result Established L2-consistency and mean squared error convergence. Improved multiclass classification with class-weighted nearest neighbors.
problem Multiclass classification with large or imbalanced classes.
method Class-weighted k-nearest neighbors algorithm, derived bounds on accuracy and risk.
result Optimized classification metrics like F1 score or Matthew's Correlation Coefficient.
Study shows SNN graph Laplacians converge to k-NN graph Laplacians under large scale asymptotics.
problem Understanding the convergence of SNN graph Laplacians to k-NN graph Laplacians.
method Analyzing the asymptotic behavior of SNN and k-NN graph Laplacians.
result The graph Laplacians of SNN and k-NN graphs converge to the same limit under large scale asymptotics.
Defense against adversarial examples using k-NN on neural network activations.
problem Adversarial examples that fool machine learning models.
method k-Nearest Neighbor (kNN) on intermediate activations of neural networks.
result Significantly outperforms state-of-the-art defenses on MNIST and CIFAR-10.
Adaptive multiclass classifier improves accuracy with varying data smoothness.
problem Multiclass classification with unknown continuous functions.
method Fixed nk-nearest neighbors, aggregation, convex combination, adaptation to unknown α. result Aggregated estimate performs as well as an oracle choice, adapts to unknown α. The paper analyzes how much data points can be altered to change their rank in nearest neighbor searches.
problem Vulnerability of nearest neighbor search in high-dimensional data.
method Statistical analysis of perturbation needed to change neighbor rank.
result Derived statistical distribution of perturbation needed to modify neighbor rank.
New analysis shows k-nearest neighbors' robustness depends on k value.
problem Understanding why adversarial examples arise in classifiers.
method Theoretical framework based on bias-variance theory.
result k-nearest neighbors' robustness properties depend on k value.
Paper proposes an approximate margin method for fast multi-class classification.
problem Challenges in multi-class classification with many classes.
method Uses ANN search structures and LSH for approximate margin estimation.
result Approximate margin method is highly competitive in time, memory, and performance.
Study nearest-neighbor radii under dependent sampling, finding they remain informative.
problem Analyzing nearest-neighbor radii under dependent sampling.
method Consider strong mixing dependent observations, establish distribution-free almost sure convergence and sharp non-asymptotic moment bounds.
result Nearest-neighbor geometry remains informative under dependence sampling.
Deep nearest neighbors outperform self-supervised methods in anomaly detection.
problem Anomaly detection using self-supervised deep methods.
method Simple nearest-neighbor approach on Imagenet pretrained features.
result Nearest-neighbor method outperforms self-supervised methods in accuracy, few shot generalization, training time, and noise robustness.
A new method estimates optimal treatment regimes using causal nearest neighbors.
problem Estimating optimal treatment regimes in precision medicine.
method Causal k-nearest neighbor method, with adaptive metric and variable selection.
result The causal k-nearest neighbor regime is universally consistent and converges as sample size increases.
Recent advances in large-margin classification of data residing in general metric spaces (rather than Hilbert spaces) enable classification under various natural metrics, such as string edit and earthmover distance. A general framework developed for this purpose by von Luxburg and Bousquet [JMLR, 2004] left open the qu…
The paper explains how nearest neighbor methods succeed in prediction.
problem Explaining the success of nearest neighbor methods in prediction.
method The paper covers both theoretical and practical aspects of nearest neighbor methods, including statistical guarantees and practical algorithms.
result The paper provides nonasymptotic statistical guarantees and practical algorithms for nearest neighbor methods.
AWNN improves matrix completion by adaptively weighting nearest neighbors.
problem Matrix completion with optimal nearest neighbor weights and radii selection.
method Adaptively weighted nearest neighbor method for matrix completion.
result Theoretical guarantees and synthetic experiments support the effectiveness of AWNN.
A new method uses nearest neighbors for importance weighting.
problem Data covariate shift problems in machine learning.
method Nearest neighbor classification scheme for determining importance weights.
result Demonstrated effectiveness through comparative experiments on various classification tasks.
Enhanced nearest neighbor improves accuracy in crowdsourced data.
problem Noise in crowdsourced labels degrades classification accuracy.
method Developed two algorithms to estimate worker quality and an enhanced nearest neighbor classifier.
result Proposed methods achieve the same regret as oracle version based on expert data.
Fast approximate nearest neighbor (NN) search in large databases is becoming popular. Several powerful learning-based formulations have been proposed recently. However, not much attention has been paid to a more fundamental question: how difficult is (approximate) nearest neighbor search in a given data set? And which …
Adaptive algorithm speeds up k-nearest-neighbor searches.
problem Finding k nearest neighbors with varying efficiency.
method Adaptive estimation of distances to optimize search.
result The algorithm achieves significant speedups compared to naive methods.
Efficiently find approximate nearest neighbors in high dimensions.
problem Finding the closest point in a high-dimensional dataset.
method Develops efficient data structures for approximate nearest neighbor search.
result Efficient solutions for approximate nearest neighbor problem.
Nearest neighbor methods are a popular class of nonparametric estimators with several desirable properties, such as adaptivity to different distance scales in different regions of space. Prior work on convergence rates for nearest neighbor classification has not fully reflected these subtle properties. We analyze the b…
Efficiently selects nearest neighbors for labeling to speed up active learning.
problem Intractable active learning and search for large-scale unlabeled data.
method Restricts candidate pool to nearest neighbors of labeled set.
result Achieved similar performance to global approach but reduced computational cost by up to 3 orders of magnitude.
Paper shows how to efficiently summarize large datasets using sparsity.
problem Efficiently summarize large datasets using submodular optimization.
method Develops a sparsified submodular optimization method and analyzes a new sparsification technique.
result Proves that sparsified submodular optimization is almost optimal under standard assumptions.
Bayesian methods improve kernel and mutual k-nearest neighbor regression.
problem Improving nonparametric regression methods for better accuracy and hyperparameter selection.
method Bayesian extensions of kernel and mutual k-nearest neighbor regression methods based on Gaussian process models.
result The proposed methods asymptotically converge to the original methods and perform better or equally well in simulations.
Python package reduces hubness in high-dimensional data.
problem Hubness in high-dimensional data affects machine learning tasks.
method Hubness analysis, reduction, and approximate neighbor search.
result Improves neighbor retrieval and learning tasks in high dimensions.
Interpolated nearest neighbor algorithms minimize bias in machine learning models.
problem Understanding and reducing overfitting in machine learning models.
method Proves the interpolated nearest neighbor algorithm achieves minimax optimal rates in regression and classification.
result Interpolated nearest neighbor algorithms are statistically optimal and perform better than traditional methods in some cases.
This paper provides fast estimates for complex option types.
problem Estimating prices for constrained multiple exercise American options.
method Lookahead search for lower estimates and nearest-neighbor martingale for upper estimates.
result Probabilistic convergence guarantees for the algorithms.
VNNGP uses nearest neighbors to approximate GPs, improving scalability and performance.
problem Scalability issues in Gaussian process approximations.
method Sparse precision structure via nearest neighbors, variational framework.
result VNNGP outperforms low-rank methods and is less prone to overfitting.
We present a simple, yet effective, approach to Semi-Supervised Learning. Our approach is based on estimating density-based distances (DBD) using a shortest path calculation on a graph. These Graph-DBD estimates can then be used in any distance-based supervised learning method, such as Nearest Neighbor methods and SVMs…
New method for clustering binary data using nearest neighbor median shift.
problem Clustering binary data effectively.
method BinNNMS based on nearest neighbor median shift.
result BinNNMS accurately discovers cluster locations in binary data.
ADSH uses asymmetric deep hashing for faster training and better performance.
problem Efficiently utilizing supervised information for large-scale database in deep supervised hashing.
method ADSH learns a deep hash function only for query points and directly learns hash codes for database points.
result ADSH achieves state-of-the-art performance in real applications.
Paper studies L2-consistency of nearest neighbor matching for biased samples.
problem Biased sampling and missing data in statistical problems.
method Nearest neighbor matching (NNM) to estimate population quantities from biased samples.
result NNM is L2-consistent in finite dimensions without smoothness and boundedness assumptions.