AWNN improves matrix completion by adaptively weighting nearest neighbors.
problem Matrix completion with optimal nearest neighbor weights and radii selection.
method Adaptively weighted nearest neighbor method for matrix completion.
result Theoretical guarantees and synthetic experiments support the effectiveness of AWNN.
Locally adaptive nearest neighbors improve automated systems' performance and are easier to interpret.
problem Improving automated systems' performance and interpretability.
method Developed a method for k nearest neighbors algorithms to learn locally adaptive metrics.
result Locally adaptive metrics improve performance and are interpretable.
Algorithms often carry out equally many computations for "easy" and "hard" problem instances. In particular, algorithms for finding nearest neighbors typically have the same running time regardless of the particular problem instance. In this paper, we consider the approximate k-nearest-neighbor problem, which is the pr…
Nearest neighbor methods are a popular class of nonparametric estimators with several desirable properties, such as adaptivity to different distance scales in different regions of space. Prior work on convergence rates for nearest neighbor classification has not fully reflected these subtle properties. We analyze the b…
A novel distributed adaptive NN classifier for large data sets.
problem Handling large and distributed data for efficient classification.
method Distributed adaptive nearest neighbor classifier with stochastic tuning parameter selection and early stopping rule.
result Achieves nearly optimal convergence rate under large sub-sample sizes.
The estimation of optimal treatment regimes is of considerable interest to precision medicine. In this work, we propose a causal k-nearest neighbor method to estimate the optimal treatment regime. The method roots in the framework of causal inference, and estimates the causal treatment effects within the nearest neig…
New algorithm learns from partial labels in general scenarios.
problem Learning from partial labels when each instance has a bag of labels.
method Adaptive nearest-neighbors algorithm PL A-kNN. result PL A-kNN outperforms state-of-the-art methods in general PLL scenarios. A new method for covariate shift adaptation using nearest neighbors.
problem Mitigating distribution shift between source and target datasets.
method Directly work on unlabeled target data, labeled by nearest neighbors in source data.
result Optimal choice of k=1 simplifies hyper-parameter tuning and improves efficiency. Information-theoretic quantities, such as conditional entropy and mutual information, are critical data summaries for quantifying uncertainty. Current widely used approaches for computing such quantities rely on nearest neighbor methods and exhibit both strong performance and theoretical guarantees in certain simple sc…
Study nearest-neighbor radii under dependent sampling, finding they remain informative.
problem Analyzing nearest-neighbor radii under dependent sampling.
method Consider strong mixing dependent observations, establish distribution-free almost sure convergence and sharp non-asymptotic moment bounds.
result Nearest-neighbor geometry remains informative under dependence sampling.
Deep nearest neighbors outperform self-supervised methods in anomaly detection.
problem Anomaly detection using self-supervised deep methods.
method Simple nearest-neighbor approach on Imagenet pretrained features.
result Nearest-neighbor method outperforms self-supervised methods in accuracy, few shot generalization, training time, and noise robustness.
Characterizes Lebesgue points using nearest neighbor methods.
problem Consistency of classification algorithms based on nearest neighbors.
method Characterization of Lebesgue points via 1-Nearest Neighbor regression.
result Proves convergence of 1-Nearest Neighbor classification algorithms in metric spaces.
k-Nearest Neighbors is one of the most fundamental but effective classification models. In this paper, we propose two families of models built on a sequence to sequence model and a memory network model to mimic the k-Nearest Neighbors model, which generate a sequence of labels, a sequence of out-of-sample feature vecto…
The paper investigates learning conditional distributions on multi-dimensional spaces using clustering and neural networks.
problem Learning conditional distributions on multi-dimensional spaces with varying dimensions.
method The approach involves clustering data near varying query points in the feature space to create empirical measures in the target space using two clustering schemes: fixed-radius ball and nearest neighbors. The convergence rates of both methods are analyzed, and the nearest neighbors method is incorporated into neural network training.
result The empirical analysis shows that the nearest neighbors method has better performance in practice and can adapt to a suitable level of Lipschitz continuity locally.
The paper explains how nearest neighbor methods succeed in prediction.
problem Explaining the success of nearest neighbor methods in prediction.
method The paper covers both theoretical and practical aspects of nearest neighbor methods, including statistical guarantees and practical algorithms.
result The paper provides nonasymptotic statistical guarantees and practical algorithms for nearest neighbor methods.
The weighted k-nearest neighbors algorithm is one of the most fundamental non-parametric methods in pattern recognition and machine learning. The question of setting the optimal number of neighbors as well as the optimal weights has received much attention throughout the years, nevertheless this problem seems to have r…
A new method uses nearest neighbors for importance weighting.
problem Data covariate shift problems in machine learning.
method Nearest neighbor classification scheme for determining importance weights.
result Demonstrated effectiveness through comparative experiments on various classification tasks.
A new method for density estimation using nearest neighbor Dirichlet mixtures.
problem Slow and unstable Bayesian density estimation methods.
method Nearest neighbor grouping, local Bayesian parametric models, Dirichlet prior, Monte Carlo sampling.
result Effective density estimation with improved computational efficiency.
New algorithms detect outliers in high-dimensional data with arbitrary shapes.
problem Challenges of high dimensionality and varying cluster shapes in traditional outlier detection methods.
method Cluster Catch Digraphs (CCDs) and their variants (U-MCCD, UN-MCCD, SU-MCCD, SUN-MCCD).
result U-MCCD efficiently identifies outliers with high true negative rates, and SU-MCCD improves handling of non-uniform clusters.
A new adaptive kNN classifier outperforms Random Forests.
problem Improving classification accuracy using nearest neighbors.
method Finding discriminant subspaces for efficient nearest neighbor classification, leveraging bagging for diversity.
result The proposed method outperforms Random Forests and other nearest neighbors ensembles.
Improves predictive algorithm performance with domain adaptation.
problem Improves performance of predictive algorithms in distributional shift scenarios.
method Domain adaptive bootstrap aggregating with iterative nearest neighbor sampling.
result Proposes a method to improve predictive algorithm performance in distributional shift scenarios.
We consider a problem of multiclass classification, where the training sample Sn={(Xi,Yi)}i=1n is generated from the model P(Y=m∣X=x)=ηm(x), 1≤m≤M, and η1(x),…,ηM(x) are unknown α-Holder continuous functions.Given a test point X, our goal is to predict its labe…
We analyze the Kozachenko--Leonenko (KL) nearest neighbor estimator for the differential entropy. We obtain the first uniform upper bound on its performance over Hölder balls on a torus without assuming any conditions on how close the density could be from zero. Accompanying a new minimax lower bound over the Hölder ba…
VNNGP uses nearest neighbors to approximate GPs, improving scalability and performance.
problem Scalability issues in Gaussian process approximations.
method Sparse precision structure via nearest neighbors, variational framework.
result VNNGP outperforms low-rank methods and is less prone to overfitting.
Generative models use kernel smoothing for conditioning on small example sets.
problem Improving generative models' performance with limited conditioning examples.
method Showed that cross-attention conditioning is equivalent to kernel smoothing, specifically a Nadaraya--Watson kernel smoother.
result The approach predicts and confirms three failure regimes for kernel-based conditioning.
Adaptive NN method improves matrix completion for non-smooth data.
problem Matrix completion with non-smooth non-linear functions under high missingness.
method Two-sided nearest neighbors with \Holder function class non-linearity.
result NN error rate matches oracle's for latent factors, non-trivial for wide range of missingness.
DANCE improves prediction set efficiency for deep learning models.
problem Inefficient, overly conservative prediction sets for pre-trained models.
method DANCE combines adaptive kernel regression and nearest-neighbor approach.
result DANCE produces more efficient and robust prediction sets.
Java implementation improves nearest neighbor algorithm complexity.
problem Improving efficiency of nearest neighbor descent algorithm.
method Parallel streams implementation with statistical termination criterion.
result Complexity up to O(nK2logK(n)) for K-nearest neighbors. The paper tackles time series data by applying conformal prediction with nearest neighbors.
problem Time series data violates the exchangeability assumption required for conformal prediction.
method The approach uses the nearest neighbors method with fast parameter tuning and weighted nearest neighbors (FPTO-WNN) to construct reliable prediction intervals.
result Data analysis shows the effectiveness of the proposed approach.
K-Nearest neighbor classifier (k-NNC) is simple to use and has little design time like finding k values in k-nearest neighbor classifier, hence these are suitable to work with dynamically varying data-sets. There exists some fundamental improvements over the basic k-NNC, like weighted k-nearest neighbors classifier (wh…
A new method for learning distance metrics for K-NN classification.
problem Improving the performance of K-NN classifier by learning an appropriate distance metric.
method Designing a continuous decision function for K-NN and minimizing its continuous empirical risk function.
result The proposed ANN algorithm outperforms existing methods like LMNN, NCA, and pairwise constraints.
The paper studies empirical processes from nearest neighbors in regression.
problem Estimating conditional cumulative distribution functions and local linear regression.
method Uniform central limit theorem and non-asymptotic bound under local bracketing entropy and uniform entropy numbers.
result Gaussian limit of empirical process with simple covariance.
This paper proposes a new hashing-based KNN technique for faster nearest neighbor selection.
problem Slowness of KNN in big datasets due to searching entire dataset.
method Divide data space into subcells, use hashing to map data points, and select nearest neighbors layer by layer.
result The proposed technique offers competitive performance with KNN and KDtree while significantly improving time efficiency.
New methods use vector search and nearest-neighbor matching for policy learning in causal inference.
problem Learning optimal policies in causal inference with limited data.
method RAG-based policy learning with vector search and nearest-neighbor matching.
result The methods bound the within-candidate choice regret and evaluate the one-step method directly as a policy.
The nearest neighbor problem is defined as follows: Given a set P of n points in some metric space (X,D), build a data structure that, given any point q, returns a point in P that is closest to q (its "nearest neighbor" in P). The data structure stores additional information about the set P, which is th…
The celebrated Monte Carlo method estimates an expensive-to-compute quantity by random sampling. Bandit-based Monte Carlo optimization is a general technique for computing the minimum of many such expensive-to-compute quantities by adaptive random sampling. The technique converts an optimization problem into a statisti…
Two methods forecast functional time series, offering competitive results.
problem Forecasting functional time series with model-free approaches.
method Two nonparametric methods: k-nearest neighbors adaptation and curve envelope selection.
result Competitive results with and often superior to benchmarks.
We propose a simple kernel based nearest neighbor approach for handwritten digit classification. The "distance" here is actually a kernel defining the similarity between two images. We carefully study the effects of different number of neighbors and weight schemes and report the results. With only a few nearest neighbo…
CGMMD generates conditional samples in one shot with MMD and nearest neighbors.
problem Generating samples from a conditional distribution unseen in training.
method CGMMD uses MMD for a direct minimization problem, producing samples in one forward pass.
result CGMMD achieves convergence to the true conditional distribution with theoretical bounds.
k Nearest Neighbor (kNN) method is a simple and popular statistical method for classification and regression. For both classification and regression problems, existing works have shown that, if the distribution of the feature vector has bounded support and the probability density function is bounded away from zero in i…
DW-KNN improves KNN by integrating distance and neighbor reliability for better prediction accuracy.
problem Standard KNN assumes all neighbors are equally reliable, leading to unreliable predictions in heterogeneous feature spaces.
method DW-KNN integrates exponential distance with neighbor validity, providing instance-level interpretability and reducing hyperparameter sensitivity.
result DW-KNN achieves 0.8988 average accuracy, ranks 2nd among six methods, and has the lowest cross-validation variance.
Both supervised and unsupervised machine learning algorithms have been used to learn partition-based index structures for approximate nearest neighbor (ANN) search. Existing supervised algorithms formulate the learning task as finding a partition in which the nearest neighbors of a training set point belong to the same…
New method speeds up k-means clustering for large k by improving nearest-neighbor search.
problem Efficiently clustering large datasets with high-dimensional points.
method Seeded Approximate Nearest-Neighbor Search methods to improve Lloyd's algorithm.
result Significantly faster k-means clustering for large k values.
Improved multiclass classification with class-weighted nearest neighbors.
problem Multiclass classification with large or imbalanced classes.
method Class-weighted k-nearest neighbors algorithm, derived bounds on accuracy and risk.
result Optimized classification metrics like F1 score or Matthew's Correlation Coefficient.
We introduce a variant of the k-nearest neighbor classifier in which k is chosen adaptively for each query, rather than supplied as a parameter. The choice of k depends on properties of each neighborhood, and therefore may significantly vary between different points. (For example, the algorithm will use larger $k…
Study shows k-NN regressor consistency in complex survey designs.
problem Lack of consistency results for k-NN regressor in complex survey data. method Analysis of regularity conditions on sampling design and data distribution.
result Consistency of k-NN regressor under complex survey designs. In this paper, we compare the performances of FAISS and FENSHSES on nearest neighbor search in Hamming space--a fundamental task with ubiquitous applications in nowadays eCommerce. Comprehensive evaluations are made in terms of indexing speed, search latency and RAM consumption. This comparison is conducted towards a b…
Algorithm identifies nearest mode in noisy data.
problem Identifying the point with the minimum k-th nearest neighbor distance in unknown multivariate probability density.
method Sequential learning algorithm using noisy oracle queries to adaptively decide which points to query.
result Upper bounds on query complexity show significant improvement over baselines.