The study evaluates and tests k k k -NN models in various applications.
problem The relation between parameters and accuracy of k k k -NN models is not well understood. method Developed a randomized algorithm to test the k k k -NN property with a complexity of O ( n k 2 / ε 2 ) O(\sqrt{n} k^2 / ε^2) O ( n k 2 / ε 2 ) . result The algorithm can detect k k k -NN models with bad accuracy in significantly less time than building the model. The k k k -nearest neighbor classification method ( k k k -NNC) is one of the simplest nonparametric classification methods. The mutual k k k -NN classification method (M k k k NNC) is a variant of k k k -NNC based on mutual neighborship. We propose another variant of k k k -NNC, the symmetric k k k -NN classification method (S k k k NNC) based …
Non-asymptotic uniform rates for k-NN regression are derived.
problem Estimating functions from noisy observations with unknown lower dimensionality.
method Derives high-probability finite-sample uniform rates of consistency for k-NN regression.
result k-NN regression rates are optimal up to logarithmic factors and adapt to unknown lower dimensions.
A novel k-NN method estimates conditional mean and variance efficiently.
problem Joint estimation of conditional mean and variance.
method Integrates k-NN with automated variance selection.
result Achieves fast convergence rates and improved precision.
Improved OOD detection using label smoothing and k-NN density estimates.
problem Detecting out-of-distribution examples in classification models.
method Label smoothing and k-NN density estimate on intermediate activations.
result Label smoothing improves OOD detection performance, both theoretically and empirically.
Paper proposes MS-k-NN for improved convergence rate in k-NN classification.
problem Improving convergence rate of k-NN classification methods.
method Proposes MS-k-NN that extrapolates unweighted k-NN estimators to k=0.
result MS-k-NN achieves improved convergence rate under certain conditions.
Image-based malware detection using transfer learning outperforms simple k-NN.
problem Malware detection and classification based on image analysis.
method Transfer learning from pre-trained deep learning models on image datasets.
result Image-based DL models outperform k-NN in zero-day experiments.
Simple k-NN filtering improves model accuracy on noisy labels.
problem Training models with noisy labels reduces performance and is hard to identify.
method A simple k k k -nearest neighbor-based filtering approach on the logit layer. result Improves model accuracy compared to recent methods.
The paper proposes a method to achieve 1-NN speed with k-NN accuracy.
problem Achieving both the speed of 1-NN and the accuracy of k-NN.
method Aggregating denoised 1-NN predictors over a small number of distributed subsamples.
result Small subsample sizes can match or improve k-NN performance without sacrificing 1-NN's computational efficiency.
A new method for learning distance metrics for K-NN classification.
problem Improving the performance of K-NN classifier by learning an appropriate distance metric.
method Designing a continuous decision function for K-NN and minimizing its continuous empirical risk function.
result The proposed ANN algorithm outperforms existing methods like LMNN, NCA, and pairwise constraints.
Implementing k k k -NN classification using Gromov--Wasserstein distances
problem Comparing metric measure spaces
method Gromov--Wasserstein and fused Gromov--Wasserstein distances
result Universal consistency of k k k -NN classifiers DS techniques outperform K-NN in classification accuracy.
problem Improving classification performance using multiple classifier systems.
method Dynamic Selection (DS) compared to K-NN, focusing on neighborhood quality and instance hardness.
result DS techniques achieve higher classification accuracy than K-NN.
Enhances k-NN accuracy through randomized hyperstructure.
problem Improves k-NN accuracy by optimizing neighbor selection.
method Constructs a random n-dimensional hyperstructure around test instances to refine neighbor selection.
result 85.71% accuracy on Haberman's Cancer Survival dataset, compared to 80.95% for conventional k-NN.
A fast method for LOOCV in k-NN regression reduces computation time.
problem Efficient computation of LOOCV for k-NN regression.
method Identical LOOCV estimate to (k+1)-NN MSE on training data.
result LOOCV computation can be done with (k+1)-NN regression once.
The study examines how K-NN and DNNs resist label noise based on noise concentration.
problem The impact of label noise on K-NN and DNN performance.
method Empirical analysis and derivation of an analytic expression for K-NN classification error.
result The concentration of label noise affects the performance of K-NN and DNNs, with more concentrated noise leading to greater degradation.
The k k k -nearest neighbour ( k k k -NN) classifier is one of the oldest and most important supervised learning algorithms for classifying datasets. Traditionally the Euclidean norm is used as the distance for the k k k -NN classifier. In this thesis we investigate the use of alternative distances for the k k k -NN classifier. We …
Improves k-NN for monotonic data with robustness against noise.
problem Class noise in real-life data violates monotonic constraints in k-NN.
method Monotonic Fuzzy k-NN (MonFkNN) with new fuzzy membership calculation.
result Significant accuracy improvements and robustness against monotonic noise.
Study shows k k k -NN classifier is not universally consistent on ( 0 , 1 ) (0,1) ( 0 , 1 ) but consistent on discrete and specific measure spaces.
problem Consistency of k k k -NN classifier under Wasserstein distance on measure spaces. method Analysis of k k k -NN classifier properties under Wasserstein distance, use of σ σ σ -finite metric dimension, geodesic structures of Wasserstein spaces. result Consistency of k k k -NN classifier on specific measure spaces (discrete, Gaussian, wavelet series) but not on ( 0 , 1 ) (0,1) ( 0 , 1 ) . New method improves crowd counting accuracy using inverse k-NN maps and multiscale upsampling.
problem Improving accuracy of crowd density maps for high-density gatherings.
method Developed MUD-i k k k NN architecture using inverse k-NN maps and multiscale upsampling. result New network architecture outperforms state-of-the-art crowd counting.
Combines k-NN and RVM for improved classification accuracy.
problem Improving k-NN's performance by considering relevancy.
method Integrates k-NN and RVM in kernel space, introduces a new stopping parameter.
result Significantly prunes irrelevant attributes and improves classification accuracy.
The k k k -NN graph has played a central role in increasingly popular data-driven techniques for various learning and vision tasks; yet, finding an efficient and effective way to construct k k k -NN graphs remains a challenge, especially for large-scale high-dimensional data. In this paper, we propose a new approach to const…
The paper optimizes k-NN for distributed learning with minimax optimal performance.
problem Minimizing error rates in classification, regression, and density estimation.
method Optimal aggregation of fixed-k nearest neighbors from multiple subsets of data.
result Achieves minimax optimal error rates up to a logarithmic factor.
The paper compares DNNs and k-NN for memorizing vs. generalizing.
problem Understanding the balance between memorizing training data and generalizing to new data.
method Comparison of DNNs and k-NN at various layers and classifiers.
result k-NN behavior on new data indicates generalization, suggesting memorization and generalization are compatible.
Paper develops robust k k k -NN algorithm for few samples.
problem Learning robust classifier from limited samples.
method Distributionally robust formulation of weighted k k k -NN. result Robust classifier improves generalization with smaller Lipschitz norm.
k k k Nearest Neighbors ( k k k NN) is one of the most widely used supervised learning algorithms to classify Gaussian distributed data, but it does not achieve good results when it is applied to nonlinear manifold distributed data, especially when a very limited amount of labeled samples are available. In this paper, we pro…
Improved convergence rate for kNN graph Laplacians with adaptive bandwidth.
problem Enhancing the efficiency of graph-based data analysis methods.
method Introducing a new class of kNN graph with adaptive bandwidth and proving operator convergence rate.
result Operator convergence rate of O ( N − 2 / ( d + 6 ) ) O(N^{-2/(d+6)}) O ( N − 2/ ( d + 6 ) ) for the kNN graph Laplacian, up to a log factor. Paper improves k k k -NN predictive performance with efficient variable selection.
problem Improving predictive performance of k k k -NN models. method Efficient forward selection of predictor variables.
result Novel approach approaches outperformance of stepwise selection models.
The problem of supervised classification (or discrimination) with functional data is considered, with a special interest on the popular k-nearest neighbors (k-NN) classifier. First, relying on a recent result by Cerou and Guyader (2006), we prove the consistency of the k-NN classifier for functional data whose distribu…
Proposes LRR and LRLR for improving stock prediction accuracy.
problem Improving stock prediction accuracy through nonparametric classification.
method Local radial regression and logistic regression variant.
result LRLR outperforms LPoR and MS- k k k -NN in real-world stock datasets. Introduction. Case Based Reasoning (CBR) is an emerg- ing decision making paradigm in medical research where new cases are solved relying on previously solved similar cases. Usually, a database of solved cases is provided, and every case is described through a set of attributes (inputs) and a label (output). Extracting…
A fast k k k -NN classifier without negative pairs.
problem Efficient k k k -NN classification without negative pairs. method Ridge regression for learning dissimilarity function.
result Better k k k -NN classification accuracy than state-of-the-art methods. A new strategy selects k in k-NN regression without hold-out data.
problem Choosing optimal k in k-NN regression without hold-out data.
method Iterative procedure over k, minimum discrepancy principle.
result Minimax-optimal over smoothness function classes.
We propose a non-parametric anomaly detection algorithm for high dimensional data. We score each datapoint by its average K K K -NN distance, and rank them accordingly. We then train limited complexity models to imitate these scores based on the max-margin learning-to-rank framework. A test-point is declared as an anomaly…
Estimating entropy and mutual information consistently is important for many machine learning applications. The Kozachenko-Leonenko (KL) estimator (Kozachenko & Leonenko, 1987) is a widely used nonparametric estimator for the entropy of multivariate continuous random variables, as well as the basis of the mutual inform…
Adaptive k-NN classifier improves accuracy over fixed k-NN.
problem Improving classification accuracy by dynamically choosing k.
method Adaptive selection of k based on local neighborhood properties.
result The adaptive k-NN classifier performs comparably to or better than fixed k-NN.
Develops a model for personalized diabetes and hypertension treatment using robust regression and K-NN.
problem Optimal personalized treatment rules for patients based on EHRs.
method Robust regression informed K-NN approach for predicting and selecting optimal treatments.
result Algorithm leads to larger reduction in HbA1c for diabetics and systolic BP for hypertensive patients compared to alternatives.
Generative models improve pose transfer between people.
problem Transferring actions from one person to another.
method Used nearest neighbor and generative models (pix2pix) for pose transfer.
result Generative models outperform k-NN in generating corresponding frames and generalizing outside the action set.
Prototype rules simplify multiclass classification in metric spaces, achieving consistency and reduced complexity.
problem Multiclass classification in metric spaces, focusing on universal consistency and convergence rates.
method Novel Proto-NN and hybrid rules for multiclass classification in metric spaces, analyzing convergence rates.
result Proto-NN is universally consistent and simpler to implement, with similar computational advantages.
PAC-Bayesian bounds improve understanding of K-NN classifier performance.
problem Improving the understanding of K-NN classifier's generalization error.
method PAC-Bayesian analysis applied to K-NN classifier in kernel space.
result PAC-Bayesian bounds provide a function of the number of redundant training examples.
Consistency of k-NN rule proven in sigma-finite dimensional metric spaces.
problem Proving consistency of k-NN rule in metric spaces.
method Direct proof using Stone's theorem, investigating metric properties.
result Universal consistency of k-NN rule in sigma-finite dimensional metric spaces.
A new k k k -NN algorithm using surprisal for robust and interpretable nonparametric learning.
problem Complex patterns and relationships in data without strong distribution assumptions.
method Surprisal-driven k k k -NN framework for classification, regression, density estimation, and anomaly detection. result State-of-the-art results in classification and anomaly detection, competitive regression results.
Paper uses K-NN resampling to simulate and evaluate LOB markets.
problem Simulating and evaluating limit order book (LOB) markets.
method Applies K K K -nearest neighbor ( K K K -NN) resampling to LOB simulation and evaluation. result Demonstrates the effectiveness and efficiency of K K K -NN resampling in LOB simulation and evaluation. Study shows SNN graph Laplacians converge to k-NN graph Laplacians under large scale asymptotics.
problem Understanding the convergence of SNN graph Laplacians to k-NN graph Laplacians.
method Analyzing the asymptotic behavior of SNN and k-NN graph Laplacians.
result The graph Laplacians of SNN and k-NN graphs converge to the same limit under large scale asymptotics.
Proposes a new metric learning method using Lie group geodesics.
problem Improving distance metrics for k-NN classification.
method Geodesic interpolation on Lie transformation group to calculate velocities and produce a diffeomorphic global transformation.
result Effective in synthetic and real datasets, improving k-NN classification.
New findings on k k k -NN algorithm's robustness under random data corruption.
problem Impact of random data corruption on k k k -NN algorithm performance. method Theoretical analysis of k k k -NN algorithm under random perturbation scenarios. result Phase transition phenomenon in asymptotic regret: small- ω \omega ω regime vs large- ω \omega ω regime. In this paper we examine the effect of applying ensemble learning to the performance of collaborative filtering methods. We present several systematic approaches for generating an ensemble of collaborative filtering models based on a single collaborative filtering algorithm (single-model or homogeneous ensemble). We pr…
The paper improves spectral convergence rates for graph Laplacians.
problem Improving spectral convergence rates for graph Laplacians.
method Utilizing regularity of continuum eigenfunctions and strong pointwise consistency results.
result Eigenvalues and eigenvectors of graph Laplacian converge to continuum at rate O ( n − 1 / ( m + 4 ) ) O(n^{-1/(m+4)}) O ( n − 1/ ( m + 4 ) ) . Improved gesture recognition using compressed domain signals.
problem Efficient gesture recognition in compressed domain.
method Direct gesture feature extraction from compressed measurements, improved DTW-based K-NN classifiers.
result Strong support for the proposed algorithm in simulations and hardware.