A defense against adversarial examples using k-Nearest Neighbor and deep learning.
problem Evaluating robustness of k-Nearest Neighbor and its deep learning combination.
method Proposed heuristic attack to find adversarial examples for kNN and DkNN.
result Our attack significantly outperforms other attacks on DkNN.
Adaptive algorithm speeds up k-nearest-neighbor searches.
problem Finding k nearest neighbors with varying efficiency.
method Adaptive estimation of distances to optimize search.
result The algorithm achieves significant speedups compared to naive methods.
A new method estimates optimal treatment regimes using causal nearest neighbors.
problem Estimating optimal treatment regimes in precision medicine.
method Causal k-nearest neighbor method, with adaptive metric and variable selection.
result The causal k-nearest neighbor regime is universally consistent and converges as sample size increases.
Study shows k-NN regressor consistency in complex survey designs.
problem Lack of consistency results for k-NN regressor in complex survey data. method Analysis of regularity conditions on sampling design and data distribution.
result Consistency of k-NN regressor under complex survey designs. In many scientific disciplines structures in high-dimensional data have to be found, e.g., in stellar spectra, in genome data, or in face recognition tasks. In this work we present a novel approach to non-linear dimensionality reduction. It is based on fitting K-nearest neighbor regression to the unsupervised regressio…
K-Nearest neighbor classifier (k-NNC) is simple to use and has little design time like finding k values in k-nearest neighbor classifier, hence these are suitable to work with dynamically varying data-sets. There exists some fundamental improvements over the basic k-NNC, like weighted k-nearest neighbors classifier (wh…
This paper proposes a new method for feature scaling in K-Nearest Neighbors.
problem Feature scaling issues in K-Nearest Neighbors algorithm.
method Assign weights to individual features using out-of-bag errors from decision tree models.
result Improves prediction accuracy by assigning weights to features based on out-of-bag errors.
We propose Bayesian extensions of two nonparametric regression methods which are kernel and mutual k-nearest neighbor regression methods. Derived based on Gaussian process models for regression, the extensions provide distributions for target value estimates and the framework to select the hyperparameters. It is show…
New estimator uses k-nearest neighbor distances for density functionals.
problem Estimating general density functionals from data.
method Asymptotically unbiased estimator using inverse Laplace transform.
result Established L2-consistency and mean squared error convergence. When using the K-nearest neighbors method, one often ignores uncertainty in the choice of K. To account for such uncertainty, Holmes and Adams (2002) proposed a Bayesian framework for K-nearest neighbors (KNN). Their Bayesian KNN (BKNN) approach uses a pseudo-likelihood function, and standard Markov chain Monte Carlo (…
Paper proposes deep learning models for k-NN classification.
problem Imbalanced datasets and complex feature vectors in classification.
method Sequence to sequence model and memory network models.
result Models outperform k-NN and other state-of-the-art models.
We introduce the speculate-correct method to derive error bounds for local classifiers. Using it, we show that k nearest neighbor classifiers, in spite of their famously fractured decision boundaries, have exponential error bounds with O(sqrt((k + ln n) / n)) error bound range for n in-sample examples.
Paper offers a new approach to locally weighted regression and classification.
problem Optimal number of neighbors and weights in k-nearest neighbors.
method Locally weighted regression with explicit bias-variance tradeoff.
result Efficiently finds optimal weights and number of neighbors for each data point.
Java implementation improves nearest neighbor algorithm complexity.
problem Improving efficiency of nearest neighbor descent algorithm.
method Parallel streams implementation with statistical termination criterion.
result Complexity up to O(nK2logK(n)) for K-nearest neighbors. DkNN combines k-NN with deep learning for robust, interpretable predictions.
problem Lack of robustness and interpretability in deep learning models.
method Hybrid classifier combining k-NN and deep learning representations.
result Confidence estimates and interpretable explanations for inputs outside the model's training manifold.
We study clustering algorithms based on neighborhood graphs on a random sample of data points. The question we ask is how such a graph should be constructed in order to obtain optimal clustering results. Which type of neighborhood graph should one choose, mutual k-nearest neighbor or symmetric k-nearest neighbor? What …
Paper proposes a k-NN classifier for detecting spike-and-wave seizures in EEG.
problem Early detection of epileptic seizures in EEG signals.
method Uses t-location-scale distribution and k-nearest neighbors classifier.
result Demonstrates improved classification accuracy, sensitivity, and specificity on real data.
Paper improves k-NN predictive performance with efficient variable selection.
problem Improving predictive performance of k-NN models. method Efficient forward selection of predictor variables.
result Novel approach approaches outperformance of stepwise selection models.
Randomized graph construction ensures giant component with fewer edges.
problem Efficiently constructing sparse graphs with good connectivity.
method Randomly connecting points to a subset of their nearest neighbors.
result A sparser graph with comparable connectivity properties.
New analysis shows k-nearest neighbors' robustness depends on k value.
problem Understanding why adversarial examples arise in classifiers.
method Theoretical framework based on bias-variance theory.
result k-nearest neighbors' robustness properties depend on k value.
Two methods forecast functional time series, offering competitive results.
problem Forecasting functional time series with model-free approaches.
method Two nonparametric methods: k-nearest neighbors adaptation and curve envelope selection.
result Competitive results with and often superior to benchmarks.
Defense against adversarial examples using k-NN on neural network activations.
problem Adversarial examples that fool machine learning models.
method k-Nearest Neighbor (kNN) on intermediate activations of neural networks.
result Significantly outperforms state-of-the-art defenses on MNIST and CIFAR-10.
This paper compares machine learning algorithms for ecological data.
problem Classifying ecological datasets into subsets with common patterns.
method Applied eight machine learning classification algorithms to ecological data.
result Linear Discriminant Analysis and k-nearest neighbors are the best methods.
Consider a weighted or unweighted k-nearest neighbor graph that has been built on n data points drawn randomly according to some density p on R^d. We study the convergence of the shortest path distance in such graphs as the sample size tends to infinity. We prove that for unweighted kNN graphs, this distance converges …
Paper compares two possibilistic segmentation methods for SAS imagery.
problem Segmenting synthetic aperture sonar images into different seafloor environments.
method Comparison of Possibilistic Fuzzy Local Information C-Means (PFLICM) and Possibilistic K-Nearest Neighbors (PKNN) algorithms.
result PKNN outperforms PFLICM in segmentation performance on SAS images.
Machine learning classifies IoT noise in smart cities.
problem Noise classification in smart cities.
method Mel-frequency cepstral coefficients for audio features, supervised classification (SVM, k-NN), parameter optimization.
result Noise classification accuracy of 85% - 100%.
Algorithm finds adversarial examples for k-NN classifiers using Voronoi diagrams.
problem Ensuring robustness of k-NN classifiers against adversarial attacks.
method Geometric approach expanding outwards from input points to find minimum-norm adversarial examples.
result Our method outperforms existing approaches on various datasets.
Predicts employee attrition using k-NN algorithm.
problem Predicting employee attrition in a company.
method Used k-Nearest Neighbors algorithm with employee performance, hours worked, and years of service as features. Split dataset 70/30 for training/testing.
result Achieved 94.32% accuracy in predictions.
Acoustic sensors identify vehicles using spectral embedding.
problem Vehicle recognition from roadside audio sensors.
method Extract frequency signatures, apply spectral embedding for dimensionality reduction.
result K-nearest neighbors achieve accurate vehicle identification after dimensionality reduction.
Improved multiclass classification with class-weighted nearest neighbors.
problem Multiclass classification with large or imbalanced classes.
method Class-weighted k-nearest neighbors algorithm, derived bounds on accuracy and risk.
result Optimized classification metrics like F1 score or Matthew's Correlation Coefficient.
Improves performance in various machine learning tasks by reparameterizing subset sampling.
problem Stochastic optimization involving subset sampling is not reparameterizable.
method Continuous relaxation of subset sampling to provide reparameterization gradients.
result Improves performance in instance-wise feature selection, deep stochastic k-nearest neighbors, and parametric t-SNE.
rpForests uses random projections to speed up kNN search.
problem Efficient kNN search in high-dimensional data.
method Random projection forests aggregating results from multiple random projection trees.
result rpForests achieves fast decay in missing rate and kNN distance discrepancy.
We provide finite-sample analysis of a general framework for using k-nearest neighbor statistics to estimate functionals of a nonparametric continuous probability density, including entropies and divergences. Rather than plugging a consistent density estimate (which requires k→∞ as the sample size $n \to \in…
Enhanced KNN-TSVM reduces noise and overfitting with stable learning rules.
problem High computational cost and overfitting in KNN-based TSVM classifiers.
method Weighted KNN with stabilizer term and LDMDBA for efficient computation.
result Effective in both classification accuracy and computational time, with up to 14x speedup.
DW-KNN improves KNN by integrating distance and neighbor reliability for better prediction accuracy.
problem Standard KNN assumes all neighbors are equally reliable, leading to unreliable predictions in heterogeneous feature spaces.
method DW-KNN integrates exponential distance with neighbor validity, providing instance-level interpretability and reducing hyperparameter sensitivity.
result DW-KNN achieves 0.8988 average accuracy, ranks 2nd among six methods, and has the lowest cross-validation variance.
A new ensemble method improves kNN performance by extending the neighborhood rule.
problem Traditional kNN's limitations when test points are outside the spherical region and ensemble's high errors.
method Determines neighbors in k steps, using bootstrap samples and optimal models selection.
result The proposed ensemble method outperforms state-of-the-art methods on 17 benchmark datasets.
Under-bagging k-NN improves performance on imbalanced classification.
problem Imbalanced classification problems where one class is significantly underrepresented.
method Proposes an under-bagging k-NN ensemble learning algorithm, analyzing convergence rates and efficiency. result Achieves optimal convergence rates under mild assumptions and reduces sub-sample size and k for highly imbalanced data. Techniques such as clusterization, neural networks and decision making usually rely on algorithms that are not well suited to deal with missing values. However, real world data frequently contains such cases. The simplest solution is to either substitute them by a best guess value or completely disregard the missing va…
Study shows SNN graph Laplacians converge to k-NN graph Laplacians under large scale asymptotics.
problem Understanding the convergence of SNN graph Laplacians to k-NN graph Laplacians.
method Analyzing the asymptotic behavior of SNN and k-NN graph Laplacians.
result The graph Laplacians of SNN and k-NN graphs converge to the same limit under large scale asymptotics.
Enhances k-NN accuracy through randomized hyperstructure.
problem Improves k-NN accuracy by optimizing neighbor selection.
method Constructs a random n-dimensional hyperstructure around test instances to refine neighbor selection.
result 85.71% accuracy on Haberman's Cancer Survival dataset, compared to 80.95% for conventional k-NN.
Paper develops robust k-NN algorithm for few samples.
problem Learning robust classifier from limited samples.
method Distributionally robust formulation of weighted k-NN. result Robust classifier improves generalization with smaller Lipschitz norm.
This study uses cloud telemetry to detect DoS attacks with machine learning.
problem Detecting DoS attacks in cloud environments is challenging.
method Use of machine learning algorithms on cloud telemetry data.
result k-Nearest Neighbors (kNN) and decision tree (CART) accurately identify DoS attacks.
IEFSVM improves SVM for imbalanced data by adjusting k-nearest neighbors entropy.
problem Imbalanced data classification challenges in machine learning.
method Instance-based IEFSVM modifies EFSVM using k-nearest neighbors entropy to prioritize samples.
result IEFSVM outperforms other SVMs and machine learning methods in high imbalance datasets.
A novel unsupervised outlier detection method using Randomized PCA Forest.
problem Unsupervised outlier detection in datasets.
method Randomized Principal Component Analysis (RPCA) Forest for deriving an outlier score.
result Superior performance compared to classical and state-of-the-art methods.
The paper compares methods for imputing missing categorical data in supervised classification tasks.
problem Missing data in categorical variables can degrade model performance.
method Experimented on two datasets with missing categorical data, comparing imputation methods with and without additional missing-data perturbation.
result Imputation methods can increase predictive accuracy in the presence of missing-data perturbation.
Interpolated nearest neighbor algorithms minimize bias in machine learning models.
problem Understanding and reducing overfitting in machine learning models.
method Proves the interpolated nearest neighbor algorithm achieves minimax optimal rates in regression and classification.
result Interpolated nearest neighbor algorithms are statistically optimal and perform better than traditional methods in some cases.
Locally adaptive nearest neighbors improve automated systems' performance and are easier to interpret.
problem Improving automated systems' performance and interpretability.
method Developed a method for k nearest neighbors algorithms to learn locally adaptive metrics.
result Locally adaptive metrics improve performance and are interpretable.
New method constructs graphs from data efficiently, suitable for large datasets.
problem Memory and runtime limitations of traditional TMFG for large datasets.
method Uses k-Nearest Neighbors Graphs and memory management for scalable graph construction.
result Provides a parsimonious way to construct graphs for learning tasks.