A defense against adversarial examples using k-Nearest Neighbor and deep learning.
problem Evaluating robustness of k-Nearest Neighbor and its deep learning combination.
method Proposed heuristic attack to find adversarial examples for kNN and DkNN.
result Our attack significantly outperforms other attacks on DkNN.
DkNN combines k-NN with deep learning for robust, interpretable predictions.
problem Lack of robustness and interpretability in deep learning models.
method Hybrid classifier combining k-NN and deep learning representations.
result Confidence estimates and interpretable explanations for inputs outside the model's training manifold.
Paper proposes deep learning models for k-NN classification.
problem Imbalanced datasets and complex feature vectors in classification.
method Sequence to sequence model and memory network models.
result Models outperform k-NN and other state-of-the-art models.
Adaptive algorithm speeds up k-nearest-neighbor searches.
problem Finding k nearest neighbors with varying efficiency.
method Adaptive estimation of distances to optimize search.
result The algorithm achieves significant speedups compared to naive methods.
A new method estimates optimal treatment regimes using causal nearest neighbors.
problem Estimating optimal treatment regimes in precision medicine.
method Causal k-nearest neighbor method, with adaptive metric and variable selection.
result The causal k-nearest neighbor regime is universally consistent and converges as sample size increases.
Study shows k-NN regressor consistency in complex survey designs.
problem Lack of consistency results for k-NN regressor in complex survey data. method Analysis of regularity conditions on sampling design and data distribution.
result Consistency of k-NN regressor under complex survey designs. Improves performance in various machine learning tasks by reparameterizing subset sampling.
problem Stochastic optimization involving subset sampling is not reparameterizable.
method Continuous relaxation of subset sampling to provide reparameterization gradients.
result Improves performance in instance-wise feature selection, deep stochastic k-nearest neighbors, and parametric t-SNE.
In many scientific disciplines structures in high-dimensional data have to be found, e.g., in stellar spectra, in genome data, or in face recognition tasks. In this work we present a novel approach to non-linear dimensionality reduction. It is based on fitting K-nearest neighbor regression to the unsupervised regressio…
K-Nearest neighbor classifier (k-NNC) is simple to use and has little design time like finding k values in k-nearest neighbor classifier, hence these are suitable to work with dynamically varying data-sets. There exists some fundamental improvements over the basic k-NNC, like weighted k-nearest neighbors classifier (wh…
Interpolated nearest neighbor algorithms minimize bias in machine learning models.
problem Understanding and reducing overfitting in machine learning models.
method Proves the interpolated nearest neighbor algorithm achieves minimax optimal rates in regression and classification.
result Interpolated nearest neighbor algorithms are statistically optimal and perform better than traditional methods in some cases.
This paper proposes a new method for feature scaling in K-Nearest Neighbors.
problem Feature scaling issues in K-Nearest Neighbors algorithm.
method Assign weights to individual features using out-of-bag errors from decision tree models.
result Improves prediction accuracy by assigning weights to features based on out-of-bag errors.
We propose Bayesian extensions of two nonparametric regression methods which are kernel and mutual k-nearest neighbor regression methods. Derived based on Gaussian process models for regression, the extensions provide distributions for target value estimates and the framework to select the hyperparameters. It is show…
New estimator uses k-nearest neighbor distances for density functionals.
problem Estimating general density functionals from data.
method Asymptotically unbiased estimator using inverse Laplace transform.
result Established L2-consistency and mean squared error convergence. When using the K-nearest neighbors method, one often ignores uncertainty in the choice of K. To account for such uncertainty, Holmes and Adams (2002) proposed a Bayesian framework for K-nearest neighbors (KNN). Their Bayesian KNN (BKNN) approach uses a pseudo-likelihood function, and standard Markov chain Monte Carlo (…
We introduce the speculate-correct method to derive error bounds for local classifiers. Using it, we show that k nearest neighbor classifiers, in spite of their famously fractured decision boundaries, have exponential error bounds with O(sqrt((k + ln n) / n)) error bound range for n in-sample examples.
Paper offers a new approach to locally weighted regression and classification.
problem Optimal number of neighbors and weights in k-nearest neighbors.
method Locally weighted regression with explicit bias-variance tradeoff.
result Efficiently finds optimal weights and number of neighbors for each data point.
Java implementation improves nearest neighbor algorithm complexity.
problem Improving efficiency of nearest neighbor descent algorithm.
method Parallel streams implementation with statistical termination criterion.
result Complexity up to O(nK2logK(n)) for K-nearest neighbors. We study clustering algorithms based on neighborhood graphs on a random sample of data points. The question we ask is how such a graph should be constructed in order to obtain optimal clustering results. Which type of neighborhood graph should one choose, mutual k-nearest neighbor or symmetric k-nearest neighbor? What …
Paper proposes a k-NN classifier for detecting spike-and-wave seizures in EEG.
problem Early detection of epileptic seizures in EEG signals.
method Uses t-location-scale distribution and k-nearest neighbors classifier.
result Demonstrates improved classification accuracy, sensitivity, and specificity on real data.
This paper improves forecasts for diverse time series by averaging similar ones.
problem Forecasting challenges in heterogeneous time series.
method Dynamic Time Warping to find similar time series, k-Nearest Neighbor averaging.
result Averaging improves forecasts of simple models.
Paper improves k-NN predictive performance with efficient variable selection.
problem Improving predictive performance of k-NN models. method Efficient forward selection of predictor variables.
result Novel approach approaches outperformance of stepwise selection models.
Randomized graph construction ensures giant component with fewer edges.
problem Efficiently constructing sparse graphs with good connectivity.
method Randomly connecting points to a subset of their nearest neighbors.
result A sparser graph with comparable connectivity properties.
New analysis shows k-nearest neighbors' robustness depends on k value.
problem Understanding why adversarial examples arise in classifiers.
method Theoretical framework based on bias-variance theory.
result k-nearest neighbors' robustness properties depend on k value.
DNAF accelerates DQL for efficient resource allocation in network slicing.
problem Efficient resource allocation in network slicing with varying demands.
method Introduced discrete normalized advantage functions (DNAF) into DQL, using a k-nearest neighbor algorithm for discrete action space.
result DNAF-based DQL converges faster through simulations.
Two methods forecast functional time series, offering competitive results.
problem Forecasting functional time series with model-free approaches.
method Two nonparametric methods: k-nearest neighbors adaptation and curve envelope selection.
result Competitive results with and often superior to benchmarks.
Defense against adversarial examples using k-NN on neural network activations.
problem Adversarial examples that fool machine learning models.
method k-Nearest Neighbor (kNN) on intermediate activations of neural networks.
result Significantly outperforms state-of-the-art defenses on MNIST and CIFAR-10.
This paper compares machine learning algorithms for ecological data.
problem Classifying ecological datasets into subsets with common patterns.
method Applied eight machine learning classification algorithms to ecological data.
result Linear Discriminant Analysis and k-nearest neighbors are the best methods.
Consider a weighted or unweighted k-nearest neighbor graph that has been built on n data points drawn randomly according to some density p on R^d. We study the convergence of the shortest path distance in such graphs as the sample size tends to infinity. We prove that for unweighted kNN graphs, this distance converges …
Paper compares two possibilistic segmentation methods for SAS imagery.
problem Segmenting synthetic aperture sonar images into different seafloor environments.
method Comparison of Possibilistic Fuzzy Local Information C-Means (PFLICM) and Possibilistic K-Nearest Neighbors (PKNN) algorithms.
result PKNN outperforms PFLICM in segmentation performance on SAS images.
Machine learning classifies IoT noise in smart cities.
problem Noise classification in smart cities.
method Mel-frequency cepstral coefficients for audio features, supervised classification (SVM, k-NN), parameter optimization.
result Noise classification accuracy of 85% - 100%.
Algorithm finds adversarial examples for k-NN classifiers using Voronoi diagrams.
problem Ensuring robustness of k-NN classifiers against adversarial attacks.
method Geometric approach expanding outwards from input points to find minimum-norm adversarial examples.
result Our method outperforms existing approaches on various datasets.
Predicts employee attrition using k-NN algorithm.
problem Predicting employee attrition in a company.
method Used k-Nearest Neighbors algorithm with employee performance, hours worked, and years of service as features. Split dataset 70/30 for training/testing.
result Achieved 94.32% accuracy in predictions.
Acoustic sensors identify vehicles using spectral embedding.
problem Vehicle recognition from roadside audio sensors.
method Extract frequency signatures, apply spectral embedding for dimensionality reduction.
result K-nearest neighbors achieve accurate vehicle identification after dimensionality reduction.
Improved multiclass classification with class-weighted nearest neighbors.
problem Multiclass classification with large or imbalanced classes.
method Class-weighted k-nearest neighbors algorithm, derived bounds on accuracy and risk.
result Optimized classification metrics like F1 score or Matthew's Correlation Coefficient.
OFTER predicts multivariate time series online, outperforming baselines.
problem Mid-sized multivariate time series forecasting challenges.
method k-nearest neighbors, Generalized Regression Neural Networks, dimensionality reduction.
result OFTER outperforms state-of-the-art baselines in financial multivariate time series forecasting.
Deep learning improves causal inference methods for treatment effects.
problem Estimating individual and average treatment effects in econometrics.
method Proposed deep learning techniques for propensity score matching and generalized neighbor matching.
result Deep learning methods outperform traditional techniques in estimating treatment effects.
rpForests uses random projections to speed up kNN search.
problem Efficient kNN search in high-dimensional data.
method Random projection forests aggregating results from multiple random projection trees.
result rpForests achieves fast decay in missing rate and kNN distance discrepancy.
We provide finite-sample analysis of a general framework for using k-nearest neighbor statistics to estimate functionals of a nonparametric continuous probability density, including entropies and divergences. Rather than plugging a consistent density estimate (which requires k→∞ as the sample size $n \to \in…
Enhanced KNN-TSVM reduces noise and overfitting with stable learning rules.
problem High computational cost and overfitting in KNN-based TSVM classifiers.
method Weighted KNN with stabilizer term and LDMDBA for efficient computation.
result Effective in both classification accuracy and computational time, with up to 14x speedup.
DW-KNN improves KNN by integrating distance and neighbor reliability for better prediction accuracy.
problem Standard KNN assumes all neighbors are equally reliable, leading to unreliable predictions in heterogeneous feature spaces.
method DW-KNN integrates exponential distance with neighbor validity, providing instance-level interpretability and reducing hyperparameter sensitivity.
result DW-KNN achieves 0.8988 average accuracy, ranks 2nd among six methods, and has the lowest cross-validation variance.
New method improves crowd counting accuracy using inverse k-NN maps and multiscale upsampling.
problem Improving accuracy of crowd density maps for high-density gatherings.
method Developed MUD-ikNN architecture using inverse k-NN maps and multiscale upsampling. result New network architecture outperforms state-of-the-art crowd counting.
Study examines neural networks for feature extraction and their impact on machine learning models.
problem Improving feature extraction for better machine learning model performance.
method Used neural networks to extract features from images and numeric data, then compared these features with SVMs and KNNs.
result Neural network-extracted features significantly enhance SVM and KNN performance in many cases.
New method quantifies deep kNN anomaly detection significance.
problem Lack of uncertainty quantification in deep kNN AD.
method Selective Inference for anomaly scoring.
result Validates AD reliability with controlled false positives.
Method detects adversarial samples using influence functions and nearest neighbors.
problem Detecting adversarial attacks on deep neural networks.
method Influence functions and k-NN model on activation layers.
result Successfully distinguishes adversarial examples with state-of-the-art results.
A new ensemble method improves kNN performance by extending the neighborhood rule.
problem Traditional kNN's limitations when test points are outside the spherical region and ensemble's high errors.
method Determines neighbors in k steps, using bootstrap samples and optimal models selection.
result The proposed ensemble method outperforms state-of-the-art methods on 17 benchmark datasets.
Paper uses K-NN resampling to simulate and evaluate LOB markets.
problem Simulating and evaluating limit order book (LOB) markets.
method Applies K-nearest neighbor (K-NN) resampling to LOB simulation and evaluation. result Demonstrates the effectiveness and efficiency of K-NN resampling in LOB simulation and evaluation. Under-bagging k-NN improves performance on imbalanced classification.
problem Imbalanced classification problems where one class is significantly underrepresented.
method Proposes an under-bagging k-NN ensemble learning algorithm, analyzing convergence rates and efficiency. result Achieves optimal convergence rates under mild assumptions and reduces sub-sample size and k for highly imbalanced data. Techniques such as clusterization, neural networks and decision making usually rely on algorithms that are not well suited to deal with missing values. However, real world data frequently contains such cases. The simplest solution is to either substitute them by a best guess value or completely disregard the missing va…