Adaptive algorithm selects optimal neighbors for manifold learning.
problem Selecting optimal neighbors in manifold learning algorithms.
method Adaptive neighboring selection algorithm based on curvature prediction.
result The proposed algorithm increases embedding quality by 45.45%.
Paper improves k-NN predictive performance with efficient variable selection.
problem Improving predictive performance of k-NN models. method Efficient forward selection of predictor variables.
result Novel approach approaches outperformance of stepwise selection models.
This paper proposes a new hashing-based KNN technique for faster nearest neighbor selection.
problem Slowness of KNN in big datasets due to searching entire dataset.
method Divide data space into subcells, use hashing to map data points, and select nearest neighbors layer by layer.
result The proposed technique offers competitive performance with KNN and KDtree while significantly improving time efficiency.
We report the results of fifteen sets of portfolio selection simulations using stocks in the ASX200 index for the period May 2000 to December 2013. We investigated five portfolio selection methods, randomly and from within industrial groups, and three based on neighbor-Net phylogenetic networks. We report that using ra…
A new method estimates optimal treatment regimes using causal nearest neighbors.
problem Estimating optimal treatment regimes in precision medicine.
method Causal k-nearest neighbor method, with adaptive metric and variable selection.
result The causal k-nearest neighbor regime is universally consistent and converges as sample size increases.
Paper proves noise-tolerant SSC using greedy methods under coherence conditions.
problem Proving noise-tolerant SSC using greedy methods under coherence conditions.
method Derives coherence-based sufficient conditions for correct neighbor identification using MP/OMP in the presence of bounded noise.
result MP/OMP succeed in identifying correct neighbors under certain noise levels, leading to higher clustering accuracy.
New methods use vector search and nearest-neighbor matching for policy learning in causal inference.
problem Learning optimal policies in causal inference with limited data.
method RAG-based policy learning with vector search and nearest-neighbor matching.
result The methods bound the within-candidate choice regret and evaluate the one-step method directly as a policy.
We propose Bayesian extensions of two nonparametric regression methods which are kernel and mutual k-nearest neighbor regression methods. Derived based on Gaussian process models for regression, the extensions provide distributions for target value estimates and the framework to select the hyperparameters. It is show…
In the panoply of pattern classification techniques, few enjoy the intuitive appeal and simplicity of the nearest neighbor rule: given a set of samples in some metric domain space whose value under some function is known, we estimate the function anywhere in the domain by giving the value of the nearest sample per the …
Improves performance in various machine learning tasks by reparameterizing subset sampling.
problem Stochastic optimization involving subset sampling is not reparameterizable.
method Continuous relaxation of subset sampling to provide reparameterization gradients.
result Improves performance in instance-wise feature selection, deep stochastic k-nearest neighbors, and parametric t-SNE.
Efficiently selects nearest neighbors for labeling to speed up active learning.
problem Intractable active learning and search for large-scale unlabeled data.
method Restricts candidate pool to nearest neighbors of labeled set.
result Achieved similar performance to global approach but reduced computational cost by up to 3 orders of magnitude.
Improves visualization of high-dimensional data by correcting misleading artifacts in neighbor embedding methods.
problem Misleading visual artifacts in t-SNE and UMAP due to lack of data-independent manifold learning interpretations.
method LOO-map framework that extends embedding maps to the entire input space, identifying and correcting map discontinuities.
result Developed point-wise diagnostic scores to detect unreliable embedding points and improve hyperparameter selection.
Enhances k-NN accuracy through randomized hyperstructure.
problem Improves k-NN accuracy by optimizing neighbor selection.
method Constructs a random n-dimensional hyperstructure around test instances to refine neighbor selection.
result 85.71% accuracy on Haberman's Cancer Survival dataset, compared to 80.95% for conventional k-NN.
New method selects recent similar periods for better electricity price forecasting.
problem Improving accuracy in forecasting electricity prices.
method Change-point detection (NOT method) to select calibration periods; estimating autoregressive models only for selected data.
result Significant improvement in forecasting accuracy compared to existing methods.
We compare three network portfolio selection methods; hierarchical clustering trees, minimum spanning trees and neighbor-Nets, with random and industry group selection methods on twelve years of data from the 30 Dow Jones Industrial Average stocks from 2001 to 2013 for very small private investor sized portfolios. We f…
Enhances classifier performance through feature space transformations and model selection.
problem Improving the accuracy of classifiers by reducing complexity.
method Combining feature mapping, prototype selection, and kernel function transformations to transform data into a more convenient distribution.
result Our methods produce competitive classifiers and are statistically different among them.
A new multilabel classification framework improves ANN search performance.
problem Efficiently finding approximate nearest neighbors in large datasets.
method Formulated ANN search as a multilabel classification problem, using partitioning classifiers.
result Natural classifier leads to strictly improved performance in ANN search.
This paper proposes a new method for feature scaling in K-Nearest Neighbors.
problem Feature scaling issues in K-Nearest Neighbors algorithm.
method Assign weights to individual features using out-of-bag errors from decision tree models.
result Improves prediction accuracy by assigning weights to features based on out-of-bag errors.
New method quantifies deep kNN anomaly detection significance.
problem Lack of uncertainty quantification in deep kNN AD.
method Selective Inference for anomaly scoring.
result Validates AD reliability with controlled false positives.
In our previous works, we proposed a physically-inspired rule to organize the data points into an in-tree (IT) structure, in which some undesired edges are allowed to occur. By removing those undesired or redundant edges, this IT structure is divided into several separate parts, each representing one cluster. In this w…
Predicts vessel destinations using AIS data and nearest neighbor search.
problem Accurately predict the destination ports and arrival times of vessel trips.
method Partitioned training routes by destination port, use nearest neighbor search, and incorporate improvements like avoiding frequent port changes and automating parameter tuning.
result Significant improvements in prediction accuracy compared to baseline methods.
Graph Denoising Policy Network learns robust representations from noisy graphs.
problem Noise sensitivity in graph representation learning.
method Reinforcement learning to select signal neighborhoods and aggregate features.
result Significantly outperforms state-of-the-art methods on node classification tasks.
Proposes a method to learn optimal neighbors and projection matrix in low-dimensional space.
problem Difficulty in precisely measuring similarity and selecting optimal neighbors in high-dimensional space.
method Models similarity and neighbors as variables, optimizing a unified objective function with nonnegative and sum-to-one constraints.
result Optimal similarity and projection matrix learned simultaneously, with adaptive regularization parameter.
DCSO dynamically selects top-performing base detectors for outlier ensembles.
problem Challenges in selecting and combining outlier scores from different detectors.
method DCSO dynamically selects top-performing base detectors based on local k-nearest neighbors.
result DCSO provides consistent performance improvement over static combination approaches.
New method accelerates large margin metric learning for nearest neighbor classification.
problem Efficiently learning metrics for nearest neighbor classification.
method Triplet mining and stratified sampling for large margin metric learning.
result Improved efficiency and scalability of optimization.
CRAD clusters data with robust depth-based dissimilarity, outperforming existing methods.
problem Clustering data with varying densities and unknown true number of clusters.
method CRAD uses a robust data depth as dissimilarity measure and a novel neighbor searching function.
result CRAD outperforms DBSCAN, OPTICS, and DBCA in detecting clusters with varying densities.
Unsupervised classifier performs as well as supervised ones on ImageNet dataset.
problem Achieving performance of supervised learning classifiers without labeled data.
method Incremental shift and rotation operations on selected hyperplanes.
result 6.2% Top 3 probability of error on ImageNet dataset.
We introduce a new principle for model selection in regression and classification. Many regression models are controlled by some smoothness or flexibility or complexity parameter c, e.g. the number of neighbors to be averaged over in k nearest neighbor (kNN) regression or the polynomial degree in regression with polyno…
A comparison-based algorithm finds nearest neighbors in metric spaces.
problem Finding nearest neighbors without direct distance information.
method Recursive splitting using random pivot points to form a comparison tree.
result The height of the comparison tree is logarithmic in the number of points, leading to efficient search performance.
A new ensemble method improves kNN performance by extending the neighborhood rule.
problem Traditional kNN's limitations when test points are outside the spherical region and ensemble's high errors.
method Determines neighbors in k steps, using bootstrap samples and optimal models selection.
result The proposed ensemble method outperforms state-of-the-art methods on 17 benchmark datasets.
Unified query framework for active metric learning and classification.
problem Combining representation learning and task-specific goals in machine learning.
method Adaptive selection of nearest neighbor queries using information theoretic criterion.
result Actively selected nearest neighbor queries outperform recent methods in deep metric learning and classification.
Proposes ALRL for single shot active learning with pseudo annotators.
problem Real-world applications where human experts are not always available.
method Substitutes human annotators with pseudo annotators providing random labels.
result ALRL outperforms state-of-the-art approaches in real-world datasets.
Study improves material similarity measures considering distinctiveness.
problem Improving similarity measures for materials science applications.
method Used machine learning techniques with specific descriptors and kernels.
result Minimizing loss of distinctiveness improves prediction accuracy.
Paper introduces a new, tractable measure of model complexity.
problem Need for a reliable measure of model complexity.
method Mathematically rigorous measure based on gradient similarities.
result Generalizes to various model types and insights into double descent.
The k-nearest neighbor classification method (k-NNC) is one of the simplest nonparametric classification methods. The mutual k-NN classification method (MkNNC) is a variant of k-NNC based on mutual neighborship. We propose another variant of k-NNC, the symmetric k-NN classification method (SkNNC) based …
CAD detects anomalies and selects prototypes using polyhedron curvature.
problem Anomaly detection and prototype selection in data.
method Curvature Anomaly Detection (CAD) and Kernel CAD approach using polyhedron curvature.
result The proposed methods are effective for anomaly detection and prototype selection.
Proposes a model selection method for t-SNE perplexity.
problem Manual tuning of t-SNE perplexity is difficult and requires expertise.
method Develops a model selection objective for t-SNE perplexity that minimizes extra computation.
result Perplexity settings found by the approach are consistent with human expert preferences.
Machine learning identifies skill levels in neurosurgery residents.
problem Assessing neurosurgical skill levels in residents.
method Machine learning applied to a VR tumor resection task with 23 skilled and 92 novice participants.
result Fuzzy K-Nearest Neighbors classifier achieved 8.3% equal error rate with 15 selected features.
Adaptive kNN method closes the gap for unbounded support distributions.
problem Inability of standard kNN to achieve minimax optimal rate for unbounded support distributions.
method Adaptive kNN method with varying k for different samples.
result The adaptive kNN method matches the minimax lower bound.
We focus in this paper on dataset reduction techniques for use in k-nearest neighbor classification. In such a context, feature and prototype selections have always been independently treated by the standard storage reduction algorithms. While this certifying is theoretically justified by the fact that each subproblem …
NNK algorithm improves neighborhood and graph construction for machine learning.
problem Ad hoc selection of k and ε parameters in kNN and ε-neighborhood methods.
method NNK algorithm for better sparse signal approximation.
result NNK leads to superior performance in local neighborhood and graph-based machine learning tasks.
New algorithms discover and utilize 'voids' in data to improve machine learning models.
problem Improving machine learning models by considering the unknown aspects of data.
method Developed algorithms to discover and utilize 'voids' in data, creating ignorance-aware prototypes.
result Improved performance of nearest neighbor classifiers through ignorance-aware prototype selection.
A new framework selects best outlier detectors locally for improved ensemble performance.
problem Challenges in combining outlier detectors without ground truth.
method Locally Selective Combination in Parallel Outlier Ensembles (LSCP) framework.
result LSCP_AOM variant consistently outperforms other methods on real-world datasets.
Two methods forecast functional time series, offering competitive results.
problem Forecasting functional time series with model-free approaches.
method Two nonparametric methods: k-nearest neighbors adaptation and curve envelope selection.
result Competitive results with and often superior to benchmarks.
This paper addresses the nearest neighbor search problem under inner product similarity and introduces a compact code-based approach. The idea is to approximate a vector using the composition of several elements selected from a source dictionary and to represent this vector by a short code composed of the indices of th…
Study predicts MSE survival using feature selection with missing data imputation methods.
problem Predicting survival of micro and small businesses (MSE).
method Feature selection with missing data imputation methods (MI, KNN, EM). Comparison of data mining techniques (logistic regression, naive Bayes, LDA, SVM).
result Developed a model to predict MSE survival.
ATPboost uses ATP feedback for binary premise selection in large-theory problem solving.
problem Learning relevant premises for ATP-based theorem proving in binary classification.
method Binary classification using XGBoost, with negative examples generated from alternative proofs.
result ATPboost outperforms k-nearest neighbors in binary premise selection.
The article proposes modified Gower's coefficients for handling mixed type variables in nearest neighbor methods.
problem Handling mixed type variables in nearest neighbor methods, especially imputation and statistical matching.
method Suggests modifications to the Gower's distance for interval and ratio scaled variables to address unbalanced contributions and outlier sensitivity.
result Improved distance calculations reduce the unbalanced contribution of different variable types and attenuate outlier effects.