The k-means++ algorithm is generalized by choosing the most distant point from the nearest center.
problem Improving the initialization of k-means clustering.
method Generalizing the center initialization process by selecting the most distant point from the nearest center.
result Choosing the most distant point from the nearest center achieves similar clustering quality to k-means++.
New method speeds up k-means clustering for large k by improving nearest-neighbor search.
problem Efficiently clustering large datasets with high-dimensional points.
method Seeded Approximate Nearest-Neighbor Search methods to improve Lloyd's algorithm.
result Significantly faster k-means clustering for large k values.
Sparse nearest-centroid classifiers detect relevant features for classification.
problem Classifying data with low computational cost and feature selection.
method Proposes ℓ1 and ℓ2 sparse variants of nearest-centroid classifiers. result Training sparse classifiers can be done exactly and at quasi-linear cost.
Paper tackles noisy comparison oracle for robust clustering algorithms.
problem Finding robust clustering algorithms under noisy comparison oracle.
method Develops algorithms for k-center clustering and agglomerative hierarchical clustering using noisy comparison oracle.
result Proves robust algorithms achieve good approximation guarantees with high probability.
DADC algorithm improves clustering for data with varying density.
problem Sparse cluster loss and cluster fragmentation in density peak clustering.
method Domain-adaptive density measurement, cluster center self-identification, and cluster self-ensemble.
result DADC achieves more reasonable clustering results on data with varying density.
We study the question of fair clustering under the {\em disparate impact} doctrine, where each protected class must have approximately equal representation in every cluster. We formulate the fair clustering problem under both the k-center and the k-median objectives, and show that even with two protected classes th…
Develops a fair clustering algorithm for datasets with outliers.
problem Fair clustering with outliers in datasets.
method Solves a linear program to identify and exclude outliers, then applies a rounding algorithm to find fair centers.
result Guaranteed approximation of fair radius and clustering cost.
Unsupervised deep learning is one of the most powerful representation learning techniques. Restricted Boltzman machine, sparse coding, regularized auto-encoders, and convolutional neural networks are pioneering building blocks of deep learning. In this paper, we propose a new building block -- distributed random models…
Many scientific and engineering challenges -- ranging from personalized medicine to customized marketing recommendations -- require an understanding of treatment effect heterogeneity. In this paper, we develop a non-parametric causal forest for estimating heterogeneous treatment effects that extends Breiman's widely us…
GBOC detects anomalies in time series data using granular-ball vectors.
problem Challenges in modeling normal behavior in dynamic, nonlinear time series data.
method Granular-ball Vector Data Description (GVDD) and Granular-ball One-Class Network (GBOC).
result GBOC improves anomaly detection in time series data.
Metric learning methods for dimensionality reduction in combination with k-Nearest Neighbors (kNN) have been extensively deployed in many classification, data embedding, and information retrieval applications. However, most of these approaches involve pairwise training data comparisons, and thus have quadratic computat…
A relatively recent advance in cognitive neuroscience has been multi-voxel pattern analysis (MVPA), which enables researchers to decode brain states and/or the type of information represented in the brain during a cognitive operation. MVPA methods utilize machine learning algorithms to distinguish among types of inform…
Survey of counterfactual explanations for time series classification.
problem Generating plausible and actionable counterfactuals for time series data.
method Review of various counterfactual generation methods for time series classification.
result Highlight unique challenges and strengths of existing methods.
Study nearest-neighbor radii under dependent sampling, finding they remain informative.
problem Analyzing nearest-neighbor radii under dependent sampling.
method Consider strong mixing dependent observations, establish distribution-free almost sure convergence and sharp non-asymptotic moment bounds.
result Nearest-neighbor geometry remains informative under dependence sampling.
Deep nearest neighbors outperform self-supervised methods in anomaly detection.
problem Anomaly detection using self-supervised deep methods.
method Simple nearest-neighbor approach on Imagenet pretrained features.
result Nearest-neighbor method outperforms self-supervised methods in accuracy, few shot generalization, training time, and noise robustness.
Characterizes Lebesgue points using nearest neighbor methods.
problem Consistency of classification algorithms based on nearest neighbors.
method Characterization of Lebesgue points via 1-Nearest Neighbor regression.
result Proves convergence of 1-Nearest Neighbor classification algorithms in metric spaces.
The paper explains how nearest neighbor methods succeed in prediction.
problem Explaining the success of nearest neighbor methods in prediction.
method The paper covers both theoretical and practical aspects of nearest neighbor methods, including statistical guarantees and practical algorithms.
result The paper provides nonasymptotic statistical guarantees and practical algorithms for nearest neighbor methods.
AWNN improves matrix completion by adaptively weighting nearest neighbors.
problem Matrix completion with optimal nearest neighbor weights and radii selection.
method Adaptively weighted nearest neighbor method for matrix completion.
result Theoretical guarantees and synthetic experiments support the effectiveness of AWNN.
A new method uses nearest neighbors for importance weighting.
problem Data covariate shift problems in machine learning.
method Nearest neighbor classification scheme for determining importance weights.
result Demonstrated effectiveness through comparative experiments on various classification tasks.
The estimation of optimal treatment regimes is of considerable interest to precision medicine. In this work, we propose a causal k-nearest neighbor method to estimate the optimal treatment regime. The method roots in the framework of causal inference, and estimates the causal treatment effects within the nearest neig…
This paper compares FAISS and FENSHSES for nearest neighbor search in Hamming space.
problem Comparing nearest neighbor search systems in Hamming space.
method Comprehensive evaluations of indexing speed, search latency, and RAM consumption.
result Better understanding of trade-offs between main memory and secondary memory systems.
A review of nearest neighbour classifiers, focusing on similarity measures and computational efficiency.
problem Improving the performance of nearest neighbour classifiers.
method Explains mechanisms for assessing similarity (distance), identifying nearest neighbours, and reducing data dimensionality.
result Added new sections on time-series similarity measures, retrieval speed-up, and intrinsic dimensionality.
Nearest neighbor methods are a popular class of nonparametric estimators with several desirable properties, such as adaptivity to different distance scales in different regions of space. Prior work on convergence rates for nearest neighbor classification has not fully reflected these subtle properties. We analyze the b…
A defense against adversarial examples using k-Nearest Neighbor and deep learning.
problem Evaluating robustness of k-Nearest Neighbor and its deep learning combination.
method Proposed heuristic attack to find adversarial examples for kNN and DkNN.
result Our attack significantly outperforms other attacks on DkNN.
Neural networks classify OOD images by their nearest neighbor in training data.
problem Understanding out-of-distribution prediction behavior of neural networks.
method Nearest category generalization (NCG) measure to assess OOD prediction accuracy.
result Adversarially robust networks have higher NCG accuracy than natural training, indicating local regularization impacts decision regions.
VNNGP uses nearest neighbors to approximate GPs, improving scalability and performance.
problem Scalability issues in Gaussian process approximations.
method Sparse precision structure via nearest neighbors, variational framework.
result VNNGP outperforms low-rank methods and is less prone to overfitting.
Java implementation improves nearest neighbor algorithm complexity.
problem Improving efficiency of nearest neighbor descent algorithm.
method Parallel streams implementation with statistical termination criterion.
result Complexity up to O(nK2logK(n)) for K-nearest neighbors. The paper tackles time series data by applying conformal prediction with nearest neighbors.
problem Time series data violates the exchangeability assumption required for conformal prediction.
method The approach uses the nearest neighbors method with fast parameter tuning and weighted nearest neighbors (FPTO-WNN) to construct reliable prediction intervals.
result Data analysis shows the effectiveness of the proposed approach.
In (\cite{zhang2014nonlinear,zhang2014nonlinear2}), we have viewed machine learning as a coding and dimensionality reduction problem, and further proposed a simple unsupervised dimensionality reduction method, entitled deep distributed random samplings (DDRS). In this paper, we further extend it to supervised learning …
A new multilabel classification framework improves ANN search performance.
problem Efficiently finding approximate nearest neighbors in large datasets.
method Formulated ANN search as a multilabel classification problem, using partitioning classifiers.
result Natural classifier leads to strictly improved performance in ANN search.
K-Nearest neighbor classifier (k-NNC) is simple to use and has little design time like finding k values in k-nearest neighbor classifier, hence these are suitable to work with dynamically varying data-sets. There exists some fundamental improvements over the basic k-NNC, like weighted k-nearest neighbors classifier (wh…
The paper studies empirical processes from nearest neighbors in regression.
problem Estimating conditional cumulative distribution functions and local linear regression.
method Uniform central limit theorem and non-asymptotic bound under local bracketing entropy and uniform entropy numbers.
result Gaussian limit of empirical process with simple covariance.
This paper proposes a new hashing-based KNN technique for faster nearest neighbor selection.
problem Slowness of KNN in big datasets due to searching entire dataset.
method Divide data space into subcells, use hashing to map data points, and select nearest neighbors layer by layer.
result The proposed technique offers competitive performance with KNN and KDtree while significantly improving time efficiency.
BigNN classifier improves nearest neighbor classification for large datasets.
problem Classification of large datasets that cannot fit into a single machine's memory.
method Divide and conquer scheme with majority voting for final decision; pre-training acceleration technique.
result Rates of convergence for bigNN classifier under minimal assumptions, proving it as optimal.
In this paper we propose and explore the k-Nearest Neighbour UCB algorithm for multi-armed bandits with covariates. We focus on a setting where the covariates are supported on a metric space of low intrinsic dimension, such as a manifold embedded within a high dimensional ambient feature space. The algorithm is concept…
The nearest neighbor problem is defined as follows: Given a set P of n points in some metric space (X,D), build a data structure that, given any point q, returns a point in P that is closest to q (its "nearest neighbor" in P). The data structure stores additional information about the set P, which is th…
Study infinite Euclidean distance discriminants of algebraic varieties.
problem Understanding the structure of data points with infinitely many critical points in Euclidean distance correspondence.
method Developed computer code to compute discriminants and proved properties of fibers.
result Infinite Euclidean distance discriminants contain all data points with infinitely many critical points for the nearest-point problem.
A new adaptive kNN classifier outperforms Random Forests.
problem Improving classification accuracy using nearest neighbors.
method Finding discriminant subspaces for efficient nearest neighbor classification, leveraging bagging for diversity.
result The proposed method outperforms Random Forests and other nearest neighbors ensembles.
New algorithm reduces regret in contextual bandits with many near-boundary contexts.
problem High regret in contextual bandits with many near-boundary contexts.
method Hierarchical nearest neighbour approach, holding out contexts for computation.
result Eradicates high regret in adversarial contextual bandits.
Locally adaptive nearest neighbors improve automated systems' performance and are easier to interpret.
problem Improving automated systems' performance and interpretability.
method Developed a method for k nearest neighbors algorithms to learn locally adaptive metrics.
result Locally adaptive metrics improve performance and are interpretable.
New methods use vector search and nearest-neighbor matching for policy learning in causal inference.
problem Learning optimal policies in causal inference with limited data.
method RAG-based policy learning with vector search and nearest-neighbor matching.
result The methods bound the within-candidate choice regret and evaluate the one-step method directly as a policy.
Improved multiclass classification with class-weighted nearest neighbors.
problem Multiclass classification with large or imbalanced classes.
method Class-weighted k-nearest neighbors algorithm, derived bounds on accuracy and risk.
result Optimized classification metrics like F1 score or Matthew's Correlation Coefficient.
New algorithm improves topological stability in non-linear dimensionality reduction.
problem Topological instability in choosing nearest neighbors in Isomap.
method Uses point and its two nearest neighbors to find subspace and orthogonal complement, then adds new points based on distance and angle.
result Improves topological stability and reduces short-circuit errors.
A hybrid K-NN and SVM technique improves classification accuracy.
problem Improving classification accuracy in pattern recognition.
method Discriminative nearest neighbour classification combined with SVM.
result The hybrid technique outperforms state-of-the-art methods.
Algorithms often carry out equally many computations for "easy" and "hard" problem instances. In particular, algorithms for finding nearest neighbors typically have the same running time regardless of the particular problem instance. In this paper, we consider the approximate k-nearest-neighbor problem, which is the pr…
Combines fast evaluation with Bayes consistency in nearest neighbors.
problem Balancing fast evaluation time with Bayes consistency in nearest neighbors.
method Combines locality-sensitive hashing (LSH) with a missing-mass argument.
result Fast and Bayes-consistent classifier with comparable risk decay rates.
A new method centers outliers in robust PCA without manual intervention.
problem Outliers in robust PCA require manual centering, complicating the analysis.
method Introduces a 'bias trick' to automatically center non-outliers.
result First optimal RPCA algorithm with automatic centering.
This paper improves nearest neighbor search by learning optimal routing functions.
problem Local minima issues in greedy routing on similarity graphs.
method Learn routing function that considers global graph structure.
result Significant improvement in search performance via learning.