DW-KNN improves KNN by integrating distance and neighbor reliability for better prediction accuracy.
problem Standard KNN assumes all neighbors are equally reliable, leading to unreliable predictions in heterogeneous feature spaces.
method DW-KNN integrates exponential distance with neighbor validity, providing instance-level interpretability and reducing hyperparameter sensitivity.
result DW-KNN achieves 0.8988 average accuracy, ranks 2nd among six methods, and has the lowest cross-validation variance.
GICDM corrects hubness in embedding spaces for better generative model evaluation.
problem Hubness phenomenon distorts distances in high-dimensional embedding spaces.
method Generative ICDM (GICDM) using multi-scale extension to correct neighborhood estimation.
result GICDM resolves hubness-induced failures and improves metric behavior.
Improves visualization of high-dimensional data by correcting misleading artifacts in neighbor embedding methods.
problem Misleading visual artifacts in t-SNE and UMAP due to lack of data-independent manifold learning interpretations.
method LOO-map framework that extends embedding maps to the entire input space, identifying and correcting map discontinuities.
result Developed point-wise diagnostic scores to detect unreliable embedding points and improve hyperparameter selection.
With the advent of Deep Learning, the field of machine learning (ML) has surpassed human-level performance on diverse classification tasks. At the same time, there is a stark need to characterize and quantify reliability of a model's prediction on individual samples. This is especially true in application of such model…
Enhanced metrics improve generative model evaluation reliability.
problem Lack of reliable quality metrics for generative models.
method Introduce Clipped Density and Clipped Coverage metrics.
result Metrics prevent out-of-distribution samples from biasing quality scores.
New methods for distributed CP improve reliability in healthcare.
problem Post-hoc calibration of models in safety-critical domains.
method Quantile-based and histogram-based distributed conformal prediction via message passing.
result Improved reliability and coverage guarantees in decentralized settings.
A predictor improves power grid frequency forecasts up to one hour.
problem Improving frequency forecast for better power system stability.
method Developed a weighted-nearest-neighbor (WNN) predictor.
result Forecasts for up to one hour are more precise than averaged daily profiles.
NESS improves neighbor embedding for smooth cell-state transitions in single-cell data.
problem Challenges in extracting smooth, low-dimensional representations from noisy single-cell data.
method Builds on PCS framework to develop NESS, a stable machine learning approach.
result NESS consistently yields useful biological insights across diverse single-cell datasets.
Although many successful ensemble clustering approaches have been developed in recent years, there are still two limitations to most of the existing approaches. First, they mostly overlook the issue of uncertain links, which may mislead the overall consensus process. Second, they generally lack the ability to incorpora…
New method quantifies deep kNN anomaly detection significance.
problem Lack of uncertainty quantification in deep kNN AD.
method Selective Inference for anomaly scoring.
result Validates AD reliability with controlled false positives.
Most density-based clustering methods largely rely on how well the underlying density is estimated. However, density estimation itself is also a challenging problem, especially the determination of the kernel bandwidth. A large bandwidth could lead to the over-smoothed density estimation in which the number of density …
Modeling interacting objects with latent Gaussian process ODEs.
problem Time uncertainty-aware modeling of continuous-time dynamics of interacting objects.
method A new model using latent Gaussian process ordinary differential equations to infer independent dynamics and interactions.
result Our model improves long-term predictions and successfully encapsulates independent dynamics and interactions.
The paper studies practical estimation and interpretation of Rényi transfer entropy.
problem Challenges in accurately estimating and interpreting Rényi transfer entropy.
method Systematic study of k-nearest neighbor estimator for Rényi entropy and transfer entropy.
result Effective estimates of effective Rényi transfer entropy can accurately capture directional information flow.
Proposes a parametric t-SNE without perplexity tuning.
problem Non-parametric t-SNE's perplexity parameter limits DR quality.
method Multi-scale parametric t-SNE with deep neural network.
result Produces reliable embeddings with competitive neighborhood preservation.
Conditional independence testing is a fundamental problem underlying causal discovery and a particularly challenging task in the presence of nonlinear and high-dimensional dependencies. Here a fully non-parametric test for continuous data based on conditional mutual information combined with a local permutation scheme …
LNPE enhances local connections in embeddings using extended neighbor propagation.
problem Improving local connections and interactions in nonlinear dimensionality reduction.
method Inspired by GCN, LNPE extends 1-hop neighbors to n-hop neighbors in LLE.
result LNPE produces more faithful and robust embeddings with better topological and geometrical properties.
The existing approaches to intrinsic dimension estimation usually are not reliable when the data are nonlinearly embedded in the high dimensional space. In this work, we show that the explicit accounting to geometric properties of unknown support leads to the polynomial correction to the standard maximum likelihood est…
Previously in 2014, we proposed the Nearest Descent (ND) method, capable of generating an efficient Graph, called the in-tree (IT). Due to some beautiful and effective features, this IT structure proves well suited for data clustering. Although there exist some redundant edges in IT, they usually have salient features …
High-dimensional inference for sparse spectral precision matrices
problem Inference on the spectral precision matrix at a fixed frequency
method Full likelihood-based inference using neighboring discrete Fourier transforms
result Simultaneous control of regularization, finite-sample truncation, and smoothing biases
New framework uses geometry of embeddings to predict robustness.
problem Monitoring robustness in models without OOD labels.
method Constructs graphs from embeddings, measures spectral complexity and curvature.
result Representation geometry predicts robustness reliably.
This work investigates how GCNs should handle local structure discrepancies in testing nodes.
problem GCNs assume homophily but real graphs often have discrepancies in local structure.
method Using causal graph analysis, the study intervenes the graph structure to assess the local structure's impact on predictions.
result The method effectively enhances GCN predictions by eliminating local structure discrepancies.
In unsupervised outlier ensembles, the absence of ground truth makes the combination of base outlier detectors a challenging task. Specifically, existing parallel outlier ensembles lack a reliable way of selecting competent base detectors, affecting accuracy and stability, during model combination. In this paper, we pr…
Study nearest-neighbor radii under dependent sampling, finding they remain informative.
problem Analyzing nearest-neighbor radii under dependent sampling.
method Consider strong mixing dependent observations, establish distribution-free almost sure convergence and sharp non-asymptotic moment bounds.
result Nearest-neighbor geometry remains informative under dependence sampling.
C-MinHash reduces the number of permutations needed for MinHash from thousands to just two.
problem Approximating Jaccard similarity in large binary datasets using many permutations.
method Initial permutation followed by circulant shifting of a second permutation to generate hashes.
result C-MinHash achieves unbiased Jaccard similarity estimation with uniformly smaller variance.
Characterizes Lebesgue points using nearest neighbor methods.
problem Consistency of classification algorithms based on nearest neighbors.
method Characterization of Lebesgue points via 1-Nearest Neighbor regression.
result Proves convergence of 1-Nearest Neighbor classification algorithms in metric spaces.
Develops MIS, a probabilistic model for multi-industry classification.
problem GICS's limitation of assigning each firm to exactly one industry, especially for diversified firms.
method Topic modeling to probabilistically assign firms to multiple industries based on business descriptions.
result Demonstrates MIS's ability to flexibly assign firms to multiple industries with relevance probabilities.
The paper explains how nearest neighbor methods succeed in prediction.
problem Explaining the success of nearest neighbor methods in prediction.
method The paper covers both theoretical and practical aspects of nearest neighbor methods, including statistical guarantees and practical algorithms.
result The paper provides nonasymptotic statistical guarantees and practical algorithms for nearest neighbor methods.
A common machine learning task is to discriminate between normal and anomalous data points. In practice, it is not always sufficient to reach high accuracy at this task, one also would like to understand why a given data point has been predicted in a certain way. We present a new principled approach for one-class SVMs …
AWNN improves matrix completion by adaptively weighting nearest neighbors.
problem Matrix completion with optimal nearest neighbor weights and radii selection.
method Adaptively weighted nearest neighbor method for matrix completion.
result Theoretical guarantees and synthetic experiments support the effectiveness of AWNN.
Hybrid model improves COVID-19 case forecasting accuracy.
problem Limited data and simplistic models for accurate prediction.
method Combining SEIR and RNN on a graph structure with local and edge features.
result Improves prediction accuracy on state-level COVID-19 data.
A new method uses nearest neighbors for importance weighting.
problem Data covariate shift problems in machine learning.
method Nearest neighbor classification scheme for determining importance weights.
result Demonstrated effectiveness through comparative experiments on various classification tasks.
Selecting and combining the outlier scores of different base detectors used within outlier ensembles can be quite challenging in the absence of ground truth. In this paper, an unsupervised outlier detector combination framework called DCSO is proposed, demonstrated and assessed for the dynamic selection of most compete…
Nearest neighbor methods are a popular class of nonparametric estimators with several desirable properties, such as adaptivity to different distance scales in different regions of space. Prior work on convergence rates for nearest neighbor classification has not fully reflected these subtle properties. We analyze the b…
GBOC detects anomalies in time series data using granular-ball vectors.
problem Challenges in modeling normal behavior in dynamic, nonlinear time series data.
method Granular-ball Vector Data Description (GVDD) and Granular-ball One-Class Network (GBOC).
result GBOC improves anomaly detection in time series data.
We propose a simple kernel based nearest neighbor approach for handwritten digit classification. The "distance" here is actually a kernel defining the similarity between two images. We carefully study the effects of different number of neighbors and weight schemes and report the results. With only a few nearest neighbo…
New methods use vector search and nearest-neighbor matching for policy learning in causal inference.
problem Learning optimal policies in causal inference with limited data.
method RAG-based policy learning with vector search and nearest-neighbor matching.
result The methods bound the within-candidate choice regret and evaluate the one-step method directly as a policy.
VNNGP uses nearest neighbors to approximate GPs, improving scalability and performance.
problem Scalability issues in Gaussian process approximations.
method Sparse precision structure via nearest neighbors, variational framework.
result VNNGP outperforms low-rank methods and is less prone to overfitting.
The estimation of optimal treatment regimes is of considerable interest to precision medicine. In this work, we propose a causal k-nearest neighbor method to estimate the optimal treatment regime. The method roots in the framework of causal inference, and estimates the causal treatment effects within the nearest neig…
Nearest neighbors is a successful and long-standing technique for anomaly detection. Significant progress has been recently achieved by self-supervised deep methods (e.g. RotNet). Self-supervised features however typically under-perform Imagenet pre-trained features. In this work, we investigate whether the recent prog…
New estimator for joint entropy outperforms existing methods in various distributions.
problem Estimating joint entropy in high-dimensional spaces.
method Partitioned sample spacing (PSS) for nonparametric estimation.
result PSS consistently outperforms k-NN and normalizing flow methods.
Deep neural networks (DNNs) enable innovative applications of machine learning like image recognition, machine translation, or malware detection. However, deep learning is often criticized for its lack of robustness in adversarial settings (e.g., vulnerability to adversarial inputs) and general inability to rationalize…
Java implementation improves nearest neighbor algorithm complexity.
problem Improving efficiency of nearest neighbor descent algorithm.
method Parallel streams implementation with statistical termination criterion.
result Complexity up to O(nK2logK(n)) for K-nearest neighbors. The paper tackles time series data by applying conformal prediction with nearest neighbors.
problem Time series data violates the exchangeability assumption required for conformal prediction.
method The approach uses the nearest neighbors method with fast parameter tuning and weighted nearest neighbors (FPTO-WNN) to construct reliable prediction intervals.
result Data analysis shows the effectiveness of the proposed approach.
Study of f-neighbors in Riemannian manifolds, proving infinite set of distances.
problem Exploring variations of Hopf theorem in Riemannian manifolds.
method Investigates continuous maps of compact Riemannian manifolds to Rm and introduces f-neighbors. result Set of distances realized as visual f-neighbors is infinite. K-Nearest neighbor classifier (k-NNC) is simple to use and has little design time like finding k values in k-nearest neighbor classifier, hence these are suitable to work with dynamically varying data-sets. There exists some fundamental improvements over the basic k-NNC, like weighted k-nearest neighbors classifier (wh…
The paper studies empirical processes from nearest neighbors in regression.
problem Estimating conditional cumulative distribution functions and local linear regression.
method Uniform central limit theorem and non-asymptotic bound under local bracketing entropy and uniform entropy numbers.
result Gaussian limit of empirical process with simple covariance.
This paper proposes a new hashing-based KNN technique for faster nearest neighbor selection.
problem Slowness of KNN in big datasets due to searching entire dataset.
method Divide data space into subcells, use hashing to map data points, and select nearest neighbors layer by layer.
result The proposed technique offers competitive performance with KNN and KDtree while significantly improving time efficiency.
This paper introduces scikit-hubness, a Python package for efficient nearest neighbor search in high-dimensional spaces. Hubness is an aspect of the curse of dimensionality, and is known to impair various learning tasks, including classification, clustering, and visualization. scikit-hubness provides algorithms for hub…