arXiv research

AWNN improves matrix completion by adaptively weighting nearest neighbors.

Locally adaptive nearest neighbors improve automated systems' performance and are easier to interpret.

Algorithms often carry out equally many computations for "easy" and "hard" problem instances. In particular, algorithms for finding nearest neighbors typically have the same running time regardless of the particular problem instance. In this paper, we consider the approximate k-nearest-neighbor problem, which is the pr…

A novel distributed adaptive NN classifier for large data sets.

Recently manifold learning algorithm for dimensionality reduction attracts more and more interests, and various linear and nonlinear, global and local algorithms are proposed. The key step of manifold learning algorithm is the neighboring region selection. However, so far for the references we know, few of which propos…

New algorithm learns from partial labels in general scenarios.

A new method for covariate shift adaptation using nearest neighbors.

Nearest neighbor methods are a popular class of nonparametric estimators with several desirable properties, such as adaptivity to different distance scales in different regions of space. Prior work on convergence rates for nearest neighbor classification has not fully reflected these subtle properties. We analyze the b…

Dimensionality reduction is an important operation in information visualization, feature extraction, clustering, regression, and classification, especially for processing noisy high dimensional data. However, most existing approaches preserve either the global or the local structure of the data, but not both. Approache…

The estimation of optimal treatment regimes is of considerable interest to precision medicine. In this work, we propose a causal $k$-nearest neighbor method to estimate the optimal treatment regime. The method roots in the framework of causal inference, and estimates the causal treatment effects within the nearest neig…

DANCE improves prediction set efficiency for deep learning models.

The weighted k-nearest neighbors algorithm is one of the most fundamental non-parametric methods in pattern recognition and machine learning. The question of setting the optimal number of neighbors as well as the optimal weights has received much attention throughout the years, nevertheless this problem seems to have r…

Information-theoretic quantities, such as conditional entropy and mutual information, are critical data summaries for quantifying uncertainty. Current widely used approaches for computing such quantities rely on nearest neighbor methods and exhibit both strong performance and theoretical guarantees in certain simple sc…

DW-KNN improves KNN by integrating distance and neighbor reliability for better prediction accuracy.

A new method for density estimation using nearest neighbor Dirichlet mixtures.

PG-EVIKAL refines molecular property predictions using neighbor fusion and evidential neural networks.

This work proposes ACTC for adaptive distributed learning under communication constraints.

Explicitly or implicitly, most of dimensionality reduction methods need to determine which samples are neighbors and the similarity between the neighbors in the original highdimensional space. The projection matrix is then learned on the assumption that the neighborhood information (e.g., the similarity) is known and f…

$K$-NN classifier is one of the most famous classification algorithms, whose performance is crucially dependent on the distance metric. When we consider the distance metric as a parameter of $K$-NN, learning an appropriate distance metric for $K$-NN can be seen as minimizing the empirical risk of $K$-NN. In this paper,…

Adaptive NN method improves matrix completion for non-smooth data.

k Nearest Neighbor (kNN) method is a simple and popular statistical method for classification and regression. For both classification and regression problems, existing works have shown that, if the distribution of the feature vector has bounded support and the probability density function is bounded away from zero in i…

We consider a problem of multiclass classification, where the training sample $S_n = \{(X_i, Y_i)\}_{i=1}^n$ is generated from the model $\mathbb P(Y = m | X = x) = η_m(x)$, $1 \leq m \leq M$, and $η_1(x), \dots, η_M(x)$ are unknown $α$-Holder continuous functions.Given a test point $X$, our goal is to predict its labe…

Study exact community recovery in noisy SBM with limited queries.

The paper investigates learning conditional distributions on multi-dimensional spaces using clustering and neural networks.

BaNk-UCB tackles batched nonparametric bandits with k-NN regression and UCB.

Graph Convolutional Networks (GCNs) have shown very powerful for graph data representation and learning tasks. Existing GCNs usually conduct feature aggregation on a fixed neighborhood graph in which each node computes its representation by aggregating the feature representations of all its neighbors which is biased by…

We analyze the Kozachenko--Leonenko (KL) nearest neighbor estimator for the differential entropy. We obtain the first uniform upper bound on its performance over Hölder balls on a torus without assuming any conditions on how close the density could be from zero. Accompanying a new minimax lower bound over the Hölder ba…

We introduce a variant of the $k$-nearest neighbor classifier in which $k$ is chosen adaptively for each query, rather than supplied as a parameter. The choice of $k$ depends on properties of each neighborhood, and therefore may significantly vary between different points. (For example, the algorithm will use larger $k…

k-Nearest Neighbors is one of the most fundamental but effective classification models. In this paper, we propose two families of models built on a sequence to sequence model and a memory network model to mimic the k-Nearest Neighbors model, which generate a sequence of labels, a sequence of out-of-sample feature vecto…

Retrieving the most similar objects in a large-scale database for a given query is a fundamental building block in many application domains, ranging from web searches, visual, cross media, and document retrievals. State-of-the-art approaches have mainly focused on capturing the underlying geometry of the data manifolds…

Distributed processing over networks relies on in-network processing and cooperation among neighboring agents. Cooperation is beneficial when agents share a common objective. However, in many applications agents may belong to different clusters that pursue different objectives. Then, indiscriminate cooperation will lea…

New algorithms detect outliers in high-dimensional data with arbitrary shapes.

Acoustic Neighbor Embeddings map speech and text to fixed dimensions for phonetic confusability.

New method for estimating counterfactual means in adaptive experiments.

Generative models use kernel smoothing for conditioning on small example sets.

The celebrated Monte Carlo method estimates an expensive-to-compute quantity by random sampling. Bandit-based Monte Carlo optimization is a general technique for computing the minimum of many such expensive-to-compute quantities by adaptive random sampling. The technique converts an optimization problem into a statisti…

A new method for online multi-label stream classification.

SPlit optimizes dataset splitting for better model performance.

Two methods forecast functional time series, offering competitive results.

CGMMD generates conditional samples in one shot with MMD and nearest neighbors.

Improves k-NN for monotonic data with robustness against noise.

Recent theory work has found that a special type of spatial partition tree - called a random projection tree - is adaptive to the intrinsic dimension of the data from which it is built. Here we examine this same question, with a combination of theory and experiments, for a broader class of trees that includes k-d trees…

LNPE enhances local connections in embeddings using extended neighbor propagation.

Paper develops multivariate time series similarity and distance measures.

Proposes a graph learning framework for clustering and semi-supervised classification.

When there is a distributional shift between data used to train a predictive algorithm and current data, performance can suffer. This is known as the domain adaptation problem. Bootstrap aggregating, or bagging, is a popular method for improving stability of predictive algorithms, while reducing variance and protecting…

Improved manifold-adaptive dimension estimator for better data complexity assessment.

AdaCAD improves semi-supervised classification by focusing on intra-class nodes.