arXiv research

Fair near neighbor search ensures equal probability of reporting points in a query's radius.

A clustering algorithm for natural hierarchical clusters with near-linear time complexity.

The method of random projections has become a standard tool for machine learning, data mining, and search with massive data at Web scale. The effective use of random projections requires efficient coding schemes for quantizing (real-valued) projected data into integers. In this paper, we focus on a simple 2-bit coding …

Study nearest-neighbor radii under dependent sampling, finding they remain informative.

If we pick $n$ random points uniformly in $[0,1]^d$ and connect each point to its $k-$nearest neighbors, then it is well known that there exists a giant connected component with high probability. We prove that in $[0,1]^d$ it suffices to connect every point to $c_{d,1} \log{\log{n}}$ points chosen randomly among its $…

We present the first sublinear memory sketch that can be queried to find the nearest neighbors in a dataset. Our online sketching algorithm compresses an N element dataset to a sketch of size $O(N^b \log^3 N)$ in $O(N^{(b+1)} \log^3 N)$ time, where $b < 1$. This sketch can correctly report the nearest neighbors of any …

Many modern machine learning models are trained to achieve zero or near-zero training error in order to obtain near-optimal (but non-zero) test error. This phenomenon of strong generalization performance for "overfitted" / interpolated classifiers appears to be ubiquitous in high-dimensional data, having been observed …

The paper explains how nearest neighbor methods succeed in prediction.

We analyze the Kozachenko--Leonenko (KL) nearest neighbor estimator for the differential entropy. We obtain the first uniform upper bound on its performance over Hölder balls on a torus without assuming any conditions on how close the density could be from zero. Accompanying a new minimax lower bound over the Hölder ba…

Improved nearest neighbors for missing data in latent factor models.

The paper investigates learning conditional distributions on multi-dimensional spaces using clustering and neural networks.

Imbalanced classification has been a major challenge for machine learning because many standard classifiers mainly focus on balanced datasets and tend to have biased results towards the majority class. We modify entropy fuzzy support vector machine (EFSVM) and introduce instance-based entropy fuzzy support vector machi…

ProbMinHash improves Jaccard similarity hashing for big data applications.

Many important applications, including signal reconstruction, parameter estimation, and signal processing in a compressed domain, rely on a low-dimensional representation of the dataset that preserves {\em all} pairwise distances between the data points and leverages the inherent geometric structure that is typically p…

DSNE visualizes data velocity in lower dimensions.

C-MinHash reduces the number of permutations needed for MinHash from thousands to just two.

SGD transitions between maxima and minima with varying time scales.

In this paper we study leveraging confidence information induced by adversarial training to reinforce adversarial robustness of a given adversarially trained model. A natural measure of confidence is $\|F({\bf x})\|_\infty$ (i.e. how confident $F$ is about its prediction?). We start by analyzing an adversarial training…

TPA-AD detects axle-box bearing anomalies using pseudo anomalies near normal boundaries.

BaNk-UCB tackles batched nonparametric bandits with k-NN regression and UCB.

New games model strategic interactions in incomplete information settings.

This work demonstrates the potential of deep reinforcement learning techniques for transmit power control in wireless networks. Existing techniques typically find near-optimal power allocations by solving a challenging optimization problem. Most of these algorithms are not scalable to large networks in real-world scena…

Study efficient graph optimization with noisy data.

LNPE enhances local connections in embeddings using extended neighbor propagation.

Recently it was shown that the problem of Maximum Inner Product Search (MIPS) is efficient and it admits provably sub-linear hashing algorithms. Asymmetric transformations before hashing were the key in solving MIPS which was otherwise hard. In the prior work, the authors use asymmetric transformations which convert th…

The detection of anomalous activity in graphs is a statistical problem that arises in many applications, such as network surveillance, disease outbreak detection, and activity monitoring in social networks. Beyond its wide applicability, graph structured anomaly detection serves as a case study in the difficulty of bal…

High-dimensional inference for sparse spectral precision matrices

In this paper, we consider the problem of classification of $M$ high dimensional queries $y^1,\cdots,y^M\in B^S$ to $N$ high dimensional classes $x^1,\cdots,x^N\in A^S$ where $A$ and $B$ are discrete alphabets and the probabilistic model that relates data to the classes $P(x,y)$ is known. This problem has applications …

Deep nearest neighbors outperform self-supervised methods in anomaly detection.

In this paper we investigate the problem of estimating the cluster tree for a density $f$ supported on or near a smooth $d$-dimensional manifold $M$ isometrically embedded in $\mathbb{R}^D$. We analyze a modified version of a $k$-nearest neighbor based algorithm recently proposed by Chaudhuri and Dasgupta. The main res…

Characterizes Lebesgue points using nearest neighbor methods.

A deterministic system of interacting agents is considered as a model for economic dynamics. The dynamics of the system is described by a coupled map lattice with near neighbor interactions. The evolution of each agent results from the competition between two factors: the agent's own tendency to grow and the environmen…

Plan2Vec learns image representations without labels, improving control tasks.

AWNN improves matrix completion by adaptively weighting nearest neighbors.

New sublinear sketches improve ANN and KDE for massive data streams.

A new method uses nearest neighbors for importance weighting.

An unsupervised learning algorithm to cluster hyperspectral image (HSI) data is proposed that exploits spatially-regularized random walks. Markov diffusions are defined on the space of HSI spectra with transitions constrained to near spatial neighbors. The explicit incorporation of spatial regularity into the diffusion…

This paper compares FAISS and FENSHSES for nearest neighbor search in Hamming space.

Nearest neighbor methods are a popular class of nonparametric estimators with several desirable properties, such as adaptivity to different distance scales in different regions of space. Prior work on convergence rates for nearest neighbor classification has not fully reflected these subtle properties. We analyze the b…

Adversarial examples are a pervasive phenomenon of machine learning models where seemingly imperceptible perturbations to the input lead to misclassifications for otherwise statistically accurate models. We propose a geometric framework, drawing on tools from the manifold reconstruction literature, to analyze the high-…

BigNN classifier improves nearest neighbor classification for large datasets.

We propose a simple kernel based nearest neighbor approach for handwritten digit classification. The "distance" here is actually a kernel defining the similarity between two images. We carefully study the effects of different number of neighbors and weight schemes and report the results. With only a few nearest neighbo…

New methods use vector search and nearest-neighbor matching for policy learning in causal inference.

VNNGP uses nearest neighbors to approximate GPs, improving scalability and performance.

The estimation of optimal treatment regimes is of considerable interest to precision medicine. In this work, we propose a causal $k$-nearest neighbor method to estimate the optimal treatment regime. The method roots in the framework of causal inference, and estimates the causal treatment effects within the nearest neig…

Python package reduces hubness in high-dimensional data.

BGNN improves GNN by modeling interactions between neighbor nodes.

MINTS uses a minimalist Bayesian framework to tackle multi-armed bandits with structural constraints.