VNNGP uses nearest neighbors to approximate GPs, improving scalability and performance.
problem Scalability issues in Gaussian process approximations.
method Sparse precision structure via nearest neighbors, variational framework.
result VNNGP outperforms low-rank methods and is less prone to overfitting.
New method speeds up k-means clustering for large k by improving nearest-neighbor search.
problem Efficiently clustering large datasets with high-dimensional points.
method Seeded Approximate Nearest-Neighbor Search methods to improve Lloyd's algorithm.
result Significantly faster k-means clustering for large k values.
Python package reduces hubness in high-dimensional data.
problem Hubness in high-dimensional data affects machine learning tasks.
method Hubness analysis, reduction, and approximate neighbor search.
result Improves neighbor retrieval and learning tasks in high dimensions.
Algorithms often carry out equally many computations for "easy" and "hard" problem instances. In particular, algorithms for finding nearest neighbors typically have the same running time regardless of the particular problem instance. In this paper, we consider the approximate k-nearest-neighbor problem, which is the pr…
A new multilabel classification framework improves ANN search performance.
problem Efficiently finding approximate nearest neighbors in large datasets.
method Formulated ANN search as a multilabel classification problem, using partitioning classifiers.
result Natural classifier leads to strictly improved performance in ANN search.
The nearest neighbor problem is defined as follows: Given a set P of n points in some metric space (X,D), build a data structure that, given any point q, returns a point in P that is closest to q (its "nearest neighbor" in P). The data structure stores additional information about the set P, which is th…
Fast approximate nearest neighbor (NN) search in large databases is becoming popular. Several powerful learning-based formulations have been proposed recently. However, not much attention has been paid to a more fundamental question: how difficult is (approximate) nearest neighbor search in a given data set? And which …
Combines fast evaluation with Bayes consistency in nearest neighbors.
problem Balancing fast evaluation time with Bayes consistency in nearest neighbors.
method Combines locality-sensitive hashing (LSH) with a missing-mass argument.
result Fast and Bayes-consistent classifier with comparable risk decay rates.
Multi-Output Dependence (MOD) learning is a generalization of standard classification problems that allows for multiple outputs that are dependent on each other. A primary issue that arises in the context of MOD learning is that for any given input pattern there can be multiple correct output patterns. This changes the…
Recently manifold learning algorithm for dimensionality reduction attracts more and more interests, and various linear and nonlinear, global and local algorithms are proposed. The key step of manifold learning algorithm is the neighboring region selection. However, so far for the references we know, few of which propos…
Java implementation improves nearest neighbor algorithm complexity.
problem Improving efficiency of nearest neighbor descent algorithm.
method Parallel streams implementation with statistical termination criterion.
result Complexity up to O(nK2logK(n)) for K-nearest neighbors. Approximate nearest neighbor algorithms are used to speed up nearest neighbor search in a wide array of applications. However, current indexing methods feature several hyperparameters that need to be tuned to reach an acceptable accuracy--speed trade-off. A grid search in the parameter space is often impractically slow…
This paper surveys various methods for dimensionality reduction and nearest neighbor search.
problem Efficiently reducing high-dimensional data to lower dimensions while preserving essential information.
method Linear and nonlinear random projections, including sparse random projections, random Fourier Features, and Random Kitchen Sinks.
result Various methods for dimensionality reduction and nearest neighbor search are explained and compared.
GraphGP: Scalable Gaussian Processes with Vecchia's Approximation
problem Naive Gaussian Process computation limits practical use
method GPU algorithm for Vecchia's approximation
result Linear time and memory requirements for nearly a billion parameters
The paper explains how nearest neighbor methods succeed in prediction.
problem Explaining the success of nearest neighbor methods in prediction.
method The paper covers both theoretical and practical aspects of nearest neighbor methods, including statistical guarantees and practical algorithms.
result The paper provides nonasymptotic statistical guarantees and practical algorithms for nearest neighbor methods.
We derive Gaussian approximations for random forest predictions using region-based stabilization.
problem Improving the accuracy of random forest predictions for Poisson process data.
method Region-based stabilization and Malliavin-Stein method for multivariate Gaussian approximation.
result Established Gaussian approximation bounds for random forest predictions under Poisson process.
Nearest Neighbors Algorithm is a Lazy Learning Algorithm, in which the algorithm tries to approximate the predictions with the help of similar existing vectors in the training dataset. The predictions made by the K-Nearest Neighbors algorithm is based on averaging the target values of the spatial neighbors. The selecti…
New methods explain NE embeddings by identifying key variables.
problem Lack of interpretability in NE techniques.
method Combining PCA, Q-residuals, Hotelling's T2, and visualization.
result Identifies discriminatory features not seen in standard approaches.
The method of random projections has become a standard tool for machine learning, data mining, and search with massive data at Web scale. The effective use of random projections requires efficient coding schemes for quantizing (real-valued) projected data into integers. In this paper, we focus on a simple 2-bit coding …
Develops LSH schemes for f-divergences and mutual information loss.
problem Approximating nearest neighbors in high-dimensional probability distributions.
method General framework and specific LSH schemes for f-divergences and mutual information loss.
result Generalized Jensen-Shannon divergence can be approximated by Hellinger distance.
Improved PCA accuracy using maximum entropy method.
problem Accuracy issues in classical PCA.
method Model uncertainty with random variables, apply Maximum Entropy Method.
result Improved estimates of distances between data items.
This paper calculates data valuation for nearest neighbor models efficiently.
problem Distributing payment for training ML models based on data contributions.
method Defined 'relative value of data' via Shapley value for fairness and decentralizability. Developed algorithms for exact and approximate computation of Shapley values for nearest neighbor models.
result Exact computation of Shapley values for nearest neighbor models in O(N log N) time, significantly faster than previous methods.
Proposes robust ABC method for outlier detection.
problem Outliers sensitivity in ABC methods.
method γ-divergence estimator with redescending property.
result Significantly higher robustness than existing methods.
New method constructs graphs from data efficiently, suitable for large datasets.
problem Memory and runtime limitations of traditional TMFG for large datasets.
method Uses k-Nearest Neighbors Graphs and memory management for scalable graph construction.
result Provides a parsimonious way to construct graphs for learning tasks.
Sparse GPs improved with nearest neighbor inducing variables.
problem Sparse GPs struggle with large numbers of inducing variables.
method Introduced a hierarchical prior for inducing variables and used nearest neighbor information for sparsity.
result Significant computational gains compared to standard sparse GPs.
ICQ improves high-dimensional similarity search without sacrificing precision.
problem High-dimensional similarity search is computationally expensive.
method Interleaved Composite Quantization (ICQ) reduces code length and quantization error.
result ICQ achieves fast similarity search without using shorter codes.
Efficient WKNN-Shapley computation improves data valuation accuracy.
problem Efficient computation of Data Shapley for WKNN algorithm.
method Reframed WKNN-Shapley as a counting problem, introduced quadratic-time algorithm.
result Quadratic-time WKNN-Shapley computation, improving from O(NK). The paper investigates learning conditional distributions on multi-dimensional spaces using clustering and neural networks.
problem Learning conditional distributions on multi-dimensional spaces with varying dimensions.
method The approach involves clustering data near varying query points in the feature space to create empirical measures in the target space using two clustering schemes: fixed-radius ball and nearest neighbors. The convergence rates of both methods are analyzed, and the nearest neighbors method is incorporated into neural network training.
result The empirical analysis shows that the nearest neighbors method has better performance in practice and can adapt to a suitable level of Lipschitz continuity locally.
Fair near neighbor search ensures equal probability of reporting points in a query's radius.
problem Ensuring equal probability of reporting points within a query's radius.
method LSH-based algorithms modified to ensure uniform probability of reporting points.
result Algorithm reports points in the query's radius with almost uniform probability.
PPC learns binary codes from data similarities and dissimilarities.
problem Creating efficient binary codes from data similarities and dissimilarities.
method PPC learns binary codes by modeling attractive and repulsive forces in a signed graph.
result PPC achieves superior results in nearest-neighbor searches compared to spectral methods.
New sublinear sketches improve ANN and KDE for massive data streams.
problem Efficiently approximate nearest neighbors and kernel density estimation in large datasets.
method Developed sublinear space and query time algorithms for ANN and A-KDE in streaming and sliding-window models.
result Achieved near-optimal trade-offs between memory size and approximation error for ANN.
Neighbor Mixture Model captures node correlations in graphs.
problem Modeling correlations between node labels in graphs.
method Neighbor Mixture Model (NMM) designed for efficient computation and scalability.
result NMM outperforms state-of-the-art models in various graph tasks.
Exemplar VAEs link generative models with nearest neighbor retrieval and data augmentation.
problem Improving generative model performance and data augmentation effectiveness.
method Exemplar VAEs with Parzen window prior, retrieval augmented training, exemplar leave-one-out, and subsampling.
result Generative data augmentation reduces classification error on MNIST and Fashion MNIST.
Acoustic Neighbor Embeddings map speech and text to fixed dimensions for phonetic confusability.
problem Mapping speech and text to fixed dimensions for phonetic confusability.
method Adapting SNE to sequential inputs, training two encoder neural networks.
result More accurate results with low-dimensional embeddings in word recognition tasks.
The Earth Mover's Distance (EMD) is a state-of-the art metric for comparing discrete probability distributions, but its high distinguishability comes at a high cost in computational complexity. Even though linear-complexity approximation algorithms have been proposed to improve its scalability, these algorithms are eit…
Generative models use kernel smoothing for conditioning on small example sets.
problem Improving generative models' performance with limited conditioning examples.
method Showed that cross-attention conditioning is equivalent to kernel smoothing, specifically a Nadaraya--Watson kernel smoother.
result The approach predicts and confirms three failure regimes for kernel-based conditioning.
Paper tackles NNS under uncertainty with improved algorithms.
problem Efficient nearest neighbor search with noisy distance estimates.
method Combines cover trees and multi-armed bandits for optimal performance.
result Optimal dependence on dataset size and unknown geometry achieved.
For classifying time series, a nearest-neighbor approach is widely used in practice with performance often competitive with or better than more elaborate methods such as neural networks, decision trees, and support vector machines. We develop theoretical justification for the effectiveness of nearest-neighbor-like clas…
Algorithm finds adversarial examples for k-NN classifiers using Voronoi diagrams.
problem Ensuring robustness of k-NN classifiers against adversarial attacks.
method Geometric approach expanding outwards from input points to find minimum-norm adversarial examples.
result Our method outperforms existing approaches on various datasets.
Prior distributions of binarized natural images are learned by using a Boltzmann machine. According the results of this study, there emerges a structure with two sublattices in the interactions, and the nearest-neighbor and next-nearest-neighbor interactions correspondingly take two discriminative values, which reflect…
Scalable Gaussian Process Operator tackles high-dimensional PDEs.
problem Scaling Gaussian Process Operators to high-dimensional, data-intensive regimes.
method Nearest-neighbor-based local kernel approximations, sparse kernel approximation, structured Kronecker factorizations, operator-aware kernel structures, task-informed mean functions.
result Consistently achieves high accuracy across varying discretization scales.
p-SNE embeds Poisson count data into low dimensions preserving structure.
problem Embedding high-dimensional sparse Poisson data into a low-dimensional space.
method p-SNE (Poisson Stochastic Neighbor Embedding) using KL divergence and Hellinger distance.
result p-SNE recovers meaningful structure in real-world count datasets.
We propose a fully distributed actor-critic algorithm approximated by deep neural networks, named \textit{Diff-DAC}, with application to single-task and to average multitask reinforcement learning (MRL). Each agent has access to data from its local task only, but it aims to learn a policy that performs well on average …
We propose a hierarchy for approximate inference based on the Dobrushin, Lanford, Ruelle (DLR) equations. This hierarchy includes existing algorithms, such as belief propagation, and also motivates novel algorithms such as factorized neighbors (FN) algorithms and variants of mean field (MF) algorithms. In particular, w…
In this paper we target the class of modal clustering methods where clusters are defined in terms of the local modes of the probability density function which generates the data. The most well-known modal clustering method is the k-means clustering. Mean Shift clustering is a generalization of the k-means clustering wh…
Paper proposes an efficient algorithm to compute minimum adversarial perturbation for NN classifiers.
problem Computing the minimum adversarial perturbation for Nearest Neighbor classifiers.
method Formulated as a list of convex quadratic programming problems, solved using efficient algorithms.
result Shows dual solutions as valid lower bounds for adversarial perturbation, aiding robustness verification.
CGMMD generates conditional samples in one shot with MMD and nearest neighbors.
problem Generating samples from a conditional distribution unseen in training.
method CGMMD uses MMD for a direct minimization problem, producing samples in one forward pass.
result CGMMD achieves convergence to the true conditional distribution with theoretical bounds.
LNPE enhances local connections in embeddings using extended neighbor propagation.
problem Improving local connections and interactions in nonlinear dimensionality reduction.
method Inspired by GCN, LNPE extends 1-hop neighbors to n-hop neighbors in LLE.
result LNPE produces more faithful and robust embeddings with better topological and geometrical properties.