arXiv research

Boosting Nyström improves accuracy of matrix approximations.

Stochastic gradient descent optimizes Nyström samples for kernel matrix approximation.

Improved Nyström approximation for kernel quadrature with theoretical guarantees.

Nyström KPCA balances computational efficiency and statistical accuracy.

This paper proposes a new Nystrom-based clustering algorithm for large-scale data.

Kernel $k$-means clustering can correctly identify and extract a far more varied collection of cluster structures than the linear $k$-means clustering algorithm. However, kernel $k$-means clustering is computationally expensive when the non-linear feature map is high-dimensional and there are many input points. Kernel …

Clarifies connections between Nyström and SVGP methods for scalable GPs.

Method uses NMF for clustering with partial distance measurements.

The Nystrom method is a popular technique that uses a small number of landmark points to compute a fixed-rank approximation of large kernel matrices that arise in machine learning problems. In practice, to ensure high quality approximations, the number of landmark points is chosen to be greater than the target rank. Ho…

Nystrom approximation speeds up kernel model training.

We give the first algorithm for kernel Nyström approximation that runs in *linear time in the number of training points* and is provably accurate for all kernel matrices, without dependence on regularity or incoherence conditions. The algorithm projects the kernel onto a set of $s$ landmark points sampled by their *rid…

Efficiently approximates kernel mean embeddings using Nyström method.

Paper uses Koopman operator and Nyström method for efficient nonlinear control.

We develop an improved bound for the approximation error of the Nyström method under the assumption that there is a large eigengap in the spectrum of kernel matrix. This is based on the empirical observation that the eigengap has a significant impact on the approximation error of the Nyström method. Our approach is bas…

This paper improves spectral clustering for large datasets using the Nystrom method.

This paper tackles scalability issues in kernel logistic regression for large datasets.

Paper proposes Nyström sketches for better adaptive compressive learning.

The Nystrom method has been popular for generating the low-rank approximation of kernel matrices that arise in many machine learning problems. The approximation quality of the Nystrom method depends crucially on the number of selected landmark points and the selection procedure. In this paper, we present a novel algori…

The Nyström methods have been popular techniques for scalable kernel based learning. They approximate explicit, low-dimensional feature mappings for kernel functions from the pairwise comparisons with the training data. However, Nyström methods are generally applied without the supervision provided by the training labe…

Paper improves clustering risk bounds for kernel k-means.

SQUEAK reduces space complexity for Nystrom approximations in KRR.

Incremental versions of batch algorithms are often desired, for increased time efficiency in the streaming data setting, or increased memory efficiency in general. In this paper we present a novel algorithm for incremental kernel PCA, based on rank one updates to the eigendecomposition of the kernel matrix, which is mo…

The Nystrom method is an efficient technique used to speed up large-scale learning applications by generating low-rank approximations. Crucial to the performance of this technique is the assumption that a matrix can be well approximated by working exclusively with a subset of its columns. In this work we relate this as…

Efficiently tests two distributions using Nyström approximation of MMD.

Nyström approximation for scalable operator learning

This paper describes a new method for low rank kernel approximation called IKA. The main advantage of IKA is that it produces a function $ψ(x)$ defined as a linear combination of arbitrarily chosen functions. In contrast the approximation produced by Nyström method is a linear combination of kernel evaluations. The pro…

Kernel methods have achieved very good performance on large scale regression and classification problems, by using the Nyström method and preconditioning techniques. The Nyström approximation -- based on a subset of landmarks -- gives a low rank approximation of the kernel matrix, and is known to provide a form of impl…

The Nyström method improves learning efficiency for convex losses.

Proposes a new method for optimizing large-scale models using Nyström approximation of the Hessian.

Nyström subsampling with Tikhonov regularization for covariate shift adaptation under misspecified case

Efficiently approximates statistical leverage scores for faster KRR.

Skyformer uses Gaussian kernel and Nyström method to speed up self-attention in transformers.

Unified framework interprets SSL models, revealing biases.

Recently, Nyström method has proved its prominence empirically and theoretically in speeding up the training of kernel machines while retaining satisfactory performances and accuracy. So far, there are several different approaches proposed to exploit Nyström method in scaling up kernel machines. However, there is no co…

New methods improve efficiency of sampling algorithms for complex systems.

Paper presents a deterministic method for diverse subset selection.

Two log-linear approximations speed up optimal transport for deep learning applications.

The Column Subset Selection Problem (CSSP) and the Nyström method are among the leading tools for constructing small low-rank approximations of large datasets in machine learning and scientific computing. A fundamental question in this area is: how well can a data subset of size k compete with the best rank k approxima…

Paper improves kernel approximations for better statistical learning.

We accelerate the power method for strong low-rank approximation using fast sketching.

Simulating the time-evolution of quantum mechanical systems is BQP-hard and expected to be one of the foremost applications of quantum computers. We consider classical algorithms for the approximation of Hamiltonian dynamics using subsampling methods from randomized numerical linear algebra. We derive a simulation tech…

New method speeds up kernel-based machine learning for force field reconstruction.

Diffusion maps are an emerging data-driven technique for non-linear dimensionality reduction, which are especially useful for the analysis of coherent structures and nonlinear embeddings of dynamical systems. However, the computational complexity of the diffusion maps algorithm scales with the number of observations. T…

Improved kernel Stein discrepancy for large-scale data.

The GMM (generalized min-max) kernel was recently proposed (Li, 2016) as a measure of data similarity and was demonstrated effective in machine learning tasks. In order to use the GMM kernel for large-scale datasets, the prior work resorted to the (generalized) consistent weighted sampling (GCWS) to convert the GMM ker…

This work connects BNNs to GPs, providing scalable inference and identifying key properties.

The Sinkhorn "distance", a variant of the Wasserstein distance with entropic regularization, is an increasingly popular tool in machine learning and statistical inference. However, the time and memory requirements of standard algorithms for computing this distance grow quadratically with the size of the data, making th…

Paper bounds the minimal rank for kernel ridge regression approximations.