arXiv research

The Nyström method improves learning efficiency for convex losses.

Nyström KPCA balances computational efficiency and statistical accuracy.

The Nystrom method has been popular for generating the low-rank approximation of kernel matrices that arise in many machine learning problems. The approximation quality of the Nystrom method depends crucially on the number of selected landmark points and the selection procedure. In this paper, we present a novel algori…

Paper uses Koopman operator and Nyström method for efficient nonlinear control.

Paper proposes Nyström sketches for better adaptive compressive learning.

New methods improve efficiency of sampling algorithms for complex systems.

We give the first algorithm for kernel Nyström approximation that runs in *linear time in the number of training points* and is provably accurate for all kernel matrices, without dependence on regularity or incoherence conditions. The algorithm projects the kernel onto a set of $s$ landmark points sampled by their *rid…

Efficiently approximates kernel mean embeddings using Nyström method.

We demonstrate that distributed block coordinate descent can quickly solve kernel regression and classification problems with millions of data points. Armed with this capability, we conduct a thorough comparison between the full kernel, the Nyström method, and random features on three large classification tasks from va…

We study Nyström type subsampling approaches to large scale kernel methods, and prove learning bounds in the statistical learning setting, where random sampling and high probability estimates are considered. In particular, we prove that these approaches can achieve optimal learning bounds, provided the subsampling leve…

Boosting Nyström improves accuracy of matrix approximations.

Stochastic gradient descent optimizes Nyström samples for kernel matrix approximation.

New method speeds up HSIC for multiple variables.

Simulating the time-evolution of quantum mechanical systems is BQP-hard and expected to be one of the foremost applications of quantum computers. We consider classical algorithms for the approximation of Hamiltonian dynamics using subsampling methods from randomized numerical linear algebra. We derive a simulation tech…

Selecting diverse and important items, called landmarks, from a large set is a problem of interest in machine learning. As a specific example, in order to deal with large training sets, kernel methods often rely on low rank matrix Nyström approximations based on the selection or sampling of landmarks. In this context, …

Paper analyzes Nyström regularization for time series forecasting with sequential sub-sampling.

Derives kernel PCA with Nyström method for scalability.

Improved Nyström approximation for kernel quadrature with theoretical guarantees.

Neumann eigenmaps improve landmark-based diffusion map embeddings.

Efficiently approximates statistical leverage scores for faster KRR.

The GMM (generalized min-max) kernel was recently proposed (Li, 2016) as a measure of data similarity and was demonstrated effective in machine learning tasks. In order to use the GMM kernel for large-scale datasets, the prior work resorted to the (generalized) consistent weighted sampling (GCWS) to convert the GMM ker…

This paper proposes a new Nystrom-based clustering algorithm for large-scale data.

We investigate regularized algorithms combining with projection for least-squares regression problem over a Hilbert space, covering nonparametric regression over a reproducing kernel Hilbert space. We prove convergence results with respect to variants of norms, under a capacity assumption on the hypothesis space and a …

This paper improves Koopman operator approximations by pruning subspaces in RKHS.

Stochastic trace estimation with tensor train random vectors

Kernel methods offer the flexibility to learn complex relationships in modern, large data sets while enjoying strong theoretical guarantees on quality. Unfortunately, these methods typically require cubic running time in the data set size, a prohibitive cost in the large-data setting. Random feature maps (RFMs) and the…

The Nystrom method is a popular technique that uses a small number of landmark points to compute a fixed-rank approximation of large kernel matrices that arise in machine learning problems. In practice, to ensure high quality approximations, the number of landmark points is chosen to be greater than the target rank. Ho…

Kernel $k$-means clustering can correctly identify and extract a far more varied collection of cluster structures than the linear $k$-means clustering algorithm. However, kernel $k$-means clustering is computationally expensive when the non-linear feature map is high-dimensional and there are many input points. Kernel …

We propose and study kernel conjugate gradient methods (KCGM) with random projections for least-squares regression over a separable Hilbert space. Considering two types of random projections generated by randomized sketches and Nyström subsampling, we prove optimal statistical results with respect to variants of norms …

Clarifies connections between Nyström and SVGP methods for scalable GPs.

Nyström subsampling with Tikhonov regularization for covariate shift adaptation under misspecified case

Method uses NMF for clustering with partial distance measurements.

This paper tackles scalability issues in kernel logistic regression for large datasets.

The Nyström methods have been popular techniques for scalable kernel based learning. They approximate explicit, low-dimensional feature mappings for kernel functions from the pairwise comparisons with the training data. However, Nyström methods are generally applied without the supervision provided by the training labe…

In this paper, we study random subsampling of Gaussian process regression, one of the simplest approximation baselines, from a theoretical perspective. Although subsampling discards a large part of training data, we show provable guarantees on the accuracy of the predictive mean/variance and its generalization ability.…

Paper proposes diagnostics for error and variance estimation in randomized matrix computations.

We consider the binary classification problem when data are large and subject to unknown but bounded uncertainties. We address the problem by formulating the nonlinear support vector machine training problem with robust optimization. To do so, we analyze and propose two bounding schemes for uncertainties associated to …

We accelerate the power method for strong low-rank approximation using fast sketching.

We develop an improved bound for the approximation error of the Nyström method under the assumption that there is a large eigengap in the spectrum of kernel matrix. This is based on the empirical observation that the eigengap has a significant impact on the approximation error of the Nyström method. Our approach is bas…

This paper improves spectral clustering for large datasets using the Nystrom method.

Nyström approximation for scalable operator learning

Nystrom approximation speeds up kernel model training.

The Nystrom method is an efficient technique used to speed up large-scale learning applications by generating low-rank approximations. Crucial to the performance of this technique is the assumption that a matrix can be well approximated by working exclusively with a subset of its columns. In this work we relate this as…

Diverse sampling improves kernel methods' performance in sparse regions.

The abstract presents a new theorem using Ross-Witt Nyström correspondence and Berndtsson's theorem.

AIRBO optimizes robustly under uncertain inputs.

Kernel-based K-means clustering has gained popularity due to its simplicity and the power of its implicit non-linear representation of the data. A dominant concern is the memory requirement since memory scales as the square of the number of data points. We provide a new analysis of a class of approximate kernel methods…

Two algorithms estimate Wasserstein distance matrices from few entries for manifold learning.