arXiv research

This work explores variably scaled kernels to improve non-stationary Gaussian processes.

Large scale online kernel learning aims to build an efficient and scalable kernel-based predictive model incrementally from a sequence of potentially infinite data points. A current key approach focuses on ways to produce an approximate finite-dimensional feature map, assuming that the kernel used has a feature map wit…

New kernel model scales to large datasets.

This work bridges two views of feature learning in neural networks.

Model shows feature learning can improve neural scaling laws for hard tasks.

RFFNet scales kernel methods to large datasets by learning kernel relevance.

Researchers parallelize neural kernels for large-scale data, achieving state-of-the-art accuracy.

Efficiently scales continuous kernels with sparse Fourier domain learning.

The paper analyzes learning curves for kernel ridge regression with dot-product kernels.

A new, fast kernel test for large data.

Kernel methods are powerful and flexible approach to solve many problems in machine learning. Due to the pairwise evaluations in kernel methods, the complexity of kernel computation grows as the data size increases; thus the applicability of kernel methods is limited for large scale datasets. Random Fourier Features (R…

Efficiently applies NTK to large-scale datasets using random features.

Derives a family of hyperparameter scaling strategies for neural networks.

Unified quadrature framework for large-scale kernel machines.

New method speeds up training of large kernel models.

This paper tackles hyperparameter tuning for large-scale kernel ridge regression.

Study on learning properties of scale-dependent kernels controlling stability and error.

Kernel clustering algorithm improved for large datasets using incomplete Cholesky factorization.

Kernel improves Gaussian process scalability for wide datasets.

Improves kernel ridge regression by optimizing scale and feature parameters.

Many real world graphs, such as the graphs of molecules, exhibit structure at multiple different scales, but most existing kernels between graphs are either purely local or purely global in character. In contrast, by building a hierarchy of nested subgraphs, the Multiscale Laplacian Graph kernels (MLG kernels) that we …

Scalable kernel methods for large datasets using Fourier representations and NUFFT.

This work extends the scaling law to multiple and kernel regression, challenging traditional machine learning principles.

Enhances Fourier estimator performance for asynchronous event-data.

The computational complexity of kernel methods has often been a major barrier for applying them to large-scale learning problems. We argue that this barrier can be effectively overcome. In particular, we develop methods to scale up kernel models to successfully tackle large-scale learning problems that are so far only …

Gaussian process regression generally does not scale to beyond a few thousands data points without applying some sort of kernel approximation method. Most approximations focus on the high eigenvalue part of the spectrum of the kernel matrix, $K$, which leads to bad performance when the length scale of the kernel is sma…

While state-of-the-art kernels for graphs with discrete labels scale well to graphs with thousands of nodes, the few existing kernels for graphs with continuous attributes, unfortunately, do not scale well. To overcome this limitation, we present hash graph kernels, a general framework to derive kernels for graphs with…

Study optimizes KSD estimation from samples, revealing Hilbert-Schmidt vs trace scales.

New method speeds up uncertainty estimation for large datasets in causal inference.

The paper improves boundary detection and density estimation on noisy data.

Wide neural networks become linear, with constant tangent kernel, due to Hessian scaling.

Study finds the number of modes in Gaussian kernel density estimators scales with sqrt(β log β).

We propose a new method for input variable selection in nonlinear regression. The method is embedded into a kernel regression machine that can model general nonlinear functions, not being a priori limited to additive models. This is the first kernel-based variable selection method applicable to large datasets. It sides…

The paper studies heat kernel asymptotics and proves Morse inequalities.

Improved kernel ridge regression for large datasets using weighted random binning.

ParK efficiently solves kernel ridge regression for large datasets.

Novel Newton method for large-scale kernel methods using random features.

Topological data analysis offers a rich source of valuable information to study vision problems. Yet, so far we lack a theoretically sound connection to popular kernel-based learning techniques, such as kernel SVMs or kernel PCA. In this work, we establish such a connection by designing a multi-scale kernel for persist…

Kernel methods are ubiquitous tools in machine learning. However, there is often little reason for the common practice of selecting a kernel a priori. Even if a universal approximating kernel is selected, the quality of the finite sample estimator may be greatly affected by the choice of kernel. Furthermore, when direc…

Study reveals an equivalence principle for the spectrum of random inner-product kernel matrices in polynomial scaling.

DSoftKI scales GP regression with full derivative observations.

Improved kernel Stein discrepancy for large-scale data.

Neural networks can learn kernel machines with a data-dependent kernel.

We analyzed optimism in linear and kernel regression models.

This paper reviews MDS, Sammon mapping, and Isomap, explaining their theory and applications.

Representations of probability measures in reproducing kernel Hilbert spaces provide a flexible framework for fully nonparametric hypothesis tests of independence, which can capture any type of departure from independence, including nonlinear associations and multivariate interactions. However, these approaches come wi…

We are interested in a framework of online learning with kernels for low-dimensional but large-scale and potentially adversarial datasets. We study the computational and theoretical performance of online variations of kernel Ridge regression. Despite its simplicity, the algorithm we study is the first to achieve the op…

String kernels are attractive data analysis tools for analyzing string data. Among them, alignment kernels are known for their high prediction accuracies in string classifications when tested in combination with SVM in various applications. However, alignment kernels have a crucial drawback in that they scale poorly du…