arXiv research

Orthogonal random features approximate a Bessel kernel, offering sharper bounds than random Fourier features.

New random feature maps for Laplacian and related kernels.

A faster graph kernel using optical random features.

Improved kernel ridge regression for large datasets using weighted random binning.

Improves efficiency of random feature approximations for dot product kernels.

Kernel ridgeless regression with random features shows good generalization without explicit regularization.

Kernel methods are powerful and flexible approach to solve many problems in machine learning. Due to the pairwise evaluations in kernel methods, the complexity of kernel computation grows as the data size increases; thus the applicability of kernel methods is limited for large scale datasets. Random Fourier Features (R…

A new method for approximating softmax and Gaussian kernels with reduced error.

HRFs adaptively linearize kernels for accurate approximations.

Random Fourier features is one of the most popular techniques for scaling up kernel methods, such as kernel ridge regression. However, despite impressive empirical results, the statistical properties of random Fourier features are still not well understood. In this paper we take steps toward filling this gap. Specifica…

New method reduces variance and bias in approximating indefinite kernels.

Devoted to multi-task learning and structured output learning, operator-valued kernels provide a flexible tool to build vector-valued functions in the context of Reproducing Kernel Hilbert Spaces. To scale up these methods, we extend the celebrated Random Fourier Feature methodology to get an approximation of operator-…

We introduce the Mondrian kernel, a fast random feature approximation to the Laplace kernel. It is suitable for both batch and online learning, and admits a fast kernel-width-selection procedure as the random features can be re-used efficiently for all kernel widths. The features are constructed by sampling trees via a…

New method improves Gaussian kernel approximations for high-frequency data.

Random features improve neural operators' generalization properties.

Random feature approximation speeds up spectral methods and improves learning rates.

Hermite polynomials improve private data generation by reducing feature count.

Lecture notes on kernel functions and Random Fourier Features.

Kernel methods form a powerful, versatile, and theoretically-grounded unifying framework to solve nonlinear problems in signal processing and machine learning. The standard approach relies on the kernel trick to perform pairwise evaluations of a kernel function, which leads to scalability issues for large datasets due …

Efficiently applies NTK to large-scale datasets using random features.

New spectral mixture representation for isotropic kernels simplifies random Fourier features.

This paper improves kernel quantile regression with random features for handling heavy-tailed noises.

Positive-definite kernel functions are fundamental elements of kernel methods and Gaussian processes. A well-known construction of such functions comes from Bochner's characterization, which connects a positive-definite function with a probability distribution. Another construction, which appears to have attracted less…

We consider the problem of improving kernel approximation via randomized feature maps. These maps arise as Monte Carlo approximation to integral representations of kernel functions and scale up kernel methods for larger datasets. Based on an efficient numerical integration technique, we propose a unifying approach that…

RFFNet scales kernel methods to large datasets by learning kernel relevance.

RQMC improves kernel-based learning by reducing deterministic error and offering computational advantages.

SRF improves kernel approximation and GP regression performance.

Random features and KRR generalize similarly when N is large enough.

End-to-end kernel learning using generative RFFs for improved performance.

The approximation of nonlinear kernels via linear feature maps has recently gained interest due to their applications in reducing the training and testing time of kernel-based learning algorithms. Current random projection methods avoid the curse of dimensionality by embedding the nonlinear feature space into a low dim…

Kernel approximation using randomized feature maps has recently gained a lot of interest. In this work, we identify that previous approaches for polynomial kernel approximation create maps that are rank deficient, and therefore do not utilize the capacity of the projected feature space effectively. To address this chal…

MetaVRF learns adaptive kernels for fast few-shot learning.

New method approximates complex kernel norms with random features, making learning tractable.

Random features provide a practical framework for large-scale kernel approximation and supervised learning. It has been shown that data-dependent sampling of random features using leverage scores can significantly reduce the number of features required to achieve optimal learning bounds. Leverage scores introduce an op…

We show that the error probability of reconstructing kernel matrices from Random Fourier Features for the Gaussian kernel function is at most $\mathcal{O}(R^{2/3} \exp(-D))$, where $D$ is the number of random features and $R$ is the diameter of the data domain. We also provide an information-theoretic method-independen…

New RFs reduce kernel approximation variance and improve Transformer performance.

Approximating non-linear kernels using feature maps has gained a lot of interest in recent years due to applications in reducing training and testing times of SVM classifiers and other kernel based learning algorithms. We extend this line of work and present low distortion embeddings for dot product kernels into linear…

Study loop corrections in random feature models affecting training and test errors.

Extends random feature analysis to spectral methods and improves learning rates.

We analyze in this paper a random feature map based on a theory of invariance I-theory introduced recently. More specifically, a group invariant signal signature is obtained through cumulative distributions of group transformed random projections. Our analysis bridges invariant feature learning with kernel methods, as …

We present an intriguing discovery related to Random Fourier Features: in Gaussian kernel approximation, replacing the random Gaussian matrix by a properly scaled random orthogonal matrix significantly decreases kernel approximation error. We call this technique Orthogonal Random Features (ORF), and provide theoretical…

Random feature method approximates operators with theoretical guarantees and reduced computation.

Kernel method has been developed as one of the standard approaches for nonlinear learning, which however, does not scale to large data set due to its quadratic complexity in the number of samples. A number of kernel approximation methods have thus been proposed in the recent years, among which the random features metho…

A new method slices and sums radial kernels faster.

Adding node feature kernels improves GCN robustness to graph perturbations.

We propose a new method for input variable selection in nonlinear regression. The method is embedded into a kernel regression machine that can model general nonlinear functions, not being a priori limited to additive models. This is the first kernel-based variable selection method applicable to large datasets. It sides…

Random features enhance control of complex systems.

RFAD uses random features to speed up dataset distillation.