arXiv research

Orthogonal random features approximate a Bessel kernel, offering sharper bounds than random Fourier features.

Random Fourier features is one of the most popular techniques for scaling up kernel methods, such as kernel ridge regression. However, despite impressive empirical results, the statistical properties of random Fourier features are still not well understood. In this paper we take steps toward filling this gap. Specifica…

This work proves convergence of adaptive resampling for random Fourier features.

Random Fourier features classification achieves fast learning rates with fewer features.

Random Fourier features model reconstructs wind fields from sparse measurements.

Two ANOVA-based algorithms boost random Fourier feature models for function approximation.

Devoted to multi-task learning and structured output learning, operator-valued kernels provide a flexible tool to build vector-valued functions in the context of Reproducing Kernel Hilbert Spaces. To scale up these methods, we extend the celebrated Random Fourier Feature methodology to get an approximation of operator-…

Random Fourier features improve tabular deep learning convergence.

Kernel methods are powerful and flexible approach to solve many problems in machine learning. Due to the pairwise evaluations in kernel methods, the complexity of kernel computation grows as the data size increases; thus the applicability of kernel methods is limited for large scale datasets. Random Fourier Features (R…

We show that the error probability of reconstructing kernel matrices from Random Fourier Features for the Gaussian kernel function is at most $\mathcal{O}(R^{2/3} \exp(-D))$, where $D$ is the number of random features and $R$ is the diameter of the data domain. We also provide an information-theoretic method-independen…

Lecture notes on kernel functions and Random Fourier Features.

This paper surveys various methods for dimensionality reduction and nearest neighbor search.

New quantization methods improve accuracy of Random Fourier Features.

Random Fourier features is a widely used, simple, and effective technique for scaling up kernel methods. The existing theoretical analysis of the approach, however, remains focused on specific learning tasks and typically gives pessimistic bounds which are at odds with the empirical results. We tackle these problems an…

New method uses tensor decompositions to overcome the curse of dimensionality for large-scale learning.

Online change detection algorithm using random Fourier features.

Improved MMD test for two-sample testing with random Fourier features.

Quantum-assisted Gaussian process speeds up data regression.

A new method slices and sums radial kernels faster.

New spectral mixture representation for isotropic kernels simplifies random Fourier features.

Improved Gaussian Process regression using TQFF over RFF and Gaussian QFF.

RFFNet scales kernel methods to large datasets by learning kernel relevance.

New method uses random features and Tikhonov regularization for operator learning from noisy data.

New random feature maps for Laplacian and related kernels.

A novel model uses ODE-based random features to model nonlinear dynamical systems.

Characterizes RFF regression in large $n,p,N$ setting, providing precise learning phases and double descent curve.

Large-scale kernel approximation is an important problem in machine learning research. Approaches using random Fourier features have become increasingly popular [Rahimi and Recht, 2007], where kernel approximation is treated as empirical mean estimation via Monte Carlo (MC) or Quasi-Monte Carlo (QMC) integration [Yang …

This paper develops a bootstrap method to estimate errors in Random Fourier Features.

Accelerates signature kernel computation for sequences.

Random non-linear Fourier features have recently shown remarkable performance in a wide-range of regression and classification applications. Motivated by this success, this article focuses on a sparse non-linear Fourier feature (NFF) model. We provide a characterization of the sufficient number of data points that guar…

This paper develops quantization algorithms for random Fourier features, simplifying the process and improving performance.

Three RFF-based methods for nonlinear causal discovery in mixed data.

A faster method for density estimation using denoising score matching with random Fourier features.

We study the statistical and computational aspects of kernel principal component analysis using random Fourier features and show that under mild assumptions, $O(\sqrt{n} \log n)$ features suffices to achieve $O(1/ε^2)$ sample complexity. Furthermore, we give a memory efficient streaming algorithm based on classical Oja…

End-to-end kernel learning using generative RFFs for improved performance.

Periodicity is often studied in timeseries modelling with autoregressive methods but is less popular in the kernel literature, particularly for higher dimensional problems such as in textures, crystallography, and quantum mechanics. Large datasets often make modelling periodicity untenable for otherwise powerful non-pa…

The paper analyzes and mitigates biases in scalable Gaussian Process methods.

Framework combines random features with CDEs for efficient time-series learning.

In this paper, we propose a fast surrogate leverage weighted sampling strategy to generate refined random Fourier features for kernel approximation. Compared to the current state-of-the-art method that uses the leverage weighted scheme [Li-ICML2019], our new strategy is simpler and more effective. It uses kernel alignm…

New method reduces variance and bias in approximating indefinite kernels.

Positive-definite kernel functions are fundamental elements of kernel methods and Gaussian processes. A well-known construction of such functions comes from Bochner's characterization, which connects a positive-definite function with a probability distribution. Another construction, which appears to have attracted less…

Kernel methods give powerful, flexible, and theoretically grounded approaches to solving many problems in machine learning. The standard approach, however, requires pairwise evaluations of a kernel function, which can lead to scalability issues for very large datasets. Rahimi and Recht (2007) suggested a popular approa…

Bayesian non-linear latent variable modeling for complex data.

New analysis proves sketching operators' RIP guarantees for mixture models without importance sampling.

We propose a principled method for kernel learning, which relies on a Fourier-analytic characterization of translation-invariant or rotation-invariant kernels. Our method produces a sequence of feature maps, iteratively refining the SVM margin. We provide rigorous guarantees for optimality and generalization, interpret…

Unified approach for interpretable regression with flexible modeling.

ARFF reduces spectral bias in SGD-trained neural networks.

New PINN architectures learn high-frequency features using Fourier features.