A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Orthogonal random features approximate a Bessel kernel, offering sharper bounds than random Fourier features.
Random Fourier features is one of the most popular techniques for scaling up kernel methods, such as kernel ridge regression. However, despite impressive empirical results, the statistical properties of random Fourier features are still not well understood. In this paper we take steps toward filling this gap. Specifica…
Two ANOVA-based algorithms boost random Fourier feature models for function approximation.
Devoted to multi-task learning and structured output learning, operator-valued kernels provide a flexible tool to build vector-valued functions in the context of Reproducing Kernel Hilbert Spaces. To scale up these methods, we extend the celebrated Random Fourier Feature methodology to get an approximation of operator-…
New quantization methods improve accuracy of Random Fourier Features.
This work proves convergence of adaptive resampling for random Fourier features.
This paper surveys various methods for dimensionality reduction and nearest neighbor search.
Lecture notes on kernel functions and Random Fourier Features.
Kernel methods are powerful and flexible approach to solve many problems in machine learning. Due to the pairwise evaluations in kernel methods, the complexity of kernel computation grows as the data size increases; thus the applicability of kernel methods is limited for large scale datasets. Random Fourier Features (R…
Random Fourier features model reconstructs wind fields from sparse measurements.
Improved Gaussian Process regression using TQFF over RFF and Gaussian QFF.
New method reduces variance and bias in approximating indefinite kernels.
This paper develops a bootstrap method to estimate errors in Random Fourier Features.
A novel model uses ODE-based random features to model nonlinear dynamical systems.
Random Fourier features classification achieves fast learning rates with fewer features.
Quantum-assisted Gaussian process speeds up data regression.
Efficiently approximates eigenspaces for symmetric and general matrices.
Large-scale kernel approximation is an important problem in machine learning research. Approaches using random Fourier features have become increasingly popular [Rahimi and Recht, 2007], where kernel approximation is treated as empirical mean estimation via Monte Carlo (MC) or Quasi-Monte Carlo (QMC) integration [Yang …
We investigate how to train kernel approximation methods that generalize well under a memory budget. Building on recent theoretical work, we define a measure of kernel approximation error which we find to be more predictive of the empirical generalization performance of kernel approximation methods than conventional me…
Improved MMD test for two-sample testing with random Fourier features.
New algorithm trains deep neural networks without global optimization.
This paper develops quantization algorithms for random Fourier features, simplifying the process and improving performance.
RFFNet scales kernel methods to large datasets by learning kernel relevance.
New random feature maps for Laplacian and related kernels.
Periodicity is often studied in timeseries modelling with autoregressive methods but is less popular in the kernel literature, particularly for higher dimensional problems such as in textures, crystallography, and quantum mechanics. Large datasets often make modelling periodicity untenable for otherwise powerful non-pa…
Random Fourier features is a widely used, simple, and effective technique for scaling up kernel methods. The existing theoretical analysis of the approach, however, remains focused on specific learning tasks and typically gives pessimistic bounds which are at odds with the empirical results. We tackle these problems an…
New method uses random features and Tikhonov regularization for operator learning from noisy data.
New method uses tensor decompositions to overcome the curse of dimensionality for large-scale learning.
Kernel methods give powerful, flexible, and theoretically grounded approaches to solving many problems in machine learning. The standard approach, however, requires pairwise evaluations of a kernel function, which can lead to scalability issues for very large datasets. Rahimi and Recht (2007) suggested a popular approa…
Random Fourier features (RFF) represent one of the most popular and wide-spread techniques in machine learning to scale up kernel algorithms. Despite the numerous successful applications of RFFs, unfortunately, quite little is understood theoretically on their optimality and limitations of their performance. Only recen…
Kernel methods represent one of the most powerful tools in machine learning to tackle problems expressed in terms of function values and derivatives due to their capability to represent and model complex relations. While these methods show good versatility, they are computationally intensive and have poor scalability t…
Fast simulates Volterra processes using RFF, focusing on S-fBM.
Three RFF-based methods for nonlinear causal discovery in mixed data.
In this paper we measured the stability of stochastic gradient method (SGM) for learning an approximated Fourier primal support vector machine. The stability of an algorithm is considered by measuring the generalization error in terms of the absolute difference between the test and the training error. Our problem is to…
The computational cost of training with softmax cross entropy loss grows linearly with the number of classes. For the settings where a large number of classes are involved, a common method to speed up training is to sample a subset of classes and utilize an estimate of the loss gradient based on these classes, known as…
Bayesian non-linear latent variable modeling for complex data.
In this paper, we study random subsampling of Gaussian process regression, one of the simplest approximation baselines, from a theoretical perspective. Although subsampling discards a large part of training data, we show provable guarantees on the accuracy of the predictive mean/variance and its generalization ability.…
Standard sparse pseudo-input approximations to the Gaussian process (GP) cannot handle complex functions well. Sparse spectrum alternatives attempt to answer this but are known to over-fit. We suggest the use of variational inference for the sparse spectrum approximation to avoid both issues. We model the covariance fu…
Improves learning of spectral mixture kernels with approximate Bayesian inference.
Proposes KMvDA for object recognition from multi-view data.
We present an intriguing discovery related to Random Fourier Features: in Gaussian kernel approximation, replacing the random Gaussian matrix by a properly scaled random orthogonal matrix significantly decreases kernel approximation error. We call this technique Orthogonal Random Features (ORF), and provide theoretical…
FMMNN combines sine activations with multi-component, multi-layer structure for high-frequency function approximation.
Random Fourier features improve tabular deep learning convergence.
A number of recent papers have provided evidence that practical design questions about neural networks may be tackled theoretically by studying the behavior of random networks. However, until now the tools available for analyzing random neural networks have been relatively ad-hoc. In this work, we show that the distrib…
Improved kernel ridge regression for large datasets using weighted random binning.
Accelerates signature kernel computation for sequences.
Positive-definite kernel functions are fundamental elements of kernel methods and Gaussian processes. A well-known construction of such functions comes from Bochner's characterization, which connects a positive-definite function with a probability distribution. Another construction, which appears to have attracted less…
We consider the binary classification problem when data are large and subject to unknown but bounded uncertainties. We address the problem by formulating the nonlinear support vector machine training problem with robust optimization. To do so, we analyze and propose two bounding schemes for uncertainties associated to …