Enhances Fourier estimator performance for asynchronous event-data.
problem Improving correlation and covariance estimation on event-data.
method Implement and test NUFFT methods with different averaging kernels.
result Demonstrates improved performance and relationship between averaging scales.
The paper introduces new estimators for multivariate functions using Fourier methods.
problem Estimating multivariate functions like densities and regression functions.
method Monte Carlo estimators based on the Fourier integral theorem.
result Established rates of convergence for new estimators, often superior to existing methods.
Random Fourier features is one of the most popular techniques for scaling up kernel methods, such as kernel ridge regression. However, despite impressive empirical results, the statistical properties of random Fourier features are still not well understood. In this paper we take steps toward filling this gap. Specifica…
Efficiently scales continuous kernels with sparse Fourier domain learning.
problem High computational and memory demands, spectral bias in continuous kernels.
method Sparse learning in the Fourier domain.
result Efficient scaling of continuous kernels, reduced computational and memory requirements, mitigated spectral bias.
We develop Fourier methods to expand translation-invariant kernels.
problem Constructing orthonormal expansions for translation-invariant kernels.
method Fourier analytic technique to derive explicit expansions.
result Explicit expansions for various kernels (Matérn, Cauchy, Gaussian).
Devoted to multi-task learning and structured output learning, operator-valued kernels provide a flexible tool to build vector-valued functions in the context of Reproducing Kernel Hilbert Spaces. To scale up these methods, we extend the celebrated Random Fourier Feature methodology to get an approximation of operator-…
Orthogonal random features approximate a Bessel kernel, offering sharper bounds than random Fourier features.
problem Approximating Gaussian kernel efficiently for large datasets.
method Use of Haar orthogonal matrices to construct orthogonal random features and analyze their bias and variance.
result Orthogonal random features approximate a Bessel kernel, not the Gaussian kernel, with sharper bounds.
A new model captures complex event data using attention and Fourier kernels.
problem Capturing complex non-linear temporal dependencies in discrete event data.
method Integrates attention mechanism into point processes' conditional intensity function and uses Fourier kernel embedding.
result Established theoretical properties and demonstrated competitive performance.
Transformers improve with Fourier integral attentions.
problem Inefficiency of dot-product attention in capturing feature dependencies.
method Interpreted attention as kernel regression, proposed FourierFormer with generalized Fourier integral kernels.
result FourierFormer achieves better accuracy and reduces redundancy.
The expressive power of Gaussian processes depends heavily on the choice of kernel. In this work we propose the novel harmonizable mixture kernel (HMK), a family of expressive, interpretable, non-stationary kernels derived from mixture models on the generalized spectral representation. As a theoretically sound treatmen…
Lecture notes on kernel functions and Random Fourier Features.
problem Understanding and approximating kernel functions in machine learning.
method Mathematical background and proofs of concentration results.
result Estimation of error in Random Fourier Features approximation.
A new method slices and sums radial kernels faster.
problem Fast computation of large kernel sums in kernel methods.
method Random projections to 1D subspaces and QMC for selecting projections.
result QMC-slicing outperforms existing methods on test datasets.
Scalable kernel methods for large datasets using Fourier representations and NUFFT.
problem Cubic complexity in kernel methods limits their use on large-scale datasets.
method Fourier representation of kernels combined with NUFFT for O(n log n) complexity.
result Achieves minimax convergence rates and processes up to tens of billions of samples.
New method for optimizing risk in financial models using Fourier transforms.
problem Optimizing risk in financial models with multi-period mean-CVaR.
method Strictly monotone 2D integration scheme via Fourier-trained transition kernels.
result Established robust and accurate optimization method for financial models.
New spectral mixture representation for isotropic kernels simplifies random Fourier features.
problem Applying Random Fourier Features to complex kernels.
method Decompose isotropic kernels into scale mixtures of α-stable random vectors.
result Constructive spectral sampling formula for various kernels.
Kernel methods are powerful and flexible approach to solve many problems in machine learning. Due to the pairwise evaluations in kernel methods, the complexity of kernel computation grows as the data size increases; thus the applicability of kernel methods is limited for large scale datasets. Random Fourier Features (R…
RFFNet scales kernel methods to large datasets by learning kernel relevance.
problem Scaling kernel methods to large datasets while maintaining interpretability.
method Designs random Fourier features for ARD kernels and uses first-order stochastic optimization for learning kernel relevances.
result RFFNet achieves low prediction error and identifies relevant features, leading to more interpretable solutions.
A new method for nonstationary Gaussian processes using Fourier features.
problem Efficient simulation of nonstationary Gaussian processes with high-dimensional distributions.
method Discretizes the spectral representation of nonstationary processes, avoiding probability measure assumptions.
result An efficient low-rank approximation of nonstationary spectral densities, consistent and positive semi-definite.
Periodicity is often studied in timeseries modelling with autoregressive methods but is less popular in the kernel literature, particularly for higher dimensional problems such as in textures, crystallography, and quantum mechanics. Large datasets often make modelling periodicity untenable for otherwise powerful non-pa…
We revisit Rahimi and Recht (2007)'s kernel random Fourier features (RFF) method through the lens of the PAC-Bayesian theory. While the primary goal of RFF is to approximate a kernel, we look at the Fourier transform as a prior distribution over trigonometric hypotheses. It naturally suggests learning a posterior on th…
Improves sparse recovery with non-linear Fourier features.
problem Sparse recovery challenges with non-linear Fourier features.
method Characterizes sufficient data points for perfect recovery.
result Sufficient data points depend on kernel matrix.
New scalable GP approximation using Fourier series decomposition.
problem Scalability and accuracy in Gaussian process approximations.
method Harmonic kernel decomposition (HKD) to decompose kernels orthogonally.
result Significantly outperforms standard variational methods in scalability and accuracy.
Random Fourier features improve tabular deep learning convergence.
problem Tabular deep learning convergence issues.
method Random Fourier projections as a pre-processing step, projecting inputs into a fixed feature space.
result Random Fourier pre-processing accelerates tabular deep learning convergence.
New Fourier features improve high-precision approximation in large-scale problems.
problem Designing scalable, high-precision Fourier features for large-scale kernel methods.
method Introducing a new family of quadrature rules that accurately approximate the Gaussian measure in higher dimensions.
result Improved approximation bounds with new Fourier features.
We show that the error probability of reconstructing kernel matrices from Random Fourier Features for the Gaussian kernel function is at most O(R2/3exp(−D)), where D is the number of random features and R is the diameter of the data domain. We also provide an information-theoretic method-independen…
Fourier representation improves KSD for infinite-dimensional data.
problem Applying KSD to infinite-dimensional data.
method Combining measure equations with kernel methods for a Fourier representation of KSD.
result KSD can separate measures in infinite-dimensional Hilbert spaces.
New quantization methods improve accuracy of Random Fourier Features.
problem Improving accuracy of Random Fourier Features for machine learning.
method Sigma-Delta and distributed noise-shaping quantization methods for 1-bit and low bit-depth quantization.
result Quantized RFFs allow high accuracy approximation of underlying kernels with polynomial error decay.
Accelerates signature kernel computation for sequences.
problem Severe computational bottleneck in computing signature kernel.
method Random Fourier features to accelerate signature kernel computation.
result Uniform approximation guarantees for unbiased estimator with linear computation time.
Quantum kernels can be efficiently embedded into classical feature spaces.
problem Can all quantum kernels be efficiently embedded into classical feature spaces?
method Invoking computational universality and using techniques like random Fourier features, the authors show that certain classes of quantum kernels can be efficiently embedded.
result For shift-invariant and composition kernels, embedding quantum kernels are universal and efficient.
New random feature maps for Laplacian and related kernels.
problem Challenges in approximating the Laplacian kernel and its generalizations.
method Developed random feature maps for Laplacian and related kernels, providing efficient sampling schemes.
result Demonstrated the efficacy of these random feature maps on real datasets.
New learning rates derived for Tikhonov-regularized problems without kernel assumptions.
problem Learning rates for Tikhonov-regularized learning problems.
method Minimax adaptive rates derived using Fourier isocapacitary condition and interpolation theory.
result Derivation of minimax adaptive rates without requiring kernel assumptions.
New method reduces variance and bias in approximating indefinite kernels.
problem Approximating non-stationary indefinite kernels with low variance and bias.
method Generalized orthogonal random features (GORF)
result GORF achieves lower variance and approximation error compared to existing methods.
New method uses tensor decompositions to overcome the curse of dimensionality for large-scale learning.
problem Large-scale machine learning problems with kernel methods.
method Deterministic Fourier features combined with low-rank tensor decomposition for tensor product structure.
result Demonstrated consistent performance and superior results compared to random Fourier features.
We establish a link between Fourier optics and a recent construction from the machine learning community termed the kernel mean map. Using the Fraunhofer approximation, it identifies the kernel with the squared Fourier transform of the aperture. This allows us to use results about the invertibility of the kernel mean m…
End-to-end kernel learning using generative RFFs for improved performance.
problem Improving kernel learning performance and generalization.
method Develops a generative network via RFFs to implicitly learn the kernel, followed by a linear classifier, jointly trained by ERM.
result Shows superior generalization performance over classical methods in real-world tasks.
Quantum-assisted Gaussian process speeds up data regression.
problem High computational complexity of Gaussian process regression for large datasets.
method Quantum-assisted sparse Gaussian process regression using random Fourier features.
result Achieves polynomial-order computational speedup compared to classical methods.
Random Fourier features (RFF) represent one of the most popular and wide-spread techniques in machine learning to scale up kernel algorithms. Despite the numerous successful applications of RFFs, unfortunately, quite little is understood theoretically on their optimality and limitations of their performance. Only recen…
Three RFF-based methods for nonlinear causal discovery in mixed data.
problem Nonlinear causal discovery in mixed data with computational constraints.
method FFML, TRFF, and FFCI methods for score-based, constraint-based, and hybrid causal discovery.
result FFML and TRFF methods provide complementary performance in causal discovery.
The study characterizes kernel spaces on hyperspheres, impacting cubature algorithms.
problem Characterizing kernel spaces on hyperspheres for cubature algorithms.
method Characterization of Sobolev spaces and reproducing kernel Hilbert spaces over hyperspheres.
result Direct consequences for kernel cubature and worst-case error rates.
We propose a Gradient Boosting algorithm for learning an ensemble of kernel functions adapted to the task at hand. Unlike state-of-the-art Multiple Kernel Learning techniques that make use of a pre-computed dictionary of kernel functions to select from, at each iteration we fit a kernel by approximating it as a weighte…
This paper develops quantization algorithms for random Fourier features, simplifying the process and improving performance.
problem Efficient quantization of random Fourier features for better performance and storage.
method Developed Lloyd-Max (LM) and LM2-RFF quantization schemes for random Fourier features. result The marginal distribution of RFF is independent of the Gaussian kernel parameter γ, simplifying quantization design.
Integrates Fourier features for faster Gaussian process regression.
problem Efficiently scaling Gaussian process regression to large datasets.
method Integrated Fourier features for Gaussian processes.
result Improves Gaussian process regression speed to O(M3) for a broad class of kernels. Kernel learning methods are among the most effective learning methods and have been vigorously studied in the past decades. However, when tackling with complicated tasks, classical kernel methods are not flexible or "rich" enough to describe the data and hence could not yield satisfactory performance. In this paper, vi…
Study examines boundedness of oscillating singular integrals on specific Lie groups.
problem Investigating boundedness of oscillating singular integrals on Lie groups of polynomial growth.
method Presented kernel criteria in terms of sub-Riemannian structure and Fourier analysis.
result Extended classical oscillating conditions for boundedness of oscillating convolution operators.
We investigate how to train kernel approximation methods that generalize well under a memory budget. Building on recent theoretical work, we define a measure of kernel approximation error which we find to be more predictive of the empirical generalization performance of kernel approximation methods than conventional me…
Quantum kernel improves solar irradiance forecasting.
problem Improving short-term solar irradiance forecasting accuracy.
method Quantum Fourier Transform kernel in KRR with feature mixing.
result Consistently improves R2 and nRMSE over classical kernels.
This paper develops a bootstrap method to estimate errors in Random Fourier Features.
problem Inability to estimate the error of Random Fourier Features approximations.
method Develops a bootstrap approach to numerically estimate the errors of RFF approximations.
result Specific, flexible, and adaptive error estimates for RFF approximations.
Quantum method improves neural density estimation in high dimensions.
problem High-dimensional density estimation with poor performance and high computational complexity.
method Adaptive Fourier features based on quantum density matrices, integrated with neural networks.
result Competitive performance compared to state-of-the-art methods in various datasets.