arXiv research

The paper introduces new estimators for multivariate functions using Fourier methods.

Random Fourier features is one of the most popular techniques for scaling up kernel methods, such as kernel ridge regression. However, despite impressive empirical results, the statistical properties of random Fourier features are still not well understood. In this paper we take steps toward filling this gap. Specifica…

Improved Gaussian Process regression using TQFF over RFF and Gaussian QFF.

BFCR detects anomalies in 1D data efficiently.

Scalable kernel methods for large datasets using Fourier representations and NUFFT.

Computing accurate estimates of the Fourier transform of analog signals from discrete data points is important in many fields of science and engineering. The conventional approach of performing the discrete Fourier transform of the data implicitly assumes periodicity and bandlimitedness of the signal. In this paper, we…

Quantum-assisted Gaussian process speeds up data regression.

This work proves convergence of adaptive resampling for random Fourier features.

Integrates Fourier features for faster Gaussian process regression.

Transformers improve with Fourier integral attentions.

Unified approach for interpretable regression with flexible modeling.

Characterizes RFF regression in large $n,p,N$ setting, providing precise learning phases and double descent curve.

SLEIPNIR improves Gaussian process regression with derivatives, scaling up efficiently and accurately.

Enhances DGPs with adaptive RKHS Fourier features for better non-stationary pattern modeling.

Paper proves L2 regression can learn k-juntas without distributional assumptions.

Random non-linear Fourier features have recently shown remarkable performance in a wide-range of regression and classification applications. Motivated by this success, this article focuses on a sparse non-linear Fourier feature (NFF) model. We provide a characterization of the sufficient number of data points that guar…

Periodicity is often studied in timeseries modelling with autoregressive methods but is less popular in the kernel literature, particularly for higher dimensional problems such as in textures, crystallography, and quantum mechanics. Large datasets often make modelling periodicity untenable for otherwise powerful non-pa…

We show that the error probability of reconstructing kernel matrices from Random Fourier Features for the Gaussian kernel function is at most $\mathcal{O}(R^{2/3} \exp(-D))$, where $D$ is the number of random features and $R$ is the diameter of the data domain. We also provide an information-theoretic method-independen…

We propose a Fourier-based learning algorithm for highly nonlinear multiclass classification. The algorithm is based on a smoothing technique to calculate the probability distribution of all classes. To obtain the probability distribution, the density distribution of each class is smoothed by a low-pass filter separate…

Random Fourier features classification achieves fast learning rates with fewer features.

New scalable GP approximation using Fourier series decomposition.

New method reduces variance and bias in approximating indefinite kernels.

Nonlinear kernel regression models are often used in statistics and machine learning because they are more accurate than linear models. Variable selection for kernel regression models is a challenge partly because, unlike the linear regression setting, there is no clear concept of an effect size for regression coeffici…

New learning rates derived for Tikhonov-regularized problems without kernel assumptions.

We consider the problem of learning a mixture of linear regressions (MLRs). An MLR is specified by $k$ nonnegative mixing weights $p_1, \ldots, p_k$ summing to $1$, and $k$ unknown regressors $w_1,...,w_k\in\mathbb{R}^d$. A sample from the MLR is drawn by sampling $i$ with probability $p_i$, then outputting $(x, y)$ wh…

Paper develops methods for estimating and simulating a Student-t Lévy regression model.

Point forecasting of univariate time series is a challenging problem with extensive work having been conducted. However, nonparametric probabilistic forecasting of time series, such as in the form of quantiles or prediction intervals is an even more challenging problem. In an effort to expand the possible forecasting p…

This note introduces a regression technique for finding a class of nonlinear integro-differential operators from data. The method parametrizes the spatial operator with neural networks and Fourier transforms such that it can fit a class of nonlinear operators without needing a library of a priori selected operators. We…

Random Fourier features is a widely used, simple, and effective technique for scaling up kernel methods. The existing theoretical analysis of the approach, however, remains focused on specific learning tasks and typically gives pessimistic bounds which are at odds with the empirical results. We tackle these problems an…

We study the volatility functional inference by Fourier transforms. This spectral framework is advantageous in that it harnesses the power of harmonic analysis to handle missing data and asynchronous observations without any artificial time alignment nor data imputation. Under conditions, this spectral approach is cons…

Bayesian model connects KMs and ELMs for multitask regression.

Convolutional Neural Networks (CNNs) have become the method of choice for learning problems involving 2D planar images. However, a number of problems of recent interest have created a demand for models that can analyze spherical images. Examples include omnidirectional vision for drones, robots, and autonomous cars, mo…

FFN addresses spectral bias in neural value approximation, improving reinforcement learning performance.

A new graph signature invariant to graph automorphisms.

Kernel methods are powerful and flexible approach to solve many problems in machine learning. Due to the pairwise evaluations in kernel methods, the complexity of kernel computation grows as the data size increases; thus the applicability of kernel methods is limited for large scale datasets. Random Fourier Features (R…

Quantum ELMs use a quantum reservoir to learn from data, with limits on expressivity and scalability.

Coded Federated Learning speeds up training in edge computing networks.

Paper introduces FNM framework for learning finite-dimensional parametrized models.

New PINN architectures learn high-frequency features using Fourier features.

Overparameterized linear model shows strong classification but weak regression, susceptible to adversarial perturbations.

Three RFF-based methods for nonlinear causal discovery in mixed data.

New method uses quotient predictor space for better PAC-Bayes bounds, reducing KL divergence and improving model performance.

New spectral mixture representation for isotropic kernels simplifies random Fourier features.

In this paper, we study random subsampling of Gaussian process regression, one of the simplest approximation baselines, from a theoretical perspective. Although subsampling discards a large part of training data, we show provable guarantees on the accuracy of the predictive mean/variance and its generalization ability.…

In this paper we study the consistency of an empirical minimum error entropy (MEE) algorithm in a regression setting. We introduce two types of consistency. The error entropy consistency, which requires the error entropy of the learned function to approximate the minimum error entropy, is shown to be always true if the…

Improved kernel ridge regression for large datasets using weighted random binning.

The study assesses low-rank approximations in Gaussian Process regression.

The study assesses low-rank approximations in Gaussian Process regression.