arXiv research

Constructs non-asymptotic confidence regions for unknown functions in RKHS.

Neural networks can approximate functionals on RKHS with error bounds.

Kernel methods are powerful tools to capture nonlinear patterns behind data. They implicitly learn high (even infinite) dimensional nonlinear features in the Reproducing Kernel Hilbert Space (RKHS) while making the computation tractable by leveraging the kernel trick. Classic kernel methods learn a single layer of nonl…

A new method for learning function parameters in operators using data-adaptive RKHS.

New analysis shows a gap between Gaussian RKHS and neural networks on unbounded domains.

Paper explores RKHS properties for derivative and integral operators.

Deep neural kernels and Laplace kernel have equivalent RKHS on spheres.

New algorithms tackle RKHS bandits with reduced complexity and improved performance.

Reconstruction of a function from noisy data is often formulated as a regularized optimization problem over an infinite-dimensional reproducing kernel Hilbert space (RKHS). The solution describes the observed data and has a small RKHS norm. When the data fit is measured using a quadratic loss, this estimator has a know…

This paper analyzes divide-and-conquer estimators for functional linear regression without assuming target function in the RKHS.

This paper improves signal reconstruction using determinantal sampling from random nodes.

Study on how sampling works for complex data functions.

We study the approximation properties of random ReLU features through their reproducing kernel Hilbert space (RKHS). We first prove a universality theorem for the RKHS induced by random features whose feature maps are of the form of nodes in neural networks. The universality result implies that the random ReLU features…

Paper tackles tensor decomposition for unaligned observations using RKHS and novel loss functions.

Kernel $ε$-Greedy optimizes multi-armed bandits with covariates for sub-linear regret.

A nonparametric kernel-based method for realizing Bayes' rule is proposed, based on representations of probabilities in reproducing kernel Hilbert spaces. Probabilities are uniquely characterized by the mean of the canonical map to the RKHS. The prior and conditional probabilities are expressed in terms of RKHS functio…

Study improves hypothesis transfer learning for functional linear models.

Despite the fundamental nature of the inhomogeneous Poisson process in the theory and application of stochastic processes, and its attractive generalizations (e.g. Cox process), few tractable nonparametric modeling approaches of intensity functions exist, especially when observed points lie in a high-dimensional space.…

Paper introduces RKHM and KME for richer data analysis.

This paper generalizes regularized regression problems in a hyper-reproducing kernel Hilbert space (hyper-RKHS), illustrates its utility for kernel learning and out-of-sample extensions, and proves asymptotic convergence results for the introduced regression models in an approximation theory view. Algorithmically, we c…

This work analyzes the role of data augmentation in self-supervised learning using RKHS approximation and regression.

Improved GP bandit algorithms for noiseless, varying noise, and RKHS norms.

Adaptive tensor modeling preserves continuity in multidimensional data.

We study quadrature rules for functions from an RKHS, using nodes sampled from a determinantal point process (DPP). DPPs are parametrized by a kernel, and we use a truncated and saturated version of the RKHS kernel. This link between the two kernels, along with DPP machinery, leads to relatively tight bounds on the qua…

We consider the problem of optimising functions in the reproducing kernel Hilbert space (RKHS) of a Matérn kernel with smoothness parameter $ν$ over the domain $[0,1]^d$ under noisy bandit feedback. Our contribution, the $π$-GP-UCB algorithm, is the first practical approach with guaranteed sublinear regret for all $ν>1…

Wide neural networks can outperform kernel methods in certain tasks.

Differential privacy is a framework for privately releasing summaries of a database. Previous work has focused mainly on methods for which the output is a finite dimensional vector, or an element of some discrete set. We develop methods for releasing functions while preserving differential privacy. Specifically, we sho…

New KQEs improve probability metrics without mean function constraints.

Several scalable sample-based methods to compute the Kullback Leibler (KL) divergence between two distributions have been proposed and applied in large-scale machine learning models. While they have been found to be unstable, the theoretical root cause of the problem is not clear. In this paper, we study a generative a…

Proposes a method for fair regression using RKHS.

Consider the problem: given the data pair $(\mathbf{x}, \mathbf{y})$ drawn from a population with $f_*(x) = \mathbf{E}[\mathbf{y} | \mathbf{x} = x]$, specify a neural network model and run gradient flow on the weights over time until reaching any stationarity. How does $f_t$, the function computed by the neural network…

Usually, complex-valued RKHS are presented as an straightforward application of the real-valued case. In this paper we prove that this procedure yields a limited solution for regression. We show that another kernel, here denoted as pseudo kernel, is needed to learn any function in complex-valued fields. Accordingly, we…

New particle-based VI algorithm expands function class and improves scalability.

New method links covariates to CTMCs using RKHS, improving state transitions modeling.

This study approximates distances between Gaussian processes and covariance operators using RKHS.

We study distributed learning with the least squares regularization scheme in a reproducing kernel Hilbert space (RKHS). By a divide-and-conquer approach, the algorithm partitions a data set into disjoint data subsets, applies the least squares regularization scheme to each data subset to produce an output function, an…

New algorithm optimizes Hölder smooth functions in RKHS with tighter regret bounds.

Algorithm optimizes cascaded functions with known structure.

Develops RKHS framework for analyzing tree ensembles.

Kernel DRO uses RKHS to optimize under distributional uncertainty.

We introduce a novel boosting algorithm called `KTBoost' which combines kernel boosting and tree boosting. In each boosting iteration, the algorithm adds either a regression tree or reproducing kernel Hilbert space (RKHS) regression function to the ensemble of base learners. Intuitively, the idea is that discontinuous …

Paper proposes a method to stabilize estimation of KL divergence using a discriminator in RKHS.

A new method for analyzing adaptive experiments using kernel treatment effects.

The paper tackles data-driven optimal control of unknown nonlinear systems using RKHS.

We present a new framework for online Least Squares algorithms for nonlinear modeling in RKH spaces (RKHS). Instead of implicitly mapping the data to a RKHS (e.g., kernel trick), we map the data to a finite dimensional Euclidean space, using random features of the kernel's Fourier transform. The advantage is that, the …

Characterizes neural kernel and NNGP for various activations.

Paper establishes lower bounds for non-stationary kernelized bandits.

The paper identifies a 'small' set of functions containing Gaussian process samples.