arXiv research

New algorithms tackle RKHS bandits with reduced complexity and improved performance.

Constructs non-asymptotic confidence regions for unknown functions in RKHS.

Reconstruction of a function from noisy data is often formulated as a regularized optimization problem over an infinite-dimensional reproducing kernel Hilbert space (RKHS). The solution describes the observed data and has a small RKHS norm. When the data fit is measured using a quadratic loss, this estimator has a know…

Deep neural kernels and Laplace kernel have equivalent RKHS on spheres.

Kernel methods are powerful tools to capture nonlinear patterns behind data. They implicitly learn high (even infinite) dimensional nonlinear features in the Reproducing Kernel Hilbert Space (RKHS) while making the computation tractable by leveraging the kernel trick. Classic kernel methods learn a single layer of nonl…

This paper develops a general framework for metric learning in RKHS with theoretical guarantees.

New framework assesses regularization norms in ill-posed problems, revealing L2 instability and proposing adaptive fractional RKHS solutions.

Enhances DGPs with adaptive RKHS Fourier features for better non-stationary pattern modeling.

We study reproducing kernel Hilbert spaces (RKHS) on a Riemannian manifold. In particular, we discuss under which condition Sobolev spaces are RKHS and characterize their reproducing kernels. Further, we introduce and discuss a class of smoother RKHS that we call diffusion spaces. We illustrate the general results with…

This work analyzes the role of data augmentation in self-supervised learning using RKHS approximation and regression.

Neural networks can approximate functionals on RKHS with error bounds.

Kernel methods have been among the most popular techniques in machine learning, where learning tasks are solved using the property of reproducing kernel Hilbert space (RKHS). In this paper, we propose a novel data analysis framework with reproducing kernel Hilbert $C^*$-module (RKHM), which is another generalization of…

This paper generalizes regularized regression problems in a hyper-reproducing kernel Hilbert space (hyper-RKHS), illustrates its utility for kernel learning and out-of-sample extensions, and proves asymptotic convergence results for the introduced regression models in an approximation theory view. Algorithmically, we c…

New approach to supervised learning in RKHS and vvRKHS using $C^*$-algebras.

Study evaluates RKHS choices for assessing graph models using KSD tests.

A new method for learning function parameters in operators using data-adaptive RKHS.

New analysis shows a gap between Gaussian RKHS and neural networks on unbounded domains.

Study online learning in RKHS with dependent processes, focusing on \(β\)- and \(φ\)-mixing.

Wide neural networks can outperform kernel methods in certain tasks.

Paper explores RKHS properties for derivative and integral operators.

Learn ODEs from noisy data using RKHS and optimization.

Develops RKHS framework for analyzing tree ensembles.

Adapts RKHS methods to estimate density ratios with optimal error.

Study on how sampling works for complex data functions.

This paper improves signal reconstruction using determinantal sampling from random nodes.

We study quadrature rules for functions from an RKHS, using nodes sampled from a determinantal point process (DPP). DPPs are parametrized by a kernel, and we use a truncated and saturated version of the RKHS kernel. This link between the two kernels, along with DPP machinery, leads to relatively tight bounds on the qua…

Kernel $ε$-Greedy optimizes multi-armed bandits with covariates for sub-linear regret.

Improved GP bandit algorithms for noiseless, varying noise, and RKHS norms.

Despite the fundamental nature of the inhomogeneous Poisson process in the theory and application of stochastic processes, and its attractive generalizations (e.g. Cox process), few tractable nonparametric modeling approaches of intensity functions exist, especially when observed points lie in a high-dimensional space.…

A Hilbert space embedding for probability measures has recently been proposed, wherein any probability measure is represented as a mean element in a reproducing kernel Hilbert space (RKHS). Such an embedding has found applications in homogeneity testing, independence testing, dimensionality reduction, etc., with the re…

Paper introduces RKHM and KME for richer data analysis.

Usually, complex-valued RKHS are presented as an straightforward application of the real-valued case. In this paper we prove that this procedure yields a limited solution for regression. We show that another kernel, here denoted as pseudo kernel, is needed to learn any function in complex-valued fields. Accordingly, we…

Develops a new framework for estimating joint probability distributions.

Research aims to improve confidence intervals for RKHS elements in online learning.

A nonparametric kernel-based method for realizing Bayes' rule is proposed, based on representations of probabilities in reproducing kernel Hilbert spaces. Probabilities are uniquely characterized by the mean of the canonical map to the RKHS. The prior and conditional probabilities are expressed in terms of RKHS functio…

Proposes CCME framework for estimating heterogeneous treatment effects.

Consider the problem: given the data pair $(\mathbf{x}, \mathbf{y})$ drawn from a population with $f_*(x) = \mathbf{E}[\mathbf{y} | \mathbf{x} = x]$, specify a neural network model and run gradient flow on the weights over time until reaching any stationarity. How does $f_t$, the function computed by the neural network…

This paper analyzes divide-and-conquer estimators for functional linear regression without assuming target function in the RKHS.

Several scalable sample-based methods to compute the Kullback Leibler (KL) divergence between two distributions have been proposed and applied in large-scale machine learning models. While they have been found to be unstable, the theoretical root cause of the problem is not clear. In this paper, we study a generative a…

Recently, some works have suggested methods to combine variational probabilistic inference with Monte Carlo sampling. One promising approach is via local optimal transport. In this approach, a gradient steepest descent method based on local optimal transport principles is formulated to transform deterministically point…

We present a new framework for online Least Squares algorithms for nonlinear modeling in RKH spaces (RKHS). Instead of implicitly mapping the data to a RKHS (e.g., kernel trick), we map the data to a finite dimensional Euclidean space, using random features of the kernel's Fourier transform. The advantage is that, the …

New method links covariates to CTMCs using RKHS, improving state transitions modeling.

Study analyzes learnability of RKHS under L∞ norm for kernel methods.

This study approximates distances between Gaussian processes and covariance operators using RKHS.

We study the approximation properties of random ReLU features through their reproducing kernel Hilbert space (RKHS). We first prove a universality theorem for the RKHS induced by random features whose feature maps are of the form of nodes in neural networks. The universality result implies that the random ReLU features…

Paper proposes a method to stabilize estimation of KL divergence using a discriminator in RKHS.

Develops a new method for learning non-parametric DAGs using RKHS.

Enhances Koopman operator estimation with intrinsic observables in RKHS.