arXiv research

New algorithms tackle RKHS bandits with reduced complexity and improved performance.

This work analyzes the role of data augmentation in self-supervised learning using RKHS approximation and regression.

Neural networks can approximate functionals on RKHS with error bounds.

Recently, some works have suggested methods to combine variational probabilistic inference with Monte Carlo sampling. One promising approach is via local optimal transport. In this approach, a gradient steepest descent method based on local optimal transport principles is formulated to transform deterministically point…

Learn ODEs from noisy data using RKHS and optimization.

This paper generalizes regularized regression problems in a hyper-reproducing kernel Hilbert space (hyper-RKHS), illustrates its utility for kernel learning and out-of-sample extensions, and proves asymptotic convergence results for the introduced regression models in an approximation theory view. Algorithmically, we c…

Consider the problem: given the data pair $(\mathbf{x}, \mathbf{y})$ drawn from a population with $f_*(x) = \mathbf{E}[\mathbf{y} | \mathbf{x} = x]$, specify a neural network model and run gradient flow on the weights over time until reaching any stationarity. How does $f_t$, the function computed by the neural network…

We study the approximation properties of random ReLU features through their reproducing kernel Hilbert space (RKHS). We first prove a universality theorem for the RKHS induced by random features whose feature maps are of the form of nodes in neural networks. The universality result implies that the random ReLU features…

Study convergence and approximations of entropic regularized Wasserstein distances for Gaussian and RKHS measures.

This paper improves signal reconstruction using determinantal sampling from random nodes.

This study approximates distances between Gaussian processes and covariance operators using RKHS.

Enhances DGPs with adaptive RKHS Fourier features for better non-stationary pattern modeling.

A scalable algorithm approximates Bayesian posteriors in RKHS with improved efficiency.

Study on how sampling works for complex data functions.

Reduced modeling of a computationally demanding dynamical system aims at approximating its trajectories, while optimizing the trade-off between accuracy and computational complexity. In this work, we propose to achieve such an approximation by first embedding the trajectories in a reproducing kernel Hilbert space (RKHS…

Develops an efficient approximation for full conformal prediction regions.

Develops a new method for learning non-parametric DAGs using RKHS.

We propose a generic spatiotemporal event forecasting method, which we developed for the National Institute of Justice's (NIJ) Real-Time Crime Forecasting Challenge. Our method is a spatiotemporal forecasting model combining scalable randomized Reproducing Kernel Hilbert Space (RKHS) methods for approximating Gaussian …

We propose a new point of view for regularizing deep neural networks by using the norm of a reproducing kernel Hilbert space (RKHS). Even though this norm cannot be computed, it admits upper and lower approximations leading to various practical strategies. Specifically, this perspective (i) provides a common umbrella f…

Wide neural networks can outperform kernel methods in certain tasks.

Develops first and second-order pseudo-mirror descent methods for nonnegative function estimation.

New algorithm learns Koopman operator online, with complexity control and convergence guarantees.

New sampling methods improve Shapley value estimation for machine learning models.

We present a new framework for online Least Squares algorithms for nonlinear modeling in RKH spaces (RKHS). Instead of implicitly mapping the data to a RKHS (e.g., kernel trick), we map the data to a finite dimensional Euclidean space, using random features of the kernel's Fourier transform. The advantage is that, the …

This paper improves Koopman operator approximations by pruning subspaces in RKHS.

Unified kernel framework extends to stochastic systems, improving numerical stability.

New KQEs improve probability metrics without mean function constraints.

Kernel ridge regression (KRR) is a well-known and popular nonparametric regression approach with many desirable properties, including minimax rate-optimality in estimating functions that belong to common reproducing kernel Hilbert spaces (RKHS). The approach, however, is computationally intensive for large data sets, d…

New method approximates complex kernel norms with random features, making learning tractable.

New algorithms improve GP inference without approximations, achieving better results.

We study distributed learning with the least squares regularization scheme in a reproducing kernel Hilbert space (RKHS). By a divide-and-conquer approach, the algorithm partitions a data set into disjoint data subsets, applies the least squares regularization scheme to each data subset to produce an output function, an…

Neural operators achieve fast convergence rates for solving PDEs.

Simplifies NL models by approximating them as LPV systems and identifying NL subterms.

New algorithm reduces online regression error in RKHS.

We consider a class of operator-induced norms, acting as finite-dimensional surrogates to the L2 norm, and study their approximation properties over Hilbert subspaces of L2 . The class includes, as a special case, the usual empirical norm encountered, for example, in the context of nonparametric regression in reproduci…

Constructs non-asymptotic confidence regions for unknown functions in RKHS.

Novel Newton method for large-scale kernel methods using random features.

Reconstruction of a function from noisy data is often formulated as a regularized optimization problem over an infinite-dimensional reproducing kernel Hilbert space (RKHS). The solution describes the observed data and has a small RKHS norm. When the data fit is measured using a quadratic loss, this estimator has a know…

Deep neural kernels and Laplace kernel have equivalent RKHS on spheres.

Kernel Dynamic Mode Decomposition reconstructs dynamical systems using Laplacian kernel.

KALE flow approximates KL divergence for distributions with disjoint support.

Kernel methods are powerful tools to capture nonlinear patterns behind data. They implicitly learn high (even infinite) dimensional nonlinear features in the Reproducing Kernel Hilbert Space (RKHS) while making the computation tractable by leveraging the kernel trick. Classic kernel methods learn a single layer of nonl…

Modeling dynamical systems with ordinary differential equations implies a mechanistic view of the process underlying the dynamics. However in many cases, this knowledge is not available. To overcome this issue, we introduce a general framework for nonparametric ODE models using penalized regression in Reproducing Kerne…

This paper develops a general framework for metric learning in RKHS with theoretical guarantees.

New framework assesses regularization norms in ill-posed problems, revealing L2 instability and proposing adaptive fractional RKHS solutions.

Survey of kernels, RKHS, and their applications in machine learning.

Paper introduces RKHM for more explicit variable structures analysis.

New particle-based VI algorithm expands function class and improves scalability.