arXiv research

New framework assesses regularization norms in ill-posed problems, revealing L2 instability and proposing adaptive fractional RKHS solutions.

A new method for learning function parameters in operators using data-adaptive RKHS.

This paper generalizes regularized regression problems in a hyper-reproducing kernel Hilbert space (hyper-RKHS), illustrates its utility for kernel learning and out-of-sample extensions, and proves asymptotic convergence results for the introduced regression models in an approximation theory view. Algorithmically, we c…

Study online learning in RKHS with dependent processes, focusing on \(β\)- and \(φ\)-mixing.

Paper explores RKHS properties for derivative and integral operators.

Regularization is used to find a solution that both fits the data and is sufficiently smooth, and thereby is very effective for designing and refining learning algorithms. But the influence of its exponent remains poorly understood. In particular, it is unclear how the exponent of the reproducing kernel Hilbert space~(…

Adapts RKHS methods to estimate density ratios with optimal error.

Reconstruction of a function from noisy data is often formulated as a regularized optimization problem over an infinite-dimensional reproducing kernel Hilbert space (RKHS). The solution describes the observed data and has a small RKHS norm. When the data fit is measured using a quadratic loss, this estimator has a know…

We study distributed learning with the least squares regularization scheme in a reproducing kernel Hilbert space (RKHS). By a divide-and-conquer approach, the algorithm partitions a data set into disjoint data subsets, applies the least squares regularization scheme to each data subset to produce an output function, an…

We propose a new point of view for regularizing deep neural networks by using the norm of a reproducing kernel Hilbert space (RKHS). Even though this norm cannot be computed, it admits upper and lower approximations leading to various practical strategies. Specifically, this perspective (i) provides a common umbrella f…

This paper analyzes divide-and-conquer estimators for functional linear regression without assuming target function in the RKHS.

Study convergence and approximations of entropic regularized Wasserstein distances for Gaussian and RKHS measures.

This study approximates distances between Gaussian processes and covariance operators using RKHS.

In this paper, we study the online learning algorithm without explicit regularization terms. This algorithm is essentially a stochastic gradient descent scheme in a reproducing kernel Hilbert space (RKHS). The polynomially decaying step size in each iteration can play a role of regularization to ensure the generalizati…

New analysis shows a gap between Gaussian RKHS and neural networks on unbounded domains.

Kernel $ε$-Greedy optimizes multi-armed bandits with covariates for sub-linear regret.

We address the problem of {\it adaptivity} in the framework of reproducing kernel Hilbert space (RKHS) regression. More precisely, we analyze estimators arising from a linear regularization scheme $g_\lam$. In practical applications, an important task is to choose the regularization parameter $\lam$ appropriately, i.e.…

This paper improves signal reconstruction using determinantal sampling from random nodes.

Novel method learns memory kernels in Langevin equations.

Study on adaptivity to kernel regularity in bandit problems.

Kernel ridge regression (KRR) is a well-known and popular nonparametric regression approach with many desirable properties, including minimax rate-optimality in estimating functions that belong to common reproducing kernel Hilbert spaces (RKHS). The approach, however, is computationally intensive for large data sets, d…

Unified analysis of kernel-based and locally adaptive bandit optimization methods.

Develops a new method for learning non-parametric DAGs using RKHS.

Consider the problem: given the data pair $(\mathbf{x}, \mathbf{y})$ drawn from a population with $f_*(x) = \mathbf{E}[\mathbf{y} | \mathbf{x} = x]$, specify a neural network model and run gradient flow on the weights over time until reaching any stationarity. How does $f_t$, the function computed by the neural network…

We introduce a novel boosting algorithm called `KTBoost' which combines kernel boosting and tree boosting. In each boosting iteration, the algorithm adds either a regression tree or reproducing kernel Hilbert space (RKHS) regression function to the ensemble of base learners. Intuitively, the idea is that discontinuous …

The paper develops divergences for Gaussian processes and RKHS settings.

Optimally tackles covariate shift in RKHS-based nonparametric regression.

Optimal CATE estimation with structured contrast functions using KRR.

Novel RKHS approach solves complex financial model equations.

New method simplifies tomographic reconstruction using RKHS.

Paper analyzes spectral algorithms under covariate shift, providing convergence rates.

New method learns kernels in nonlocal operators robustly.

CurvSSL improves SSL by aligning local manifold curvature.

We construct a Wasserstein gradient flow of the maximum mean discrepancy (MMD) and study its convergence properties. The MMD is an integral probability metric defined for a reproducing kernel Hilbert space (RKHS), and serves as a metric on probability measures for a sufficiently rich RKHS. We obtain conditions for conv…

End-to-end graph SVM with graph convolutions and RKHS.

We propose a generic spatiotemporal event forecasting method, which we developed for the National Institute of Justice's (NIJ) Real-Time Crime Forecasting Challenge. Our method is a spatiotemporal forecasting model combining scalable randomized Reproducing Kernel Hilbert Space (RKHS) methods for approximating Gaussian …

New geometric approach for analyzing compositional data like gut microbiomes.

New algorithms tackle RKHS bandits with reduced complexity and improved performance.

Constructs non-asymptotic confidence regions for unknown functions in RKHS.

Regularized approaches have been successfully applied to linear system identification in recent years. Many of them model unknown impulse responses exploiting the so called Reproducing Kernel Hilbert spaces (RKHSs) that enjoy the notable property of being in one-to-one correspondence with the class of positive semidefi…

New particle-based VI algorithm expands function class and improves scalability.

Deep neural kernels and Laplace kernel have equivalent RKHS on spheres.

Kernel methods are powerful tools to capture nonlinear patterns behind data. They implicitly learn high (even infinite) dimensional nonlinear features in the Reproducing Kernel Hilbert Space (RKHS) while making the computation tractable by leveraging the kernel trick. Classic kernel methods learn a single layer of nonl…

We study generalization properties of distributed algorithms in the setting of nonparametric regression over a reproducing kernel Hilbert space (RKHS). We first investigate distributed stochastic gradient methods (SGM), with mini-batches and multi-passes over the data. We show that optimal generalization error bounds c…

We propose a vector-valued regression problem whose solution is equivalent to the reproducing kernel Hilbert space (RKHS) embedding of the Bayesian posterior distribution. This equivalence provides a new understanding of kernel Bayesian inference. Moreover, the optimization problem induces a new regularization for the …

This paper develops a general framework for metric learning in RKHS with theoretical guarantees.

Enhances DGPs with adaptive RKHS Fourier features for better non-stationary pattern modeling.

Paper introduces RKHM for more explicit variable structures analysis.