Paper explores RKHS properties for derivative and integral operators.
problem Establishing sufficient conditions for reproducing property in RKHS.
method Establishing reproducing property for combinations of composition operators.
result Provides framework for regularized learning algorithms involving function values, gradients, or operators.
A new method for learning function parameters in operators using data-adaptive RKHS.
problem Learning function parameters in operators with robustness to noise and numerical error.
method Data Adaptive RKHS Tikhonov Regularization (DARTR) method.
result DARTR leads to an accurate estimator robust to noise and numerical error, converging at a consistent rate as data refines.
Paper introduces RKHM for more explicit variable structures analysis.
problem Explicitly analyzing structures among variables.
method Orthonormal systems in Hilbert C∗-modules, RKHM. result Theoretical and practical procedures for RKHM orthonormalization.
This study approximates distances between Gaussian processes and covariance operators using RKHS.
problem Approximating distances between Gaussian processes and covariance operators from finite samples.
method Using reproducing kernel Hilbert space (RKHS) covariance and cross-covariance operators, the study shows how to consistently and efficiently estimate Sinkhorn divergence from finite samples.
result Convergence rates are dimension-independent and of the same order as Hilbert-Schmidt distance.
Enhances Koopman operator estimation with intrinsic observables in RKHS.
problem Accurate estimation of Koopman operator and its spectrum.
method Jet Extended Dynamic Mode Decomposition (JetEDMD) leveraging RKHS jets.
result Proves JetEDMD's superiority with error bounds and convergence rate.
New algorithm learns Koopman operator online, with complexity control and convergence guarantees.
problem Online learning of Koopman operator for general nonlinear systems.
method Sparse online learning via stochastic approximation, RKHS action, CME operator.
result Provably convergent algorithm with finite-time guarantees in mis-specified setting.
Kernel Dynamic Mode Decomposition reconstructs dynamical systems using Laplacian kernel.
problem Reconstructing spatial-temporal dynamics of complex systems.
method Kernel Dynamic Mode Decomposition with Laplacian kernel.
result Laplacian kernel allows for the closability of Koopman operators in RKHS, enabling reconstruction.
The paper tackles data-driven optimal control of unknown nonlinear systems using RKHS.
problem Unknown nonlinear dynamics and stage cost functions.
method Embed state densities into RKHS, learn Markov operators, solve Hamilton-Jacobi-Bellman recursions.
result Solves a wide range of nonlinear control problems, including depth regulation.
Enhances DGPs with adaptive RKHS Fourier features for better non-stationary pattern modeling.
problem Capturing complex non-stationary patterns in non-linear dynamical systems.
method Integrates ODE-based RKHS Fourier features into DGPs using convolution operations for adaptive amplitude and phase modulation. Uses a doubly stochastic variational inference framework.
result Improved predictive performance across various regression tasks.
Neural operators achieve fast convergence rates for solving PDEs.
problem Solving partial differential equations (PDEs) efficiently.
method Two-layer neural operators with gradient descent analysis in RKHS.
result Fast convergence rates are minimax optimal for early-stopped GD.
Kernel methods are powerful tools to capture nonlinear patterns behind data. They implicitly learn high (even infinite) dimensional nonlinear features in the Reproducing Kernel Hilbert Space (RKHS) while making the computation tractable by leveraging the kernel trick. Classic kernel methods learn a single layer of nonl…
This paper analyzes divide-and-conquer estimators for functional linear regression without assuming target function in the RKHS.
problem Functional linear regression without target function in RKHS.
method Integral operator approach to establish upper bounds and prove asymptotic optimality.
result Sharp finite sample upper bounds and asymptotic optimality of divide-and-conquer estimators.
Study on estimating distances between covariance operators and Gaussian processes.
problem Estimating distances between covariance operators and Gaussian processes.
method Riemannian distances, concentration results for Hilbert space-valued random variables, RKHS covariance and cross-covariance operators.
result Both distances converge in the Hilbert-Schmidt norm and can be consistently and efficiently estimated.
Recently, some works have suggested methods to combine variational probabilistic inference with Monte Carlo sampling. One promising approach is via local optimal transport. In this approach, a gradient steepest descent method based on local optimal transport principles is formulated to transform deterministically point…
This work analyzes the role of data augmentation in self-supervised learning using RKHS approximation and regression.
problem Limited theoretical understanding of the role of data augmentation in self-supervised learning.
method Geometric characterization of the target function given by augmentation, proving generalization bounds.
result Two generalization bounds are derived, one free of model complexity, the other specific to near-optimal encoders.
The paper develops divergences for Gaussian processes and RKHS settings.
problem Estimating divergences in infinite-dimensional spaces.
method Formulations of Alpha Log-Det divergences, continuity in norm, laws of large numbers, consistent estimation from finite samples.
result Infinite-dimensional divergences can be estimated from finite-dimensional versions with dimension-independent sample complexities.
In this paper, we specify what functions induce the bounded composition operators on a reproducing kernel Hilbert space (RKHS) associated with an analytic positive definite function defined on Rd. We prove that only affine transforms can do so in a pretty large class of RKHS. Our result covers not only the …
We study distributed learning with the least squares regularization scheme in a reproducing kernel Hilbert space (RKHS). By a divide-and-conquer approach, the algorithm partitions a data set into disjoint data subsets, applies the least squares regularization scheme to each data subset to produce an output function, an…
New method simplifies tomographic reconstruction using RKHS.
problem Tomographic reconstruction challenges.
method RKHS framework for X-ray transform.
result Sharp stability results without Fourier transform.
The main contribution of the paper is to show that Gaussian sketching of a kernel-Gram matrix K yields an operator whose counterpart in an RKHS H, is a \emph{random projection} operator---in the spirit of Johnson-Lindenstrauss (J-L) lemma. To be precise, given a random matrix Z with i.i.d. Ga…
Study convergence and approximations of entropic regularized Wasserstein distances for Gaussian and RKHS measures.
problem Convergence and approximations of entropic regularized Wasserstein distances in Gaussian and RKHS settings.
method Analysis of convergence and finite sample approximations of entropic regularized Wasserstein distances in Gaussian and RKHS settings.
result Strictly weaker convergence in 2-Sinkhorn divergence for Gaussian measures compared to exact 2-Wasserstein distance.
Modeling dynamical systems with ordinary differential equations implies a mechanistic view of the process underlying the dynamics. However in many cases, this knowledge is not available. To overcome this issue, we introduce a general framework for nonparametric ODE models using penalized regression in Reproducing Kerne…
This paper improves Koopman operator approximations by pruning subspaces in RKHS.
problem Improving predictive accuracy of Koopman operator approximations.
method Computes principal angles and vectors in RKHS to prune subspaces.
result Validated approach enhances Koopman operator approximations for large datasets.
New method learns kernels in nonlocal operators robustly.
problem Learning kernels in nonlocal operators is ill-posed.
method Nonparametric regression with Tikhonov regularization.
result Robust estimator of kernel yields homogenized model.
Continuum transformers learn operators in context via gradient descent.
problem Generalizing transformers to handle infinite-dimensional inputs for in-context learning.
method Gradient descent in an operator RKHS, leveraging generalized representer theorems and gradient flows.
result Operator learned in context is Bayes Optimal Predictor in infinite depth limit.
Unified kernel framework extends to stochastic systems, improving numerical stability.
problem Extending kernel methods to stochastic dynamical systems with diffusion.
method Unified kernel framework, Feynman-Kac path-integral representations, collocation-based computational framework.
result Kernel equivalence under uniform ellipticity assumptions and improved numerical stability with moderate diffusion.
New learning rates for embeddings in RKHSs, even when the target is not Hilbert-Schmidt.
problem Applying conditional mean embeddings to complex ML/RL settings with infinite-dimensional RKHSs.
method Developed novel learning rates using interpolation theory for RKHSs, derived explicit adaptive rates for sample estimator.
result Achieved uniform convergence rates in the output RKHS for certain parameter regimes.
End-to-end graph SVM with graph convolutions and RKHS.
problem Graph classification with complex feature spaces.
method End-to-end training of graph convolutions, kernel function, and SVM parameters.
result Outperforms existing deep learning models on graph classification tasks.
We consider a class of operator-induced norms, acting as finite-dimensional surrogates to the L2 norm, and study their approximation properties over Hilbert subspaces of L2 . The class includes, as a special case, the usual empirical norm encountered, for example, in the context of nonparametric regression in reproduci…
KKR uses Koopman theory to improve forecasting in complex systems.
problem Forecasting complex, nonlinear dynamical systems in decision-making.
method Derives a universal Koopman-invariant RKHS for LTI dynamical systems.
result KKR framework provides convergence results and generalization error bounds.
New algorithms tackle RKHS bandits with reduced complexity and improved performance.
problem Adversarial and stochastic RKHS bandit problems with high computational complexity.
method Combining approximation theory with misspecified linear bandit methods.
result First general algorithm for adversarial RKHS bandit problem.
Constructs non-asymptotic confidence regions for unknown functions in RKHS.
problem Global probabilistic confidence regions for unknown functions in RKHS.
method Reduces confidence region construction to estimating RKHS norm.
result Valid confidence regions can be constructed non-asymptotically.
Simplifies NL models by approximating them as LPV systems and identifying NL subterms.
problem Complex NL models are hard to interpret and impractical.
method Linear approximation around operating points, sparse estimation in RKHS, LPV model reduction.
result Identifies NL subterms and their input spaces in sparse additive NL models.
We propose a representation of Gaussian processes (GPs) based on powers of the integral operator defined by a kernel function, we call these stochastic processes integral Gaussian processes (IGPs). Sample paths from IGPs are functions contained within the reproducing kernel Hilbert space (RKHS) defined by the kernel fu…
Reconstruction of a function from noisy data is often formulated as a regularized optimization problem over an infinite-dimensional reproducing kernel Hilbert space (RKHS). The solution describes the observed data and has a small RKHS norm. When the data fit is measured using a quadratic loss, this estimator has a know…
Deep neural kernels and Laplace kernel have equivalent RKHS on spheres.
problem Comparing RKHS of deep neural tangent and Laplace kernels.
method Proof of RKHS equivalence using sphere restrictions and kernel properties.
result RKHS of deep neural tangent kernel and Laplace kernel are the same on Sd−1. Two adaptive kernel selection methods improve the accuracy of Kernelized Diffusion Maps.
problem Selecting an appropriate kernel for Kernelized Diffusion Maps.
method Two complementary approaches: variational outer loop and unsupervised cross-validation.
result Both methods improve the quality and stability of the recovered eigenfunctions.
Reproducing kernel Hilbert spaces (RKHSs) play an important role in many statistics and machine learning applications ranging from support vector machines to Gaussian processes and kernel embeddings of distributions. Operators acting on such spaces are, for instance, required to embed conditional probability distributi…
This paper develops a general framework for metric learning in RKHS with theoretical guarantees.
problem Learning a metric in RKHS from triplet comparisons.
method Develops a general RKHS framework for metric learning with theoretical guarantees.
result Provides novel generalization guarantees and sample complexity bounds for metric learning in RKHS.
New algorithms compute Koopman operators on RKHSs efficiently and accurately.
problem Data-driven spectral analysis of Koopman operators on RKHSs.
method General, provably convergent algorithms for RKHSs.
result Optimal algorithms with error control and spectral measures.
New framework assesses regularization norms in ill-posed problems, revealing L2 instability and proposing adaptive fractional RKHS solutions.
problem Comparative analysis of regularization norms in ill-posed problems.
method Small noise analysis framework for Tikhonov and RKHS regularizations.
result Optimal convergence rates achieved with adaptive fractional RKHS, but hyper-parameters decay too fast.
The report analyzes infinite-dimensional output space regression.
problem Learning theory in vector-valued RKHS regression.
method Integral operator technique with spectral theory for non-compact operators.
result Results with minimal assumptions using Chebyshev's inequality.
We study reproducing kernel Hilbert spaces (RKHS) on a Riemannian manifold. In particular, we discuss under which condition Sobolev spaces are RKHS and characterize their reproducing kernels. Further, we introduce and discuss a class of smoother RKHS that we call diffusion spaces. We illustrate the general results with…
Neural networks can approximate functionals on RKHS with error bounds.
problem Approximating functionals on RKHS using neural networks.
method Interpolating orthogonal projections in RKHS using point evaluations.
result Explicit error bounds for various kernels (inverse multiquadric, Gaussian, Sobolev).
This paper generalizes regularized regression problems in a hyper-reproducing kernel Hilbert space (hyper-RKHS), illustrates its utility for kernel learning and out-of-sample extensions, and proves asymptotic convergence results for the introduced regression models in an approximation theory view. Algorithmically, we c…
New approach to supervised learning in RKHS and vvRKHS using C∗-algebras.
problem Traditional supervised learning in RKHS and vvRKHS.
method Generalizing supervised learning to RKHM using C∗-algebras. result Constructing RKHMs with enhanced representation power.
Study evaluates RKHS choices for assessing graph models using KSD tests.
problem Effect of RKHS choice on KSD tests for graph model assessment.
method Investigated power performance and computational runtime of KSD tests for ERGMs and synthetic graph generators.
result Different RKHS choices affect KSD test performance and computational runtime.
Paper learns Koopman operator from sparse data, escaping function space constraints.
problem Learning Koopman operator from non-closed function spaces.
method Operator stochastic approximation algorithm using conditional mean embeddings (CME).
result Online sparse learning algorithm with trajectory-based sampling guarantees.