Develops a framework for learning nonlinear operators using Mercer kernels.
problem Learning nonlinear operators between infinite-dimensional spaces.
method Stochastic approximation framework with Mercer operator-valued kernels.
result Establishes dimension-free polynomial convergence rates for nonlinear operator learning.
New kernels allow learning from non-separable data.
problem Learning from non-separable data.
method Introducing entangled kernels and a two-step algorithm.
result Efficient algorithm for learning entangled kernels.
Positive definite operator-valued kernels generalize the well-known notion of reproducing kernels, and are naturally adapted to multi-output learning situations. This paper addresses the problem of learning a finite linear combination of infinite-dimensional operator-valued kernels which are suitable for extending func…
Deep learning framework for kernel methods using RKHM and Perron-Frobenius operators.
problem Kernel methods in deep learning with potential overfitting issues.
method Combining RKHM and Perron-Frobenius operator to derive a new Rademacher bound and analyze deep kernel methods.
result Theoretical interpretation of benign overfitting and milder dependency on output dimension.
Devoted to multi-task learning and structured output learning, operator-valued kernels provide a flexible tool to build vector-valued functions in the context of Reproducing Kernel Hilbert Spaces. To scale up these methods, we extend the celebrated Random Fourier Feature methodology to get an approximation of operator-…
Optimizes learning Hilbert-Schmidt operators between Sobolev spaces.
problem Statistical limits of learning mappings between infinite-dimensional function spaces.
method Minimax optimal regularization and multilevel training.
result Multilevel kernel operator learning achieves optimal learning rate.
Kernel method learns PDEs from noisy data.
problem Discovering and solving PDEs from noisy data.
method Kernel smoothing, regression, and operator learning.
result Competitive performance compared to state-of-the-art algorithms.
Random features improve neural operators' generalization properties.
problem Improving generalization of neural operators.
method Unified framework for spectral regularization techniques and operator-valued kernels.
result Established optimal learning rates and required number of neurons.
New method learns kernels in nonlocal operators robustly.
problem Learning kernels in nonlocal operators is ill-posed.
method Nonparametric regression with Tikhonov regularization.
result Robust estimator of kernel yields homogenized model.
Active data collection improves convergence rates in operator learning.
problem Improving convergence rates in operator learning with linear target and stochastic input.
method Active data collection strategies with mean-zero stochastic process and continuous covariance kernels.
result Achieves arbitrarily fast error convergence rates with eigenvalue decay of covariance kernels.
Random feature method approximates operators with theoretical guarantees and reduced computation.
problem Approximating operators between infinite dimensional Banach spaces using machine learning.
method Random feature operator learning method with theoretical guarantees and error bounds.
result The random feature method can achieve similar or better test errors than kernel-based methods and neural networks with significantly reduced training times.
Nyström approximation for scalable operator learning
problem Scalability of operator learning for large datasets
method Nyström subsampling with operator learning
result Minimax-optimal convergence rates for functional outputs
We consider the problem of learning a vector-valued function f in an online learning setting. The function f is assumed to lie in a reproducing Hilbert space of operator-valued kernels. We describe two online algorithms for learning f while taking into account the output structure. A first contribution is an algorithm,…
Kernel methods outperform neural nets in operator learning tasks.
problem Learning operators between Banach spaces from partial observations.
method Kernel-based framework with a priori error analysis and numerical comparisons.
result Kernel methods are competitive with neural nets in cost-accuracy trade-off.
To the best of our knowledge, there are no general well-founded robust methods for statistical unsupervised learning. Most of the unsupervised methods explicitly or implicitly depend on the kernel covariance operator (kernel CO) or kernel cross-covariance operator (kernel CCO). They are sensitive to contaminated data, …
We study the problem of structured output learning from a regression perspective. We first provide a general formulation of the kernel dependency estimation (KDE) problem using operator-valued kernels. We show that some of the existing formulations of this problem are special cases of our framework. We then propose a c…
Scalable Gaussian Process Operator tackles high-dimensional PDEs.
problem Scaling Gaussian Process Operators to high-dimensional, data-intensive regimes.
method Nearest-neighbor-based local kernel approximations, sparse kernel approximation, structured Kronecker factorizations, operator-aware kernel structures, task-informed mean functions.
result Consistently achieves high accuracy across varying discretization scales.
A new data-adaptive prior stabilizes kernel learning in operators.
problem Learning kernels in operators from data is ill-posed due to nonlocal dependence.
method Introduces a data-adaptive prior to stabilize the Bayesian posterior mean.
result The data-adaptive prior achieves a stable posterior with small noise limits.
New kernels capture both local and non-local interactions efficiently.
problem Designing kernels that capture both local and non-local interactions while remaining computationally tractable.
method Spectral truncation kernels based on C∗-algebra. result Spectral truncation kernels induce interactions across the data function domain and reduce computational cost.
Quantum kernel machines need to use more complex kernels to fully exploit their potential.
problem Current quantum kernels struggle with complex learning tasks due to limited degrees of freedom.
method Propose using operator-valued kernels and C∗-algebraic representations to enhance quantum kernels. result Quantum operator-valued kernels can reveal structural dependencies that scalar-valued kernels miss.
Complex analysis techniques link Gaussian RBF kernels to quantum mechanics.
problem Understanding the Gaussian RBF kernel in machine learning and SVMs.
method Using Fock space and Segal-Bargmann theories in complex analysis.
result Proves connections between Gaussian RBF kernels and quantum mechanics operators.
We study the stability properties of nonlinear multi-task regression in reproducing Hilbert spaces with operator-valued kernels. Such kernels, a.k.a. multi-task kernels, are appropriate for learning prob- lems with nonscalar outputs like multi-task learning and structured out- put prediction. We show that multi-task ke…
Kernel-based Bayesian filter for nonlinear systems using infinite-dimensional operators.
problem Modeling and predicting nonlinear dynamical systems.
method Functional Bayesian perspective, reproducing kernel Hilbert space, Gaussian kernel.
result Effective approximation and accurate results for nonlinear systems.
Reproducing kernel Hilbert spaces (RKHSs) play an important role in many statistics and machine learning applications ranging from support vector machines to Gaussian processes and kernel embeddings of distributions. Operators acting on such spaces are, for instance, required to embed conditional probability distributi…
MCNO learns PDE solution operators using Monte Carlo sampling.
problem Learning solution operators for PDEs efficiently and flexibly.
method Directly learns kernel function using Monte Carlo sampling of input-output pairs.
result Competitive accuracy with efficient computational cost on 1D PDE benchmarks.
Kernel transfer operators, which can be regarded as approximations of transfer operators such as the Perron-Frobenius or Koopman operator in reproducing kernel Hilbert spaces, are defined in terms of covariance and cross-covariance operators and have been shown to be closely related to the conditional mean embedding fr…
Many unsupervised kernel methods rely on the estimation of the kernel covariance operator (kernel CO) or kernel cross-covariance operator (kernel CCO). Both kernel CO and kernel CCO are sensitive to contaminated data, even when bounded positive definite kernels are used. To the best of our knowledge, there are few well…
Paper tackles conditional expectation estimation using compactification operators.
problem Estimating conditional expectations from product of two random variables.
method Operator theoretic approach using kernel integral operators in reproducing kernel Hilbert space.
result Solutions allow numerical approximation and convergence of data-driven implementations.
Although operator-valued kernels have recently received increasing interest in various machine learning and functional data analysis problems such as multi-task learning or functional regression, little attention has been paid to the understanding of their associated feature spaces. In this paper, we explore the potent…
Kernel Dynamic Mode Decomposition reconstructs dynamical systems using Laplacian kernel.
problem Reconstructing spatial-temporal dynamics of complex systems.
method Kernel Dynamic Mode Decomposition with Laplacian kernel.
result Laplacian kernel allows for the closability of Koopman operators in RKHS, enabling reconstruction.
RKUM is an R package for robust kernel-based unsupervised methods.
problem Robust analysis under contaminated or noisy data conditions.
method Robust kernel covariance and cross-covariance operators using generalized loss functions.
result RKUM reduces sensitivity to contamination and effectively identifies outliers.
Automates GPU kernel optimization for diverse applications.
problem Lack of systematic evaluation for multi-scenario GPU kernel optimization.
method Introduces MSKernelBench and CUDAMaster for automated optimization.
result Significant speedups across various operators, outperforming existing tools.
MetaNOR learns common nonlocal kernels for efficient metamaterial modeling.
problem Efficiently modeling wave propagation in new metamaterials.
method Meta-learns a common nonlocal kernel from existing tasks and transfers this knowledge to new tasks with minimal data.
result Substantial improvements in sampling efficiency for new metamaterials.
A new algorithm for high-dimensional hedging problems.
problem High-dimensional, path-dependent hedging problems.
method Signature-based algorithm using operator-valued kernels and geometric rough paths.
result Theoretical guarantees on existence and uniqueness of a global minimum.
Neural operators achieve fast convergence rates for solving PDEs.
problem Solving partial differential equations (PDEs) efficiently.
method Two-layer neural operators with gradient descent analysis in RKHS.
result Fast convergence rates are minimax optimal for early-stopped GD.
Transfer operators such as the Perron--Frobenius or Koopman operator play an important role in the global analysis of complex dynamical systems. The eigenfunctions of these operators can be used to detect metastable sets, to project the dynamics onto the dominant slow processes, or to separate superimposed signals. We …
Formula for Toeplitz operator kernel on CR manifolds.
problem Analyzing Toeplitz operators on CR manifolds.
method Formula for the symbol of the kernel, asymptotic expansions.
result Formula for the values at the diagonal of the second coefficient in the expansion of the symbol of the kernel.
Kernel VICReg improves SSL in RKHS, capturing nonlinear structures.
problem Limited ability of existing SSL methods to handle nonlinear dependencies.
method Kernel VICReg framework in RKHS, kernelizing VICReg objectives.
result Kernel VICReg mitigates representational collapse and improves performance.
Paper explores RKHS properties for derivative and integral operators.
problem Establishing sufficient conditions for reproducing property in RKHS.
method Establishing reproducing property for combinations of composition operators.
result Provides framework for regularized learning algorithms involving function values, gradients, or operators.
The paper reformulates regression in infinite dimensions as an inverse problem, showing it's equivalent to compact inverse problems.
problem Learning a linear operator between Hilbert spaces from empirical observations.
method Reformulates regression as an inverse problem, proving equivalence to compact inverse problems under specific conditions.
result The inverse problem is equivalent to compact inverse problems in terms of spectral properties and regularisation theory.
Convolution and pooling improve kernel methods in image classification.
problem Understanding the interplay between approximation and generalization in convolutional architectures.
method Characterized RKHS of kernels with convolution, pooling, and downsampling, computed generalization error.
result Convolution and pooling operations trade off approximation with generalization power.
Study uses SGD to learn operators in Hilbert spaces with convergence analysis.
problem Learning operators in general Hilbert spaces with SGD.
method Proposes weak and strong regularity conditions for convergence analysis.
result SGD converges to best linear approximation of nonlinear operators.
Paper learns Koopman operator from sparse data, escaping function space constraints.
problem Learning Koopman operator from non-closed function spaces.
method Operator stochastic approximation algorithm using conditional mean embeddings (CME).
result Online sparse learning algorithm with trajectory-based sampling guarantees.
In this paper we consider the problems of supervised classification and regression in the case where attributes and labels are functions: a data is represented by a set of functions, and the label is also a function. We focus on the use of reproducing kernel Hilbert space theory to learn from such functional data. Basi…
A new method estimates SDEs using occupation kernels.
problem Learning multivariate stochastic differential equations (SDEs).
method Two-step procedure: estimate drift, then diffusion. Occupation kernels used in RKHS.
result Validated on simulated and real-world data.
This work extends Gaussian process priors to neural operators for function space mappings.
problem Improving uncertainty quantification in deep neural networks.
method Extending Gaussian process priors to neural operators with conditions for convergence and computation of covariance functions.
result Arbitrary-depth neural operators with Gaussian kernels converge to function-valued GPs, enabling posterior computation in regression scenarios.
A new distance metric compares probability distributions using kernel covariance operators.
problem Comparing probability distributions in machine learning tasks.
method Introduces a novel distance metric based on Schatten norm of kernel covariance operators.
result The new distance metric is more discriminative and robust to hyperparameters.
The Fredholm integral equation of the first kind improves solutions for ill-posed supervised learning problems with limited data.
problem Ill-posed supervised learning problems with insufficient data.
method Using the Fredholm integral equation of the first kind (FIFK) with semi-supervised assumptions and MSDF methods.
result Improved accuracy and stability in solutions for ill-posed problems.