A Hilbert space embedding for probability measures has recently been proposed, wherein any probability measure is represented as a mean element in a reproducing kernel Hilbert space (RKHS). Such an embedding has found applications in homogeneity testing, independence testing, dimensionality reduction, etc., with the re…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “universal RKHS”
We study the approximation properties of random ReLU features through their reproducing kernel Hilbert space (RKHS). We first prove a universality theorem for the RKHS induced by random features whose feature maps are of the form of nodes in neural networks. The universality result implies that the random ReLU features…
Neural networks can approximate functionals on RKHS with error bounds.
problem Approximating functionals on RKHS using neural networks.
method Interpolating orthogonal projections in RKHS using point evaluations.
result Explicit error bounds for various kernels (inverse multiquadric, Gaussian, Sobolev).
Paper introduces RKHM and KME for richer data analysis.
problem Lack of rich data structures in kernel methods.
method Proposes RKHM and KME for functional data analysis.
result RKHM captures structural properties in functional data.
We present a universal algorithm for online trading in Stock Market which performs asymptotically at least as good as any stationary trading strategy that computes the investment at each step using a fixed function of the side information that belongs to a given RKHS (Reproducing Kernel Hilbert Space). Using a universa…
ULFS-KDPE estimates parameters efficiently without influence functions.
problem Estimating pathwise differentiable parameters in nonparametric models.
method Kernel debiased plug-in estimator based on universal least favorable submodel.
result Semiparametric efficiency achieved without influence function derivation.
Develops RKHS framework for analyzing tree ensembles.
problem Analyzing the theoretical properties of tree ensembles.
method Reproducing Kernel Hilbert Spaces (RKHS) for tree ensembles.
result Characterizes Random Forest predictor as minimizer of a penalized empirical risk functional in RKHS.
A short note on extension theorems and their connection to universal consistency in machine learningstat.ML
Statistical machine learning plays an important role in modern statistics and computer science. One main goal of statistical machine learning is to provide universally consistent algorithms, i.e., the estimator converges in probability or in some stronger sense to the Bayes risk or to the Bayes decision function. Kerne…
Researchers improve Gaussian processes to model inconsistent preferences.
problem Model inconsistent preferences and clusters of comparable items.
method Generalized Gaussian processes with spectral decomposition and universal RKHS.
result Competitive with state-of-the-art methods on simulated and real-world data.
Kernel DRO uses RKHS to optimize under distributional uncertainty.
problem Optimizing under distributional uncertainty with limited knowledge.
method Kernel DRO using RKHS ambiguity sets and duality theory.
result Unified approach to robust and stochastic optimization.
Kernel mean embeddings have recently attracted the attention of the machine learning community. They map measures from some set to functions in a reproducing kernel Hilbert space (RKHS) with kernel . The RKHS distance of two mapped measures is a semi-metric over . We study three questions. (I) For a…
Survey of kernels, RKHS, and their applications in machine learning.
problem Understanding kernels and their applications in machine learning.
method Review of historical context, mathematical definitions, and practical applications of kernels.
result Comprehensive overview of kernels, RKHS, and their applications.
Geometric theory connects machine learning classifiers to differential geometry.
problem Classifying data points in machine learning.
method Mapping binary classification to vector bundles and differential geometry.
result Harmonic interpolation solves RKHS interpolation problems.
New algorithms tackle RKHS bandits with reduced complexity and improved performance.
problem Adversarial and stochastic RKHS bandit problems with high computational complexity.
method Combining approximation theory with misspecified linear bandit methods.
result First general algorithm for adversarial RKHS bandit problem.
Constructs non-asymptotic confidence regions for unknown functions in RKHS.
problem Global probabilistic confidence regions for unknown functions in RKHS.
method Reduces confidence region construction to estimating RKHS norm.
result Valid confidence regions can be constructed non-asymptotically.
Study the cost of overfitting in noisy KRR models.
problem Cost of overfitting in noisy kernel ridge regression.
method An agnostic view of overfitting cost as a function of sample size for any target function, using Gaussian universality ansatz and task eigenstructure.
result Characterization of benign, tempered, and catastrophic overfitting.
Reconstruction of a function from noisy data is often formulated as a regularized optimization problem over an infinite-dimensional reproducing kernel Hilbert space (RKHS). The solution describes the observed data and has a small RKHS norm. When the data fit is measured using a quadratic loss, this estimator has a know…
Deep neural kernels and Laplace kernel have equivalent RKHS on spheres.
problem Comparing RKHS of deep neural tangent and Laplace kernels.
method Proof of RKHS equivalence using sphere restrictions and kernel properties.
result RKHS of deep neural tangent kernel and Laplace kernel are the same on .
Stacked Kernel Networkstat.ML
Kernel methods are powerful tools to capture nonlinear patterns behind data. They implicitly learn high (even infinite) dimensional nonlinear features in the Reproducing Kernel Hilbert Space (RKHS) while making the computation tractable by leveraging the kernel trick. Classic kernel methods learn a single layer of nonl…
Paper generalizes kernel mean embedding to von Neumann-algebra-valued measures.
problem Analyzing complex multivariate distributions and quantum mechanics.
method Generalizes kernel mean embedding to von Neumann-algebra-valued measures in reproducing kernel Hilbert modules.
result Injectivity and universality of the generalized KME are confirmed.
Metric Learning in an RKHSstat.ML
This paper develops a general framework for metric learning in RKHS with theoretical guarantees.
problem Learning a metric in RKHS from triplet comparisons.
method Develops a general RKHS framework for metric learning with theoretical guarantees.
result Provides novel generalization guarantees and sample complexity bounds for metric learning in RKHS.
New framework assesses regularization norms in ill-posed problems, revealing L2 instability and proposing adaptive fractional RKHS solutions.
problem Comparative analysis of regularization norms in ill-posed problems.
method Small noise analysis framework for Tikhonov and RKHS regularizations.
result Optimal convergence rates achieved with adaptive fractional RKHS, but hyper-parameters decay too fast.
Enhances DGPs with adaptive RKHS Fourier features for better non-stationary pattern modeling.
problem Capturing complex non-stationary patterns in non-linear dynamical systems.
method Integrates ODE-based RKHS Fourier features into DGPs using convolution operations for adaptive amplitude and phase modulation. Uses a doubly stochastic variational inference framework.
result Improved predictive performance across various regression tasks.
We study reproducing kernel Hilbert spaces (RKHS) on a Riemannian manifold. In particular, we discuss under which condition Sobolev spaces are RKHS and characterize their reproducing kernels. Further, we introduce and discuss a class of smoother RKHS that we call diffusion spaces. We illustrate the general results with…
This work analyzes the role of data augmentation in self-supervised learning using RKHS approximation and regression.
problem Limited theoretical understanding of the role of data augmentation in self-supervised learning.
method Geometric characterization of the target function given by augmentation, proving generalization bounds.
result Two generalization bounds are derived, one free of model complexity, the other specific to near-optimal encoders.
KKR uses Koopman theory to improve forecasting in complex systems.
problem Forecasting complex, nonlinear dynamical systems in decision-making.
method Derives a universal Koopman-invariant RKHS for LTI dynamical systems.
result KKR framework provides convergence results and generalization error bounds.
Kernel methods have been among the most popular techniques in machine learning, where learning tasks are solved using the property of reproducing kernel Hilbert space (RKHS). In this paper, we propose a novel data analysis framework with reproducing kernel Hilbert -module (RKHM), which is another generalization of…
This paper generalizes regularized regression problems in a hyper-reproducing kernel Hilbert space (hyper-RKHS), illustrates its utility for kernel learning and out-of-sample extensions, and proves asymptotic convergence results for the introduced regression models in an approximation theory view. Algorithmically, we c…
New approach to supervised learning in RKHS and vvRKHS using -algebras.
problem Traditional supervised learning in RKHS and vvRKHS.
method Generalizing supervised learning to RKHM using -algebras.
result Constructing RKHMs with enhanced representation power.
Study evaluates RKHS choices for assessing graph models using KSD tests.
problem Effect of RKHS choice on KSD tests for graph model assessment.
method Investigated power performance and computational runtime of KSD tests for ERGMs and synthetic graph generators.
result Different RKHS choices affect KSD test performance and computational runtime.
A new method for learning function parameters in operators using data-adaptive RKHS.
problem Learning function parameters in operators with robustness to noise and numerical error.
method Data Adaptive RKHS Tikhonov Regularization (DARTR) method.
result DARTR leads to an accurate estimator robust to noise and numerical error, converging at a consistent rate as data refines.
New analysis shows a gap between Gaussian RKHS and neural networks on unbounded domains.
problem Understanding the function space bias of neural networks compared to Gaussian RKHS.
method Infinite-center asymptotic analysis of neural network Banach space and Gaussian RKHS on unbounded domains.
result Certain functions in Gaussian RKHS have infinite norm in neural network Banach space on unbounded domains.
Study online learning in RKHS with dependent processes, focusing on \(β\)- and \(φ\)-mixing.
problem Online learning in RKHS with dependent data.
method Online regularized learning algorithm in RKHS, analyzing \(β\)- and \(φ\)-mixing sequences.
result Probabilistic upper bounds and convergence rates for mixing coefficients.
Wide neural networks can outperform kernel methods in certain tasks.
problem Understanding when neural networks outperform kernel methods in classification tasks.
method Analyzing the performance of wide neural networks and kernel methods on various tasks, considering the initialization of SGD and the structure of covariates.
result Wide neural networks can outperform kernel methods in tasks where covariates have a low-dimensional structure similar to the target function.
Paper explores RKHS properties for derivative and integral operators.
problem Establishing sufficient conditions for reproducing property in RKHS.
method Establishing reproducing property for combinations of composition operators.
result Provides framework for regularized learning algorithms involving function values, gradients, or operators.
Learn ODEs from noisy data using RKHS and optimization.
problem Learning nonparametric ODEs from noisy data.
method Using RKHS theory, solve a constrained optimization problem iteratively with penalty methods and Euler approximations.
result Prove a generalization bound for L2 distance between true and estimated solutions.
Adapts RKHS methods to estimate density ratios with optimal error.
problem Estimating density ratios from limited data.
method Minimizes regularized Bregman divergence in RKHS, with Lepskii type parameter choice.
result Adaptive minimax optimal error rate for quadratic loss.
Study on how sampling works for complex data functions.
problem Analyzing convergence of sampling algorithms for RKHS functions.
method Minimalistic assumptions on kernel and data, error estimates in RKHS norm, uniform convergence on compact domains.
result New convergence rates for Lipschitz and Hölder continuous kernels.
This paper improves signal reconstruction using determinantal sampling from random nodes.
problem Approximating square-integrable functions from random node evaluations.
method Combines determinantal point processes and mixtures thereof for RKHS-adapted approximations.
result Proves mean-square guarantees in norm and shows faster convergence rates.
Kernel quadrature with DPPsstat.ML
We study quadrature rules for functions from an RKHS, using nodes sampled from a determinantal point process (DPP). DPPs are parametrized by a kernel, and we use a truncated and saturated version of the RKHS kernel. This link between the two kernels, along with DPP machinery, leads to relatively tight bounds on the qua…
Kernel -Greedy optimizes multi-armed bandits with covariates for sub-linear regret.
problem Optimizing multi-armed bandits with covariates in a reproducing kernel Hilbert space.
method Online weighted kernel ridge regression estimator for mean reward function estimation.
result Achieves sub-linear regret rate and optimal regret rate under margin condition.
Improved GP bandit algorithms for noiseless, varying noise, and RKHS norms.
problem Minimizing regret in Gaussian process bandits with unknown reward functions.
method New upper bound on maximum posterior variance, refined MVR and PE algorithms.
result Optimal regret bounds for noiseless, varying noise, and RKHS norms.
Despite the fundamental nature of the inhomogeneous Poisson process in the theory and application of stochastic processes, and its attractive generalizations (e.g. Cox process), few tractable nonparametric modeling approaches of intensity functions exist, especially when observed points lie in a high-dimensional space.…
Usually, complex-valued RKHS are presented as an straightforward application of the real-valued case. In this paper we prove that this procedure yields a limited solution for regression. We show that another kernel, here denoted as pseudo kernel, is needed to learn any function in complex-valued fields. Accordingly, we…
Develops a new framework for estimating joint probability distributions.
problem Estimating joint probability distributions from large sample sizes.
method Tensor product reproducing kernel Hilbert spaces (RKHS) with normalized and positive model.
result Fast computation and applicability to prediction and classification problems.
Research aims to improve confidence intervals for RKHS elements in online learning.
problem Improper confidence intervals lead to suboptimal regret bounds in kernel-based bandit and reinforcement learning.
method Formalizes the open problem of online confidence intervals in RKHS and reviews existing results.
result Identifies the online nature of observation points as the main challenge for tight confidence intervals.
Kernel Bayes' rulestat.ML
A nonparametric kernel-based method for realizing Bayes' rule is proposed, based on representations of probabilities in reproducing kernel Hilbert spaces. Probabilities are uniquely characterized by the mean of the canonical map to the RKHS. The prior and conditional probabilities are expressed in terms of RKHS functio…
Proposes CCME framework for estimating heterogeneous treatment effects.
problem Estimating heterogeneous treatment effects in complex distributions.
method Embeds conditional distributions into RKHS, develops meta-estimators for CCME.
result Establishes finite-sample convergence rates and double robustness for CCME estimators.