Kernel ridge regression (KRR) is a well-known and popular nonparametric regression approach with many desirable properties, including minimax rate-optimality in estimating functions that belong to common reproducing kernel Hilbert spaces (RKHS). The approach, however, is computationally intensive for large data sets, d…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “truncated RKHS”
Kernel quadrature with DPPsstat.ML
We study quadrature rules for functions from an RKHS, using nodes sampled from a determinantal point process (DPP). DPPs are parametrized by a kernel, and we use a truncated and saturated version of the RKHS kernel. This link between the two kernels, along with DPP machinery, leads to relatively tight bounds on the qua…
A method for interpreting SVMs using polynomial kernels, revealing model complexity.
problem Interpreting SVMs built with truncated orthogonal polynomial kernels.
method Orthogonal Representation Contribution Analysis (ORCA) with normalized Orthogonal Kernel Contribution (OKC) indices.
result The method reveals structural aspects of model complexity not captured by predictive accuracy.
New kernels capture both local and non-local interactions efficiently.
problem Designing kernels that capture both local and non-local interactions while remaining computationally tractable.
method Spectral truncation kernels based on -algebra.
result Spectral truncation kernels induce interactions across the data function domain and reduce computational cost.
Study improves hypothesis transfer learning for functional linear models.
problem Incompatible TL techniques for high-dimensional FLR methods due to infinite-dimensional nature of functional data.
method Proposes two algorithms for hypothesis transfer learning in RKHS framework, leveraging RKHS distance and aggregation techniques.
result Establishes asymptotic lower bounds and matching upper bounds for the proposed algorithms, demonstrating their effectiveness.
Optimally tackles covariate shift in RKHS-based nonparametric regression.
problem Covariate shift in nonparametric regression over RKHS.
method Two families of covariate shift problems defined using likelihood ratios. Minimax rate-optimal estimators for KRR and reweighted KRR.
result KRR is minimax rate-optimal and strictly sub-optimal compared to naive estimator under covariate shift.
Paper analyzes spectral algorithms under covariate shift, providing convergence rates.
problem Addressing distributional mismatch in regression models.
method Incorporates importance weights into spectral algorithms in RKHS.
result Establishes minimax-optimal convergence rates for misspecified cases.
In this paper, we consider the nonparametric least square regression in a Reproducing Kernel Hilbert Space (RKHS). We propose a new randomized algorithm that has optimal generalization error bounds with respect to the square loss, closing a long-standing gap between upper and lower bounds. Moreover, we show that our al…
New method approximates complex kernel norms with random features, making learning tractable.
problem Complexity of learning with kernel methods in high dimensions.
method Random features approximations to norms, focusing on .
result For , the number of random features required is polynomial in the sample size, making learning tractable.
New algorithms tackle RKHS bandits with reduced complexity and improved performance.
problem Adversarial and stochastic RKHS bandit problems with high computational complexity.
method Combining approximation theory with misspecified linear bandit methods.
result First general algorithm for adversarial RKHS bandit problem.
Constructs non-asymptotic confidence regions for unknown functions in RKHS.
problem Global probabilistic confidence regions for unknown functions in RKHS.
method Reduces confidence region construction to estimating RKHS norm.
result Valid confidence regions can be constructed non-asymptotically.
Reconstruction of a function from noisy data is often formulated as a regularized optimization problem over an infinite-dimensional reproducing kernel Hilbert space (RKHS). The solution describes the observed data and has a small RKHS norm. When the data fit is measured using a quadratic loss, this estimator has a know…
Deep neural kernels and Laplace kernel have equivalent RKHS on spheres.
problem Comparing RKHS of deep neural tangent and Laplace kernels.
method Proof of RKHS equivalence using sphere restrictions and kernel properties.
result RKHS of deep neural tangent kernel and Laplace kernel are the same on .
Positive weights improve kernel quadrature's accuracy.
problem Improving kernel quadrature weights to be positive and stable.
method Using convex geometry to approximate the kernel mean embedding with positive weights.
result Positive weights lead to improved kernel quadrature bounds with Monte-Carlo-beating rates.
Stacked Kernel Networkstat.ML
Kernel methods are powerful tools to capture nonlinear patterns behind data. They implicitly learn high (even infinite) dimensional nonlinear features in the Reproducing Kernel Hilbert Space (RKHS) while making the computation tractable by leveraging the kernel trick. Classic kernel methods learn a single layer of nonl…
A new kernel test reduces noise in MMD by focusing on leading eigen-directions.
problem Noise in trailing directional components degrades power of standard kernel two-sample tests.
method Truncate MMD spectral decomposition, retaining only leading eigen-directions.
result Our method achieves superior power and robustness, especially in high-dimensional and unbalanced settings.
Metric Learning in an RKHSstat.ML
This paper develops a general framework for metric learning in RKHS with theoretical guarantees.
problem Learning a metric in RKHS from triplet comparisons.
method Develops a general RKHS framework for metric learning with theoretical guarantees.
result Provides novel generalization guarantees and sample complexity bounds for metric learning in RKHS.
New framework assesses regularization norms in ill-posed problems, revealing L2 instability and proposing adaptive fractional RKHS solutions.
problem Comparative analysis of regularization norms in ill-posed problems.
method Small noise analysis framework for Tikhonov and RKHS regularizations.
result Optimal convergence rates achieved with adaptive fractional RKHS, but hyper-parameters decay too fast.
Enhances DGPs with adaptive RKHS Fourier features for better non-stationary pattern modeling.
problem Capturing complex non-stationary patterns in non-linear dynamical systems.
method Integrates ODE-based RKHS Fourier features into DGPs using convolution operations for adaptive amplitude and phase modulation. Uses a doubly stochastic variational inference framework.
result Improved predictive performance across various regression tasks.
Paper introduces RKHM for more explicit variable structures analysis.
problem Explicitly analyzing structures among variables.
method Orthonormal systems in Hilbert -modules, RKHM.
result Theoretical and practical procedures for RKHM orthonormalization.
This work analyzes the role of data augmentation in self-supervised learning using RKHS approximation and regression.
problem Limited theoretical understanding of the role of data augmentation in self-supervised learning.
method Geometric characterization of the target function given by augmentation, proving generalization bounds.
result Two generalization bounds are derived, one free of model complexity, the other specific to near-optimal encoders.
We study reproducing kernel Hilbert spaces (RKHS) on a Riemannian manifold. In particular, we discuss under which condition Sobolev spaces are RKHS and characterize their reproducing kernels. Further, we introduce and discuss a class of smoother RKHS that we call diffusion spaces. We illustrate the general results with…
Neural networks can approximate functionals on RKHS with error bounds.
problem Approximating functionals on RKHS using neural networks.
method Interpolating orthogonal projections in RKHS using point evaluations.
result Explicit error bounds for various kernels (inverse multiquadric, Gaussian, Sobolev).
This paper generalizes regularized regression problems in a hyper-reproducing kernel Hilbert space (hyper-RKHS), illustrates its utility for kernel learning and out-of-sample extensions, and proves asymptotic convergence results for the introduced regression models in an approximation theory view. Algorithmically, we c…
New approach to supervised learning in RKHS and vvRKHS using -algebras.
problem Traditional supervised learning in RKHS and vvRKHS.
method Generalizing supervised learning to RKHM using -algebras.
result Constructing RKHMs with enhanced representation power.
Study evaluates RKHS choices for assessing graph models using KSD tests.
problem Effect of RKHS choice on KSD tests for graph model assessment.
method Investigated power performance and computational runtime of KSD tests for ERGMs and synthetic graph generators.
result Different RKHS choices affect KSD test performance and computational runtime.
A new method for learning function parameters in operators using data-adaptive RKHS.
problem Learning function parameters in operators with robustness to noise and numerical error.
method Data Adaptive RKHS Tikhonov Regularization (DARTR) method.
result DARTR leads to an accurate estimator robust to noise and numerical error, converging at a consistent rate as data refines.
New analysis shows a gap between Gaussian RKHS and neural networks on unbounded domains.
problem Understanding the function space bias of neural networks compared to Gaussian RKHS.
method Infinite-center asymptotic analysis of neural network Banach space and Gaussian RKHS on unbounded domains.
result Certain functions in Gaussian RKHS have infinite norm in neural network Banach space on unbounded domains.
Study online learning in RKHS with dependent processes, focusing on \(β\)- and \(φ\)-mixing.
problem Online learning in RKHS with dependent data.
method Online regularized learning algorithm in RKHS, analyzing \(β\)- and \(φ\)-mixing sequences.
result Probabilistic upper bounds and convergence rates for mixing coefficients.
Wide neural networks can outperform kernel methods in certain tasks.
problem Understanding when neural networks outperform kernel methods in classification tasks.
method Analyzing the performance of wide neural networks and kernel methods on various tasks, considering the initialization of SGD and the structure of covariates.
result Wide neural networks can outperform kernel methods in tasks where covariates have a low-dimensional structure similar to the target function.
Paper explores RKHS properties for derivative and integral operators.
problem Establishing sufficient conditions for reproducing property in RKHS.
method Establishing reproducing property for combinations of composition operators.
result Provides framework for regularized learning algorithms involving function values, gradients, or operators.
The problem of an arbitrary truncated Levy flight description using the method of cumulant approach has been solved. The set of cumulants of the truncated Levy distribution given the assumption of arbitrary truncation has been found. The influence of truncation shape on the truncated Levy flight properties in the Gauss…
Embedding RL policies in RKHS for robustness and theoretical guarantees.
problem Stability and theoretical guarantees in RL policy representation.
method Low-dimensional embedding of RL policies in RKHS.
result Embedded policies maintain high return with strong theoretical guarantees.
Learn ODEs from noisy data using RKHS and optimization.
problem Learning nonparametric ODEs from noisy data.
method Using RKHS theory, solve a constrained optimization problem iteratively with penalty methods and Euler approximations.
result Prove a generalization bound for L2 distance between true and estimated solutions.
We consider black box optimization of an unknown function in the nonparametric Gaussian process setting when the noise in the observed function values can be heavy tailed. This is in contrast to existing literature that typically assumes sub-Gaussian noise distributions for queries. Under the assumption that the unknow…
Efficiently estimate Boolean product distribution parameters from truncated samples.
problem Estimating parameters of Boolean product distributions from truncated samples.
method Introducing fatness of truncation set, using membership queries, and adapting Stochastic Gradient Descent.
result Efficiently learn Boolean product distributions from truncated samples with small sample complexity.
In the paper "On Truncated Variation of Brownian Motion with Drift" (Bull. Pol. Acad. Sci. Math. 56 (2008), no.4, 267 - 281) we defined truncated variation of Brownian motion with drift, where is a standard Brownian motion. Truncated variation differs from regular variation by neglect…
Fast algorithms for learning a Gaussian under halfspace truncation with optimal sample complexitycs.DS
Optimal algorithm learns Gaussian under halfspace truncation with minimal samples.
problem Learning a Gaussian distribution truncated to an unknown halfspace.
method Efficient algorithm using samples and runtime dominated by empirical covariance matrix computation.
result Optimal sample and time complexity bounds for learning a Gaussian under halfspace truncation.
Develops RKHS framework for analyzing tree ensembles.
problem Analyzing the theoretical properties of tree ensembles.
method Reproducing Kernel Hilbert Spaces (RKHS) for tree ensembles.
result Characterizes Random Forest predictor as minimizer of a penalized empirical risk functional in RKHS.
Adapts RKHS methods to estimate density ratios with optimal error.
problem Estimating density ratios from limited data.
method Minimizes regularized Bregman divergence in RKHS, with Lepskii type parameter choice.
result Adaptive minimax optimal error rate for quadratic loss.
New method for constructing truncated vine copulas.
problem High-dimensional parameter space in vine copulas.
method Propose a new score and algorithm for constructing truncated vines.
result New algorithms exploit conditional independences.
Non-negative matrix factorization (NMF) minimizes the Euclidean distance between the data matrix and its low rank approximation, and it fails when applied to corrupted data because the loss function is sensitive to outliers. In this paper, we propose a Truncated CauchyNMF loss that handle outliers by truncating large e…
Study on how sampling works for complex data functions.
problem Analyzing convergence of sampling algorithms for RKHS functions.
method Minimalistic assumptions on kernel and data, error estimates in RKHS norm, uniform convergence on compact domains.
result New convergence rates for Lipschitz and Hölder continuous kernels.
This paper improves signal reconstruction using determinantal sampling from random nodes.
problem Approximating square-integrable functions from random node evaluations.
method Combines determinantal point processes and mixtures thereof for RKHS-adapted approximations.
result Proves mean-square guarantees in norm and shows faster convergence rates.
New algorithm optimizes functions in Matérn kernel RKHS with noisy feedback.
problem Optimizing functions in RKHS of Matérn kernel with noisy bandit feedback.
method π-GP-UCB algorithm with guaranteed sublinear regret for all ν > 1 and d ≥ 1.
result First practical approach with guaranteed sublinear regret for all ν > 1 and d ≥ 1.
Kernel -Greedy optimizes multi-armed bandits with covariates for sub-linear regret.
problem Optimizing multi-armed bandits with covariates in a reproducing kernel Hilbert space.
method Online weighted kernel ridge regression estimator for mean reward function estimation.
result Achieves sub-linear regret rate and optimal regret rate under margin condition.
Improved GP bandit algorithms for noiseless, varying noise, and RKHS norms.
problem Minimizing regret in Gaussian process bandits with unknown reward functions.
method New upper bound on maximum posterior variance, refined MVR and PE algorithms.
result Optimal regret bounds for noiseless, varying noise, and RKHS norms.
Despite the fundamental nature of the inhomogeneous Poisson process in the theory and application of stochastic processes, and its attractive generalizations (e.g. Cox process), few tractable nonparametric modeling approaches of intensity functions exist, especially when observed points lie in a high-dimensional space.…