Deep networks are shown to be equivalent to a new type of kernel chain.
problem Identifying an appropriate function space for deep neural networks.
method Extending Reproducing Kernel Banach Spaces (RKBS) to chain RKBS (cRKBS), which composes kernels rather than functions.
result Any deep neural network function is a neural cRKBS function, and conversely, any neural cRKBS function corresponds to a deep neural network.
Study Markov chain gradient descent in Hilbert spaces for quadratic loss.
problem Approximating optimal solutions for quadratic loss functions.
method Developed a Markov chain-based stochastic gradient algorithm in Hilbert spaces.
result Established probabilistic upper bounds on convergence.
Stein importance sampling is a widely applicable technique based on kernelized Stein discrepancy, which corrects the output of approximate sampling algorithms by reweighting the empirical distribution of the samples. A general analysis of this technique is conducted for the previously unconsidered setting where samples…
Kernel-based methods improve policy evaluation in MRP models.
problem Estimating value functions in infinite-horizon discounted MRP models.
method Kernel-based temporal difference methods using reproducing kernel Hilbert spaces.
result Optimal error bounds derived for the kernel-based LSTD estimate.
Kernel thinning compresses distributions more effectively than i.i.d. sampling or standard thinning.
problem Efficiently compressing distributions for better sampling and integration accuracy.
method Introduces kernel thinning, a procedure that compresses an n-point approximation of a distribution into a sqrt(n)-point approximation with comparable integration error.
result Kernel thinning achieves a maximum discrepancy in integration error of O_d(n^(-1/2) sqrt(log n)) in probability for compactly supported distributions and O_d(n^(-1/2) (log n)^(d+1/2) sqrt(log log n)) for sub-exponential distributions.
We propose kernel sequential Monte Carlo (KSMC), a framework for sampling from static target densities. KSMC is a family of sequential Monte Carlo algorithms that are based on building emulator models of the current particle system in a reproducing kernel Hilbert space. We here focus on modelling nonlinear covariance s…
New rates for GLD and SGLD in infinite-dimensional spaces without dimensionality issues.
problem Gradient Langevin dynamics and SGLD convergence rates in high-dimensional spaces.
method Analysis of GLD and SGLD in infinite-dimensional Hilbert spaces, using stochastic differential equations and Markov chains.
result Derivation of dimension-free convergence rates for GLD and SGLD.
In this paper we solve support vector machines in reproducing kernel Banach spaces with reproducing kernels defined on nonsymmetric domains instead of the traditional methods in reproducing kernel Hilbert spaces. Using the orthogonality of semi-inner-products, we can obtain the explicit representations of the dual (nor…
Study online learning in RKHS with dependent processes, focusing on \(β\)- and \(φ\)-mixing.
problem Online learning in RKHS with dependent data.
method Online regularized learning algorithm in RKHS, analyzing \(β\)- and \(φ\)-mixing sequences.
result Probabilistic upper bounds and convergence rates for mixing coefficients.
Kernel methods are studied in a mean field limit for high-dimensional data.
problem Analyzing kernel methods in high-dimensional data with many variables.
method Investigation of kernel methods in the mean field limit of interacting particle systems.
result Rigorous mean field limit of kernels and detailed analysis of the limiting reproducing kernel Hilbert space.
This paper extends mirror descent to Banach spaces with reproducing kernels.
problem Optimizing in Banach spaces with reproducing kernels.
method Mirror descent algorithm adapted for Banach spaces with reproducing kernels.
result Mirror descent achieves linear convergence in certain conditions and standard convergence in a constrained setting.
We propose a nonparametric statistical test for goodness-of-fit: given a set of samples, the test determines how likely it is that these were generated from a target density function. The measure of goodness-of-fit is a divergence constructed via Stein's method using functions from a Reproducing Kernel Hilbert Space. O…
We study reproducing kernel Hilbert spaces (RKHS) on a Riemannian manifold. In particular, we discuss under which condition Sobolev spaces are RKHS and characterize their reproducing kernels. Further, we introduce and discuss a class of smoother RKHS that we call diffusion spaces. We illustrate the general results with…
We construct a canonical correspondence from a wide class of reproducing kernels on infinite-dimensional Hermitian vector bundles to linear connections on these bundles. The linear connection in question is obtained through a pull-back operation involving the tautological universal bundle and the classifying morphism o…
Approximate Markov chain Monte Carlo (MCMC) offers the promise of more rapid sampling at the cost of more biased inference. Since standard MCMC diagnostics fail to detect these biases, researchers have developed computable Stein discrepancy measures that provably determine the convergence of a sample to its target dist…
Derives properties of heat kernel for Rumin complex on Heisenberg groups.
problem Analyzing heat kernel properties for Rumin complex.
method Derives properties of heat equation with Hodge operator on Heisenberg groups.
result Constructs Calderón reproducing formula using heat kernel for Rumin forms.
A Kernel Adaptive Metropolis-Hastings algorithm is introduced, for the purpose of sampling from a target distribution with strongly nonlinear support. The algorithm embeds the trajectory of the Markov chain into a reproducing kernel Hilbert space (RKHS), such that the feature space covariance of the samples informs the…
Paper characterizes embeddability of function spaces into Lp-type RKBS via metric entropy.
problem Characterizing embeddability of function spaces into Lp-type RKBS. method Establishes a connection between metric entropy growth and embeddability.
result A bound on metric entropy growth allows embedding into Lp-type RKBS. Paper introduces RKHM and KME for richer data analysis.
problem Lack of rich data structures in kernel methods.
method Proposes RKHM and KME for functional data analysis.
result RKHM captures structural properties in functional data.
Stochastic kernel based dimensionality reduction approaches have become popular in the last decade. The central component of many of these methods is a symmetric kernel that quantifies the vicinity between pairs of data points and a kernel-induced Markov chain on the data. Typically, the Markov chain is fully specified…
The paper uses Banach spaces to analyze neural networks.
problem Understanding the function spaces of neural networks.
method Theory of reproducing kernel Banach spaces.
result Representer theorem for wide class of Banach spaces.
The study uses reproducing kernels to model bond discount curves.
problem Estimating bond discount curves under no-arbitrage conditions.
method Introduced reproducing kernels as a regression basis for estimating bond discount curves.
result Reproducing kernels provide a tractable solution for calibrating models to market data.
The paper improves probabilistic herding methods using Gibbs distributions.
problem Improving integration accuracy over Monte Carlo quadrature in infinite-dimensional RKHS.
method Developed a Gibbs distribution over quadrature nodes to minimize MMD.
result The Gibbs distribution outperforms i.i.d. Monte Carlo in integration accuracy.
Paper proposes a method for early stopping in regression using reproducing kernels.
problem Early stopping for iterative learning algorithms in nonparametric regression.
method Data-driven rule based on minimum discrepancy principle, validated by fixed-point analysis of localized Rademacher complexities.
result The proposed rule is minimax-optimal and performs comparably to cross-validation.
The paper develops a uniform function estimator in RKHS for regression.
problem Reconstructing functions from noisy data at random locations.
method Using reproducing kernel Hilbert spaces and Gaussian random fields.
result The estimator converges uniformly to the conditional expectation.
The paper develops methods to handle missing data using regularized M-estimation in reproducing kernel Hilbert space.
problem Handling missing data in statistical analysis.
method Kernel ridge regression for imputation and maximum entropy method for propensity score estimation.
result The proposed methods achieve statistical consistency and asymptotic equivalence.
New method for learning with non-Euclidean data using decomposable kernels.
problem Difficulty in using classical kernels for non-Euclidean data.
method Reproducing kernel Krein space (RKKS) methods for kernels that admit a positive decomposition.
result Invariant kernels can be used for learning in non-Euclidean spaces.
Study of regularized least squares in RKKS with indefinite kernels.
problem Asymptotic properties of regularized least squares with indefinite kernels in RKKS.
method Introducing a bounded hyper-sphere constraint, theoretical demonstration of globally optimal solution, modified error decomposition techniques, matrix perturbation theory.
result Derivation of learning rates in RKKS, same as RKHS under certain conditions.
Paper introduces RKHM for more explicit variable structures analysis.
problem Explicitly analyzing structures among variables.
method Orthonormal systems in Hilbert C∗-modules, RKHM. result Theoretical and practical procedures for RKHM orthonormalization.
This study connects Gaussian processes and RKHS, bridging two machine learning communities.
problem Understanding the relationship between Gaussian processes and RKHS.
method Examining connections and equivalences in regression, interpolation, and other topics.
result Established the equivalence between Gaussian Hilbert space and RKHS.
Motivated by multi-task machine learning with Banach spaces, we propose the notion of vector-valued reproducing kernel Banach spaces (RKBS). Basic properties of the spaces and the associated reproducing kernels are investigated. We also present feature map constructions and several concrete examples of vector-valued RK…
Estimates kernel eigenvalues for compositional dot-product kernels.
problem Improving estimates for kernel eigenvalues.
method Eigenvalue decay estimates of integral operators associated with dot-product kernels.
result Improved estimates for kernel volumes in reproducing kernel Hilbert spaces.
This note explains when neural networks can be seen as Gaussian processes.
problem Understanding the relationship between neural networks and Gaussian processes.
method Formulating a Gaussian process regression based on neural network outputs and analyzing the resulting posterior mean functions.
result The posterior mean functions of neural networks follow a Gaussian process in certain cases, providing an interpretation of reproducing kernel Hilbert spaces.
Study on how sampling works for complex data functions.
problem Analyzing convergence of sampling algorithms for RKHS functions.
method Minimalistic assumptions on kernel and data, error estimates in RKHS norm, uniform convergence on compact domains.
result New convergence rates for Lipschitz and Hölder continuous kernels.
Develops vector-valued RKBS for neural networks and operators.
problem Understanding function spaces of Rd-valued neural networks and neural operators. method Defines and constructs vector-valued RKBS (vv-RKBS) without restrictive assumptions.
result Establishes Representer Theorem for neural architectures.
Deep neural networks define suitable reproducing kernel Banach spaces.
problem Characterizing the function spaces of deep neural networks.
method Reproducing kernel Banach spaces and variational results.
result Deep neural networks define suitable reproducing kernel Banach spaces.
Recently, there has been emerging interest in constructing reproducing kernel Banach spaces (RKBS) for applied and theoretical purposes such as machine learning, sampling reconstruction, sparse approximation and functional analysis. Existing constructions include the reflexive RKBS via a bilinear form, the semi-inner-p…
Prefix consistency improves model reliability by weighting answers based on their reproducibility.
problem Improving the reliability of large language models' reasoning traces.
method Use prefix consistency to weight candidate answers based on their reproducibility during regeneration.
result Prefix consistency is the best correctness predictor, reaching Standard MV plateau accuracy with up to 21x fewer tokens.
New algorithm improves dynamic mode decomposition for high-dimensional data.
problem Reduced modeling in high-dimensional spaces.
method Low rank constraint optimization and kernel-based computation.
result Gain in approximation accuracy and computational efficiency.
A typical approach in estimating the learning rate of a regularized learning scheme is to bound the approximation error by the sum of the sampling error, the hypothesis error and the regularization error. Using a reproducing kernel space that satisfies the linear representer theorem brings the advantage of discarding t…
Paper explores RKHS properties for derivative and integral operators.
problem Establishing sufficient conditions for reproducing property in RKHS.
method Establishing reproducing property for combinations of composition operators.
result Provides framework for regularized learning algorithms involving function values, gradients, or operators.
Kernel interpolation is inconsistent for norms with smoothness above a constant.
problem Inconsistency of kernel interpolation in reproducing kernel Hilbert spaces.
method Lower bounds for generalization error in Sobolev norms.
result Kernel interpolation is always inconsistent for norms with smoothness above a constant.
New approach to supervised learning in RKHS and vvRKHS using C∗-algebras.
problem Traditional supervised learning in RKHS and vvRKHS.
method Generalizing supervised learning to RKHM using C∗-algebras. result Constructing RKHMs with enhanced representation power.
New algorithm optimizes functions in Matérn kernel RKHS with noisy feedback.
problem Optimizing functions in RKHS of Matérn kernel with noisy bandit feedback.
method π-GP-UCB algorithm with guaranteed sublinear regret for all ν > 1 and d ≥ 1.
result First practical approach with guaranteed sublinear regret for all ν > 1 and d ≥ 1.
In this paper we propose a bivariate generalization of a weighted indexed semi-Markov chains to study the high frequency price dynamics of traded stocks. We assume that financial returns are described by a weighted indexed semi-Markov chain model. We show, through Monte Carlo simulations, that the model is able to repr…
We study the complex geometry of generalized Kepler manifolds, defined in Jordan theoretic terms, introduce Hilbert spaces of holomorphic functions defined by radial measures, and find the complete asymptotic expansion of the corresponding reproducing kernels for Kähler potentials, both in the flat and bounded setting.
Extends Gaussian process theory to Banach spaces.
problem Extending Gaussian process theory to Banach spaces.
method Investigates the connection between Gaussian processes and Gaussian random elements in reproducing kernel Banach spaces.
result Characterizes positive definite functions that arise from covariance operators in Banach space setting.
The role of kernels is central to machine learning. Motivated by the importance of power-law distributions in statistical modeling, in this paper, we propose the notion of power-law kernels to investigate power-laws in learning problem. We propose two power-law kernels by generalizing Gaussian and Laplacian kernels. Th…