arXiv research

We show in this note that the Sobolev Discrepancy introduced in Mroueh et al in the context of generative adversarial networks, is actually the weighted negative Sobolev norm $||.||_{\dot{H}^{-1}(ν_q)}$, that is known to linearize the Wasserstein $W_2$ distance and plays a fundamental role in the dynamic formulation of…

Large scale online kernel learning aims to build an efficient and scalable kernel-based predictive model incrementally from a sequence of potentially infinite data points. A current key approach focuses on ways to produce an approximate finite-dimensional feature map, assuming that the kernel used has a feature map wit…

We introduce a class of regularisable infinite dimensional principal fibre bundles which includes fibre bundles arising in gauge field theories like Yang-Mills and string theory and which generalise finite dimensional Riemannian principal fibre bundles induced by an isometric action. We show that the orbits of regulari…

Uniform-in-time analysis for Stein Variational Gradient Descent across various metrics.

This paper solves nonparametric estimation of continuous DPPs using kernel methods.

In this survey article, we review the relation between heat kernels and path integrals. In particular, we review recent results on the approximation of the Wiener measure on compact manifold by measures on (finite-dimensional) spaces of piece-wise geodesics.

We propose a novel supervised learning method to optimize the kernel in the maximum mean discrepancy generative adversarial networks (MMD GANs), and the kernel support vector machines (SVMs). Specifically, we characterize a distributionally robust optimization problem to compute a good distribution for the random featu…

Generalizes neural networks for infinite-dimensional mappings, including PDE solutions.

Kernel methods are powerful tools to capture nonlinear patterns behind data. They implicitly learn high (even infinite) dimensional nonlinear features in the Reproducing Kernel Hilbert Space (RKHS) while making the computation tractable by leveraging the kernel trick. Classic kernel methods learn a single layer of nonl…

Kernel methods form a powerful, versatile, and theoretically-grounded unifying framework to solve nonlinear problems in signal processing and machine learning. The standard approach relies on the kernel trick to perform pairwise evaluations of a kernel function, which leads to scalability issues for large datasets due …

Study on infinite-dimensional Heisenberg groups using hypoelliptic heat kernels.

We consider a class of operator-induced norms, acting as finite-dimensional surrogates to the L2 norm, and study their approximation properties over Hilbert subspaces of L2 . The class includes, as a special case, the usual empirical norm encountered, for example, in the context of nonparametric regression in reproduci…

Paper introduces FDM for efficient training of Neural SDEs.

The popular cubic smoothing spline estimate of a regression function arises as the minimizer of the penalized sum of squares $\sum_j(Y_j - μ(t_j))^2 + λ\int_a^b [μ"(t)]^2 dt$, where the data are $t_j,Y_j$, $j=1,..., n$. The minimization is taken over an infinite-dimensional function space, the space of all functions wi…

Hermite polynomials improve private data generation by reducing feature count.

We develop a new theoretical framework to analyze the generalization error of deep learning, and derive a new fast learning rate for two representative algorithms: empirical risk minimization and Bayesian deep learning. The series of theoretical analyses of deep learning has revealed its high expressive power and unive…

New kernels on symmetric groups enable efficient Gaussian process sampling.

Let M be a compact Riemannian manifold without boundary and let H be a self-adjoint generalized Laplace operator acting on sections in a bundle over M. We give a path integral formula for the solution to the corresponding heat equation. This is based on approximating path space by finite dimensional spaces of geodesic …

Physics-informed neural networks improve by measuring effective dimensionality of constraints.

Conditional diffusion models can approximate target distributions well with Gaussian-mixture reverse kernels.

The paper calculates heat kernel and closed geodesic asymptotics for nilpotent coverings.

The study uses reproducing kernels to model bond discount curves.

We study heat kernel measures on sub-Riemannian infinite-dimensional Heisenberg-like Lie groups. In particular, we show that Cameron-Martin type quasi-invariance results hold in this subelliptic setting and give $L^p$-estimates for the Radon-Nikodym derivatives. The main ingredient in our proof is a generalized curvatu…

Entropy study on synthetic spaces with curvature bounds.

We propose kernel-based collocation methods for numerical solutions to Heath-Jarrow-Morton models with Musiela parametrization. The methods can be seen as the Euler-Maruyama approximation of some finite dimensional stochastic differential equations, and allow us to compute the derivative prices by the usual Monte Carlo…

Kernel-based Bayesian filter for nonlinear systems using infinite-dimensional operators.

Kernel $ε$-Greedy optimizes multi-armed bandits with covariates for sub-linear regret.

Proposes a new K-means method for efficient clustering of nonlinear data.

The study characterizes kernel spaces on hyperspheres, impacting cubature algorithms.

The paper develops divergences for Gaussian processes and RKHS settings.

Differential privacy is a framework for privately releasing summaries of a database. Previous work has focused mainly on methods for which the output is a finite dimensional vector, or an element of some discrete set. We develop methods for releasing functions while preserving differential privacy. Specifically, we sho…

GPs' decisions can vary significantly with different kernels, even if kernels are qualitatively similar.

A method for interpreting SVMs using polynomial kernels, revealing model complexity.

We examine groups whose resonance varieties, characteristic varieties and Sigma-invariants have a natural arithmetic group symmetry, and we explore implications on various finiteness properties of subgroups. We compute resonance varieties, characteristic varieties and Alexander polynomials of Torelli groups, and we sho…

This work provides closed-form solutions and minimum achievable errors for a large class of low-rank approximation problems in Hilbert spaces. The proposed theorem generalizes to the case of bounded linear operators the previous results obtained in the finite dimensional case for the Frobenius norm. The theorem provide…

We prove that the kernels of the restrictions of symplectic Dirac or symplectic Dirac-Dolbeault operators on natural subspaces of polynomial valued spinor fields are finite dimensional on a compact symplectic manifold. We compute those kernels for the complex projective spaces. We construct injections of subgroups of t…

Study finds maximal symmetry groups for CR structures with specific properties.

We present a new framework for online Least Squares algorithms for nonlinear modeling in RKH spaces (RKHS). Instead of implicitly mapping the data to a RKHS (e.g., kernel trick), we map the data to a finite dimensional Euclidean space, using random features of the kernel's Fourier transform. The advantage is that, the …

We associate certain probability measures on $\R$ to geodesics in the space $\H_L$ of positively curved metrics on a line bundle $L$, and to geodesics in the finite dimensional symmetric space of hermitian norms on $H^0(X, kL)$. We prove that the measures associated to the finite dimensional spaces converge weakly to t…

Study small-time CLTs for stochastic Volterra equations with various kernels.

Study shows how to effectively predict functions on manifolds using kernel methods.

This paper improves Koopman operator approximations by pruning subspaces in RKHS.

Deep neural networks for structured prediction using kernel-induced losses.

We consider the problem of streaming kernel regression, when the observations arrive sequentially and the goal is to recover the underlying mean function, assumed to belong to an RKHS. The variance of the noise is not assumed to be known. In this context, we tackle the problem of tuning the regularization parameter ada…

New algorithms improve GP inference without approximations, achieving better results.

Paper develops efficient estimator for Hawkes processes using representer theorem.

Kernelized convex clustering handles non-linear and non-convex data.

Develops vector-valued RKBS for neural networks and operators.