arXiv research

Kernel method is a very powerful tool in machine learning. The trick of kernel has been effectively and extensively applied in many areas of machine learning, such as support vector machine (SVM) and kernel principal component analysis (kernel PCA). Kernel trick is to define a kernel function which relies on the inner-…

Study on KRR with power-law data, showing better sample complexity.

A Monte Carlo k-nearest neighbours (KNN) and a multi-resolution convolutional neural network (CNN) were developed to detect the presences of multiple gasses in near infrared (IR) spectrums. High Resolution Transmission database was used to synthesize the near IR spectrums. Monte Carlo KNN determined the optimal kernel …

A new method warps inputs to learn nonstationary kernels efficiently.

Proposes a Gaussian process for graph signals using adaptive spectral kernels.

We study the existence and uniqueness of the heat kernel on infinite, locally finite, connected graphs. For general graphs, a uniqueness criterion, shown to be optimal, is given in terms of the maximal valence on spheres about a fixed vertex. A sufficient condition for non-uniqueness is also presented. Furthermore, we …

Study analyzes learnability of RKHS under L∞ norm for kernel methods.

Proposes a novel graph signal model using narrowband kernels.

We consider a complete noncompact smooth Riemannian manifold $M$ with a weighted measure and the associated drifting Laplacian. We demonstrate that whenever the $q$-Bakry-Émery Ricci tensor on $M$ is bounded below, then we can obtain an upper bound estimate for the heat kernel of the drifting Laplacian from the upper b…

Study reveals an equivalence principle for the spectrum of random inner-product kernel matrices in polynomial scaling.

We present an approach to model time series data from resting state fMRI for autism spectrum disorder (ASD) severity classification. We propose to adopt kernel machines and employ graph kernels that define a kernel dot product between two graphs. This enables us to take advantage of spatio-temporal information to captu…

NTK-SAP improves neural network pruning by aligning training dynamics.

Introduces tunable basis functions for Gaussian processes.

This paper shows universality in spectrum behavior for random inner-product kernel matrices in polynomial regime.

In this paper we study the concentration properties for the eigenvalues of kernel matrices, which are central objects in a wide range of kernel methods and, more recently, in network analysis. We present a set of concentration inequalities tailored for each individual eigenvalue of the kernel matrix with respect to its…

Using Weitzenböck techniques on any compact Riemannian spin manifold we derive a general inequality depending on a real parameter and joining the spectrum of the Dirac operator with terms depending on the Ricci tensor and its first covariant derivatives. The discussion of this inequality yields vanishing theorems for t…

We empirically investigate the best trade-off between sparse and uniformly-weighted multiple kernel learning (MKL) using the elastic-net regularization on real and simulated datasets. We find that the best trade-off parameter depends not only on the sparsity of the true kernel-weight spectrum but also on the linear dep…

This study improves graph coarsening methods by preserving graph spectrum and distances.

Paper provides unbiased spectral moment estimates from finite data.

Researchers resolve string theory ambiguities and define a new metric for massless spectrum.

Let X=G/K be a symmetric space of noncompact type and let L be the Laplacian associated with a G-invariant metric on X. We show that the resolvent kernel of L admits a holomorphic extension to a Riemann surface depending on the rank of the symmetric space. This Riemann surface is a branched cover of the complex plane w…

We study the relationship between the geometry and the Laplace spectrum of a Riemannian orbifold O via its heat kernel; as in the manifold case, the time-zero asymptotic expansion of the heat kernel furnishes geometric information about O. In the case of a good Riemannian orbifold (i.e., an orbifold arising as the orbi…

We study quadrature rules for functions from an RKHS, using nodes sampled from a determinantal point process (DPP). DPPs are parametrized by a kernel, and we use a truncated and saturated version of the RKHS kernel. This link between the two kernels, along with DPP machinery, leads to relatively tight bounds on the qua…

Study the Bochner-Schrödinger operator on symplectic manifolds, proving gap existence and asymptotic kernel behavior.

Gaussian process regression generally does not scale to beyond a few thousands data points without applying some sort of kernel approximation method. Most approximations focus on the high eigenvalue part of the spectrum of the kernel matrix, $K$, which leads to bad performance when the length scale of the kernel is sma…

Expressiveness and generalization of deep models was recently addressed via the connection between neural networks (NNs) and kernel learning, where first-order dynamics of NN during a gradient-descent (GD) optimization were related to gradient similarity kernel, also known as Neural Tangent Kernel (NTK). In the majorit…

We give results about the L^2 kernel and the spectrum of the Dirac operator on a complete Riemannian manifold which is conformally equivalent to the interior of a Riemannian manifold with nonempty boundary.

We study the sub-Laplacian of the $15$-dimensional unit sphere which is obtained by lifting with respect to the Hopf fibration the Laplacian of the octonionic projective space. We obtain in particular explicit formulas for its heat kernel and deduce an expression for the Green function of a related sub-Laplacian. As a …

Opportunistic spectrum access is one of the emerging techniques for maximizing throughput in congested bands and is enabled by predicting idle slots in spectrum. We propose a kernel-based reinforcement learning approach coupled with a novel budget-constrained sparsification technique that efficiently captures the envir…

We analyze kernel matrices in polynomial high-dimensional settings and explain double descent in KRR.

Study the spectral properties of Laplacian on warped product manifolds.

In this paper we propose a family of tractable kernels that is dense in the family of bounded positive semi-definite functions (i.e. can approximate any bounded kernel with arbitrary precision). We start by discussing the case of stationary kernels, and propose a family of spectral kernels that extends existing approac…

We investigate if kernel regularization methods can achieve minimax convergence rates over a source condition regularity assumption for the target function. These questions have been considered in past literature, but only under specific assumptions about the decay, typically polynomial, of the spectrum of the the kern…

Nowadays, hyperspectral image classification widely copes with spatial information to improve accuracy. One of the most popular way to integrate such information is to extract hierarchical features from a multiscale segmentation. In the classification context, the extracted features are commonly concatenated into a lon…

We derive analytical expressions for the generalization performance of kernel regression as a function of the number of training samples using theoretical methods from Gaussian processes and statistical physics. Our expressions apply to wide neural networks due to an equivalence between training them and kernel regress…

The dynamics of DNNs during gradient descent is described by the so-called Neural Tangent Kernel (NTK). In this article, we show that the NTK allows one to gain precise insight into the Hessian of the cost of DNNs. When the NTK is fixed during training, we obtain a full characterization of the asymptotics of the spectr…

We present a novel attention-based model for discrete event data to capture complex non-linear temporal dependence structures. We borrow the idea from the attention mechanism and incorporate it into the point processes' conditional intensity function. We further introduce a novel score function using Fourier kernel emb…

GNTK reveals convergence of GNNs on large graphs.

In this paper, we study the spectrum and the eigenvectors of radial kernels for mixtures of distributions in $\mathbb{R}^n$. Our approach focuses on high dimensions and relies solely on the concentration properties of the components in the mixture. We give several results describing of the structure of kernel matrices …

New method interpolates high-dimensional scattered data using kernel theory.

The study bounds heat kernel for manifolds with specific curvature conditions.

In this thesis, we analyze the stochastic completeness of a heat kernel on graphs which is a function of three variables: a pair of vertices and a continuous time, for infinite, locally finite, connected graphs. For general graphs, a sufficient condition for stochastic completeness is given in terms of the maximum vale…

HRFs adaptively linearize kernels for accurate approximations.

A fundamental task in kernel methods is to pick nodes and weights, so as to approximate a given function from an RKHS by the weighted sum of kernel translates located at the nodes. This is the crux of kernel density estimation, kernel quadrature, or interpolation from discrete samples. Furthermore, RKHSs offer a conven…

Existing approaches to analyzing the asymptotics of graph Laplacians typically assume a well-behaved kernel function with smoothness assumptions. We remove the smoothness assumption and generalize the analysis of graph Laplacians to include previously unstudied graphs including kNN graphs. We also introduce a kernel-fr…

Manifolds with fibered cusps are a class of complete noncompact Riemannian manifolds including all locally symmetric spaces of rank one. We study the spectrum of the Hodge Laplacian with coefficients in a flat bundle on a closed manifold undergoing degeneration to a manifold with fibered cusps. We obtain precise asympt…

Study of homogeneous spaces in Hartree-Fock-Bogoliubov theory.

We study the spectrum and heat kernel of the Hodge Laplacian with coefficients in a flat bundle on a closed manifold degenerating to a manifold with wedge singularities. Provided the Hodge Laplacians in the fibers of the wedge have an appropriate spectral gap, we give uniform constructions of the resolvent and heat ker…