Proposes a novel graph signal model using narrowband kernels.
problem Graph signals with multiple concentrated frequency regions.
method Jointly learns graph signal model parameters and coefficients.
result Joint learning improves signal interpolation accuracy.
New spectral mixture representation for isotropic kernels simplifies random Fourier features.
problem Applying Random Fourier Features to complex kernels.
method Decompose isotropic kernels into scale mixtures of α-stable random vectors.
result Constructive spectral sampling formula for various kernels.
In this paper we propose a family of tractable kernels that is dense in the family of bounded positive semi-definite functions (i.e. can approximate any bounded kernel with arbitrary precision). We start by discussing the case of stationary kernels, and propose a family of spectral kernels that extends existing approac…
New method for spectral and Bergman kernels under local spectral gap condition.
problem Analyzing spectral and Bergman kernels for complex manifolds.
method Developed a new scaling method to study spectral and Bergman kernels.
result Established pointwise asymptotics of spectral and Bergman kernels.
Standard kernels such as Matérn or RBF kernels only encode simple monotonic dependencies within the input space. Spectral mixture kernels have been proposed as general-purpose, flexible kernels for learning and discovering more complicated patterns in the data. Spectral mixture kernels have recently been generalized in…
The generalization performance of kernel methods is largely determined by the kernel, but common kernels are stationary thus input-independent and output-independent, that limits their applications on complicated tasks. In this paper, we propose a powerful and efficient spectral kernel learning framework and learned ke…
New inequalities for spectral zeta kernels on spheres and manifolds.
problem Establishing new inequalities for spectral zeta functions.
method Applying Kato's inequalities and majorisation techniques.
result Generalized Kato's comparison inequalities to higher dimensions.
Unified framework for spectral methods, kernel learning, and manifold unfolding.
problem Tackles the unification and optimization of spectral dimensionality reduction methods.
method Unified spectral methods as kernel PCA, kernel learning by SDP, and detailed explanation of MVU variants.
result Unified understanding and optimization of manifold learning techniques.
New kernel models multi-output Gaussian processes accurately.
problem Challenges in modelling cross-covariances for multiple-output Gaussian processes.
method Replaced Gaussian components with block components of finite bandwidth in spectral mixture kernel.
result First multi-output generalization of spectral mixture kernel that can approximate any stationary multi-output kernel to arbitrary precision.
The paper develops heat kernel comparison theorems and applies them to spectral geometry.
problem Developing mathematical tools for spectral geometry.
method Established weighted heat kernel comparison theorems for manifolds with bounded radial curvatures.
result Two eigenvalue comparison theorems for the first Dirichlet eigenvalue of the Witten-Laplacian.
New kernels capture both local and non-local interactions efficiently.
problem Designing kernels that capture both local and non-local interactions while remaining computationally tractable.
method Spectral truncation kernels based on C∗-algebra. result Spectral truncation kernels induce interactions across the data function domain and reduce computational cost.
Multi-output Gaussian processes (MOGPs) are an extension of Gaussian Processes (GPs) for predicting multiple output variables (also called channels, tasks) simultaneously. In this paper we use the convolution theorem to design a new kernel for MOGPs, by modeling cross channel dependencies through cross convolution of t…
Improves learning of spectral mixture kernels with approximate Bayesian inference.
problem Difficult optimization of large number of SM kernel parameters.
method Approximate Bayesian inference using variational distribution of spectral points and random Fourier features.
result Accelerates convergence and leads to better optimal parameters.
Efficiently scales continuous kernels with sparse Fourier domain learning.
problem High computational and memory demands, spectral bias in continuous kernels.
method Sparse learning in the Fourier domain.
result Efficient scaling of continuous kernels, reduced computational and memory requirements, mitigated spectral bias.
Proposes a Gaussian process for graph signals using adaptive spectral kernels.
problem Predicting signals on graph nodes with various structures.
method Spectral kernel learning approach that incorporates a polynomial function in the graph spectral domain.
result The model accurately recovers ground truth spectral filters and outperforms in real-world graph data.
Study Bergman and spectral kernels for non-compact complex manifolds.
problem Analyze asymptotic behavior of kernels over non-compact complex manifolds.
method Generalize scaling method to study Bergman and spectral kernels.
result Derive leading term of Bergman and spectral kernels under local convergence of Chern curvatures.
The study analyzes spectral algorithms for kernel methods and derives generalization error.
problem Estimating generalization error of spectral algorithms for kernel methods.
method Considered spectral algorithms including KRR and GD, derived generalization error as a functional of learning profile.
result Showed the loss localizes on certain spectral scales and conjectured universality of the loss for noisy observations.
Recently, non-stationary spectral kernels have drawn much attention, owing to its powerful feature representation ability in revealing long-range correlations and input-dependent characteristics. However, non-stationary spectral kernels are still shallow models, thus they are deficient to learn both hierarchical featur…
A novel nonstationary permanental process relaxes kernel constraints and captures complex data patterns.
problem Limitations of existing permanental processes in terms of kernel types and stationarity.
method Sparse spectral representation of nonstationary kernels and hierarchical stacking of spectral feature mappings.
result Enhanced model expressiveness and reduced computational complexity.
SRF improves kernel approximation and GP regression performance.
problem Efficient kernel approximation and Bayesian kernel learning in large-scale regression problems.
method Stein variational gradient descent to generate high-quality random features and approximate spectral measure posteriors.
result SRF outperforms traditional approaches in kernel approximation and GP regression.
A new method for nonstationary Gaussian processes using Fourier features.
problem Efficient simulation of nonstationary Gaussian processes with high-dimensional distributions.
method Discretizes the spectral representation of nonstationary processes, avoiding probability measure assumptions.
result An efficient low-rank approximation of nonstationary spectral densities, consistent and positive semi-definite.
We propose non-stationary spectral kernels for Gaussian process regression. We propose to model the spectral density of a non-stationary kernel function as a mixture of input-dependent Gaussian process frequency density surfaces. We solve the generalised Fourier transform with such a model, and present a family of non-…
We propose a new method to model multi-way similarities into hypergraphs for clustering.
problem Clustering real-valued data using hypergraphs with multi-way similarities.
method Formulate multi-way similarities using kernel functions, establish connections to hypergraph cut, and develop a fast spectral clustering algorithm.
result Our method outperforms existing graph and heuristic modeling methods in clustering performance.
Develops nonstationary MOGP kernels for better performance.
problem Limited applicability of existing MOGP kernels for nonstationary data.
method Harmonizable spectral mixture kernels for nonstationary MOGP.
result Automatic identification of nonstationary behavior in data.
Study reveals how spectral bias affects learnability on real-world data.
problem Understanding how well complex datasets can be learned using kernel methods.
method Use eigenvalues and eigenfunctions from idealized data to reveal spectral bias on real-world data.
result Bound learnability on real-world data using symmetries of realistic kernels.
Proposes a deep network for multi-class classification using spectral training and Gaussian kernel.
problem Multi-class classification with deep networks.
method Spectral training with linear weights and Gaussian kernel activation, constrained on Stiefel Manifold.
result Theoretical guarantee of global optimum and insight into network generalization.
Nystrom approximation speeds up kernel model training.
problem Slow convergence in kernel models due to poor conditioning.
method Spectral preconditioning with Nystrom approximation for scalability.
result Nystrom approximation accelerates gradient descent nearly as well as exact preconditioner.
Random feature approximation speeds up spectral methods and improves learning rates.
problem Improving the efficiency and generalization of spectral methods in large-scale algorithms.
method Combining random feature approximation with spectral regularization methods.
result Optimal learning rates for estimators over various regularity classes, including those not in the RKHS.
We derive analytical expressions for the generalization performance of kernel regression as a function of the number of training samples using theoretical methods from Gaussian processes and statistical physics. Our expressions apply to wide neural networks due to an equivalence between training them and kernel regress…
We consider the problem of learning regression functions from pairwise data when there exists prior knowledge that the relation to be learned is symmetric or anti-symmetric. Such prior knowledge is commonly enforced by symmetrizing or anti-symmetrizing pairwise kernel functions. Through spectral analysis, we show that …
Study spectral properties of graph Laplacian for manifold data.
problem Understanding spectral properties of graph Laplacian for manifold data.
method Non-asymptotic error bounds on spectral properties of empirical graph Laplacian.
result Eigenvalues and eigenspaces of empirical graph Laplacian are close to Laplace-Beltrami operator of manifold.
Spectral algorithms on manifolds using diffusion kernels improve convergence rates.
problem The limitations of existing spectral algorithms in RKHSs for data on manifolds.
method Integrating manifold structure into spectral algorithms using heat kernel diffusion spaces.
result Spectral algorithms converge to the target function and its derivatives in a strong sense, with rates dependent on manifold intrinsic dimension.
Study spectral properties of sub-Riemannian Laplacians, proving quantum ergodicity and heat kernel asymptotics.
problem Spectral properties of sub-Riemannian Laplacians.
method Quantum ergodicity results, small-time asymptotics of sub-Riemannian heat kernels, Weyl law.
result Weyl law and spectral concentration on Lie brackets of length r-1.
Graph spectral techniques for measuring graph similarity, or for learning the cluster number, require kernel smoothing. The choice of kernel function and bandwidth are typically chosen in an ad-hoc manner and heavily affect the resulting output. We prove that kernel smoothing biases the moments of the spectral density.…
Study heat kernel on manifolds with fibred boundary metrics.
problem Analyzing spectral problems in manifolds with fibred boundary metrics.
method Construct heat kernel as polyhomogeneous conormal distribution.
result Fundamental step towards analysis of Ray-Singer torsion, eta-invariants and index theorems.
Enhances GPLVM for multi-view data with scalable latent representation learning.
problem Limited kernel expressiveness and computational inefficiency in multi-view GPLVM.
method Introduces a new duality between spectral density and kernel function, uses NG-SM kernel, and applies random Fourier feature approximation for scalability.
result Consistently outperforms state-of-the-art models in learning meaningful latent representations across diverse datasets.
Due to the growing ubiquity of unlabeled data, learning with unlabeled data is attracting increasing attention in machine learning. In this paper, we propose a novel semi-supervised kernel learning method which can seamlessly combine manifold structure of unlabeled data and Regularized Least-Squares (RLS) to learn a ne…
Paper provides unbiased spectral moment estimates from finite data.
problem Challenges in estimating spectral moments from limited data.
method Dynamic programming approach to estimate spectral moments of kernel integral operator.
result Demonstrates consistency with theoretical spectra and practical utility in neural networks.
Simple proof that stable minimal hypersurfaces in R^4 are hyperplanes.
problem Proving stable minimal hypersurfaces in R^4 are hyperplanes.
method Using spectral Ricci curvature bounds and Green kernel estimates.
result Complete, two-sided stable minimal hypersurfaces in R^4 are hyperplanes.
Spectral mixture (SM) kernels comprise a powerful class of generalized kernels for Gaussian processes (GPs) to describe complex patterns. This paper introduces model compression and time- and phase (TP) modulated dependency structures to the original (SM) kernel for improved generalization of GPs. Specifically, by adop…
Efficiently marginalizes over Gaussian Process kernels for better model flexibility and uncertainty.
problem Inefficient marginalization over Gaussian Process kernels for large datasets.
method Bayesian Quadrature scheme with maximum mean discrepancies and invariances between Spectral Mixture kernels.
result Achieves more accurate predictions and better calibrated uncertainty than state-of-the-art baselines.
New bounds for KRR condition number reveal overfitting phenomena.
problem Characterizing overfitting in KRR with varying kernel spectral decay.
method Derived new bounds for kernel matrices, enhanced test error bounds, and identified feature independence role.
result Identified tempered and catastrophic overfitting phenomena.
Quantum stochastic flow computes heat kernel traces for Ricci flat manifolds.
problem Computing heat kernel traces for Ricci flat manifolds.
method Quantum stochastic differential equation (qsde) on Fock space over L2 differential 1-forms, adapted flow construction. result Trace of the connection Laplacian heat kernel can be computed over any compact Ricci-flat Riemannian manifold.
Extends random feature analysis to spectral methods and improves learning rates.
problem Improving generalization properties of spectral methods in large-scale learning.
method Extends random feature analysis to a broad class of spectral regularization techniques, including gradient descent and Nesterov method.
result Obtains optimal learning rates for regularity classes, including those not in the RKHS.
FFN addresses spectral bias in neural value approximation, improving reinforcement learning performance.
problem Spectral bias in neural value approximation, leading to slow convergence and poor performance.
method Proposes Fourier feature networks (FFN) to overcome spectral bias by using a composite neural tangent kernel.
result FFN achieves state-of-the-art performance on challenging continuous control domains with faster convergence and better stability.
New STKR estimators use unlabeled data for smoother function learning.
problem Leveraging unlabeled data for smoother function learning.
method Spectrally transformed kernel regression (STKR) with scalable implementations.
result STKR can learn any sufficiently smooth function.
Study shows deterministic equivalent for neural network kernel convergence.
problem Understanding convergence of neural network kernels.
method Analyzes empirical spectral distribution of Conjugate Kernel, proving convergence to a deterministic limit.
result Obtains a deterministic equivalent for the Stieltjes transform and resolvent of the Conjugate Kernel.
New non-separable covariance kernels for spatiotemporal data derived from harmonic oscillator physics.
problem Capturing complex spatiotemporal dependencies in Gaussian processes.
method Hybrid spectral method based on the harmonic oscillator, deriving explicit covariance kernels.
result Explicit non-separable covariance kernels with space-time interactions.