Many unsupervised kernel methods rely on the estimation of the kernel covariance operator (kernel CO) or kernel cross-covariance operator (kernel CCO). Both kernel CO and kernel CCO are sensitive to contaminated data, even when bounded positive definite kernels are used. To the best of our knowledge, there are few well…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Kernelization”
Survey of kernels, RKHS, and their applications in machine learning.
problem Understanding kernels and their applications in machine learning.
method Review of historical context, mathematical definitions, and practical applications of kernels.
result Comprehensive overview of kernels, RKHS, and their applications.
To the best of our knowledge, there are no general well-founded robust methods for statistical unsupervised learning. Most of the unsupervised methods explicitly or implicitly depend on the kernel covariance operator (kernel CO) or kernel cross-covariance operator (kernel CCO). They are sensitive to contaminated data, …
Deep neural kernels and Laplace kernel have equivalent RKHS on spheres.
problem Comparing RKHS of deep neural tangent and Laplace kernels.
method Proof of RKHS equivalence using sphere restrictions and kernel properties.
result RKHS of deep neural tangent kernel and Laplace kernel are the same on .
Kernel methods linked to feature subspaces and maximal correlation kernels.
problem Understanding kernel methods and their relationship to feature extraction.
method Established a correspondence between feature subspaces and kernels, introduced maximal correlation kernels, and demonstrated their optimality.
result Kernel SVM on maximal correlation kernel achieves minimum prediction error.
PGF kernels analyze spherical data using generalized RBF kernels.
problem Analysis of spherical data.
method Introduced PGF kernels and a semi-parametric learning algorithm.
result PGF kernels generalize RBF kernels for spherical data.
Adapts manifold structure for better clustering performance.
problem Lack of consideration for local manifold structure in existing multiple kernel k-means methods.
method Adopts manifold adaptive kernel to integrate local manifold structure of kernels.
result Proposed method outperforms state-of-the-art methods.
Optimal kernel in KR can be data-dependent, improving model performance.
problem Fixed kernel in KR limits model performance.
method Considered data-dependent kernels for KR, using posterior covariance.
result Data-dependent kernel choice leads to optimal performance.
Quantum kernels can be efficiently embedded into classical feature spaces.
problem Can all quantum kernels be efficiently embedded into classical feature spaces?
method Invoking computational universality and using techniques like random Fourier features, the authors show that certain classes of quantum kernels can be efficiently embedded.
result For shift-invariant and composition kernels, embedding quantum kernels are universal and efficient.
New random feature maps for Laplacian and related kernels.
problem Challenges in approximating the Laplacian kernel and its generalizations.
method Developed random feature maps for Laplacian and related kernels, providing efficient sampling schemes.
result Demonstrated the efficacy of these random feature maps on real datasets.
New method for learning with non-Euclidean data using decomposable kernels.
problem Difficulty in using classical kernels for non-Euclidean data.
method Reproducing kernel Krein space (RKKS) methods for kernels that admit a positive decomposition.
result Invariant kernels can be used for learning in non-Euclidean spaces.
New estimator reduces kernel mean estimation error.
problem Kernel mean estimation in reproducing kernel Hilbert spaces.
method Corrupt data with known distributions and estimate kernel mean under the corrupted distribution.
result The marginalized kernel mean estimator achieves lower estimation error.
We present Random Partition Kernels, a new class of kernels derived by demonstrating a natural connection between random partitions of objects and kernels between those objects. We show how the construction can be used to create kernels from methods that would not normally be viewed as random partitions, such as Random…
The NNGP kernel's predictions closely match those of the Matern kernel under certain conditions.
problem Comparing NNGP kernels to Matern kernels in practical applications.
method Demonstrated the necessity of normalization for NNGP kernels, explored numerical challenges, and compared predictions and performance.
result NNGP kernel predictions closely match Matern kernel predictions under specific circumstances.
In this paper, we compare 5 different nonlinear kernels: min-max, RBF, fRBF (folded RBF), acos, and acos-, on a wide range of publicly available datasets. The proposed fRBF kernel performs very similarly to the RBF kernel. Both RBF and fRBF kernels require an important tuning parameter (). Interestingly, for a …
New kernels allow learning from non-separable data.
problem Learning from non-separable data.
method Introducing entangled kernels and a two-step algorithm.
result Efficient algorithm for learning entangled kernels.
Laplace kernel and Neural Tangent Kernels are shown to be nearly identical for normalized data.
problem Understanding the similarity between Laplace and Neural Tangent Kernels.
method Theoretical analysis and experiments on normalized data.
result Laplace kernel and Neural Tangent Kernels have nearly identical eigenfunctions and RKHS for normalized data.
Kernel smoothing on unknown manifolds with bounds and asymptotic normality.
problem Data on unknown manifolds without boundaries.
method Finite sample bounds and asymptotic normality for kernel smoothing and its derivatives.
result Established finite sample bounds and asymptotic normality for kernel smoothing.
Generalized Spectral Kernelsstat.ML
In this paper we propose a family of tractable kernels that is dense in the family of bounded positive semi-definite functions (i.e. can approximate any bounded kernel with arbitrary precision). We start by discussing the case of stationary kernels, and propose a family of spectral kernels that extends existing approac…
The success of kernel-based learning methods depend on the choice of kernel. Recently, kernel learning methods have been proposed that use data to select the most appropriate kernel, usually by combining a set of base kernels. We introduce a new algorithm for kernel learning that combines a {\em continuous set of base …
New kernels capture both local and non-local interactions efficiently.
problem Designing kernels that capture both local and non-local interactions while remaining computationally tractable.
method Spectral truncation kernels based on -algebra.
result Spectral truncation kernels induce interactions across the data function domain and reduce computational cost.
Quantum kernel machines need to use more complex kernels to fully exploit their potential.
problem Current quantum kernels struggle with complex learning tasks due to limited degrees of freedom.
method Propose using operator-valued kernels and -algebraic representations to enhance quantum kernels.
result Quantum operator-valued kernels can reveal structural dependencies that scalar-valued kernels miss.
Constructing the adjacency graph is fundamental to graph-based clustering. Graph learning in kernel space has shown impressive performance on a number of benchmark data sets. However, its performance is largely determined by the chosen kernel matrix. To address this issue, the previous multiple kernel learning algorith…
Optimal kernel improves estimation accuracy in modal statistical methods.
problem Estimation accuracy of kernel-based modal statistical methods depends on the kernel used.
method The study theoretically shows an optimal kernel that minimizes asymptotic error criterion.
result An optimal kernel minimizes the error criterion when using an optimal bandwidth.
Efficiently searches through Gaussian process kernels using symbolic representation and Bayesian optimization.
problem Manual selection of kernels in Gaussian processes is complex and computationally expensive.
method Proposes a novel method using symbolic representation and Bayesian optimization to search through a structured kernel space.
result Empirically shows a computationally more efficient way of searching through a discrete kernel space.
CoRE Kernelsstat.ML
The term "CoRE kernel" stands for correlation-resemblance kernel. In many applications (e.g., vision), the data are often high-dimensional, sparse, and non-binary. We propose two types of (nonlinear) CoRE kernels for non-binary sparse data and demonstrate the effectiveness of the new kernels through a classification ex…
MKLpy simplifies Multiple Kernel Learning in Python.
problem Learning optimal kernel functions from data.
method Python-based framework for Multiple Kernel Learning algorithms.
result Maximizes usability and simplifies development of novel solutions.
Sparse Kernel Flows learns dynamical systems from data.
problem Learning dynamical systems from limited data.
method Sparse Kernel Flows: trains optimal kernel from a dictionary of kernels.
result Sparse Kernel Flows can learn from 132 chaotic systems.
A Framework for an Assessment of the Kernel-target Alignment in Tree Ensemble Kernel Learningstat.ML
The study investigates kernel-target alignment in tree ensemble kernels.
problem The degree of kernel-target alignment affects the performance of tree ensemble kernels in kernel learning.
method Eigenanalysis of the kernel matrix and sensitivity analysis via landmark learning.
result Good performance of tree ensemble kernels is associated with strong kernel-target alignment.
Many real world graphs, such as the graphs of molecules, exhibit structure at multiple different scales, but most existing kernels between graphs are either purely local or purely global in character. In contrast, by building a hierarchy of nested subgraphs, the Multiscale Laplacian Graph kernels (MLG kernels) that we …
Two adaptive kernel selection methods improve the accuracy of Kernelized Diffusion Maps.
problem Selecting an appropriate kernel for Kernelized Diffusion Maps.
method Two complementary approaches: variational outer loop and unsupervised cross-validation.
result Both methods improve the quality and stability of the recovered eigenfunctions.
This study examines the practical equivalence of Laplace and neural tangent kernels.
problem Understanding the practical equivalence of Laplace and neural tangent kernels.
method The study matches the kernels exactly and by matching posteriors of a Gaussian process. It also analyzes the kernels in R^d and experiments with them in regression tasks.
result The Laplace and neural tangent kernels are practically equivalent.
The paper provides consistency results for KDE on manifolds with irregular kernels.
problem Analyzing density estimation on manifolds with complex kernels.
method Strong uniform consistency with rates for KDE on Riemannian manifolds with Riemann integrable kernels.
result Strong uniform consistency with rates for KDE on manifolds.
Kernel methods are studied in a mean field limit for high-dimensional data.
problem Analyzing kernel methods in high-dimensional data with many variables.
method Investigation of kernel methods in the mean field limit of interacting particle systems.
result Rigorous mean field limit of kernels and detailed analysis of the limiting reproducing kernel Hilbert space.
New quantum kernels avoid overfitting by combining local and global components.
problem Exponential concentration in quantum kernels leads to overfitting.
method Local-global quantum kernels combining small subsystem and full-system measurements.
result Demonstrated benign overfitting in local-global quantum kernels.
IDK improves anomaly detection for points and groups without explicit learning.
problem Anomaly detection for points and groups using kernel methods.
method Isolation Distributional Kernel (IDK) addresses data independence and intractable dimensionality issues.
result IDK outperforms existing methods for both point and group anomaly detection.
Taming Nonconvexity in Kernel Feature Selection -- Favorable Properties of the Laplace Kernelmath.ST
Laplace kernel feature selection offers statistical guarantees for nonparametric models with few samples.
problem Statistical guarantees for kernel-based feature selection in nonconvex optimization problems.
method Sharp characterization of the gradient of the objective function for Laplace kernel feature selection.
result Model-selection consistency for Laplace kernel-based feature selection in nonparametric settings with samples.
New algorithm speeds up polynomial kernel approximations.
problem Efficiently approximating polynomial kernels of high degree.
method Oblivious sketching combined with novel sampling.
result Polynomial factor slowdown removed in running time.
Large scale online kernel learning aims to build an efficient and scalable kernel-based predictive model incrementally from a sequence of potentially infinite data points. A current key approach focuses on ways to produce an approximate finite-dimensional feature map, assuming that the kernel used has a feature map wit…
Following the very recent line of work on the ``generalized min-max'' (GMM) kernel, this study proposes the ``generalized intersection'' (GInt) kernel and the related ``normalized generalized min-max'' (NGMM) kernel. In computer vision, the (histogram) intersection kernel has been popular, and the GInt kernel generaliz…
New method reduces variance and bias in approximating indefinite kernels.
problem Approximating non-stationary indefinite kernels with low variance and bias.
method Generalized orthogonal random features (GORF)
result GORF achieves lower variance and approximation error compared to existing methods.
Unified framework combines trace-induced quantum kernels for improved machine learning models.
problem Improving performance of quantum machine learning models using trace-induced kernels.
method Developed a unified framework combining various trace-induced quantum kernels, including global fidelity and local projected kernels, as Lego kernels.
result Local projected kernels can achieve comparable performance to global fidelity kernels with fewer quantum resources.
We study strictly proper scoring rules in the Reproducing Kernel Hilbert Space. We propose a general Kernel Scoring rule and associated Kernel Divergence. We consider conditions under which the Kernel Score is strictly proper. We then demonstrate that the Kernel Score includes the Maximum Mean Discrepancy as a special …
We study the expressive power of kernel methods and the algorithmic feasibility of multiple kernel learning for a special rich class of kernels. Specifically, we define \emph{Euclidean kernels}, a diverse class that includes most, if not all, families of kernels studied in literature such as polynomial kernels and radi…
Learning the kernel functions used in kernel methods has been a vastly explored area in machine learning. It is now widely accepted that to obtain 'good' performance, learning a kernel function is the key challenge. In this work we focus on learning kernel representations for structured regression. We propose use of po…
RKUM is an R package for robust kernel-based unsupervised methods.
problem Robust analysis under contaminated or noisy data conditions.
method Robust kernel covariance and cross-covariance operators using generalized loss functions.
result RKUM reduces sensitivity to contamination and effectively identifies outliers.
New method uses multiple kernels to improve SVGD performance.
problem Sub-optimal performance of single kernel in SVGD.
method Combines multiple kernels to approximate optimal kernel, using Kernelized Stein Discrepancy (KSD) and constructing Multiple Kernel SVGD (MK-SVGD).
result Consistently matches or outperforms competing methods in experiments.
Kernel alignment measures the degree of similarity between two kernels. In this paper, inspired from kernel alignment, we propose a new Linear Discriminant Analysis (LDA) formulation, kernel alignment LDA (kaLDA). We first define two kernels, data kernel and class indicator kernel. The problem is to find a subspace to …