A new kernel, Isolation Kernel, simplifies large scale online kernel learning without sacrificing accuracy.
problem Building efficient and scalable kernel-based models from large datasets with high accuracy.
method Introducing Isolation Kernel, which creates an exact, sparse, and finite-dimensional feature map of a kernel, allowing for efficient large scale online kernel learning without accuracy loss.
result Large scale online kernel learning can be achieved efficiently and accurately using Isolation Kernel.
Kernel fusion is a popular and effective approach for combining multiple features that characterize different aspects of data. Traditional approaches for Multiple Kernel Learning (MKL) attempt to learn the parameters for combining the kernels through sophisticated optimization procedures. In this paper, we propose an a…
New approach to learning kernels from data using AIT principles.
problem Learning kernels from data in machine learning.
method Sparse Kernel Flows method based on AIT principles.
result Sparse Kernel Flows aligns with MDL principle and offers a robust theoretical foundation.
In this paper we propose a family of tractable kernels that is dense in the family of bounded positive semi-definite functions (i.e. can approximate any bounded kernel with arbitrary precision). We start by discussing the case of stationary kernels, and propose a family of spectral kernels that extends existing approac…
Multi-kernel learning has been well explored in the recent past and has exhibited promising outcomes for multi-class classification and regression tasks. In this paper, we present a multiple kernel learning approach for the One-class Classification (OCC) task and employ it for anomaly detection. Recently, the basic mul…
We develop a multi-kernel based regression method for graph signal processing where the target signal is assumed to be smooth over a graph. In multi-kernel regression, an effective kernel function is expressed as a linear combination of many basis kernel functions. We estimate the linear weights to learn the effective …
The success of kernel-based learning methods depend on the choice of kernel. Recently, kernel learning methods have been proposed that use data to select the most appropriate kernel, usually by combining a set of base kernels. We introduce a new algorithm for kernel learning that combines a {\em continuous set of base …
A new method slices and sums radial kernels faster.
problem Fast computation of large kernel sums in kernel methods.
method Random projections to 1D subspaces and QMC for selecting projections.
result QMC-slicing outperforms existing methods on test datasets.
A new method for kernel tests without data splitting increases power.
problem Lack of power in kernel-based tests due to data splitting.
method Selective inference framework to learn hyperparameters and test on full sample.
result Empirically larger test power without data splitting, regardless of split proportion.
New approach interprets Nyström for kernel machines with geometric insight.
problem No comparative study over Nyström-based kernel machine approaches.
method Developed a new approach with geometric interpretation, showing equivalence to existing methods.
result Proposed approach offers insights into approximation errors and accuracy.
In this paper, we propose a random projection approach to estimate variance in kernel ridge regression. Our approach leads to a consistent estimator of the true variance, while being computationally more efficient. Our variance estimator is optimal for a large family of kernels, including cubic splines and Gaussian ker…
Multiple Kernel Learning, or MKL, extends (kernelized) SVM by attempting to learn not only a classifier/regressor but also the best kernel for the training task, usually from a combination of existing kernel functions. Most MKL methods seek the combined kernel that performs best over every training example, sacrificing…
MKL-based models outperform complex multi-omics integrative approaches.
problem Integrating diverse omics data sources.
method Supervised multiple kernel learning with different kernel fusion strategies.
result MKL-based models outperform more complex architectures.
For applications as varied as Bayesian neural networks, determinantal point processes, elliptical graphical models, and kernel learning for Gaussian processes (GPs), one must compute a log determinant of an n×n positive definite matrix, and its derivatives - leading to prohibitive O(n3) computatio…
New quantum kernels avoid overfitting by combining local and global components.
problem Exponential concentration in quantum kernels leads to overfitting.
method Local-global quantum kernels combining small subsystem and full-system measurements.
result Demonstrated benign overfitting in local-global quantum kernels.
Paper improves kernel approximations for better statistical learning.
problem Improving kernel approximations for better statistical learning.
method Taylor series approximations of radial kernel functions.
result Establishes upper bounds for eigenfunctions, leading to better approximations.
Deep kernels learn from embeddings to capture data similarity efficiently.
problem Capturing similarity between high-dimensional data points with small labeled data.
method Probabilistic neural network to learn deep kernels on probabilistic embeddings.
result Our approach outperforms state-of-the-art GP kernel learning in various settings.
The computational and storage complexity of kernel machines presents the primary barrier to their scaling to large, modern, datasets. A common way to tackle the scalability issue is to use the conjugate gradient algorithm, which relieves the constraints on both storage (the kernel matrix need not be stored) and computa…
Models like support vector machines or Gaussian process regression often require positive semi-definite kernels. These kernels may be based on distance functions. While definiteness is proven for common distances and kernels, a proof for a new kernel may require too much time and effort for users who simply aim at prac…
Study Kleinian groups using orbital functions and heat kernels.
problem Understanding Kleinian groups through orbital functions and heat kernels.
method Using the heat kernel approach developed in \cite{artmoiheatcounting1}.
result Developed a method to study Kleinian groups using orbital functions and heat kernels.
RFFNet scales kernel methods to large datasets by learning kernel relevance.
problem Scaling kernel methods to large datasets while maintaining interpretability.
method Designs random Fourier features for ARD kernels and uses first-order stochastic optimization for learning kernel relevances.
result RFFNet achieves low prediction error and identifies relevant features, leading to more interpretable solutions.
We present a novel framework for kernel learning with sequential data of any kind, such as time series, sequences of graphs, or strings. Our approach is based on signature features which can be seen as an ordered variant of sample (cross-)moments; it allows to obtain a "sequentialized" version of any static kernel. The…
Develops a new method to learn kernels directly over function space.
problem Learning kernels for flexible function approximators.
method Functional Kernel Learning (FKL) using transformed Gaussian processes over spectral densities.
result Direct inference of functional posteriors over kernels enables rich representations.
Optimal scoring framework for kernel classification with feature selection.
problem Two-group classification problem.
method Optimal scoring framework, structured sparsity using weighted kernels, automated parameter selection.
result Superior classification performance compared to existing nonparametric classifiers.
Detecting the emergence of abrupt property changes in time series is a challenging problem. Kernel two-sample test has been studied for this task which makes fewer assumptions on the distributions than traditional parametric approaches. However, selecting kernels is non-trivial in practice. Although kernel selection fo…
Kernel methods detect coherent structures in dynamical data.
problem Detecting coherent structures in complex dynamical systems.
method Kernel-based dimensionality reduction techniques and eigendecompositions of RKHS operators.
result Coherent sets of particle trajectories can be computed by kernel CCA.
New kernels for set optimization improve combinatorial optimization without jitter.
problem Optimizing with set inputs, especially in combinatorial problems.
method Developed new set kernels based on Reproducing Kernel Hilbert Space embeddings.
result Subclasses of Deep Embedding kernels are strictly positive definite, enabling combinatorial optimization.
Unified analysis of kernel-based and locally adaptive bandit optimization methods.
problem Performance of bandit optimization algorithms in RKHS functions.
method Investigates the relationship between kernel regularity and algorithmic performance, characterizing spectral properties of various kernels.
result Unified framework for analyzing kernel-based and locally adaptive bandit algorithms, deriving explicit regret bounds.
KSOS improves kernel learning for dynamical systems via global optimization.
problem Challenges in selecting optimal kernels and tuning parameters in traditional kernel-based methods.
method Global optimization framework with kernel-based surrogate functions.
result KSOS consistently outperforms gradient descent in predicting dynamical systems.
Kernel methods are powerful and flexible approach to solve many problems in machine learning. Due to the pairwise evaluations in kernel methods, the complexity of kernel computation grows as the data size increases; thus the applicability of kernel methods is limited for large scale datasets. Random Fourier Features (R…
Meta-learning framework uses task similarity through nonparametric kernel regression.
problem Limited tasks and outliers/dissimilar tasks hinder meta-learning performance.
method Nonparametric kernel regression to quantify and use task similarity.
result Meta-learning algorithm outperforms existing methods in task-limited settings.
Kernel Bayesian inference is a principled approach to nonparametric inference in probabilistic graphical models, where probabilistic relationships between variables are learned from data in a nonparametric manner. Various algorithms of kernel Bayesian inference have been developed by combining kernelized basic probabil…
Paper develops a novel approach for optimal control using kernel methods.
problem Optimal control of nonlinear stochastic systems.
method Infinitesimal generator approach in reproducing kernel Hilbert spaces.
result Data-driven solution to optimal control problems.
Recent research on multiple kernel learning has lead to a number of approaches for combining kernels in regularized risk minimization. The proposed approaches include different formulations of objectives and varying regularization strategies. In this paper we present a unifying general optimization criterion for multip…
We offer a new, rigorous approach to conditional mean embeddings without operator constraints.
problem Lack of rigorous, operator-free approach to conditional mean embeddings.
method Measure-theoretic approach to conditional mean embeddings.
result Natural regression interpretation and universal consistency of empirical estimates.
A novel kernel approach for model selection in simulator-based models.
problem Model selection for simulator-based statistical models with limited prior knowledge.
method Iteratively updates model weights and parameters using Bayes' rule and kernel recursive ABC algorithm.
result Demonstrates effectiveness on dynamical systems in ecology and epidemiology.
The heat kernel for the Cauchy-Riemann subLaplacian on S(2n+1) is derived in a manner which is completely analogous to the classical derivation of elliptic heat kernels. This suggests that the classical hamiltonian construction of elliptic heat kernels, with appropriate modifications, does yield heat kernels for subell…
Kernel methods linked to feature subspaces and maximal correlation kernels.
problem Understanding kernel methods and their relationship to feature extraction.
method Established a correspondence between feature subspaces and kernels, introduced maximal correlation kernels, and demonstrated their optimality.
result Kernel SVM on maximal correlation kernel achieves minimum prediction error.
To cluster data that are not linearly separable in the original feature space, k-means clustering was extended to the kernel version. However, the performance of kernel k-means clustering largely depends on the choice of kernel function. To mitigate this problem, multiple kernel learning has been introduced into th…
Combines kernels to create flexible priors in BNNs for seasonal and trend data.
problem Creating flexible priors in Bayesian neural networks for complex data.
method Derives BNN architectures from kernel combinations and periodic functions.
result BNNs can produce periodic kernels useful for capturing seasonal and trend data.
Paper develops a novel kernel-based method for MRI data recovery.
problem Reconstructing dynamic MRI data on manifolds.
method Kernel bi-linear modeling in reproducing kernel Hilbert spaces.
result Validated on synthetic dMRI data, the method outperforms state-of-the-art approaches.
This paper examines the problem of learning with a finite and possibly large set of p base kernels. It presents a theoretical and empirical analysis of an approach addressing this problem based on ensembles of kernel predictors. This includes novel theoretical guarantees based on the Rademacher complexity of the corres…
Develops a kernel method for computing Wasserstein distance.
problem Lack of kernel methods for nonlinear data in Wasserstein distance.
method Kernel trick to compute L2-Wasserstein distance in a kernel space.
result Kernel approach outperforms classical non-kernel methods in identifying CT slices with artifacts.
We propose new positive definite kernels for permutations. First we introduce a weighted version of the Kendall kernel, which allows to weight unequally the contributions of different item pairs in the permutations depending on their ranks. Like the Kendall kernel, we show that the weighted version is invariant to rela…
MKPN predicts varying-sized kernels for burst image denoising.
problem Denoising burst images corrupted by noise.
method Deep neural network (MKPN) predicts and fuses kernels of varying sizes.
result MKPN outperforms state-of-the-art on synthetic datasets.
HKRR adapts to MIM, overcoming the curse of dimensionality.
problem Understanding when deep networks outperform kernel methods in high dimensions.
method Hyper-kernel ridge regression (HKRR) for multi-index models (MIM).
result HKRR can adaptively learn MIM, overcoming the curse of dimensionality.
Meta-KeL learns kernels from offline data to improve sequential decision-making.
problem Adaptive confidence sets for prediction functions in sequential decision-making tasks.
method Meta-KeL: meta-learning a kernel from offline data; structured sparsity estimator for unknown kernel combinations.
result Valid confidence sets that become as tight as those given the true unknown kernel with increasing offline data.
New theoretical tools simplify kernel-based tests analysis.
problem Asymptotic behavior of kernel-based tests in various scenarios.
method Avoids complex expansions and limit theorems, works directly with Hilbert spaces random functionals.
result Framework leads to simpler analysis with minimal regularity conditions.