arXiv research

Random Machines improves SVM performance with free kernel choice.

GPs' decisions can vary significantly with different kernels, even if kernels are qualitatively similar.

Study evaluates RKHS choices for assessing graph models using KSD tests.

Optimal kernel in KR can be data-dependent, improving model performance.

Improves kernel ridge regression by optimizing scale and feature parameters.

Gaussian kernel fails on circle and related spaces.

The generalization properties of Gaussian processes depend heavily on the choice of kernel, and this choice remains a dark art. We present the Neural Kernel Network (NKN), a flexible family of kernels represented by a neural network. The NKN architecture is based on the composition rules for kernels, so that each unit …

Develops UKP for comparing feature representations in multitask learning.

This paper tackles the problem of selecting among several linear estimators in non-parametric regression; this includes model selection for linear regression, the choice of a regularization parameter in kernel ridge regression, spline smoothing or locally weighted regression, and the choice of a kernel in multiple kern…

NVGD uses neural networks to infer distributions without kernel choices.

Alternative proof of coisotropic embedding theorem for pre-symplectic manifolds.

Kernel methods are one of the mainstays of machine learning, but the problem of kernel learning remains challenging, with only a few heuristics and very little theory. This is of particular importance in methods based on estimation of kernel mean embeddings of probability measures. For characteristic kernels, which inc…

New ensemble SVM model reduces prediction error without choosing best kernel.

A new framework improves kernel Stein discrepancy tests for validating distributions.

Kernel Dynamic Mode Decomposition reconstructs dynamical systems using Laplacian kernel.

Bayesian Optimization (BO) has become a core method for solving expensive black-box optimization problems. While much research focussed on the choice of the acquisition function, we focus on online length-scale adaption and the choice of kernel function. Instead of choosing hyperparameters in view of maximum likelihood…

The article introduces practical estimators for kernel discrepancies.

We show that minimum-norm interpolation in the Reproducing Kernel Hilbert Space corresponding to the Laplace kernel is not consistent if input dimension is constant. The lower bound holds for any choice of kernel bandwidth, even if selected based on data. The result supports the empirical observation that minimum-norm …

This paper tackles scalability issues in kernel logistic regression for large datasets.

The expressive power of Gaussian processes depends heavily on the choice of kernel. In this work we propose the novel harmonizable mixture kernel (HMK), a family of expressive, interpretable, non-stationary kernels derived from mixture models on the generalized spectral representation. As a theoretically sound treatmen…

New framework explains neural network bias in solving differential equations.

Study proves existence, uniqueness, and positivity of solutions to a complex volatility model.

We establish the first nonasymptotic error bounds for Kaplan-Meier-based nearest neighbor and kernel survival probability estimators where feature vectors reside in metric spaces. Our bounds imply rates of strong consistency for these nonparametric estimators and, up to a log factor, match an existing lower bound for c…

The success of kernel-based learning methods depend on the choice of kernel. Recently, kernel learning methods have been proposed that use data to select the most appropriate kernel, usually by combining a set of base kernels. We introduce a new algorithm for kernel learning that combines a {\em continuous set of base …

Information-theoretic Bayesian optimisation techniques have demonstrated state-of-the-art performance in tackling important global optimisation problems. However, current information-theoretic approaches require many approximations in implementation, introduce often-prohibitive computational overhead and limit the choi…

A family of maximum mean discrepancy (MMD) kernel two-sample tests is introduced. Members of the test family are called Block-tests or B-tests, since the test statistic is an average over MMDs computed on subsets of the samples. The choice of block size allows control over the tradeoff between test power and computatio…

The paper examines how kernel approximations affect Gaussian process regression in large data applications.

Identifies a gradient flow to solve kernel learning problems with noise reduction.

Bayesian Gaussian Processes improve exoplanet transit and Hubble constant inference.

Kernel K-means clusters probability distributions.

Support Vector Machine (SVM) is powerful classification technique based on the idea of structural risk minimization. Use of kernel function enables curse of dimensionality to be addressed. However, proper kernel function for certain problem is dependent on specific dataset and as such there is no good method on choice …

We introduce a simulation method for dynamic portfolio valuation and risk management building on machine learning with kernels. We learn the dynamic value process of a portfolio from a finite sample of its cumulative cash flow. The learned value process is given in closed form thanks to a suitable choice of the kernel.…

Conditional kernel mean embeddings are nonparametric models that encode conditional expectations in a reproducing kernel Hilbert space. While they provide a flexible and powerful framework for probabilistic inference, their performance is highly dependent on the choice of kernel and regularization hyperparameters. Neve…

New method controls false discoveries in structured hypothesis spaces.

Kernel methods are ubiquitous tools in machine learning. However, there is often little reason for the common practice of selecting a kernel a priori. Even if a universal approximating kernel is selected, the quality of the finite sample estimator may be greatly affected by the choice of kernel. Furthermore, when direc…

Performing exact posterior inference in complex generative models is often difficult or impossible due to an expensive to evaluate or intractable likelihood function. Approximate Bayesian computation (ABC) is an inference framework that constructs an approximation to the true likelihood based on the similarity between …

The paper examines Gaussian process means under misspecified likelihoods and smoothness.

A new kernel for probability measures based on optimal transport.

Geometric theory connects machine learning classifiers to differential geometry.

This thesis improves kernel-based distances for statistical inference and integration.

Study provides guarantees for kernel clustering under non-parametric mixtures.

High-dimensional kernel regression struggles due to rotational invariance.

Many investment models in discrete or continuous-time settings boil down to maximizing an objective of the quantile function of the decision variable. This quantile optimization problem is known as the quantile formulation of the original investment problem. Under certain monotonicity assumptions, several schemes to so…

Kernel $ε$-Greedy optimizes multi-armed bandits with covariates for sub-linear regret.

Kernel methods are among the most popular techniques in machine learning. From a frequentist/discriminative perspective they play a central role in regularization theory as they provide a natural choice for the hypotheses space and the regularization functional through the notion of reproducing kernel Hilbert spaces. F…

A new method uses multifidelity Gaussian process regression to solve nonlinear PDEs.

New method uses multiple kernels to improve SVGD performance.

Sparse Kernel Flows learns dynamical systems from data.