A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Bayesian optimization for set inputs using approximate set kernels.
We study the problem of column selection in large-scale kernel canonical correlation analysis (KCCA) using the Nyström approximation, where one approximates two positive semi-definite kernel matrices using "landmark" points from the training set. When building low-rank kernel approximations in KCCA, previous work mostl…
Paper improves kernel approximations for better statistical learning.
Kernel-based function approximation improves reinforcement learning performance.
New method improves Gaussian kernel approximations for high-frequency data.
SRF improves kernel approximation and GP regression performance.
Adapts deep learning with kernel methods for efficient learning.
Ad-SVGD optimizes kernel parameters for SVGD, improving inference performance.
Accelerated RPCholesky speeds up kernel matrix approximations.
We study the construction of coresets for kernel density estimates. That is we show how to approximate the kernel density estimate described by a large point set with another kernel density estimate with a much smaller point set. For characteristic kernels (including Gaussian and Laplace kernels), our approximation pre…
We investigate how to train kernel approximation methods that generalize well under a memory budget. Building on recent theoretical work, we define a measure of kernel approximation error which we find to be more predictive of the empirical generalization performance of kernel approximation methods than conventional me…
Extends Tanimoto kernel to real-valued functions.
New sparse GP model learns compositional kernels efficiently.
The paper improves error bounds for Bayesian quadrature in noisy settings.
Stochastic gradient descent optimizes Nyström samples for kernel matrix approximation.
Positive definite kernels and their associated Reproducing Kernel Hilbert Spaces provide a mathematically compelling and practically competitive framework for learning from data. In this paper we take the approximation theory point of view to explore various aspects of smooth kernels related to their inferential proper…
We give the first algorithm for kernel Nyström approximation that runs in *linear time in the number of training points* and is provably accurate for all kernel matrices, without dependence on regularity or incoherence conditions. The algorithm projects the kernel onto a set of landmark points sampled by their *rid…
Improved kernel ridge regression using conjugate gradients.
A novel Laplace-approximated Bayesian Tensor Network Kernel Machine (LA-TNKM) provides principled uncertainty estimates.
In this paper we measured the stability of stochastic gradient method (SGM) for learning an approximated Fourier primal support vector machine. The stability of an algorithm is considered by measuring the generalization error in terms of the absolute difference between the test and the training error. Our problem is to…
Gaussian processes are powerful, yet analytically tractable models for supervised learning. A Gaussian process is characterized by a mean function and a covariance function (kernel), which are determined by a model selection criterion. The functions to be compared do not just differ in their parametrization but in thei…
New algorithms learn in complex decision-making problems with smooth transitions.
Kernel approximation using randomized feature maps has recently gained a lot of interest. In this work, we identify that previous approaches for polynomial kernel approximation create maps that are rank deficient, and therefore do not utilize the capacity of the projected feature space effectively. To address this chal…
Proposes landmark selection for kernel methods.
Kernel methods offer the flexibility to learn complex relationships in modern, large data sets while enjoying strong theoretical guarantees on quality. Unfortunately, these methods typically require cubic running time in the data set size, a prohibitive cost in the large-data setting. Random feature maps (RFMs) and the…
Deep networks can be approximated as Gaussian processes, linking training methods and kernel learning.
We consider fast kernel summations in high dimensions: given a large set of points in dimensions (with ) and a pair-potential function (the {\em kernel} function), we compute a weighted sum of all pairwise kernel interactions for each point in the set. Direct summation is equivalent to a (dense) matrix-vec…
Random feature approximation speeds up spectral methods and improves learning rates.
Many interesting machine learning problems are best posed by considering instances that are distributions, or sample sets drawn from distributions. Previous work devoted to machine learning tasks with distributional inputs has done so through pairwise kernel evaluations between pdfs (or sample sets). While such an appr…
Efficiently approximates kernel mean embeddings using Nyström method.
We consider the problem of improving the efficiency of randomized Fourier feature maps to accelerate training and testing speed of kernel methods on large datasets. These approximate feature maps arise as Monte Carlo approximations to integral representations of shift-invariant kernel functions (e.g., Gaussian kernel).…
Functional input neural networks approximate continuous functions on weighted spaces.
We present an approximation scheme for support vector machine models that use an RBF kernel. A second-order Maclaurin series approximation is used for exponentials of inner products between support vectors and test instances. The approximation is applicable to all kernel methods featuring sums of kernel evaluations and…
New method uses kernel Stein discrepancy for measure transport without strict continuity constraints.
Convolution and pooling improve kernel methods in image classification.
New method reduces variance and bias in approximating indefinite kernels.
We introduce two versions of a new sketch for approximately embedding the Gaussian kernel into Euclidean inner product space. These work by truncating infinite expansions of the Gaussian kernel, and carefully invoking the RecursiveTensorSketch [Ahle et al. SODA 2020]. After providing concentration and approximation pro…
This work studies nonnegativity-preserving kernels for stochastic equations and their applications.
New kernel enables exact GP analysis of massive datasets.
We provide a fast approximation to eNTKs for neural networks.
Framework extends neural operators to handle functions outside training set.
Kernel method learns PDEs from noisy data.
In this paper, we propose a data-adaptive non-parametric kernel learning framework in margin based kernel methods. In model formulation, given an initial kernel matrix, a data-adaptive matrix with two constraints is imposed in an entry-wise scheme. Learning this data-adaptive matrix in a formulation-free strategy enlar…
Nonlinear kernel regression models are often used in statistics and machine learning because they are more accurate than linear models. Variable selection for kernel regression models is a challenge partly because, unlike the linear regression setting, there is no clear concept of an effect size for regression coeffici…
New framework estimates eigenvalues of kernel matrices without full matrix construction.
Advanced kernels improve Gaussian process accuracy by incorporating domain knowledge.
A new method for approximating softmax and Gaussian kernels with reduced error.
McKernel introduces a framework to use kernel approximates in the mini-batch setting with Stochastic Gradient Descent (SGD) as an alternative to Deep Learning. Based on Random Kitchen Sinks [Rahimi and Recht 2007], we provide a C++ library for Large-scale Machine Learning. It contains a CPU optimized implementation of …