A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Adaptive kernels from neural networks improve model performance.
RF models implicitly regularize kernel methods as feature count increases.
This paper examines the problem of learning with a finite and possibly large set of p base kernels. It presents a theoretical and empirical analysis of an approach addressing this problem based on ensembles of kernel predictors. This includes novel theoretical guarantees based on the Rademacher complexity of the corres…
Bayesian neural networks explore rare fluctuations for better feature learning.
KernelCobra combines multiple predictors using a kernel to improve prediction performance.
New probabilistic complexity measures for linear and kernel methods.
Paper develops a method to predict cancer patient survival using molecular profiles.
Kernel Three-Pass Regression Filter improves forecasting efficiency for nonlinear dependencies.
In this work, we propose the kernel Pitman-Yor process (KPYP) for nonparametric clustering of data with general spatial or temporal interdependencies. The KPYP is constructed by first introducing an infinite sequence of random locations. Then, based on the stick-breaking construction of the Pitman-Yor process, we defin…
Prediction of dynamical time series with additive noise using support vector machines or kernel based regression has been proved to be consistent for certain classes of discrete dynamical systems. Consistency implies that these methods are effective at computing the expected value of a point at a future time given the …
Proposes a method to create fair, robust predictors that remain consistent across different scenarios.
The neural tangent kernel equivalence theorem fails in practice.
The paper describes an application of Aggregating Algorithm to the problem of regression. It generalizes earlier results concerned with plain linear regression to kernel techniques and presents an on-line algorithm which performs nearly as well as any oblivious kernel predictor. The paper contains the derivation of an …
We propose a specialized string kernel for small bio-molecules, peptides and pseudo-sequences of binding interfaces. The kernel incorporates physico-chemical properties of amino acids and elegantly generalize eight kernels, such as the Oligo, the Weighted Degree, the Blended Spectrum, and the Radial Basis Function. We …
Study on how initialization scale affects neural network training regimes.
New method for reducing dimensions of distributional data.
Estimates KRR risk from training data for various kernels and hyperparameters.
Kernel methods can learn hierarchical polynomials efficiently.
The study reveals how attention paths in Transformers influence learning outcomes.
Given a reproducing kernel Hilbert space H of real-valued functions and a suitable measure mu over the source space D (subset of R), we decompose H as the sum of a subspace of centered functions for mu and its orthogonal in H. This decomposition leads to a special case of ANOVA kernels, for which the functional ANOVA r…
A well-recognized limitation of kernel learning is the requirement to handle a kernel matrix, whose size is quadratic in the number of training examples. Many methods have been proposed to reduce this computational cost, mostly by using a subset of the kernel matrix entries, or some form of low-rank matrix approximatio…
Optimal Biweight kernel and computationally efficient Epanechnikov kernel for modal linear regression.
Paper speeds up Gaussian process inference using Matérn kernels.
New method uses machine learning to improve statistical inference.
Study shows that ridgeless Gaussian kernel regression overfits even with varying bandwidth or dimensionality.
Kernel models learn low-dimensional predictive subspaces from input data.
For supervised and unsupervised learning, positive definite kernels allow to use large and potentially infinite dimensional feature spaces with a computational cost that only depends on the number of observations. This is usually done through the penalization of predictor functions by Euclidean or Hilbertian norms. In …
Study optimizes prediction error for growing-dimensional PFLM models.
Quantum kernel improves solar irradiance forecasting.
New insights into simple kernel smoothing reveal surprising asymptotics.
A recent line of work studies overparametrized neural networks in the "kernel regime," i.e. when the network behaves during training as a kernelized linear predictor, and thus training with gradient descent has the effect of finding the minimum RKHS norm solution. This stands in contrast to other studies which demonstr…
Needlets have been recognized as state-of-the-art tools to tackle spherical data, due to their excellent localization properties in both spacial and frequency domains. This paper considers developing kernel methods associated with the needlet kernel for nonparametric regression problems whose predictor variables are de…
AGOP from KRR recovers central subspace in fewer samples than needed for prediction.
Learning from examples is one of the key problems in science and engineering. It deals with function reconstruction from a finite set of direct and noisy samples. Regularization in reproducing kernel Hilbert spaces (RKHSs) is widely used to solve this task and includes powerful estimators such as regularization network…
Variable selection is central to high-dimensional data analysis, and various algorithms have been developed. Ideally, a variable selection algorithm shall be flexible, scalable, and with theoretical guarantee, yet most existing algorithms cannot attain these properties at the same time. In this article, a three-step va…
We establish optimal convergence rates for a decomposition-based scalable approach to kernel ridge regression. The method is simple to describe: it randomly partitions a dataset of size N into m subsets of equal size, computes an independent kernel ridge regression estimator for each subset, then averages the local sol…
In this paper, we consider the nonparametric least square regression in a Reproducing Kernel Hilbert Space (RKHS). We propose a new randomized algorithm that has optimal generalization error bounds with respect to the square loss, closing a long-standing gap between upper and lower bounds. Moreover, we show that our al…
Study loop corrections in random feature models affecting training and test errors.
Extends neural network training framework to handle noise and uncertainty.
Paper proposes a robust LPR method using similarity kernels.
A key question in modern statistics is how to make fast and reliable inferences for complex, high-dimensional data. While there has been much interest in sparse techniques, current methods do not generalize well to data with nonlinear structure. In this work, we present an orthogonal series estimator for predictors tha…
Improves regression efficiency by separating material and immaterial parts of responses.
Survival analysis is a fundamental tool in medical research to identify predictors of adverse events and develop systems for clinical decision support. In order to leverage large amounts of patient data, efficient optimisation routines are paramount. We propose an efficient training algorithm for the kernel survival su…
New bounds prevent degradation in high-dimensional signal estimation.
We consider the problem of simultaneously learning to linearly combine a very large number of kernels and learn a good predictor based on the learnt kernel. When the number of kernels to be combined is very large, multiple kernel learning methods whose computational cost scales linearly in are intractable. We p…
Paper connects risk consistency to L_p consistency for broader loss functions.
This research improves online learning by correcting for target shift in machine learning.
New insights into CI tests reveal key factors for practical performance.