The kernel least mean squares (KLMS) algorithm is a computationally efficient nonlinear adaptive filtering method that "kernelizes" the celebrated (linear) least mean squares algorithm. We demonstrate that the least mean squares algorithm is closely related to the Kalman filtering, and thus, the KLMS can be interpreted…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,341 papers · 148 categories
Trend · papers per month
48 results for “kernel least mean squares”
The ratio of two probability densities can be used for solving various machine learning tasks such as covariate shift adaptation (importance sampling), outlier detection (likelihood-ratio test), and feature selection (mutual information). Recently, several methods of directly estimating the density ratio have been deve…
Study reveals KLMS algorithms as simplified GP regression models.
problem Understanding KLMS algorithms and their performance differences.
method Examined the relationship between online Gaussian process regression and KLMS algorithms.
result KLMS algorithms correspond to specific cases of a parametric model of posterior covariance.
Kernel adaptive filters (KAF) are a class of powerful nonlinear filters developed in Reproducing Kernel Hilbert Space (RKHS). The Gaussian kernel is usually the default kernel in KAF algorithms, but selecting the proper kernel size (bandwidth) is still an open important issue especially for learning with small sample s…
This paper presents a stochastic behavior analysis of a kernel-based stochastic restricted-gradient descent method. The restricted gradient gives a steepest ascent direction within the so-called dictionary subspace. The analysis provides the transient and steady state performance in the mean squared error criterion. It…
Learning interaction kernels in mean-field equations of 1st-order systems of interacting particlesstat.ML
Algorithm learns interaction kernels for particle systems from data.
problem Understanding and modeling interactions in systems of interacting particles.
method Nonparametric algorithm using least squares with regularization, probabilistic error functional, and reproducing kernel Hilbert space convergence.
result The algorithm converges optimally and accurately learns interaction kernels.
Improved kernel ridge regression using conjugate gradients.
problem Efficiently solving large-scale kernel ridge regression problems.
method Structured Gaussian regression model with low-rank approximation and conjugate gradients.
result Enhanced approximation of kernel ridge regressor/Gaussian process posterior mean.
Optimal Biweight kernel and computationally efficient Epanechnikov kernel for modal linear regression.
problem Finding the best kernel for modal linear regression.
method Refined analysis of asymptotic statistical behavior and IRLS algorithm convergence.
result Biweight kernel minimizes asymptotic mean squared error, Epanechnikov kernel guarantees IRLS convergence.
New methods improve accuracy and scalability for large datasets in multi-class classification.
problem Improving accuracy and scalability for multi-class classification with large datasets.
method Randomized block kernel matrices for approximation of least-squares support vector machines.
result The proposed methods provide good accuracy and reliable scaling for multi-class classification problems with large data sets.
Sparse LR-LSSVM improves kernel machine performance.
problem Improving kernel machine performance with controlled model size.
method Introduces LR-LSSVM with low rank kernels and a two-step optimization algorithm.
result Proposed algorithm's performance is comparable or superior to existing kernel machines.
Study of regularized least squares in RKKS with indefinite kernels.
problem Asymptotic properties of regularized least squares with indefinite kernels in RKKS.
method Introducing a bounded hyper-sphere constraint, theoretical demonstration of globally optimal solution, modified error decomposition techniques, matrix perturbation theory.
result Derivation of learning rates in RKKS, same as RKHS under certain conditions.
Adaptive filtering algorithms operating in reproducing kernel Hilbert spaces have demonstrated superiority over their linear counterpart for nonlinear system identification. Unfortunately, an undesirable characteristic of these methods is that the order of the filters grows linearly with the number of input data. This …
We prove the statistical consistency of kernel Partial Least Squares Regression applied to a bounded regression learning problem on a reproducing kernel Hilbert space. Partial Least Squares stands out of well-known classical approaches as e.g. Ridge Regression or Principal Components Regression, as it is not defined as…
Improved robustness in kernel-based regression via novel loss function and IRLS.
problem Noise sensitivity in kernel-based regression methods.
method Proposed -loss function and iteratively reweighted least squares (IRLS) optimization.
result Improved noise robustness in kernel-based regression methods.
Optimizes learning rates for kernel-based expectile regression.
problem Estimating conditional expectiles efficiently.
method Support vector machine type approach using Gaussian RBF kernels.
result Learning rates are minimax optimal with a logarithmic factor.
Distributed learning with least squares regularization achieves good performance without eigenfunction assumptions.
problem Efficiently learning from large datasets distributed across multiple machines.
method Divide-and-conquer approach, least squares regularization, RKHS, error bounds in expectation.
result The global estimator is a good approximation to the full data estimator, with sharp error bounds.
A new method for complex-valued signals improves convergence and performance.
problem Nonlinear channel equalization and complex-valued signals with different properties.
method Generalized complex kernel least-mean-square (gCKLMS) algorithm.
result The gCKLMS algorithm converges faster and performs better than previous methods.
We prove rates of convergence in the statistical sense for kernel-based least squares regression using a conjugate gradient algorithm, where regularization against overfitting is obtained by early stopping. This method is directly related to Kernel Partial Least Squares, a regression method that combines supervised dim…
We propose a method for nonparametric density estimation that exhibits robustness to contamination of the training sample. This method achieves robustness by combining a traditional kernel density estimator (KDE) with ideas from classical -estimation. We interpret the KDE based on a radial, positive semi-definite ke…
Kernel methods with random projections improve least-squares regression efficiency.
problem Efficiently solving least-squares regression problems in high-dimensional spaces.
method Kernel conjugate gradient methods with randomized sketches and Nyström subsampling.
result Optimal generalization and computational advantages with proportional projection dimensions.
Kernel Conjugate Gradient achieves fast convergence rates for regression.
problem Statistical rates of convergence for kernel-based regression.
method Kernel Conjugate Gradient algorithm with early stopping for regularization.
result Upper bounds for and Hilbert norms, matching minimax lower bounds.
LS-SVM performance analyzed for large p and n, showing Gaussian approximation.
problem Performance analysis of LS-SVM in high-dimensional settings.
method Random matrix theory applied to LS-SVM under Gaussian mixture model.
result LS-SVM decision function approximates normally distributed under large p and n.
This paper presents regression models obtained from a process of blind prediction of peptide binding affinity from provided descriptors for several distinct datasets as part of the 2006 Comparative Evaluation of Prediction Algorithms (COEPRA) contest. This paper finds that kernel partial least squares, a nonlinear part…
New algorithm improves regression error bounds and accelerates performance for low noise.
problem Nonparametric least square regression in RKHS with optimal error bounds.
method Kernel Truncated Randomized Ridge Regression (KTRRR) with optimal generalization error bounds.
result Faster finite-time and asymptotic rates on low noise problems.
This note optimizes distributions using kernel mean embeddings with a new parameterization.
problem Optimizing distributions using kernel mean embeddings is challenging due to the difficulty of characterizing probability distribution vectors.
method Proposes a new parameterization of positive functions using kernel sums-of-squares to fit distributions in the MMD geometry.
result Distributions with kernel sum-of-squares densities are dense in the MMD geometry, allowing optimization in the finite-sample setting.
A new hybrid federated learning algorithm for combining clinical and omics data.
problem Combining clinical and omics data in federated learning settings.
method Reformulated Kernel Regularized Least Squares algorithm for hybrid federated learning.
result Validation of two variants of the hybrid algorithm on well-established datasets.
Paper presents novel online MTL methods using WRLS and OSLSSVR.
problem Online Multi-Task Learning (MTL) Regression Problems
method Develops recursive versions of WRLS and OSLSSVR for MTL.
result Achieves exact and approximate recursions with quadratic cost.
Although operator-valued kernels have recently received increasing interest in various machine learning and functional data analysis problems such as multi-task learning or functional regression, little attention has been paid to the understanding of their associated feature spaces. In this paper, we explore the potent…
Paper studies a robust online learning algorithm for regression.
problem Develops a robust online learning algorithm for regression problems.
method Introduces an online learning algorithm with a robust loss function over RKHS.
result The algorithm achieves optimal convergence rates in mean square and RKHS.
Study on gradient dynamics of shallow ReLU networks for least-squares interpolation.
problem Understanding the gradient dynamics of shallow ReLU networks for interpolation.
method Theoretical and empirical analysis of gradient flow in non-redundant parameterization.
result Identification of two learning regimes: kernel and adaptive, with distinct interpolant shapes.
Kernel regression predicts graph signals in noisy environments.
problem Predicting smooth graph signals in the presence of sparse noise.
method Kernel regression with -norm and -norm optimization using IRLS.
result Efficacy demonstrated on real-world temperature data.
Paper develops efficient estimator for Hawkes processes using representer theorem.
problem Estimating latent triggering kernels for Hawkes processes from event sequences.
method Penalized least squares minimization in RKHS framework.
result Efficient estimator with competitive accuracy and improved computational efficiency.
The runtime for Kernel Partial Least Squares (KPLS) to compute the fit is quadratic in the number of examples. However, the necessity of obtaining sensitivity measures as degrees of freedom for model selection or confidence intervals for more detailed analysis requires cubic runtime, and thus constitutes a computationa…
Adaptive Gaussian kernel filtering with updated parameters.
problem Improving kernel adaptive filtering for better performance.
method Adaptive updating of Gaussian kernel parameters on an SPD manifold.
result Validation of the proposed method through experimental results.
A new learning method for complex functions using kernel and range space.
problem Learning complex functions represented as systems of linear equations.
method Manipulation in the kernel and range space for least squares error approximation.
result The method successfully learns a deep feedforward network.
Paper criticizes mFLMS algorithm for power signal estimation.
problem Criticism of mFLMS algorithm for power signal parameter estimation.
method Recently proposed mFLMS algorithm for power signal parameter estimation.
result Simulation results show mFLMS does not outperform LMS.
The study analyzes LS-SVM performance using random matrix theory.
problem Performance analysis of LS-SVM in large dimensional settings.
method Random matrix theory applied to LS-SVM performance analysis.
result The LS-SVM decision function is asymptotically normal.
Paper introduces a new multi-kernel algorithm for better gradient approximation.
problem Improving gradient approximation in high-dimensional problems.
method Develops a multi-kernel passive stochastic gradient algorithm with variance reduction.
result The multi-kernel algorithm performs better in high-dimensional problems.
gKRLS accelerates KRLS estimation for complex models.
problem Limited flexibility and high computation for KRLS.
method Re-formulate KRLS as a hierarchical model and implement random sketching.
result gKRLS can fit models on large datasets in minutes.
Rational Krigingstat.ME
New kriging method improves mean estimation and uncertainty.
problem Improving mean estimation and uncertainty in kriging.
method Proposes a rational kriging method using generalized least squares and Gaussian process.
result Generalized least squares estimate is more well-behaved than ordinary kriging.
The paper analyzes learning schemes for various stationary stochastic processes.
problem Analyzing learning schemes with different stationary stochastic processes.
method Unified treatment of various mixing processes using generalized Bernstein-type inequality.
result Sharp oracle inequalities and convergence rates for learning schemes.
We consider adaptive system identification problems with convex constraints and propose a family of regularized Least-Mean-Square (LMS) algorithms. We show that with a properly selected regularization parameter the regularized LMS provably dominates its conventional counterpart in terms of mean square deviations. We es…
If pricing kernels are assumed non-negative then the inverse problem of finding the pricing kernel is well-posed. The constrained least squares method provides a consistent estimate of the pricing kernel. When the data are limited, a new method is suggested: relaxed maximization of the relative entropy. This estimator …
Paper optimizes prediction in semi-functional linear models using kernel methods.
problem Optimizing prediction in semi-functional linear models with functional and nonparametric components.
method Double-penalized least squares method in reproducing kernel Hilbert spaces, with regularization parameter selection via generalized cross validation.
result Achieves minimax optimal rates of convergence for both functional and nonparametric components.
A joint optimization approach to identifying sparse dynamics using least squares kernel collocationstat.ME
Develops a new method for learning ODEs from sparse data.
problem Learning systems of ODEs from scarce, partial, and noisy data.
method Combines sparse recovery and RKHS techniques.
result Significant gains in accuracy, sample efficiency, and robustness to noise.
Optimal hedging strategy found in markets with incomplete pricing kernels.
problem Finding optimal hedging in markets with incomplete pricing kernels.
method Demonstrated existence of an optimal hedge portfolio using an expected least squared-error criterion.
result Existence of an optimal hedge portfolio in Lévy-Ito markets.
New deep learning solver for high-dimensional derivative pricing.
problem High-dimensional derivatives pricing problems.
method Combines deep learning with least square regression for backward SDE solving.
result Accurate and efficient pricing of complex derivatives.
A new kernel method improves Poisson process intensity estimation.
problem Estimating intensity functions of inhomogeneous Poisson processes.
method Kernel method-based intensity estimator using least squares loss.
result KIE achieves comparable predictive performance with improved efficiency.