The kernel least mean squares (KLMS) algorithm is a computationally efficient nonlinear adaptive filtering method that "kernelizes" the celebrated (linear) least mean squares algorithm. We demonstrate that the least mean squares algorithm is closely related to the Kalman filtering, and thus, the KLMS can be interpreted…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,932 papers · 148 categories
Trend · papers per month
48 results for “Least-mean squares”
We consider adaptive system identification problems with convex constraints and propose a family of regularized Least-Mean-Square (LMS) algorithms. We show that with a properly selected regularization parameter the regularized LMS provably dominates its conventional counterpart in terms of mean square deviations. We es…
We study the relationship between online Gaussian process (GP) regression and kernel least mean squares (KLMS) algorithms. While the latter have no capacity of storing the entire posterior distribution during online learning, we discover that their operation corresponds to the assumption of a fixed posterior covariance…
The purpose of this paper is to indicate that the recently proposed Momentum fractional least mean squares (mFLMS) algorithm has some serious flaws in its design and analysis. Our apprehensions are based on the evidence we found in the derivation and analysis in the paper titled: \textquotedblleft \textit{Momentum frac…
Kernel adaptive filters (KAF) are a class of powerful nonlinear filters developed in Reproducing Kernel Hilbert Space (RKHS). The Gaussian kernel is usually the default kernel in KAF algorithms, but selecting the proper kernel size (bandwidth) is still an open important issue especially for learning with small sample s…
Constant Step Size Least-Mean-Square: Bias-Variance Trade-offs and Optimal Sampling Distributionscs.LG
We consider the least-squares regression problem and provide a detailed asymptotic analysis of the performance of averaged constant-step-size stochastic gradient descent (a.k.a. least-mean-squares). In the strongly-convex case, we provide an asymptotic expansion up to explicit exponentially decaying terms. Our analysis…
New algorithm tightens sensitivity bounds for coresets, reducing size by up to 10x.
problem Finding efficient coresets for least-mean-squares problems.
method Algorithm based on iterative sensitivity computation and reduction to higher-dimensional subspaces.
result Significantly smaller coresets with data-dependent sensitivity bounds.
Solves fundamental machine learning problems with a faster algorithm.
problem Fundamental machine learning problems and their building blocks.
method A novel algorithm that computes a subset of vectors whose sum is exactly the same, using sketches and coresets.
result Computes a subset of vectors in time, up to x100 faster than existing LMS solvers.
This paper presents a stochastic behavior analysis of a kernel-based stochastic restricted-gradient descent method. The restricted gradient gives a steepest ascent direction within the so-called dictionary subspace. The analysis provides the transient and steady state performance in the mean squared error criterion. It…
Paper analyzes robustness of data-selective Volterra NLMS algorithm.
problem Robustness analysis of data-selective Volterra NLMS algorithm.
method The paper analyzes the local robustness and proposes a global bound for the error in the coefficient vector.
result The DS-VNLMS algorithm is robust against noise and improves parameter estimation for most iterations.
Adaptive filtering algorithms operating in reproducing kernel Hilbert spaces have demonstrated superiority over their linear counterpart for nonlinear system identification. Unfortunately, an undesirable characteristic of these methods is that the order of the filters grows linearly with the number of input data. This …
Improved backpropagation with consequentialism weight updates for neural networks.
problem Improving backpropagation for neural networks, especially with mini-batch training.
method Introducing consequentialism weight updates derived from NLMS for multi-layer neural networks.
result The proposed method outperforms traditional BP and mini-batch training.
This letter presents an improved version of diffusion least mean ppower (LMP) algorithm for distributed estimation. Instead of sum of mean square errors, a weighted sum of mean square error is defined as the cost function for global and local cost functions of a network of sensors. The weight coefficients are updated b…
We propose a version of least-mean-square (LMS) algorithm for sparse system identification. Our algorithm called online linearized Bregman iteration (OLBI) is derived from minimizing the cumulative prediction error squared along with an l1-l2 norm regularizer. By systematically treating the non-differentiable regulariz…
We propose a novel adaptive kernel based regression method for complex-valued signals: the generalized complex-valued kernel least-mean-square (gCKLMS). We borrow from the new results on widely linear reproducing kernel Hilbert space (WL-RKHS) for nonlinear regression and complex-valued signals, recently proposed by th…
We introduce a probabilistic approach to the LMS filter. By means of an efficient approximation, this approach provides an adaptable step-size LMS algorithm together with a measure of uncertainty about the estimation. In addition, the proposed approximation preserves the linear complexity of the standard LMS. Numerical…
Robust diffusion adaptive estimation algorithms based on the maximum correntropy criterion (MCC), including adaptation to combination MCC and combination to adaptation MCC, are developed to deal with the distributed estimation over network in impulsive (long-tailed) noise environments. The cost functions used in distri…
In this paper, we propose to apply object detection methods from the vision domain on the speech recognition domain, by treating audio fragments as objects. More specifically, we present SpeechYOLO, which is inspired by the YOLO algorithm for object detection in images. The goal of SpeechYOLO is to localize boundaries …
Constrained adaptive filtering algorithms inculding constrained least mean square (CLMS), constrained affine projection (CAP) and constrained recursive least squares (CRLS) have been extensively studied in many applications. Most existing constrained adaptive filtering algorithms are developed under mean square error (…
New algorithm improves learning in noisy networks with robust performance.
problem Improving learning in noisy, distributed networks.
method Diffusion normalized least mean M-estimate algorithm with sparse-aware variant.
result The proposed algorithms outperform existing diffusion algorithms in impulsive noise scenarios.
Proposed in 1991, Least Mean Square Error Reconstruction for self-organizing network, shortly Lmser, was a further development of the traditional auto-encoder (AE) by folding the architecture with respect to the central coding layer and thus leading to the features of symmetric weights and neurons, as well as jointly s…
The kernel least-mean-square (KLMS) algorithm is an appealing tool for online identification of nonlinear systems due to its simplicity and robustness. In addition to choosing a reproducing kernel and setting filter parameters, designing a KLMS adaptive filter requires to select a so-called dictionary in order to get a…
Paper introduces a new multi-kernel algorithm for better gradient approximation.
problem Improving gradient approximation in high-dimensional problems.
method Develops a multi-kernel passive stochastic gradient algorithm with variance reduction.
result The multi-kernel algorithm performs better in high-dimensional problems.
Randomized algorithms that base iteration-level decisions on samples from some pool are ubiquitous in machine learning and optimization. Examples include stochastic gradient descent and randomized coordinate descent. This paper makes progress at theoretically evaluating the difference in performance between sampling wi…
Iterative algorithms are ubiquitous in the field of data mining. Widely known examples of such algorithms are the least mean square algorithm, backpropagation algorithm of neural networks. Our contribution in this paper is an improvement upon this iterative algorithms in terms of their respective performance metrics an…
Mid-price Prediction Based on Machine Learning Methods with Technical and Quantitative Indicatorsq-fin.ST
Study predicts stock mid-prices using machine learning with selected features.
problem Predicting mid-price movements in stock markets.
method Wrapper feature selection using entropy, LMS, LDA, and adaptive logistic regression.
result Best performance achieved with a small set of advanced features.
On Data-Selective Learningeess.SP
New adaptive filters reduce energy consumption in electronic devices.
problem Reducing energy consumption in adaptive filtering algorithms.
method Data-selective adaptive filters, set-membership (SM) filters, trinion and quaternion systems, partial-updating, LMS and recursive LMS algorithms.
result Improved adaptive filtering algorithms with reduced computational complexity and enhanced stability.
The NLMS algorithm with time-variant optimum stepsize derived from a Bayesian network perspectivestat.ML
In this article, we derive a new stepsize adaptation for the normalized least mean square algorithm (NLMS) by describing the task of linear acoustic echo cancellation from a Bayesian network perspective. Similar to the well-known Kalman filter equations, we model the acoustic wave propagation from the loudspeaker to th…
In the last decade, a considerable research effort has been devoted to developing adaptive algorithms based on kernel functions. One of the main features of these algorithms is that they form a family of universal approximation techniques, solving problems with nonlinearities elegantly. In this paper, we present data-s…
In this paper we introduce a completely continuous and time-variate model of the evolution of market limit orders based on the existence, uniqueness, and regularity of the solutions to a type of stochastic partial differential equations obtained in Zheng and Sowers (2012). In contrary to several models proposed and res…
A distance-squared function is one of the most significant functions in the application of singularity theory to differential geometry. Moreover, distance-squared mappings are naturally extended mappings of distance-squared functions, wherein each component is a distance-squared function. In this paper, compositions of…
Square-tiled torimath.GT
We study square-tiled tori, that is, tori obtained from a finite collection of unit squares by parallel side identifications. Square-tiled tori can be parametrized in a natural way that allows to count the number of square-tiled tori tiled by a given number of square tiles. There is a natural $\mathrm{SL}(2,\mathbf{Z})…
Study finds Hilbert square of real surfaces can be maximal even when the surface has disconnected real locus.
problem Exploring conditions for maximality of Hilbert square of real surfaces.
method Analyzing Hilbert square of maximal real surfaces and examining specific examples.
result Hilbert square can be maximal even for surfaces with disconnected real locus.
Directly simulates squared Bessel processes efficiently.
problem Simulating squared Bessel processes accurately and efficiently.
method Two-dimensional Chebyshev expansion for non-central chi-square distribution inverse.
result Accurate and efficient simulation for various degrees of freedom.
CD converges linearly for MCP/SCAD penalized least squares.
problem Recovering sparse signals from data.
method Coordinate descent for MCP/SCAD penalized least squares.
result CD converges linearly to solutions of MCP/SCAD penalized least squares.
This paper uses CausalGANs and RL with LLM to predict bond yields.
problem Challenges in financial bond yield forecasting due to data scarcity and market conditions.
method Proposes a novel framework combining CausalGANs, RL, and LLM for synthetic data generation and trading signals.
result Improves forecasting performance over existing methods with low Mean Absolute Error.
Study on Nijenhuis tensor forms and vanishing properties.
problem Understanding the properties of Nijenhuis tensor.
method Provided strong and weak forms of the square of Nijenhuis tensor, and identified vanishing results.
result Identified new vanishing results for the square of Nijenhuis tensor.
Study differential properties of matrix square roots in specific cases.
problem Understanding matrix square roots in semi-simple, symmetric, and orthogonal cases.
method Analysis of differential and metric structures of real square roots of matrices under specific conditions.
result Differential properties of matrix square roots in semi-simple, symmetric, and orthogonal cases.
Square-like quadrilaterals inscribed in space curves proven for finite total curvature.
problem Square-peg problem in space curves.
method Local curvature analysis and limiting argument on approximating curves.
result Every embedded curve with finite total curvature has an inscribed square-like quadrilateral.
Shear moves connect square-tiled surfaces in quadratic differentials.
problem Connecting square-tiled surfaces via specific moves.
method Shear moves corresponding to diagonal flips preserving square-tiled properties.
result Connected components of reconfiguration problem are in bijection with moduli space of quadratic differentials.
A square complex is a 2-complex formed by gluing squares together. This article is concerned with the fundamental group of certain square complexes of nonpositive curvature, related to quaternion algebras. The abelian subgroup structure of is studied in some detail.
On the square peg problemmath.GT
Square can fit inside curves close to smooth ones.
problem Finding inscribed squares in nearly smooth curves.
method Using curvature and a map to relate curves, proving existence of inscribed squares.
result Curves close to smooth ones contain inscribed squares.
Distance-squared mappingsmath.GT
A distance-squared function is one of the most significant functions in the application of singularity theory to differential geometry. In this paper, we define naturally extended mappings of distance-squared functions, wherein each component is a distance-squared function. We investigate the properties of these mappin…
Arithmeticity of the Kontsevich--Zorich monodromies of certain families of square-tiled surfacesmath.DS
The study finds arithmetic groups often in square-tiled surface monodromies.
problem Understanding arithmetic properties of square-tiled surfaces.
method Analyzing variations of Hodge structures and Kontsevich-Zorich monodromies.
result Arithmetic groups are frequent in low genus square-tiled surfaces.
New expressions for Nijenhuis tensor squares found.
problem Understanding Nijenhuis tensor squares.
method Expressed dual forms in terms of exterior derivative derivatives.
result New vanishing results for Nijenhuis tensor squares.
In the context of Synthetic Differential Geometry, we describe the square volume of a ``second-infinitesimal simplex'', in terms of square-distance between its vertices. The square-volume function thus described is symmetric in the vertices. The square-volume gives rise to a characterization of the volume form in the t…
A new method simulates square-root processes efficiently.
problem Simulating square-root processes accurately and efficiently.
method Simulate the integrated square-root process instead of the square-root process itself.
result High precision with low number of time steps, and exact limiting Inverse Gaussian distributions.
Square metrics is an important class of Finsler metrics. Recently, we introduced a special class of non-regular Finsler metrics called singular square metrics. The main purpose of this paper is to provide a necessary and sufficient condition for singular square metrics to be of constant Ricci or flag curvature when dim…