Kernel-based K-means clustering has gained popularity due to its simplicity and the power of its implicit non-linear representation of the data. A dominant concern is the memory requirement since memory scales as the square of the number of data points. We provide a new analysis of a class of approximate kernel methods…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “memory kernel”
Enhanced Hopfield model boosts memory retrieval capacity.
problem Memory retrieval in modern Hopfield models with limited capacity.
method Introduces a learnable feature map transforming energy function into kernel space, minimizing separation loss for uniform memory distribution.
result Significant reduction in metastable states, enhancing memory capacity and retrieval accuracy.
The study analyzes prediction errors in systems with memory kernels, providing bounds and stability results.
problem Prediction errors in stochastic dynamical systems with memory kernels.
method Analysis of generalized Langevin equations (GLEs) with Volterra equations, integrating synchronized noise coupling and weighted norms.
result Prediction discrepancies decay at a rate determined by the memory kernel's decay, quantitatively bounded by kernel estimation errors.
We investigate how to train kernel approximation methods that generalize well under a memory budget. Building on recent theoretical work, we define a measure of kernel approximation error which we find to be more predictive of the empirical generalization performance of kernel approximation methods than conventional me…
Efficiently scales continuous kernels with sparse Fourier domain learning.
problem High computational and memory demands, spectral bias in continuous kernels.
method Sparse learning in the Fourier domain.
result Efficient scaling of continuous kernels, reduced computational and memory requirements, mitigated spectral bias.
Sigmoid autoencoders can implement associative memory with certain conditions.
problem Implementing associative memory in neural networks.
method Theoretical analysis of overparameterized sigmoid autoencoders using the NTK and iterative maps.
result Overparameterized sigmoid autoencoders can have attractors in the NTK limit, leading to associative memory.
Novel method learns memory kernels in Langevin equations.
problem Estimating memory kernels in Langevin equations.
method Regularized Prony method for correlation functions, followed by regression over Sobolev norm-based loss function with RKHS regularization.
result Method outperforms other regression estimators in exponentially weighted L^2 space.
New GPU kernels boost deep learning speed and memory efficiency.
problem Sparse deep learning matrices are not well-suited for existing sparse kernels.
method Identified favorable properties of sparse matrices from deep learning, developed high-performance GPU kernels for sparse matrix operations.
result 27% of single-precision peak performance on Nvidia V100 GPUs achieved with new kernels.
Efficient tensor kernel method reduces memory usage and computational cost for sparse regression.
problem Memory and computational limitations in tensor kernel methods for sparse regression.
method Proposes a new tensor data layout and Nystrom subsampling approach to reduce memory and computational requirements.
result Improvements lead to more efficient tensor kernel methods for sparse regression.
Kernel methods have great promise for learning rich statistical representations of large modern datasets. However, compared to neural networks, kernel methods have been perceived as lacking in scalability and flexibility. We introduce a family of fast, flexible, lightly parametrized and general purpose kernel learning …
A new method uses neural tangent kernel to efficiently compute MMD statistic.
problem Efficiently computing Maximum Mean Discrepancy (MMD) statistic with low memory and computational complexity.
method Identifies a connection between neural tangent kernel (NTK) and MMD to develop a computationally and memory-efficient approach.
result The proposed NTK-MMD statistic is validated through numerical experiments on synthetic and real-world datasets.
StreaMRAK improves KRR for streaming data.
problem Streaming data with memory constraints.
method Divides problem into levels of resolution, sub-sampling.
result Efficiently integrates new samples, reduces memory and complexity.
Incremental versions of batch algorithms are often desired, for increased time efficiency in the streaming data setting, or increased memory efficiency in general. In this paper we present a novel algorithm for incremental kernel PCA, based on rank one updates to the eigendecomposition of the kernel matrix, which is mo…
Kernel-based clustering algorithm can identify and capture the non-linear structure in datasets, and thereby it can achieve better performance than linear clustering. However, computing and storing the entire kernel matrix occupy so large memory that it is difficult for kernel-based clustering to deal with large-scale …
We present memory-efficient and scalable algorithms for kernel methods used in machine learning. Using hierarchical matrix approximations for the kernel matrix the memory requirements, the number of floating point operations, and the execution time are drastically reduced compared to standard dense linear algebra routi…
This study examines memory effects in S&P500 market correlations using Langevin models.
problem The neglect of memory effects in market correlations for optimal portfolio selection.
method Fit a generalised Langevin equation (GLE) to S&P500 market correlation data.
result Memory effects in market correlations significantly improve forecasting accuracy and suggest a hidden slow time scale.
Despite their attractiveness, popular perception is that techniques for nonparametric function approximation do not scale to streaming data due to an intractable growth in the amount of storage they require. To solve this problem in a memory-affordable way, we propose an online technique based on functional stochastic …
New analysis tightens memory capacity of Hopfield models using spherical codes.
problem Optimizing memory capacity in modern Hopfield models and Kernelized Hopfield Models.
method Connecting Hopfield models to spherical codes in information theory, establishing an optimal capacity bound and a sub-linear algorithm.
result First tight and optimal asymptotic memory capacity for modern Hopfield models, matching known lower bounds.
We investigate time-dependent data analysis from the perspective of recurrent kernel machines, from which models with hidden units and gated memory cells arise naturally. By considering dynamic gating of the memory cell, a model closely related to the long short-term memory (LSTM) recurrent neural network is derived. E…
Linear RNNs exhibit a bias towards shorter memory due to initialization variance.
problem Understanding the performance limitations of RNNs, especially linear ones.
method Kernel regime analysis to show equivalence to 1D-convolutional networks and analyze weightings.
result Linear RNNs with random initialization have a bias towards shorter memory periods.
Parameterized state space models in the form of recurrent networks are often used in machine learning to learn from data streams exhibiting temporal dependencies. To break the black box nature of such models it is important to understand the dynamical features of the input driving time series that are formed in the sta…
RFFNet scales kernel methods to large datasets by learning kernel relevance.
problem Scaling kernel methods to large datasets while maintaining interpretability.
method Designs random Fourier features for ARD kernels and uses first-order stochastic optimization for learning kernel relevances.
result RFFNet achieves low prediction error and identifies relevant features, leading to more interpretable solutions.
CoRE Kernelsstat.ML
The term "CoRE kernel" stands for correlation-resemblance kernel. In many applications (e.g., vision), the data are often high-dimensional, sparse, and non-binary. We propose two types of (nonlinear) CoRE kernels for non-binary sparse data and demonstrate the effectiveness of the new kernels through a classification ex…
NANSDE-Net models time series with memory using neural ARMA-type noise.
problem Modeling time series with long- or short-memory characteristics.
method Developed NANSDE-Net, a generative model that incorporates Neural Network-kernel ARMA-type noise.
result NANSDE-Net matches or outperforms existing models in reproducing long- and short-memory features of data.
We extend the herding algorithm to continuous spaces by using the kernel trick. The resulting "kernel herding" algorithm is an infinite memory deterministic process that learns to approximate a PDF with a collection of samples. We show that kernel herding decreases the error of expectations of functions in the Hilbert …
Efficient online kernel CUSUM detects changes quickly and accurately.
problem Detecting changes in online data streams efficiently.
method Online kernel CUSUM using maximum kernel statistics.
result Increased sensitivity to small changes compared to existing methods.
Kernel approximation methods create explicit, low-dimensional kernel feature maps to deal with the high computational and memory complexity of standard techniques. This work studies a supervised kernel learning methodology to optimize such mappings. We utilize the Discriminant Information criterion, a measure of class …
New algorithms for multitask learning with long-term memory.
problem Learning from tasks partitioned into unknown segments with associated hypotheses.
method Online multitask learning algorithms exploiting segmentation and hypothesis association.
result Regret bounds and efficient algorithms for various hypothesis classes.
Kernel-based methods enjoy powerful generalization capabilities in handling a variety of learning tasks. When such methods are provided with sufficient training data, broadly-applicable classes of nonlinear functions can be approximated with desired accuracy. Nevertheless, inherent to the nonparametric nature of kernel…
Gaussian process regression generally does not scale to beyond a few thousands data points without applying some sort of kernel approximation method. Most approximations focus on the high eigenvalue part of the spectrum of the kernel matrix, , which leads to bad performance when the length scale of the kernel is sma…
Kernel methods provide a principled way to perform non linear, nonparametric learning. They rely on solid functional analytic foundations and enjoy optimal statistical properties. However, at least in their basic form, they have limited applicability in large scale scenarios because of stringent computational requireme…
New algorithm scales NDPP learning and inference to large item collections.
problem Memory and runtime limitations in existing NDPP learning and inference algorithms.
method Introduced a new NDPP kernel decomposition for learning and a linear-complexity MAP inference algorithm.
result Our algorithms scale linearly in , matching prior work's predictive performance.
New algorithms improve GP inference without approximations, achieving better results.
problem Inexact stochastic optimization methods in Gaussian Processes leading to biased results.
method Exact stochastic inference for GPs with finite dimensional RKHS, extending to infinite dimensions.
result Achieves better experimental results than existing methods in constrained resource settings.
Despite their successes, what makes kernel methods difficult to use in many large scale problems is the fact that storing and computing the decision function is typically expensive, especially at prediction time. In this paper, we overcome this difficulty by proposing Fastfood, an approximation that accelerates such co…
We study the statistical and computational aspects of kernel principal component analysis using random Fourier features and show that under mild assumptions, features suffices to achieve sample complexity. Furthermore, we give a memory efficient streaming algorithm based on classical Oja…
GraphGP: Scalable Gaussian Processes with Vecchia's Approximation
problem Naive Gaussian Process computation limits practical use
method GPU algorithm for Vecchia's approximation
result Linear time and memory requirements for nearly a billion parameters
In most adaptive signal processing applications, system linearity is assumed and adaptive linear filters are thus used. The traditional class of supervised adaptive filters rely on error-correction learning for their adaptive capability. The kernel method is a powerful nonparametric modeling tool for pattern analysis a…
Streaming Approach to Quadratic Covariation Estimation Using Financial Ultra-High-Frequency Dataq-fin.CP
A streaming algorithm estimates quadratic covariation from financial data efficiently.
problem Estimating quadratic covariation from ultra-high-frequency financial data with limited memory.
method Formulated multi-scale, realized kernel, pre-averaging, and modulated realized covariance estimators with fixed bandwidth.
result Fixed bandwidth estimators require higher bandwidth for positive semidefiniteness.
Derives variance kernel for reaction boundary in financial models.
problem Separating components in financial volatility models.
method Operational-time variance kernel, damped Abel response kernel, closed asymptotic form.
result Operational variance has a closed asymptotic form involving various parameters.
Derives operational-time variance kernel for reaction boundaries in financial markets.
problem Separating components in volatility models to better understand market dynamics.
method Derives a variance kernel for a latent-order-book reaction boundary, separating structural boundary cumulant, clock projection, and pricing-measure choice.
result Operational variance has a closed asymptotic form for long-memory forcing, with effective signed-forcing intensity and resilience.
Researchers develop a generalised geometric Brownian motion for better asset pricing.
problem Irregularities in simple geometric Brownian motion for asset dynamics.
method Introduce a memory kernel to generalise GBM, derive moments and probability density functions.
result The performance of kernels in pricing options depends on option maturity and moneyness.
Scaling Limits of Bivariate Nearly-Unstable Hawkes Processes and Applications to Rough Volatilitymath.PR
Study of coupled Hawkes processes with rough-volatility limits.
problem Understanding coupled Hawkes processes with rough-volatility limits.
method Proving weak convergence of rescaled intensity vector to stochastic Volterra equations.
result Limiting components exhibit different degrees of roughness and cross-decorrelation law.
Clustering samples according to an effective metric and/or vector space representation is a challenging unsupervised learning task with a wide spectrum of applications. Among several clustering algorithms, k-means and its kernelized version have still a wide audience because of their conceptual simplicity and efficacy.…
KM method reduces ConvNet parameters to 9% higher accuracy with minimal additional memory.
problem Expensive memory usage for training ConvNets on embedded devices.
method Kernel Modulation (KM) method that adapts all network parameters for each task.
result KM delivers up to 9% higher accuracy than other parameter-efficient methods.
Real time application of deep learning algorithms is often hindered by high computational complexity and frequent memory accesses. Network pruning is a promising technique to solve this problem. However, pruning usually results in irregular network connections that not only demand extra representation efforts but also …
Kernel methods are widespread in machine learning; however, they are limited by the quadratic complexity of the construction, application, and storage of kernel matrices. Low-rank matrix approximation algorithms are widely used to address this problem and reduce the arithmetic and storage cost. However, we observed tha…
Novel algorithm speeds up log-determinant estimation for large matrices.
problem Efficiently estimating log-determinants of large positive definite matrices under memory constraints.
method Hierarchical algorithm based on block-wise computation of LDL decomposition.
result Accurate estimation of NTK log-determinants from a tiny fraction of the full dataset.
Support Vector Machines (SVMs) are powerful learners that have led to state-of-the-art results in various computer vision problems. SVMs suffer from various drawbacks in terms of selecting the right kernel, which depends on the image descriptors, as well as computational and memory efficiency. This paper introduces a n…