Topological data analysis and its main method, persistent homology, provide a toolkit for computing topological information of high-dimensional and noisy data sets. Kernels for one-parameter persistent homology have been established to connect persistent homology with machine learning techniques. We contribute a kernel…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,786 papers · 148 categories
Trend · papers per month
48 results for “kernel parameters”
Gaussian kernel fails on circle and related spaces.
problem Gaussian kernel's positive definiteness on non-Euclidean spaces.
method Analyzing the Gaussian kernel on the circle and related metric spaces.
result Gaussian kernel is not positive definite on the circle or spaces admitting circle embeddings.
Due to the growing ubiquity of unlabeled data, learning with unlabeled data is attracting increasing attention in machine learning. In this paper, we propose a novel semi-supervised kernel learning method which can seamlessly combine manifold structure of unlabeled data and Regularized Least-Squares (RLS) to learn a ne…
Adaptive RBF-KAN: A Comparative Evaluation of Dynamic Shape Parameters in Kolmogorov-Arnold Networksstat.ML
Adaptive RBF-KAN improves KANs by dynamically adjusting kernel parameters.
problem Efficiently approximating multivariate functions using learnable univariate edge functions.
method Integrates LOOCV-based kernel scale estimation with adaptive kernel learning.
result Adaptive RBF-KAN outperforms fixed kernel KANs on various benchmark functions.
Raiders of the Lost Architecture: Kernels for Bayesian Optimization in Conditional Parameter Spacesstat.ML
In practical Bayesian optimization, we must often search over structures with differing numbers of parameters. For instance, we may wish to search over neural network architectures with an unknown number of layers. To relate performance data gathered for different architectures, we define a new kernel for conditional p…
Improves learning of spectral mixture kernels with approximate Bayesian inference.
problem Difficult optimization of large number of SM kernel parameters.
method Approximate Bayesian inference using variational distribution of spectral points and random Fourier features.
result Accelerates convergence and leads to better optimal parameters.
Improves kernel ridge regression by optimizing scale and feature parameters.
problem Kernel ridge regression with fixed kernel.
method Introduces a matrix parameter U to optimize scale and feature parameters.
result Solves a nonlinear variational problem to optimize U.
The present paper proposes generalized Gaussian kernel adaptive filtering, where the kernel parameters are adaptive and data-driven. The Gaussian kernel is parametrized by a center vector and a symmetric positive definite (SPD) precision matrix, which is regarded as a generalization of the scalar width parameter. These…
We investigate iterated compositions of weighted sums of Gaussian kernels and provide an interpretation of the construction that shows some similarities with the architectures of deep neural networks. On the theoretical side, we show that these kernels are universal and that SVMs using these kernels are universally con…
Kernel based methods have shown effective performance in many remote sensing classification tasks. However their performance significantly depend on its hyper-parameters. The conventional technique to estimate the parameter comes with high computational complexity. Thus, the objective of this letter is to propose an fa…
We define a family of kernels for mixed continuous/discrete hierarchical parameter spaces and show that they are positive definite.
Paper proposes adaptive parameter selection for KGD algorithms.
problem Improving parameter selection for kernel-based gradient descent.
method Integrates bias-variance analysis with splitting method, introduces empirical effective dimension.
result Adaptive parameter selection strategy achieves optimal generalization error bound.
A new method debiases multiple target parameters without IFs.
problem Debiasing multiple target parameters in nonparametric models.
method Kernel Debiased Plug-in Estimation (KDPE) using TMLE and reproducing kernel Hilbert spaces.
result KDPE simultaneously debiases all pathwise differentiable target parameters.
Paper studies kernel hyperparameters for clustering, proposing an efficient search method.
problem Challenges in tuning kernel parameters for clustering, especially for RBF kernels.
method Derives a lower bound for RBF kernel parameters, proposes an efficient hyperparameter search algorithm.
result Proposes an efficient algorithm for hyperparameter search in kernel clustering, improving upon grid search.
This work simplifies SVM parameter selection using S&S ratio.
problem SVM parameter tuning for optimal performance.
method S&S ratio to model SVM performance; automatic RP, kernel, and parameter selection.
result Optimized SVM parameters with reduced computational complexity.
Ad-SVGD optimizes kernel parameters for SVGD, improving inference performance.
problem Efficiently approximating posterior distributions in Bayesian inference.
method Adaptive kernel selection for SVGD dynamics.
result Ad-SVGD outperforms standard heuristics in various tasks.
This work addresses two main issues of the standard Kernel Entropy Component Analysis (KECA) algorithm: the optimization of the kernel decomposition and the optimization of the Gaussian kernel parameter. KECA roughly reduces to a sorting of the importance of kernel eigenvectors by entropy instead of by variance as in K…
Estimates system parameters from a single observation using kernel-based score.
problem Estimating parameters of a dynamical system from a high-dimensional signal.
method Kernel-based score to compare temporal dependencies between signal and model.
result Accuracy and efficiency demonstrated on chaotic systems.
Paper introduces new regression methods for consistent estimation of biophysical parameters.
problem Estimating biophysical parameters while respecting auxiliary variables.
method Linear and nonlinear kernel-based regression models with consistency constraints.
result Models provide closed-form solutions and successfully estimate chlorophyll content.
Behavioral analysis of support vector machine classifier with Gaussian kernel and imbalanced datacs.LG
Analyzes SVM classifier behavior with different parameters and data types.
problem Tuning SVM parameters for balanced and imbalanced data.
method Behavioral analysis of SVM with different parameters and data types, proposing a novel search algorithm.
result Proposed search algorithm reduces computational time and provides expected kernel function range.
A new algorithm for differential privacy in kernelized contextual bandits reduces error rate.
problem Joint differential privacy in kernelized contextual bandits.
method Proposes a novel algorithm with a specific error rate and privacy parameter dependence.
result Achieves an error rate of $\mathcal{O}\left(\sqrt{\frac{γ_T}{T}} + \frac{γ_T}{T \varepsilon}
ight)$ after queries.
KM method reduces ConvNet parameters to 9% higher accuracy with minimal additional memory.
problem Expensive memory usage for training ConvNets on embedded devices.
method Kernel Modulation (KM) method that adapts all network parameters for each task.
result KM delivers up to 9% higher accuracy than other parameter-efficient methods.
Kernel Quantization improves CNN compression without sacrificing performance.
problem Efficiently compressing CNN models without significant performance loss.
method Quantizes convolution kernels as the unit, learning a codebook for low-bit indexes.
result Significant compression ratio achieved with minimal accuracy loss.
We consider the two-group classification problem and propose a kernel classifier based on the optimal scoring framework. Unlike previous approaches, we provide theoretical guarantees on the expected risk consistency of the method. We also allow for feature selection by imposing structured sparsity using weighted kernel…
While tree methods have been popular in practice, researchers and practitioners are also looking for simple algorithms which can reach similar accuracy of trees. In 2010, (Ping Li UAI'10) developed the method of "abc-robust-logitboost" and compared it with other supervised learning methods on datasets used by the deep …
Gaussian kernel tests are optimal against smooth alternatives.
problem Understanding the statistical properties of nonparametric tests using Gaussian kernels.
method Analysis of Gaussian kernel-based goodness-of-fit, homogeneity, and independence tests.
result Gaussian kernel tests are minimax optimal against smooth alternatives in all three settings.
Efficient estimators for smooth Hilbert-valued parameters with theoretical guarantees.
problem Estimating smooth Hilbert-valued parameters with theoretical guarantees.
method Pathwise differentiable Hilbert-valued parameters, efficient influence functions, regularized one-step estimators.
result Theoretical guarantees for efficient estimators even when nuisance functions are arbitrary.
Support Vector Data Description (SVDD) is a machine-learning technique used for single class classification and outlier detection. SVDD formulation with kernel function provides a flexible boundary around data. The value of kernel function parameters affects the nature of the data boundary. For example, it is observed …
A well-recognized limitation of kernel learning is the requirement to handle a kernel matrix, whose size is quadratic in the number of training examples. Many methods have been proposed to reduce this computational cost, mostly by using a subset of the kernel matrix entries, or some form of low-rank matrix approximatio…
Proposes a new hyperprior and predictive criterion for weakly informative hyperprior in relevance vector machine.
problem Capturing non-homogeneous data structure with limited kernel functions.
method Uses inverse gamma hyperprior with a shape parameter close to zero and a scale parameter not close to zero. Applies multiple kernel method with different widths. Proposes extended predictive information criterion for scale parameter selection.
result Obtains a multiple kernel relevance vector regression model with good predictive accuracy.
Deterministic equivalent of the Conjugate Kernel matrix associated to Artificial Neural Networksmath.PR
Study shows deterministic equivalent for neural network kernel convergence.
problem Understanding convergence of neural network kernels.
method Analyzes empirical spectral distribution of Conjugate Kernel, proving convergence to a deterministic limit.
result Obtains a deterministic equivalent for the Stieltjes transform and resolvent of the Conjugate Kernel.
We propose a novel approach to parameter estimation for simulator-based statistical models with intractable likelihood. Our proposed method involves recursive application of kernel ABC and kernel herding to the same observed data. We provide a theoretical explanation regarding why the approach works, showing (for the p…
Kernel methods are widespread in machine learning; however, they are limited by the quadratic complexity of the construction, application, and storage of kernel matrices. Low-rank matrix approximation algorithms are widely used to address this problem and reduce the arithmetic and storage cost. However, we observed tha…
Support Vector Data Description (SVDD) provides a useful approach to construct a description of multivariate data for single-class classification and outlier detection with various practical applications. Gaussian kernel used in SVDD formulation allows flexible data description defined by observations designated as sup…
pGMM kernel outperforms ordinary ridge regression and RBF kernel ridge regression without tuning.
problem Comparing pGMM kernel regression with other ridge regression methods.
method Implemented and compared pGMM kernel regression with ordinary ridge regression and RBF kernel ridge regression.
result pGMM kernel performs well without tuning and can match boosted trees with parameter tuning.
CoRE Kernelsstat.ML
The term "CoRE kernel" stands for correlation-resemblance kernel. In many applications (e.g., vision), the data are often high-dimensional, sparse, and non-binary. We propose two types of (nonlinear) CoRE kernels for non-binary sparse data and demonstrate the effectiveness of the new kernels through a classification ex…
A new method for faster bandwidth selection in Gaussian kernel ridge regression.
problem Efficiently selecting the bandwidth in Gaussian kernel ridge regression.
method Formulated an approximate Jacobian expression for bandwidth selection, proposing a closed-form heuristic.
result Our method is as accurate as cross-validation and marginal likelihood maximization but up to six orders of magnitude faster.
Wider neural networks perform better than deeper ones with the same number of parameters.
problem Understanding the role of network width versus the number of parameters in neural network performance.
method Comparing models with different ways of increasing width while keeping the number of parameters constant, analyzing their performance and using Gaussian Process kernels for analysis.
result Network width is the determining factor for good performance, while the number of weights is secondary as long as trainability is ensured.
Two-sample tests using MMD control type I error and achieve optimal power.
problem Developing reliable nonparametric two-sample tests for small sample sizes.
method Maximum Mean Discrepancy (MMD) for constructing novel nonparametric tests, proving non-asymptotic error control and optimality.
result MMDAgg test controls type I error and achieves minimax rate over Sobolev balls, outperforming other tests.
Tensor networks constrain kernel machines to Gaussian processes.
problem Speeding up kernel machines with reduced model complexity.
method Proving CPD and TT-constrained models recover Gaussian processes with i.i.d. priors.
result TT-constrained models exhibit more Gaussian process behavior than CPD for the same parameters.
Needlets have been recognized as state-of-the-art tools to tackle spherical data, due to their excellent localization properties in both spacial and frequency domains. This paper considers developing kernel methods associated with the needlet kernel for nonparametric regression problems whose predictor variables are de…
We show that the output of a (residual) convolutional neural network (CNN) with an appropriate prior over the weights and biases is a Gaussian process (GP) in the limit of infinitely many convolutional filters, extending similar results for dense networks. For a CNN, the equivalent kernel can be computed exactly and, u…
At initialization, artificial neural networks (ANNs) are equivalent to Gaussian processes in the infinite-width limit, thus connecting them to kernel methods. We prove that the evolution of an ANN during training can also be described by a kernel: during gradient descent on the parameters of an ANN, the network functio…
The paper analyzes how to estimate Gaussian process parameters accurately.
problem Estimating parameters of Gaussian process kernels from noisy data.
method Maximum likelihood estimation of the scale parameter of a Sobolev kernel.
result Maximum likelihood estimation provides significant adaptation against misspecification.
Flexible nonstationary Gaussian process with neural network parameters.
problem Limited expressiveness of stationary Gaussian processes.
method Nonstationary kernels with neural network parameters trained jointly.
result Better accuracy and log-score compared to stationary and hierarchical models.
Analyzes feature learning in neural networks using a self-consistent dynamical field theory.
problem Feature learning in infinite-width neural networks.
method Constructs deterministic dynamical order parameters as inner-product kernels for hidden unit activations and gradients.
result Reveals the hidden layer activation distribution, neural tangent kernel evolution, and output predictions.
Quantum tangent kernelquant-ph
This work introduces a new quantum kernel, quantum tangent kernel, for improved performance.
problem Improving quantum machine learning performance beyond conventional methods.
method Developed a deep parameterized quantum circuit and used first-order expansion for training.
result The quantum tangent kernel outperforms conventional quantum kernel methods for ansatz-generated datasets.
Paper analyzes consistency of Bayesian and machine learning methods for hierarchical parameter estimation.
problem Learning hierarchical parameters in complex and real-world problems.
method Empirical Bayes and Kernel Flow approaches.
result Consistency results for Matérn-like model on the torus, and comparison of algorithms.