Kernel methods are studied in a mean field limit for high-dimensional data.
problem Analyzing kernel methods in high-dimensional data with many variables.
method Investigation of kernel methods in the mean field limit of interacting particle systems.
result Rigorous mean field limit of kernels and detailed analysis of the limiting reproducing kernel Hilbert space.
Study on kernel methods in large-scale machine learning problems.
problem Large-scale machine learning with many interacting variables.
method Mean field limit analysis of kernels and their Hilbert spaces.
result Mean field convergence of empirical and infinite-sample solutions.
Adaptive kernels from neural networks improve model performance.
problem Improving neural network performance through adaptive kernels.
method Deriving adaptive kernels from infinite-width neural networks using feature learning and gradient flow training.
result Adaptive kernels achieve lower test loss compared to traditional kernels.
In this paper, we study the large time behavior of the heat kernel on complete Riemannian manifolds with nonnegative Ricci curvature, which was studied by P. Li with additional maximum volume growth assumption. Following Y. Ding's original strategy, by blowing down the metric, using Cheeger and Colding's theory about l…
Study compares exponential and power-law kernels in modeling high-frequency trading data.
problem Modeling high-frequency trading data with specific kernel types.
method Proposes and analyzes two bivariate Hawkes processes with exponential and power-law kernels.
result Identifies strengths and limitations of exponential and power-law kernels for high-frequency trading data.
At initialization, artificial neural networks (ANNs) are equivalent to Gaussian processes in the infinite-width limit, thus connecting them to kernel methods. We prove that the evolution of an ANN during training can also be described by a kernel: during gradient descent on the parameters of an ANN, the network functio…
New algorithm optimizes tessellated kernels for larger datasets and improved performance.
problem Limited accuracy and complexity in machine learning algorithms based on kernel optimization.
method 2-step algorithm for optimizing tessellated kernels, scaling to 10,000 data points and extending to regression.
result Significant improvement in performance over Neural Nets and SimpleMKL with similar computation time.
CKA with Gaussian RBF kernels converges linearly as bandwidth increases.
problem Understanding the behavior of CKA with large bandwidth Gaussian kernels.
method Analyzing the convergence of CKA based on Gaussian RBF kernels in the large-bandwidth limit.
result CKA based on Gaussian RBF kernels converges linearly as bandwidth increases.
New method for spot volatility estimation with reduced microstructure noise.
problem Estimating spot volatility from noisy high-frequency data.
method Pre-averaging/kernel estimator to handle microstructure noise.
result Optimal bandwidth selection and kernel functions for minimal variance.
Functional central limit theorem for kernel gradient flow and infinitesimal gradient boosting
problem Fluctuations of boosting processes around their deterministic limit
method Stochastic perturbation analysis of ODEs in Banach spaces
result Rescaled deviations converge to a Gaussian process
Study small-time CLTs for stochastic Volterra equations with various kernels.
problem Understanding the behavior of stochastic Volterra equations with different kernels.
method Proved convergence of finite-dimensional distributions, functional CLT, and limit theorems for smooth transformations.
result Derived asymptotic pricing formulae for digital calls in rough volatility models.
Meta two-sample testing uses auxiliary data to quickly find powerful tests from limited samples.
problem Challenges in identifying powerful kernels for distinguishing complex distributions with limited data.
method Introduces meta two-sample testing (M2ST) to leverage abundant auxiliary data on related tasks.
result Proposed algorithms improve over baselines and identify powerful tests from scarce observations.
Kernelized Taylor diagram visualizes data populations with fewer assumptions.
problem Limitations of Taylor diagram in capturing non-linear relationships and sensitivity to outliers.
method Proposes a kernelized version of the Taylor diagram that uses maximum mean discrepancy and kernel mean embedding.
result Kernelized Taylor diagram visualizes data populations with minimal assumptions of data distributions.
New framework for understanding infinite-width neural networks.
problem Understanding the infinite-width limit behavior of neural networks.
method General framework to study limit behavior of neural models based on hyperparameter scaling.
result Derives scaling for existing mean-field and neural tangent kernel limits and introduces new dynamically stable limits.
Efficiently scales continuous kernels with sparse Fourier domain learning.
problem High computational and memory demands, spectral bias in continuous kernels.
method Sparse learning in the Fourier domain.
result Efficient scaling of continuous kernels, reduced computational and memory requirements, mitigated spectral bias.
Study shows deterministic equivalent for neural network kernel convergence.
problem Understanding convergence of neural network kernels.
method Analyzes empirical spectral distribution of Conjugate Kernel, proving convergence to a deterministic limit.
result Obtains a deterministic equivalent for the Stieltjes transform and resolvent of the Conjugate Kernel.
New theory explains deep learning's success in transforming inputs.
problem Standard theoretical approaches eliminate representation learning.
method Developed a new infinite width limit for representation learning.
result Deep Gaussian processes (DGPs) have multivariate Gaussian posteriors.
Study of coupled Hawkes processes with rough-volatility limits.
problem Understanding coupled Hawkes processes with rough-volatility limits.
method Proving weak convergence of rescaled intensity vector to stochastic Volterra equations.
result Limiting components exhibit different degrees of roughness and cross-decorrelation law.
This paper provides a dictionary of closed-form kernel mean embeddings.
problem Challenges in deriving closed-form kernel mean embeddings.
method Comprehensive dictionary and practical tools for deriving new embeddings.
result Provides a Python library with minimal implementations of embeddings.
Convolutional DKMs improve kernel methods on MNIST, CIFAR-10, and CIFAR-100.
problem Improving kernel methods for image classification.
method Developed a novel inter-domain inducing point approximation and introduced various techniques to extend DKMs to convolutional networks.
result Achieved state-of-the-art performance on image classification benchmarks.
Study eigenvalue distributions of neural kernels for linear-width networks.
problem Eigenvalue distributions of neural kernels in linear-width networks.
method Asymptotic analysis of Conjugate Kernel and Neural Tangent Kernel under random initialization and approximate orthogonality.
result Eigenvalue distributions converge to deterministic limits, described by recursive fixed-point equations.
Kernel discriminant analysis uses nonlinear embeddings to improve classification.
problem Limited effectiveness of linear discriminant analysis in capturing nonlinear features.
method Study of nonlinear embeddings in kernel discriminant analysis using polynomial and Gaussian kernels, solving generalized eigenvalue problems.
result Polynomial and Gaussian discriminants capture class differences through population moments and randomized projections.
Two-layer neural networks learn efficiently using kernel methods in mean-field analysis.
problem Feature learning ability of two-layer neural networks in the mean-field regime.
method Mean-field analysis through kernel methods, focusing on dynamics of the first layer's kernel.
result Two-layer neural networks can learn a union of multiple reproducing kernel Hilbert spaces more efficiently than kernel methods.
If pricing kernels are assumed non-negative then the inverse problem of finding the pricing kernel is well-posed. The constrained least squares method provides a consistent estimate of the pricing kernel. When the data are limited, a new method is suggested: relaxed maximization of the relative entropy. This estimator …
A novel nonstationary permanental process relaxes kernel constraints and captures complex data patterns.
problem Limitations of existing permanental processes in terms of kernel types and stationarity.
method Sparse spectral representation of nonstationary kernels and hierarchical stacking of spectral feature mappings.
result Enhanced model expressiveness and reduced computational complexity.
Recent studies utilize multiple kernel learning to deal with incomplete-data problem. In this study, we introduce new methods that do not only complete multiple incomplete kernel matrices simultaneously, but also allow control of the flexibility of the model by parameterizing the model matrix. By imposing restrictions …
New methods improve Reservoir Computing for chaotic time series prediction.
problem Chaotic time series prediction in Reservoir Computing.
method Established Recurrent Kernel limit, introduced Structured Reservoir Computing.
result Structured Reservoir Computing is faster and more memory-efficient.
Building highly non-linear and non-parametric models is central to several state-of-the-art machine learning systems. Kernel methods form an important class of techniques that induce a reproducing kernel Hilbert space (RKHS) for inferring non-linear models through the construction of similarity functions from data. The…
We propose a novel class of Gaussian processes (GPs) whose spectra have compact support, meaning that their sample trajectories are almost-surely band limited. As a complement to the growing literature on spectral design of covariance kernels, the core of our proposal is to model power spectral densities through a rect…
Study on deep neural networks using branching processes and Mehler's formula.
problem Understanding the mathematical role of activation functions in compositional neural networks.
method Connection between compositional kernels and branching processes via Mehler's formula; new random features algorithm.
result Explicit formulas for eigenvalues of compositional kernels quantify complexity.
Researchers transform equations and define integral operators on a ball.
problem Transforming equations from half space to ball.
method Identify Poisson kernel, define extension operator, prove inequalities.
result Uniqueness of extremal functions in limit case.
Study on zeros of Gaussian sections on semipositive line bundles on punctured Riemann surfaces.
problem Distribution of zeros of Gaussian sections on semipositive line bundles.
method Analysis of Bergman kernels and random zeros in high tensor powers.
result Equidistribution, large deviation estimates, central limit theorem, and number variances for zeros in the semi-classical limit.
Kernel balancing weights are generalized as KRRR, providing better confidence intervals for treatment effects.
problem Lack of generalization error, correct feature specification, and limited to average effects.
method Interpreting kernel balancing weights as KRRR, relaxing feature specification, and extending Gaussian approximation.
result KRRR provides strong generalization properties and justifies confidence sets for causal functions.
Quantum neural tangent kernels help understand variational quantum circuits in machine learning.
problem Designing and predicting performance of variational quantum circuits.
method Using quantum neural tangent kernels and dynamical equations for loss functions.
result Analytical solutions for training dynamics in variational quantum circuits.
We establish a link between Fourier optics and a recent construction from the machine learning community termed the kernel mean map. Using the Fraunhofer approximation, it identifies the kernel with the squared Fourier transform of the aperture. This allows us to use results about the invertibility of the kernel mean m…
Kernel VICReg improves SSL in RKHS, capturing nonlinear structures.
problem Limited ability of existing SSL methods to handle nonlinear dependencies.
method Kernel VICReg framework in RKHS, kernelizing VICReg objectives.
result Kernel VICReg mitigates representational collapse and improves performance.
We study the expressive power of kernel methods and the algorithmic feasibility of multiple kernel learning for a special rich class of kernels. Specifically, we define \emph{Euclidean kernels}, a diverse class that includes most, if not all, families of kernels studied in literature such as polynomial kernels and radi…
This paper improves computational efficiency in kernel ridge regression under covariate shift.
problem Covariate shift in nonparametric regression.
method Random projections in RKHS to reduce computational demands.
result Significant computational savings can be achieved without compromising learning performance under covariate shift.
New method speeds up neural kernel computations for various activations.
problem Inefficient computation of neural kernels for general activations.
method Fast sketching method using truncated Hermite expansion.
result 106x speedup for approximate CNTK computation on CIFAR-10.
Kernel-based learning predicts ICU escalation from COVID-19 chest X-rays.
problem Predicting ICU escalation from chest X-rays using complex data patterns.
method Generalized Linear Models with Integrated Multiple Additive Regression with Kernels (GLIMARK).
result GLIMARK effectively predicts ICU escalation from chest X-rays.
High-dimensional U-statistics show surprising phase transitions, impacting kernel-based tests.
problem Understanding phase transitions in high-dimensional U-statistics.
method Proved a convergence theorem for U-statistics of degree two in high dimensions.
result High-dimensional U-statistics can have non-Gaussian limits with larger variance and asymmetry.
We investigate the capabilities and limitations of Gaussian process models by jointly exploring three complementary directions: (i) scalable and statistically efficient inference; (ii) flexible kernels; and (iii) objective functions for hyperparameter learning alternative to the marginal likelihood. Our approach outper…
Meta-learning framework uses task similarity through nonparametric kernel regression.
problem Limited tasks and outliers/dissimilar tasks hinder meta-learning performance.
method Nonparametric kernel regression to quantify and use task similarity.
result Meta-learning algorithm outperforms existing methods in task-limited settings.
Develops EFT for ResNets, revealing limitations of kernel-only approach.
problem Limitations of kernel-only approach in deep neural networks.
method Collective kernel EFT for pre-activation ResNets based on G-only closure hierarchy. result Numerical findings show V4 equation residual accumulates to an O(1) error. Permutation-valued features arise in a variety of applications, either in a direct way when preferences are elicited over a collection of items, or an indirect way in which numerical ratings are converted to a ranking. To date, there has been relatively limited study of regression, classification, and testing problems …
The study proves a central limit theorem for Gaussian holomorphic sections on Kähler manifolds.
problem Understanding statistical properties of zeros of random holomorphic sections.
method Proves a central limit theorem for smooth linear statistics of zero divisors of Gaussian sections in line bundles over Kähler manifolds.
result Derives first-order asymptotics and upper decay estimates for Bergman kernels.
Constructing the adjacency graph is fundamental to graph-based clustering. Graph learning in kernel space has shown impressive performance on a number of benchmark data sets. However, its performance is largely determined by the chosen kernel matrix. To address this issue, the previous multiple kernel learning algorith…
The paper discusses convergence of Bergman kernels on complex manifolds.
problem Convergence of Bergman kernels on complex manifolds with given conditions.
method Analysis of Gromov-Hausdorff limits, Hermitian line bundles, and positive currents.
result Uniform asymptotic expansion and estimates of Bergman kernels.