arXiv research

In this paper we continue our study of the Laplacian on manifolds with axial analytic asymptotically cylindrical ends initiated in~arXiv:1003.2538. By using the complex scaling method and the Phragmén-Lindelöf principle we prove exponential decay of the eigenfunctions corresponding to the non-threshold eigenvalues of t…

Paper calculates eigenvalue decay rates for neural network kernels on general domains.

Under the quadratic-decay-conditions of the radial curvatures of an end, we shall derive growth estimates of solutions to the eigenvalue equation and show the absence of eigenvalues.

Active data collection improves convergence rates in operator learning.

The concern of this paper is to clarify a relationship between the curvatures at infinity and the spectral structure of the Laplacian. In particular, this paper discusses the question of whether there is an eigenvalue of the Laplacian embedded in the essential spectrum or not. The borderline-behavior of the radial curv…

Two new algorithms reduce online kernel regression's computational cost while maintaining optimal regret bounds.

Upper bound found for Steklov eigenvalues counting function.

Study small eigenvalues of Toeplitz operators and their relation to Mabuchi geodesics.

Deep ReLU networks approximate as well as shallow ones in kernel regimes.

Spectral feature learning improves IV regression for causal effect estimation.

The salient properties of large empirical covariance and correlation matrices are studied for three datasets of size 54, 55 and 330. The covariance is defined as a simple cross product of the returns, with weights that decay logarithmically slowly. The key general properties of the covariance matrices are the following…

New framework estimates eigenvalues of kernel matrices without full matrix construction.

Eigenvalues of random hyperbolic surface covers converge to hyperbolic plane's.

This purpose of this write-up is to share an idea for accurate computation of Laplace eigenvalues on a broad class of smooth domains. We represent the eigenfunction $u$ as a linear combination of eigenfunctions corresponding to the common eigenvalue $ρ^{2}$:\EQN{6}{1}{}{0}{\RD{\CELL{u(r,θ) =\sum_{n=0}^{N}P_{n}J_{n}(ρ) …

ASGD outperforms SGD in overparameterized linear regression, especially in subspaces of small eigenvalues.

Stability inequalities for specific solutions in high dimensions.

We introduce a novel algorithm that computes the $k$-sparse principal component of a positive semidefinite matrix $A$. Our algorithm is combinatorial and operates by examining a discrete set of special vectors lying in a low-dimensional eigen-subspace of $A$. We obtain provable approximation guarantees that depend on t…

We investigate if kernel regularization methods can achieve minimax convergence rates over a source condition regularity assumption for the target function. These questions have been considered in past literature, but only under specific assumptions about the decay, typically polynomial, of the spectrum of the the kern…

New energy functional bounds Ricci flows on ancient spaces.

Model proposes neural network for continuous time dynamics with inductive biases.

The authors prove that the logarithmic Monge-Ampère flow with uniformly bound and convex initial data satisfies uniform decay estimates away from time $t=0$. Then applying the decay estimates, we conclude that every entire classical strictly convex solution of the equation {equation*} \det D^{2}u=\exp\{n(-u+1/2\sum_{i=…

Universal algorithm learns unknown distribution for various decision-making problems.

The study examines Kernel Ridge Regression error rates across noiseless and noisy conditions.

High-dimensional kernel regression struggles due to rotational invariance.

Power-law spectrum of random feature model is preserved in neural networks.

Study on eigenvalue distribution of correlated time series, showing deformation of Marchenko-Pastur distribution.

Gradient descent outperforms ridge regression under certain covariance matrix decay conditions.

Positive definite kernels and their associated Reproducing Kernel Hilbert Spaces provide a mathematically compelling and practically competitive framework for learning from data. In this paper we take the approximation theory point of view to explore various aspects of smooth kernels related to their inferential proper…

In this paper, we give a new sharp generalization bound of lp-MKL which is a generalized framework of multiple kernel learning (MKL) and imposes lp-mixed-norm regularization instead of l1-mixed-norm regularization. We utilize localization techniques to obtain the sharp learning rate. The bound is characterized by the d…

Kernel Density Estimation is a very popular technique of approximating a density function from samples. The accuracy is generally well-understood and depends, roughly speaking, on the kernel decay and local smoothness of the true density. However concrete statements in the literature are often invoked in very specific …

The paper studies eigenvalues of graph Laplacians on data clouds and proves central limit theorems.

Theoretical framework explains why few epochs are enough for LLM fine-tuning.

We present eigenvalue decay estimates of integral operators associated with compositional dot-product kernels. The estimates improve on previous ones established for power series kernels on spheres. This allows us to obtain the volumes of balls in the corresponding reproducing kernel Hilbert spaces. We discuss the cons…

We consider the learning algorithms under general source condition with the polynomial decay of the eigenvalues of the integral operator in vector-valued function setting. We discuss the upper convergence rates of Tikhonov regularizer under general source condition corresponding to increasing monotone index function. T…

We consider the $SL(2,R)$ action on moduli spaces of quadratic differentials. If $μ$ is an $SL(2,R)$-invariant probability measure, crucial information about the associated representation on $L^2(μ)$ (and in particular, fine asymptotics for decay of correlations of the diagonal action, the Teichmüller flow) is encoded …

Two methods solve kernel ridge regression problems efficiently.

The Novikov-Shubin invariants for a non-compact Riemannian manifold M can be defined in terms of the large time decay of the heat operator of the Laplacian on square integrable p-forms on M. For the (2n+1)-dimensional Heisenberg group H, the Laplacian can be decomposed into operators in the conjugate of the generalised…

PCA-Net combines PCA and neural networks for operator approximation, with new bounds on complexity.

Signatures of universality are detected by comparing individual eigenvalue distributions and level spacings from financial covariance matrices to random matrix predictions. A chopping procedure is devised in order to produce a statistical ensemble of asset-price covariances from a single instance of financial data sets…

We analyze the size of the dictionary constructed from online kernel sparsification, using a novel formula that expresses the expected determinant of the kernel Gram matrix in terms of the eigenvalues of the covariance operator. Using this formula, we are able to connect the cardinality of the dictionary with the eigen…

We study the spectral gap of the Erdős--Rényi random graph through the connectivity threshold. In particular, we show that for any fixed $δ> 0$ if $p \ge \frac{(1/2 + δ) \log n}{n},$ then the normalized graph Laplacian of an Erdős--Rényi graph has all of its nonzero eigenvalues tightly concentrated around $1$. We est…

Bayesian approach learns linear operators from noisy data.

We find a deterministic equivalent for random feature regression's test error, independent of feature map dimension.

Kernel interpolation is inconsistent for norms with smoothness above a constant.

We derive an upper bound on the local Rademacher complexity of $\ell_p$-norm multiple kernel learning, which yields a tighter excess risk bound than global approaches. Previous local approaches aimed at analyzed the case $p=1$ only while our analysis covers all cases $1\leq p\leq\infty$, assuming the different feature …

Study elliptic operators on glued manifolds, reducing to finite-dimensional systems.

In this paper, we consider the nonparametric least square regression in a Reproducing Kernel Hilbert Space (RKHS). We propose a new randomized algorithm that has optimal generalization error bounds with respect to the square loss, closing a long-standing gap between upper and lower bounds. Moreover, we show that our al…

In the absence of explicit regularization, Kernel "Ridgeless" Regression with nonlinear kernels has the potential to fit the training data perfectly. It has been observed empirically, however, that such interpolated solutions can still generalize well on test data. We isolate a phenomenon of implicit regularization for…