arXiv research

Active data collection improves convergence rates in operator learning.

Paper calculates eigenvalue decay rates for neural network kernels on general domains.

In this paper we continue our study of the Laplacian on manifolds with axial analytic asymptotically cylindrical ends initiated in~arXiv:1003.2538. By using the complex scaling method and the Phragmén-Lindelöf principle we prove exponential decay of the eigenfunctions corresponding to the non-threshold eigenvalues of t…

In this paper, we give a new sharp generalization bound of lp-MKL which is a generalized framework of multiple kernel learning (MKL) and imposes lp-mixed-norm regularization instead of l1-mixed-norm regularization. We utilize localization techniques to obtain the sharp learning rate. The bound is characterized by the d…

Model proposes neural network for continuous time dynamics with inductive biases.

We investigate if kernel regularization methods can achieve minimax convergence rates over a source condition regularity assumption for the target function. These questions have been considered in past literature, but only under specific assumptions about the decay, typically polynomial, of the spectrum of the the kern…

The study examines Kernel Ridge Regression error rates across noiseless and noisy conditions.

Universal algorithm learns unknown distribution for various decision-making problems.

Two new algorithms reduce online kernel regression's computational cost while maintaining optimal regret bounds.

Stability inequalities for specific solutions in high dimensions.

We consider the learning algorithms under general source condition with the polynomial decay of the eigenvalues of the integral operator in vector-valued function setting. We discuss the upper convergence rates of Tikhonov regularizer under general source condition corresponding to increasing monotone index function. T…

ASGD outperforms SGD in overparameterized linear regression, especially in subspaces of small eigenvalues.

Under the quadratic-decay-conditions of the radial curvatures of an end, we shall derive growth estimates of solutions to the eigenvalue equation and show the absence of eigenvalues.

High-dimensional kernel regression struggles due to rotational invariance.

Theoretical framework explains why few epochs are enough for LLM fine-tuning.

The concern of this paper is to clarify a relationship between the curvatures at infinity and the spectral structure of the Laplacian. In particular, this paper discusses the question of whether there is an eigenvalue of the Laplacian embedded in the essential spectrum or not. The borderline-behavior of the radial curv…

Bayesian approach learns linear operators from noisy data.

We find a deterministic equivalent for random feature regression's test error, independent of feature map dimension.

We derive an upper bound on the local Rademacher complexity of $\ell_p$-norm multiple kernel learning, which yields a tighter excess risk bound than global approaches. Previous local approaches aimed at analyzed the case $p=1$ only while our analysis covers all cases $1\leq p\leq\infty$, assuming the different feature …

In this paper, we consider the nonparametric least square regression in a Reproducing Kernel Hilbert Space (RKHS). We propose a new randomized algorithm that has optimal generalization error bounds with respect to the square loss, closing a long-standing gap between upper and lower bounds. Moreover, we show that our al…

Gradient descent outperforms ridge regression under certain covariance matrix decay conditions.

Optimal learning rates decay to zero in easy tasks and maintain a warmup phase in hard tasks.

Upper bound found for Steklov eigenvalues counting function.

The paper studies eigenvalues of graph Laplacians on data clouds and proves central limit theorems.

Study on massless Vlasov equation on Reissner-Nordström spacetimes, showing decay rates and non-decay phenomena.

Introduces gradient decay in Softmax for better generalization.

Study small eigenvalues of Toeplitz operators and their relation to Mabuchi geodesics.

Deep ReLU networks approximate as well as shallow ones in kernel regimes.

Spectral feature learning improves IV regression for causal effect estimation.

Kernel interpolation is inconsistent for norms with smoothness above a constant.

WSD schedule improves model training efficiency by adapting learning rates dynamically.

The salient properties of large empirical covariance and correlation matrices are studied for three datasets of size 54, 55 and 330. The covariance is defined as a simple cross product of the returns, with weights that decay logarithmically slowly. The key general properties of the covariance matrices are the following…

New framework estimates eigenvalues of kernel matrices without full matrix construction.

Step decay schedules improve convergence in non-convex optimization.

Ridge regression performs optimally in noisy environments with heavy-tailed distributions.

Minimax optimal convergence rates for classes of stochastic convex optimization problems are well characterized, where the majority of results utilize iterate averaged stochastic gradient descent (SGD) with polynomially decaying step sizes. In contrast, SGD's final iterate behavior has received much less attention desp…

We get decay rate of higher derivatives of nonlinear massless Dirac equations with a kind of "good" spin null form. The method we rely on is similar to that of Li and Zang. However, they only give the decay rate of solution itself to nonlinear massless Dirac system.

We study the spectral gap of the Erdős--Rényi random graph through the connectivity threshold. In particular, we show that for any fixed $δ> 0$ if $p \ge \frac{(1/2 + δ) \log n}{n},$ then the normalized graph Laplacian of an Erdős--Rényi graph has all of its nonzero eigenvalues tightly concentrated around $1$. We est…

Improved TD learning with tail averaging and regularization achieves optimal convergence rates.

The paper studies harmonic map heat flow stability and decay rates.

Eigenvalues of random hyperbolic surface covers converge to hyperbolic plane's.

Learning rate decay (lrDecay) is a \emph{de facto} technique for training modern neural networks. It starts with a large learning rate and then decays it multiple times. It is empirically observed to help both optimization and generalization. Common beliefs in how lrDecay works come from the optimization analysis of (S…

This paper concerns integral varifolds of arbitrary dimension in an open subset of Euclidean space satisfying integrability conditions on their first variation. Firstly, the study of pointwise power decay rates almost everywhere of the quadratic tilt-excess is completed by establishing the precise decay rate for two-di…

This purpose of this write-up is to share an idea for accurate computation of Laplace eigenvalues on a broad class of smooth domains. We represent the eigenfunction $u$ as a linear combination of eigenfunctions corresponding to the common eigenvalue $ρ^{2}$:\EQN{6}{1}{}{0}{\RD{\CELL{u(r,θ) =\sum_{n=0}^{N}P_{n}J_{n}(ρ) …

Study elliptic operators on glued manifolds, reducing to finite-dimensional systems.

We consider any pseudo holomorphic integral 2-cycle in an arbitrary almost complex manifold and perform a blow up analysis at an arbitrary point. Building upon a pseudo algebraic blow up (previously introduced by the author) we prove a geometric rate of decay for the mass ratio towards the limiting density, with an exp…

We introduce a novel algorithm that computes the $k$-sparse principal component of a positive semidefinite matrix $A$. Our algorithm is combinatorial and operates by examining a discrete set of special vectors lying in a low-dimensional eigen-subspace of $A$. We obtain provable approximation guarantees that depend on t…

Optimal CATE estimation with structured contrast functions using KRR.