arXiv research

SALR improves deep learning generalization by dynamically adjusting learning rates.

New insights into network generalization show learning rate affects both norm and sharpness.

Sharp lower bounds on shallow neural networks' approximation rates are derived.

We prove sharp bounds for the growth rate of eigenfunctions of the Ornstein-Uhlenbeck operator and its natural generalizations. The bounds are sharp even up to lower order terms and have important applications to geometric flows.

In this paper, we give a new sharp generalization bound of lp-MKL which is a generalized framework of multiple kernel learning (MKL) and imposes lp-mixed-norm regularization instead of l1-mixed-norm regularization. We utilize localization techniques to obtain the sharp learning rate. The bound is characterized by the d…

Study on convergence rates for optimal transport with regularization.

New adaptive scheduler improves SAM for better model training.

Sharp Lipschitz bounds for flow-matching and diffusion models with optimal sampling rates.

Study shows how a curve shortens to a half-circle under specific flow.

EM algorithm converges linearly and achieves sharp rate in estimating mixtures of pairwise differences.

Sharp rates found for learning with dependent data, avoiding sample size deflation.

Sharp representation theorems show depth benefits for ReLU networks.

Sharp bounds on neural network approximation rates and widths.

Study sharp convergence rates of empirical UOT for spatio-temporal point processes.

Paper finds sharpness differences in transformer blocks accelerating LLM training.

Forward regression is a statistical model selection and estimation procedure which inductively selects covariates that add predictive power into a working statistical regression model. Once a model is selected, unknown regression parameters are estimated by least squares. This paper analyzes forward regression in high-…

Deep linear networks minimize sharpness, avoiding large eigenvalues.

Gradient descent trains neural networks to match kernel regression's sharp generalization rate.

Sharp-MAML improves MAML by reducing saddle points in few-shot learning.

Perpetual futures offer leverage without maturity, with prices influenced by funding rates.

Stochastic (sub)gradient methods require step size schedule tuning to perform well in practice. Classical tuning strategies decay the step size polynomially and lead to optimal sublinear rates on (strongly) convex problems. An alternative schedule, popular in nonconvex optimization, is called \emph{geometric step decay…

This paper closes the gap on matching pursuit's convergence rate.

Classifiers built with neural networks handle large-scale high dimensional data, such as facial images from computer vision, extremely well while traditional statistical methods often fail miserably. In this paper, we attempt to understand this empirical success in high dimensional classification by deriving the conver…

Investment strategy using fractional Kelly portfolios for better growth expectations.

Maximal initial learning rate for deep ReLU networks identified.

Poly-GNNs achieve similar performance regardless of depth, highlighting graph noise's dominance.

Sharp convergence rate for curvature stability in planar free elastic flow.

The study analyzes sharpness dynamics in neural networks, revealing mechanisms and conditions.

New algorithm optimizes convex functions with noisy evaluations in one dimension.

Sharp bounds on quasimode norms on compact space forms.

Sharp bounds on weak convergence rate for rough volatility models.

We describe a post hoc test for the Sharpe ratio, analogous to Tukey's test for pairwise equality of means. The test can be applied after rejection of the hypothesis that all population Signal-Noise ratios are equal. The test is applicable under a simple correlation structure among asset returns. Simulations indicate t…

Noise in linear networks minimizes sharpness and leads to shrinkage-thresholding.

Novel oracle-type inequality for logistic loss in DNNs achieves sharp convergence rates.

We prove that the boundary of a (not necessarily connected) bounded smooth set with constant nonlocal mean curvature is a sphere. More generally, and in contrast with what happens in the classical case, we show that the Lipschitz constant of the nonlocal mean curvature of such a boundary controls its $C^2$-distance fro…

Sharp comparison theorems for 3D manifolds with scalar curvature bound.

We observe that a sharp result on the exponential growth rate of the number of primitive elements exists for the free group on two generators.

Study on convergence rate of Bergman metrics on Kähler manifolds.

Stochastic Gradient Descent (SGD) and its variants are mainstream methods for training deep networks in practice. SGD is known to find a flat minimum that often generalizes well. However, it is mathematically unclear how deep learning can select a flat minimum among so many minima. To answer the question quantitatively…

We construct continuous-time equilibrium models based on a finite number of exponential utility investors. The investors' income rates as well as the stock's dividend rate are governed by discontinuous Levy processes. Our main result provides the equilibrium (i.e., bond and stock price dynamics) in closed-form. As an a…

Truncated SGD with heavy-tailed noise eliminates sharp local minima.

We introduce a scalable measure of curvature for analyzing training dynamics of large language models.

We show that if $F$ is a convex class of functions that is $L$-subgaussian, the error rate of learning problems generated by independent noise is equivalent to a fixed point determined by `local' covering estimates of the class, rather than by the gaussian averages. To that end, we establish new sharp upper and lower e…

Sharp bounds established for Federated Averaging (FedAvg), improving convergence rates.

We study statistical risk minimization problems under a privacy model in which the data is kept confidential even from the learner. In this local privacy framework, we establish sharp upper and lower bounds on the convergence rates of statistical estimation procedures. As a consequence, we exhibit a precise tradeoff be…

Neural networks trained with PGD achieve sharp regression rates in interpolation spaces.

Sharp convergence analysis for nonconvex regression models.

We apply the procedure of Lee et al. to the problem of performing inference on the signal-noise ratio of the asset which displays maximum sample Sharpe ratio over a set of possibly correlated assets. We find a multivariate analogue of the commonly used approximate standard error of the Sharpe ratio to use in this condi…