On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,932 papers · 148 categories

Trend · papers per month

count % of corpus

4 results for “AdaGrad-Norm”

Linear Convergence of Adaptive Stochastic Gradient Descentstat.ML

AdaGrad-Norm achieves linear convergence for certain functions.

problem Proving linear convergence for specific types of functions.

method Introducing RUIG, a measure of gradient balance; developing a two-stage framework.

result AdaGrad-Norm achieves linear convergence for certain functions.

AdaGrad-Norm linear convergence stochastic gradient descent Polyak Lojasiewicz inequality gradient balance

2019-08-28abs ↗pdf ↗

The Power of Adaptivity in SGD: Self-Tuning Step Sizes with Unbounded Gradients and Affine Variancestat.ML

AdaGrad-Norm achieves optimal convergence rates for non-convex objectives without tuning.

problem Optimal convergence rates for non-convex, smooth objectives with adaptive step sizes.

method Adaptive SGD (AdaGrad-Norm) with self-tuning step sizes, analyzing under unbounded gradients and affine variance scaling.

result AdaGrad-Norm achieves order optimal convergence rate of $\mathcal{O}\left(\frac{\mathrm{poly}\log(T)}{\sqrt{T}} ight)$ under optimal assumptions.

AdaGrad SGD Adaptive Optimization Convergence Rates Non-Convex Optimization

2022-02-11abs ↗pdf ↗

AdaGrad stepsizes: Sharp convergence over nonconvex landscapesstat.ML

Adaptive gradient methods such as AdaGrad and its variants update the stepsize in stochastic gradient descent on the fly according to the gradients received along the way; such methods have gained widespread use in large-scale optimization for their ability to converge robustly, without the need to fine-tune the stepsi…

2018-06-05abs ↗pdf ↗

The High Line: Exact Risk and Learning Rate Curves of Stochastic Adaptive Learning Rate Algorithmsmath.OC

Exact risk and learning rate curves derived for adaptive SGD on high-dimensional problems.

problem Analyzing risk and learning rate dynamics in high-dimensional optimization problems.

method Developed a framework to give exact expressions for risk and learning rate curves using ODEs.

result Exact expressions for risk and learning rate curves, with detailed analysis of two adaptive learning rates.

high-dimensional optimization stochastic gradient descent adaptive learning rates risk curves learning rate curves

2024-05-30abs ↗pdf ↗