Unified AdaGrad with momentum accelerates stochastic optimization in non-convex settings.
problem Theoretical gaps in convergence for AdaGrad and related algorithms in non-convex stochastic settings.
method Proposes AdaUSM, a unified momentum scheme for AdaGrad with adaptive learning rates.
result AdaUSM achieves a convergence rate of in non-convex stochastic settings.