A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Gradient descent at edge of stability stabilizes implicitly, following projected gradient descent.
Gradient descent with large momentum finds flatter minima.
SGD works well with large learning rates at the edge of stability.
A standard belief on emerging collective behavior is that it emerges from simple individual rules. Most of the mathematical research on such collective behavior starts from imperative individual rules, like always go to the center. But how could an (optimal) individual rule emerge during a short period within the group…
A new ODE model explains gradient descent dynamics near edge of stability.
The study explains delayed spikes in batch-normalized models.
Asynchronous distributed machine learning solutions have proven very effective so far, but always assuming perfectly functioning workers. In practice, some of the workers can however exhibit Byzantine behavior, caused by hardware failures, software bugs, corrupt data, or even malicious attacks. We introduce \emph{Karda…