A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
RTRL optimizes long sequences without truncation, converging to loss minima.
A new algorithm reduces the computational cost of RTRL while maintaining performance.
Despite all the impressive advances of recurrent neural networks, sequential data is still in need of better modelling. Truncated backpropagation through time (TBPTT), the learning algorithm most widely used in practice, suffers from the truncation bias, which drastically limits its ability to learn long-term dependenc…
SnAp approximates RTRL for online training of sparse recurrent networks.
UORO reduces gradient variance in online RNN learning.
We cast Amari's natural gradient in statistical learning as a specific case of Kalman filtering. Namely, applying an extended Kalman filter to estimate a fixed unknown parameter of a probabilistic model from a series of observations, is rigorously equivalent to estimating this parameter via an online stochastic natural…
New neural network theory mimics physics laws, making computations more plausible.
New RL framework improves real-time control performance.
Proves local convergence of various online and recurrent optimization algorithms.