A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Adaptive TBPTT controls gradient bias in RNNs for faster convergence.
In this paper, we revisit the recurrent back-propagation (RBP) algorithm, discuss the conditions under which it applies as well as how to satisfy them in deep neural networks. We show that RBP can be unstable and propose two variants based on conjugate gradient on the normal equations (CG-RBP) and Neumann series (Neuma…
A new algorithm reduces the computational cost of RTRL while maintaining performance.
Despite all the impressive advances of recurrent neural networks, sequential data is still in need of better modelling. Truncated backpropagation through time (TBPTT), the learning algorithm most widely used in practice, suffers from the truncation bias, which drastically limits its ability to learn long-term dependenc…
New method reduces memory usage in deep HRNNs by replacing gradient backpropagation with local losses.
Analyzes learning dynamics of RNNs under locality constraints.
RTRL optimizes long sequences without truncation, converging to loss minima.