Bayesian filtering unifies adaptive and non-adaptive neural network optimization methods.
problem Optimizing neural networks using standard methods like Adam and RMSprop.
method Formulated as Bayesian filtering, accounting for temporal dynamics of all parameters.
result Recover Adam and AdamW optimizers with competitive generalization performance.