HyperAdam learns to optimize neural networks by combining Adam's updates with varying decay rates.
problem Limitation of learned black-box optimizers in generalization ability.
method HyperAdam combines learned and traditional Adam optimizer, adaptively learning weights and decay rates.
result HyperAdam outperforms traditional optimizers in various network training tasks.