Flexpoint improves deep learning training efficiency by using adaptive 16-bit format.
problem Training deep neural networks in low bit-width formats is challenging.
method Flexpoint uses a shared exponent dynamically adjusted to minimize overflows and maximize dynamic range.
result 16-bit Flexpoint tensors closely match 32-bit floating point in training deep networks without tuning.