15 Oct
2019
15 Oct
'19
10:30 a.m.
Is this the only 16-bit floating-point format in use?
IEEE 16bit (fp16) and bfloat16 are both around, but bfloat16 seems to be
the new leader in modern implementations thanks to ML use. I haven't experienced both used together but I wouldn't rule it out given bfloat16 may be accelerator specific. Google and intel have support for bfloat16 in some hardware. bfloat16 makes it easy to move to fp32 as they have the same exponent size. Refs: https://en.wikipedia.org/wiki/Bfloat16_floating-point_format https://nickhigham.wordpress.com/2018/12/03/half-precision-arithmetic-fp16-v... --Matt.