Is this the only 16-bit floating-point format in use?

IEEE 16bit (fp16) and bfloat16 are both around, but bfloat16 seems to be

the new leader in modern implementations thanks to ML use. I haven't experienced both used together but I wouldn't rule it out given bfloat16 may be accelerator specific. Google and intel have support for bfloat16 in some hardware. bfloat16 makes it easy to move to fp32 as they have the same exponent size. Refs: https://en.wikipedia.org/wiki/Bfloat16_floating-point_format https://nickhigham.wordpress.com/2018/12/03/half-precision-arithmetic-fp16-v... --Matt.

Thanks for these useful references.

n.p.

Are bloat16 and IEEE float16 I:\boost\libs\math\include\boost\math\cstdfloat\cstdfloat_types.hpp

the two layouts that we need to consider?

Arm also supports another in that it has two similar formats __fp16 and _Float16 :-( "ARM processors support (via a floating point control register https://en.wikipedia.org/wiki/Control_register bit) an "alternative half-precision" format, which does away with the special case for an exponent value of 31 (111112).[10] https://en.wikipedia.org/wiki/Half-precision_floating-point_format#cite_note... It is almost identical to the IEEE format, but there is no encoding for infinity or NaNs; instead, an exponent of 31 encodes normalized numbers in the range 65536 to 131008." from wiki: https://en.wikipedia.org/wiki/Half-precision_floating-point_format Arm reference: http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.100067_0612_00_e... gcc ref: https://gcc.gnu.org/onlinedocs/gcc/Half-Precision.html Strange complexity within Arm: "The ARM target provides hardware support for conversions between __fp16 and float values as an extension to VFP and NEON (Advanced SIMD), and from ARMv8-A provides hardware support for conversions between __fp16 and double values. GCC generates code using these hardware instructions if you compile with options to select an FPU that provides them; for example, -mfpu=neon-fp16 -mfloat-abi=softfp, in addition to the -mfp16-format option to select a half-precision format." Unpleasant, sorry. More bfloat16 FWIW (hardware support list is longer): https://en.wikichip.org/wiki/brain_floating-point_format So that makes at least 3 I guess. Facebook has been experimenting with an alternate format Gustafson's post which is quite neat and rational and perhaps better than ieee at 64 bit too: https://www.nextplatform.com/2019/07/08/new-approach-could-sink-floating-poi... Facebook reference: https://engineering.fb.com/ai-research/floating-point-math/ posit "land": https://posithub.org/news but posit only lives in FPGA land AFAICT and not yet something to worry about. --Matt.

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

...

Strange complexity within Arm: "The ARM target provides hardware support for conversions between __fp16 and float values as an extension to VFP and NEON (Advanced SIMD), and from ARMv8-A provides hardware support for conversions between __fp16 and double values. GCC generates code using these hardware instructions if you compile with options to select an FPU that provides them; for example, -mfpu=neon-fp16 -mfloat-abi=softfp, in addition to the -mfp16-format option to select a half-precision format."

Unpleasant, sorry. I really, really don't wish to know all that! 😉 Paul

Me neither. My first reaction was "yes of course we should do that", but on reflection I'm not so sure: The intention of was to provide a *portable* set of typedefs such that each is mapped to the corresponding IEEE defined type. And indeed, if numeric_limits::is_iec559 happens to be false, then the header will trigger a static_assert and complain that it's incorrectly configured. So... we could provide a float16_t typedef, but if we're being consistent (and we should be), then it would only be for IEEE float16 compatible types. My concern is quite how we detect/configure/test that. I also don't think we currently have any tester running on arm (for example), and we certainly don't have arm CI (is there any?). Mostly thinking out loud yours, John. -- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

John Maddock wrote:

The intention of was to provide a *portable* set of typedefs such that each is mapped to the corresponding IEEE defined type.

And indeed, if numeric_limits::is_iec559 happens to be false, then the header will trigger a static_assert and complain that it's incorrectly configured.

So... we could provide a float16_t typedef, but if we're being consistent (and we should be), then it would only be for IEEE float16 compatible types.

My concern is quite how we detect/configure/test that.

#if (__ARM_FP & 2) && defined(__ARM_FP16_FORMAT_IEEE) using float16_t = __fp16; struct std::numeric_limits { const bool is_iec559 = true; etc. etc. }; #endif I think the difficult issues are (a) Do we need a richer vocabulary than floatNN_t to describe the different formats, when more than one is supported at the same time? (b) Can we provide a software fallback for platforms where there is no hardware support? Right now, I'm trying to make a binary file containing these 16-bit floats that will be read (memory-mapped) on an ARM system, built on an x86 system.

I also don't think we currently have any tester running on arm (for example)

I was running ARM64 testers when I had an account with Scaleway, but I don't think anyone was looking at the results; certainly no-one ever asked me anything about them or seemed to notice when I turned them off. I have considered reviving this now that AWS has ARM instances, but their smallest ARM instances are quite a lot larger (and more expensive) than their smallest x86 instances for some reason. That may change eventually. If anyone's interested, get in touch. Regards, Phil.

BFloat16 conversion to Float32 is not 100% trivial, because of NaN and rounding modes. I think https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/lib/bfl... does a good job at documenting this (the linked code is Apache 2.0 licensed, in case you worry about such issues). However, if one ignores these complexities, conversion is as simple as bitshifting. The public Intel BF16 spec is https://software.intel.com/sites/default/files/managed/40/8b/bf16-hardware-n..., which describes the details of the Intel definition. There are some implementation specific details in https://reviews.llvm.org/D60550. I can't comment on other hardware implementations. The important distinction between __fp16 and _Float16 is the former is a storage format type, not an arithmetic type, whereas the latter is an arithmetic type. The former is more easily implementable, e.g.Intel CPUs since Ivy Bridge use the F16C to do fast conversion to/from the 16-bit storage format to float32, but all arithmetic is done with float32 hardware. Folks who are interested in this topic may enjoy reading https://arxiv.org/abs/1904.06376. The methods described therein are not necessarily applicable to Boost Multiprecision, but may be relevant if uBLAS gets involved. Jeff, who works for Intel On Tue, Oct 15, 2019 at 7:04 AM Phil Endecott via Boost < boost@lists.boost.org> wrote:

Matt Hurd wrote:

...

IEEE 16bit (fp16) and bfloat16 are both around, but bfloat16 seems to be the new leader in modern implementations thanks to ML use. I haven't experienced both used together but I wouldn't rule it out given bfloat16 may be accelerator specific. Google and intel have support for bfloat16 in some hardware. bfloat16 makes it easy to move to fp32 as they have the same exponent size.

Refs: https://en.wikipedia.org/wiki/Bfloat16_floating-point_format

https://nickhigham.wordpress.com/2018/12/03/half-precision-arithmetic-fp16-v...

According to section 4.1.2 of this ARM document:

http://infocenter.arm.com/help/topic/com.arm.doc.ihi0053d/IHI0053D_acle_2_1....

implementations support both the IEEE format (1 sign, 5 exponent and 10 mantissa) and an alternative format which is similar except that it doesn't support Inf and NaN, and gains slightly more range. Apparently the bfloat16 format is supported in ARMv8.6-A, but I don't believe that is deployed anywhere yet.

The other place where I've used 16-bit floats is in OpenGL textures, ( https://www.khronos.org/registry/OpenGL/extensions/OES/OES_texture_float.txt ), which use the 1-5-10 format.

I was a bit surprised by the 1-5-10 choice; the maximum value that can be represented is only 65504, i.e. less than the maximum value for an unsigned int of the same size.

bfloat16 can be trivially implemented (as a storage-only type) simply by truncating a 32-bit float; perhaps support for that would be useful too?

Regards, Phil.

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

-- Jeff Hammond jeff.science@gmail.com http://jeffhammond.github.io/