- What is your evaluation of the design? For the original scope (i.e. integers), very good. However, I'd like to see additional support for float, double, and user defined types added. Without it, this library isn't very useful to me personally since a large percentage of the values that I have to deal with are either float or double. ---------------------------------------------------------------------- - What is your evaluation of the documentation? Excellent. Is a tutorial section going to be added? I'm not sure one is needed given the existing examples in the rest of the documentation, but the documentation's menu bar implies there will be one. ---------------------------------------------------------------------- - What is your evaluation of the implementation? Very good. At -O2, I was impressed with the performance relative to GCC's builtin byteswap routines. When I first saw the implementation, I honestly didn't think it would perform very well given its complexity compared to a naive implementation (e.g. the version used in the endian::reorder functions). I was pleasantly surprised. I guess that's a testament to the level of optimization compilers are capable of these days. On an Intel Core2, the performance penalty of the uint32 endian<> template and the reorder() function relative to GCC-4.4 builtins (i.e. runtime divided by runtime of builtin) is as follows: endian<> reorder() ------------------------ -O0 5.4 3.0 -O2 2.3 6.5 It seems like the reorder functions could be implemented in terms of the integer types for improved performance (and also maintainability). Something like: template<typename T> inline void reorder(T& x) { x = *reinterpret_cast< endian<endianness::nonnative, T, sizeof(T)/8> *>(&x); } template<typename T> inline void reorder(T source, T& target) { target = *reinterpret_cast< endian<endianness::nonnative, T, sizeof(T)/8> *>(&source); } Obviously the above requires extending the endianess enum with a "nonnative" value. Performance isn't that critical for me, but it does seem like providing platform specific implementations that take advantage of things like GCC's builtin byteswap functions should also be considered to get the performance up even further. Finally, for my last performance related comment, I was surprised to see that the native types had basically exact same performance as the nonnative types. Couldn't the native types be specialized to effectively be a no-op? Would "byteswap" or maybe "swapbytes" be a better name than "reorder"? ---------------------------------------------------------------------- - What is your evaluation of the potential usefulness of the library? For those who deal with non-native endian data, this library is extremely useful. It automatically eliminates a whole class of common programming errors when dealing with such data. ---------------------------------------------------------------------- - Did you try to use the library? With what compiler? Yes; gcc-4.4.3 on Linux ---------------------------------------------------------------------- - Are you knowledgeable about the problem domain? Yes. I've used a similar home-grown library for the past 10 years, and have been looking forward this being added to Boost for a long time. ----------------------------------------------------------------------
- Do you think the library should be accepted as a Boost library?
Absolutely. I'd like to see support for float and double, but even without those additions, I still vote yes.
On Wed, Sep 7, 2011 at 1:03 PM, john filo <filo@arlut.utexas.edu> wrote:
- What is your evaluation of the design?
For the original scope (i.e. integers), very good. However, I'd like to see additional support for float, double, and user defined types added. Without it, this library isn't very useful to me personally since a large percentage of the values that I have to deal with are either float or double.
Several others have also expressed interest in FP, so I'll give it a try. Issues: * Because FP formats vary, just dealing with endianness doesn't ensure portability. * The endianness of FP and integer values differs on some platforms, so we will have to build up a config file with separate entries for each platform, and that will take time to mature. * Ditto FP sizes. * I'm only willing to provide conversion.hpp FP support. Providing types that mimic FP types is far beyond my knowledge of how to deal with floating point's notorious arithmetic issues.
---------------------------------------------------------------------- - What is your evaluation of the documentation?
Excellent.
Is a tutorial section going to be added? I'm not sure one is needed given the existing examples in the rest of the documentation, but the documentation's menu bar implies there will be one.
Yes, but it will be short and I won't have a chance to write it until after the formal review.
---------------------------------------------------------------------- - What is your evaluation of the implementation?
Very good.
At -O2, I was impressed with the performance relative to GCC's builtin byteswap routines. When I first saw the implementation, I honestly didn't think it would perform very well given its complexity compared to a naive implementation (e.g. the version used in the endian::reorder functions). I was pleasantly surprised. I guess that's a testament to the level of optimization compilers are capable of these days.
On an Intel Core2, the performance penalty of the uint32 endian<> template and the reorder() function relative to GCC-4.4 builtins (i.e. runtime divided by runtime of builtin) is as follows:
endian<> reorder() ------------------------ -O0 5.4 3.0 -O2 2.3 6.5
It seems like the reorder functions could be implemented in terms of the integer types for improved performance (and also maintainability). Something like:
template<typename T> inline void reorder(T& x) { x = *reinterpret_cast< endian<endianness::nonnative, T, sizeof(T)/8> *>(&x); }
template<typename T> inline void reorder(T source, T& target) { target = *reinterpret_cast< endian<endianness::nonnative, T, sizeof(T)/8> *>(&source); }
Obviously the above requires extending the endianess enum with a "nonnative" value.
Performance isn't that critical for me, but it does seem like providing platform specific implementations that take advantage of things like GCC's builtin byteswap functions should also be considered to get the performance up even further.
Finally, for my last performance related comment, I was surprised to see that the native types had basically exact same performance as the nonnative types. Couldn't the native types be specialized to effectively be a no-op?
There have been a lot of other suggestions about performance, and I'm actively working on performance issues right now. Stay tuned.
Would "byteswap" or maybe "swapbytes" be a better name than "reorder"?
I've tried a bunch of names, and am pretty happy with "reorder". But "swapbytes" would also be a decent name.
---------------------------------------------------------------------- - What is your evaluation of the potential usefulness of the library?
For those who deal with non-native endian data, this library is extremely useful. It automatically eliminates a whole class of common programming errors when dealing with such data.
---------------------------------------------------------------------- - Did you try to use the library? With what compiler?
Yes; gcc-4.4.3 on Linux
---------------------------------------------------------------------- - Are you knowledgeable about the problem domain?
Yes. I've used a similar home-grown library for the past 10 years, and have been looking forward this being added to Boost for a long time.
----------------------------------------------------------------------
- Do you think the library should be accepted as a Boost library?
Absolutely. I'd like to see support for float and double, but even without those additions, I still vote yes.
Thanks for the kind words, --Beman
On 8 September 2011 21:31, Beman Dawes <bdawes@acm.org> wrote:
Several others have also expressed interest in FP, so I'll give it a try. Issues:
* Because FP formats vary, just dealing with endianness doesn't ensure portability. * The endianness of FP and integer values differs on some platforms, so we will have to build up a config file with separate entries for each platform, and that will take time to mature. * Ditto FP sizes. * I'm only willing to provide conversion.hpp FP support. Providing types that mimic FP types is far beyond my knowledge of how to deal with floating point's notorious arithmetic issues.
I would also like to add my +1 for float support. You could limit support to IEEE754 format (32 bit, 64 bit). I'm not sure how widespread differing endianness between ints and floats is across the various platforms. Would it be acceptable to provide a #define that will cause ints and floats to be treated as having reversed endianness relative to each other (i.e. if native int is little-endian, float would be big-endian)?
On Thu, Sep 8, 2011 at 7:50 AM, Vitaly Budovski <vbudovski+news@gmail.com> wrote:
On 8 September 2011 21:31, Beman Dawes <bdawes@acm.org> wrote:
Several others have also expressed interest in FP, so I'll give it a try. Issues:
* Because FP formats vary, just dealing with endianness doesn't ensure portability. * The endianness of FP and integer values differs on some platforms, so we will have to build up a config file with separate entries for each platform, and that will take time to mature. * Ditto FP sizes. * I'm only willing to provide conversion.hpp FP support. Providing types that mimic FP types is far beyond my knowledge of how to deal with floating point's notorious arithmetic issues.
I would also like to add my +1 for float support. You could limit support to IEEE754 format (32 bit, 64 bit).
The plan is to support the most common platforms. I suspect that's IEEE754 format (32 bit, 64 bit), but I haven't had a chance to check yet.
I'm not sure how widespread differing endianness between ints and floats is across the various platforms.
The Wikipedia article says it's uncommon, IIRC, but we need some real data on that.
Would it be acceptable to provide a #define that will cause ints and floats to be treated as having reversed endianness relative to each other (i.e. if native int is little-endian, float would be big-endian)?
Yes, and that might be a good way to deal with less common platforms until our config file coverage matures. Thanks, --Beman
Beman Dawes wrote:
I'm not sure how widespread differing endianness between ints and floats is across the various platforms.
The Wikipedia article says it's uncommon, IIRC, but we need some real data on that.
I think it's vanishingly rare and can be ignored (as can any non-ieee754 formats). The only case I'm aware of is the format of doubles on the original (20-year-old) ARM FPA chip. This would store the bytes of a double as 45670123, i.e the bytes are little-endian within the words (like ints) but the two words are ordered big endian. <anecdote>This chip was designed by 3 guys, and I shared an office with 2 of them at the time. One came from Acorn who had been using little-endian ARM chips for years, and the other came from Apple (who were about to put an ARM chip in the first Newton) and had a 68000 big-endian background. So perhaps it's not surprising that it got muddled. (The third guy designed the divide-and-square-root unit, nocturnally.) I doubt that more than a few hundred of these chips were ever made, but the format was a bit more widespread because the chip could be emulated by the OS through illegal instruction traps.</anecdote> Cheers, Phil.
On Thu, Sep 8, 2011 at 12:44 PM, Phil Endecott <spam_from_boost_dev@chezphil.org> wrote:
Beman Dawes wrote:
I'm not sure how widespread differing endianness between ints and floats is across the various platforms.
The Wikipedia article says it's uncommon, IIRC, but we need some real data on that.
I think it's vanishingly rare and can be ignored (as can any non-ieee754 formats).
Let's hope so! Perhaps the only real impact of such historical curiosities is a need to be sure there are test cases that would fail if we ran into one of them.But normal test cases probably do that anyhow.
The only case I'm aware of is the format of doubles on the original (20-year-old) ARM FPA chip. This would store the bytes of a double as 45670123, i.e the bytes are little-endian within the words (like ints) but the two words are ordered big endian.
I worked briefly in the early 1980's on a 16-bit system that stored 32-bit integers as two 16-bit words in big endian order but the bytes within the words in little endian order:-)
<anecdote>This chip was designed by 3 guys, and I shared an office with 2 of them at the time. One came from Acorn who had been using little-endian ARM chips for years, and the other came from Apple (who were about to put an ARM chip in the first Newton) and had a 68000 big-endian background. So perhaps it's not surprising that it got muddled. (The third guy designed the divide-and-square-root unit, nocturnally.) I doubt that more than a few hundred of these chips were ever made, but the format was a bit more widespread because the chip could be emulated by the OS through illegal instruction traps.</anecdote>
:-) --Beman
participants (4)
-
Beman Dawes -
john filo -
Phil Endecott -
Vitaly Budovski