On Sat, Sep 10, 2011 at 10:40 AM, Vicente J. Botet Escriba <vicente.botet@wanadoo.fr> wrote:

...

   - What is your evaluation of the design?

The design of the endian class is a clasic integer wrapper, that respond to almost all the needs. No major problem with it excep:

I find the explicit conversion (constructor) and the implicit assignement a little bit confuising. I don't think the the explicit constructor make it safer.

As you know, the conventional wisdom is to never write a class that includes both an implicit conversion constructor from another type and an implicit conversion operator back to that other type. If you think that for some reason this conventional wisdom doesn't apply here, please explain:-) The usual way to deal with this in C++03 of course is to make the converting constructor implicit. Although it was years ago, I have a vague memory that I forgot to do that initially, and then either someone pointed it out or I got into unwanted conversion trouble when writing test cases. With C++11, we could instead mark the conversion operator implicit. I have no idea if that is a better approach, and would like to wait until more experience develops before using it.

The implicit conversion to the underlying type has the drawback that different endiannes will print always in native format. The library must overload the input and output operators, and as Beman has suggested add a facet and a some manipulators.

I'll make sure printing issues are covered in the docs, and make sure the tests cover these as actually delivered by the code.

The endian class should be splited into endian_pack/buffer aware of the endianness and and endian integer providing the arithmetic operators.

I'm weakly in favor of separating the concerns, but want to hear what others say before making any commitment. One irritation with that approach is that the cleanest way to do so is probably via inheritance, but in C++03 that breaks PODness. That was what motivated me to propose the POD relaxations that are in C++11. Not a big thing, but unpleasant from the implementation viewpoint because we need to support both 03 and 11.

The endianness scoped enum native should be replaced so it is defined depending on the endianess of the platform.

 BOOST_SCOPED_ENUM_START(endianness) { big, little, native=(big or little) }; BOOST_SCOPED_ENUM_END

This will have the advantage to reduce of 1/3 the number of specializations.

Interesting idea! I'll give it a try!

The endian classes should be able to take as underlying type a C++11 scoped enum.

Any UDT providing access to the underlying integer type could also be good candidates as template parameters of the endian classes.

One issue is whether UDT's should "just work", or require some traits helpers such as a way to get at the underlying type. I've just started to run tests on UDT's with the current code. So far, it "just works". But that's without considering efficiency - knowing more may be required to get reasonable performance.

The conversion part could be improved: * The reorder fuction should provide both reorder in place and returned.

Interesting, but providing two ways of doing something is often considered the not best design practice. And the tests I've run so far don't indicate any efficiency concern. Once all other aspects of the interface and implementation are settled, let's revisit that question.

* conversion of at least c-arrays, boost::arrays, tuples, pair, ... should be provided as these types are the most used while defining messages or binary formats.

I may be misunderstanding your suggestion, but that sounds like a recipe for interface bloat to me. Many other operations are commonly performed on the contents of boost::arrays, tuples, and pairs, but the providers of those operations don't usually provide any special support for when they are to be called with arguments that happen to live in these types. Could you give an example of what you have in mind?

The library should provide in addition endian conversion functions that have the endiannes as template parameters to make possible generic functions.

Compile time dispatch on an endianness enum was also requested in another review. That's fine with me, but I haven't had a chance to figure out the interface details.

The use of a macro to choose between a POD implementation or not could introduce problems when two libraries using Boost.Endian expect that this macro takes different values. I don't know whether a policy parameter or duplicating the classes would be the best choice.

Since relaxed POD's and extended unions are now officially blessed by C++11, the whole POD issue goes away. Remember that the 03 non-POD version works fine with C++03 compilers except in unions. So the separate versions goe away eventually. Thus I don't want to do anything that enshrines two versions in the interface.

...

   - What is your evaluation of the implementation?

Quite correct. As other have signaled, * the endian class must use the reorder functions whenever possible.

As long as that doesn't degrade performance!

* the reorder functions should use intrinsics when available and provide a better performance.

Sure, all other things being equal.

Some performances tests should be added.

I've already started work on a benchmark program, although it isn't ready for prime time yet.

...

   - What is your evaluation of the documentation?

Short, very short.

The scope of the library should be clarified, limited to integer types,

There have been a lot of requests for extending the library to handle floating point types and UDT's, and I intend to do that if it can be done safely.

explaining the other builting types are not considered. Why only big/little endianness has been taken in account?

I'll add FAQ and/or add more entries to the final docs. Only big/little endianness is taken into account because these are the only endian schemes that have any practical value. All the others are just historical curiosities.

I would like to see a tutorial and examples sections that show how the library should be used.

A tutorial is planned, as are more examples.

* Use with a 3rd party UDT that doesn't have as parameter the underlying type

See above. That seems to be working already. The issue is nailing down and documenting the template parameter requirements.

* Use with Boost.Chrono and Boost.Units

Hum... Why wouldn't it already work with Boost.Chrono and Boost.Units types that are integers?

* Some examples reading a relatively complex data type (endian unaware) from a file, changing a field and then writing the data.

Yes! That's the example I have in mind for the tutorial.

The reference section must document the constraints of the template parameters.

Yes!

...

   - What is your evaluation of the potential usefulness of the      library?

Very useful. I've used it and my extension in a lot of projects. It's a must for the Boost.Bitfields which manage with endiansess for bitfields.

...

   - Did you try to use the library?  With what compiler?  Did you      have any problems?

I have not tried the review version. I adapted the initial version in Boost.EndianExt to take care of some of the requests made 1 year ago in this ML. So yes, I can say that I have used it.

...

   - How much effort did you put into your evaluation? A glance? A      quick reading? In-depth study?

I know quite well the library internals since a long time. I proposed Beman to take in account some of the extension I did without success. Happy to see that other have provided the good arguments so the future Endian library (if accepted) will be closer to what I was looking for.

The Endian library has had contributions from an unusually large number of people, and Vicente is one of those. I'll be updating the Acknowledgements in the docs again before any actual release.

...

   - Are you knowledgeable about the problem domain?

Yes, I think so.

...

And finally, every review should answer this question:

   - Do you think the library should be accepted as a Boost library?

No in its current state. Once the library takes in account the requested modification (that Beman has already accepted) ...

a mini-review will be necessary to improve the library before release.

No problem.

Beman, be free to adapt whatever you need from Boost.EndianExt.

Thanks, Vicente, and thanks for the detailed review! --Beman

Hi Vicente, There isn't time before the review ends to go into much detail, but I'll try to answer a few of the easier questions: On Thu, Sep 15, 2011 at 2:16 PM, Vicente J. Botet Escriba <vicente.botet@wanadoo.fr> wrote:

...

...

The implicit conversion to the underlying type has the drawback that different endiannes will print always in native format. The library must overload the input and output operators, and as Beman has suggested add a facet and a some manipulators.

I'll make sure printing issues are covered in the docs, and make sure the tests cover these as actually delivered by the code.

Could you tell us where you are related to i/o?

The binary I/O concerns seem to me to be best addressed by a bin() (or binary(), or whatever name) function that looks like an I/O manipulator. Boring, but it works. The other approaches (overloading operators beyond << and >> to do binary I/O) were too hard to explain, too error-prone, or both.

...

One issue is whether UDT's should "just work", or require some traits helpers such as a way to get at the underlying type.

I've just started to run tests on UDT's with the current code. So far, it "just works". But that's without considering efficiency - knowing more may be required to get reasonable performance.

Could you show some examples? Have you tried with scoped enums in c++11?

Not yet.

...

I've already started work on a benchmark program, although it isn't ready for prime time yet.

Could you show your test now so we can test on other platforms?

Nothing ready just yet.

...

...

* Some examples reading a relatively complex data type (endian unaware) from a file, changing a field and then writing the data.

Yes! That's the example I have in mind for the tutorial.

Have you choosen already the data type?

I've been mentally working out an example of a program that just uses built-in types for some record-oriented I/O. Then, say, a requirement arises for it to work portably. Maybe the boss bought some new servers with different native endianness. Then show how the program could be changed to use just endian integers, then just endian buffers, then endian conversion functions. And then a final version that uses a mixture of approaches. Part of the example is timings; they show the motivation and rationale for having multiple ways to handle endianness. That approach would also meet the request for a timing or benchmark program, since working versions of the example programs would be provided.

Could you show us, before the review ends, how the library will take care of conversions?

I don't really know yet! I think I'm going to develop the example application described above, then try the conversion-oriented version of it with several of the suggested approaches. If I was smarter, I might be able to figure out ahead of time which was best. But I need to see the code first.

It would be great if you had time to give some details of how you see the evolution of your library before the review ends, otherwise we will have too much things to discuss/review during the the mini review.

Here is the rough sequence: * Re-read all messages, and extract Acknowledgements and revise TODO list. * Develop the use-cases example programs, applying revised conversion functions. Iterate until happy. * Apply the integers to the example programs, and also see if using the revised conversion functions to implement some or all of the integers. * Apply mockups of the buffers to the example programs, to get a firmer idea if the buffer idea actually seems to do be worthwhile in practice. * Post work-in-progress for feedback. * Finalize and hold mini-review. Thanks for the continued interest, --Beman