On Thu, Sep 8, 2011 at 1:19 PM, Phil Endecott < spam_from_boost_dev@chezphil.org> wrote:

Dear All,

I have had a quick look at the proposed endian::big|littleN_t types. A couple of comments:

- I was expecting to find that these types would use the conversion functions that I've already looked at at the lowest level, so that optimisations made to those functions would be useful here too. But instead, these classes seem to use their own byte-shuffling code. This seems like an odd design.

The conversion functions work on aligned built-in integers. Class endian, except for the aligned specialization, works internally on unaligned char data. Thus using the conversion functions would involve conversion to or from a temporary, and a byte copy from or to the temporary. I'm wondering why you think this is worthwhile?

- I'm not convinced that these types cause the conversion to happen at the optimum time. For example:

struct file_header { big32_t n_things; .... };

int main() { file_header h; h.n_things = 0; while (....) { ++h.n_things; .... } write(h); .... }

Here the conversion is happening twice every time that n_things is incremented. It would be better to instead do the conversion once, just before the file_header is written. The same thing applies in reverse when reading from a file.

Yes.

(Surely this is the most common use-case?)

Not for the real-world applications I've worked on. In those applications, several things are different: * Most of the variables in the record are never touched. (They are, of course, touched when the file the record is in is built, but that happens once. The file is used many, where "many" may be in the millions, times but the record is only built once.) * If a variable is touched, it is only to access the value once. * A few variables do have much more complex uses, but these uses are not in performance critical code. There are certainly plenty or real-world applications where the use pattern you give above is typical. Is there are reason you couldn't use the conversion functions if you want to do a bulk reorder once after reading a record, and then again once before writing a record? Particularly assuming that the conversion functions also have templated forms that will support any reasonable built-in or UDT type.

I have been wondering if there is a better design that can avoid this. If we had some sort of struct introspection, we could store the fields in native byte order and then

template <typename T> T external_representation(const T& t) { T res(t); for each field of T { res.field.reorder(); } }

++h.n_things; // cheap write(external_representation(**h)); // conversion happens once here.

Of course we don't have introspection of structs so we can't do that. Maybe someone else has another idea.

I've always provided read and write functions that take care of any reordering needed. struct file_header { big32_t thing_1; big32_t thing_2; .... void reorder(){ thing_1 = endian::reorder(thing_1); // assume conversion.hpp uses value returning reorder() thing_2 = endian::reorder(thing_2); ... } bool read(... file) { // return false on eof file.read(...); if (file.eof()) return false; reorder(); return true; } void write(... file) { reorder(); file.write(...); } }; Is that the best we can do with today's language? I haven't worried about it because the apps I work on have only a few formats like the above, so having to add a few additional functions isn't a big deal. Thanks for the comments, --Beman