----- Original Message ----- From: "Jonathan Franklin" <franklin.jonathan@gmail.com> Newsgroups: gmane.comp.lib.boost.devel To: <boost@lists.boost.org> Sent: Tuesday, June 01, 2010 4:41 PM Subject: Re: [boost::endian] Request for comments/interest Sorry for my late replies. I have been offline for a bit. On Fri, May 28, 2010 at 2:27 PM, Terry Golubiewski <tjgolubi@netins.net> wrote:
Since IP packets cannot be 10GB, I submit that you're going to have to break your 10GB array down into messages. Then you're going to copy portions of the 10GB array into those messages and send them. In the type-base approach the message may indeed contain an array.
As Tomas already pointed out, explicitly breaking a blob of data into chunks for network transfer is superfluous in streaming protocols such as TCP. The protocol will take care of this more efficiently at the network and/or transport layer than I can at the application layer. Further, if I break my data blob into k segments and insert my own message header as a preamble to each segment, then I unnecessarily add k*sizeof(my_header) to the stream.
In the message-based interfaces that I am used to, one always must copy some data structures into a message before you send it.
As Tomas also pointed out, this is not a marshaling library. However, a marshaling library could make good use of an "endian" library such as this. In addition to my network needs, I also have at least two image file formats that have the option of storing the arbitrarily sized data on disk in big- or little-endian format. A library that allows me to slurp up the image data into memory and swap in place if needed is terribly useful to me. Jon In another post, I finally understood this use-case. Again (being the non-swap advocate) to address this, I would suggest... template<endian_t E1, class T, endian_t E2> endian<E2, T>* copy(const endian<E1, T>* first, const endian<E1, T>* last, endian<E2, T>* dest) { if (E1 == E2 && first == dest) return dest + distance(first, last); while (first != last) *dest++ = *first++; return dest; } Assuming you just read a TCP stream of big-endian T's into memory at a location 'char* buf', then you could endian-convert the T's, if necessary, like this... const endian<big, T>* src = reinterpret_cast<const endian<big, T>*>(buf); endian<native, T>* dst = reinterpret_cast<endian<native, T>*>(buf); (void) copy(src, src+numTs, dst); Would this do what you want? terry