"Peter Dimov" <pdimov@mmltd.net> writes:
Matthias Troyer wrote:
Indeed this sounds like a lot of work and that's why this mechanism for message passing was rarely used in the past. The hard part is to manually build up the custom MPI datatype, i.e. to inform MPI about what the offsets and types of the various data members in a struct are.
This is where the serialization library fits in and makes the task extraordinarily easy. Saving a data member with such an MPI archive will register its address, type (as well as the number of identical consecutive elements in an array) with the MPI library. Thus the serialization library does all the hard work already!
I still don't see the 10x speedup in the subject. For a X[], the two approaches are:
1. for each x in X[], "serialize" into an MPI descriptor
Do you mean "datatype?" ------------------------^^^^^^^^^^
2. serialize X[0] into an MPI descriptor, construct an array descriptor from it
Conceptual issues with (2) aside (the external format of X is determined by X itself and you have no idea whether the structure of X[0] also describes X[1]),
I don't know what you mean.
I'm not sure that there will be such a major speedup compared to the naive (1).
Why the doubt? We've measured. Furthermore, as we stated earlier, in our application we can't afford to make an in-memory copy because it would take more than the available memory.
Robert's point also deserves attention; a portable binary archive that writes directly into a socket eliminates the MPI middleman and will probably achieve a similar performance as your two-pass MPI approach.
I'll let the experts say more, but AFAICT MPI provides essential services that aren't provided by plain sockets. See http://www.open-mpi.org/papers/euro-pvmmpi-2004-overview/euro-pvmmpi-2004-ov...
It also supports versioned non-PODs and other nontrivial types.
What does? Such an archive?
As an example, I have a type which is saved as
save( x ):
save( x.anim.name() ); // std::string
and loaded as
load( x ):
string tmp; load( tmp ); x.set_animation( tmp );
Not everything is field-based.
Yes of course. Our approach supports those types too; but you can only take advantage of a subset of the optimizations for that part of your communication. -- Dave Abrahams Boost Consulting www.boost-consulting.com