On Sep 16, 2006, at 4:43 AM, Matthias Troyer wrote:
On Sep 16, 2006, at 9:21 AM, Geoffrey Irving wrote:
3. Send it as a datatype with the structure determined up to the total size of the message. This requires getting the size with MPI_Probe, then building a datatype, then receiving it.
The third option allows you to send a variable size vector with no extra explicit buffer. The same applies to a vector plus a constant amount of data (such as pair<int,vector<int> >). That would be quite useful, but probably difficult to work out automatically.
For std::vector and std::valarray of MPI datatypes I can see your trick being generally useful and it can be implemented as a special optimization (Doug, what do you think?). For more general types I believe that the skeleton&content mechanism is the appropriate generalization of the third option.
We can do this, although MPI_Probe() is both a problem for threading (as Geoffrey noted) and for performance (with some MPI implementations). Our best bet would be to do the same thing we do with serialized types, but without the serialization: send a small message containing the size of the vector, followed by the data itself. In a multithreaded environment, we need to send the second message with a unique tag and send that tag value as part of the first message. Again, it's the same thing we would do for serialized types. It would be so much easier if the MPI committee had come up with an MPI_Probe that actually worked :) Doug