On Sep 15, 2006, at 9:12 PM, Markus Blatt wrote:
- What is your evaluation of the design?
The design is pretty straight forward. Therefore I think everybody familiar with the C-Interface of MPI will feel at home at once.
One thing that struck me at once is the missing of tags for the communication. In the MPI standard they are everywhere and usually are used to distinguish different communications (as they might get to the target process in arbitrary order). How can the programmer make sure that the message that the process is receiving is the message that he wants to receive at this point of the program? If there is no such means I would consider this a serious flaw in the design.
I don't fully understand that. There are tags in all the point-to- point communications, which is where the MPI standard supports them.
- What is your evaluation of the implementation?
One thing I do not understand is why the collective operations use std:vector to collect the data. I always thought that there is now garanty by the standard that the values are stored in contiguous memory (which is what MPI needs).
Section 23.2.4 of C++03 states "The elements of a vector are stored contiguously"
What if the user wants to comunicate from/to a given data structure represented by a standard C-Array? This might be the case very often in scientific computing where you tend to reuse existent libraries (probably implemented C or by a novice C++ programmer).
This is a good point. Doug, can we extend the interface to also allow a C-array?
As I see the implementation is using the boost serialization framework to send data types that do not correspond to a standard MPI_Datatype. While in this way it is still possible to send values between different architectures, I fear that there is (in some cases) a great efficiency loss: E. g. with none-PODs broadcasts (and probably other collective operations) do not use MPI's collective operations, but independent asynchronous MPI send and receives. This might circumvent applications of special optimization based on the underlying network topology (e. g. on a hypercube the broadcast is just O(ld P) where P is the number of processes).
This is a knock-out criterium for people doing High-Performance-Computing where efficiency is everything unless there is a way of sending at least 1-dimensional arrays the usual way. But I fear that even then what they gain by using Boost.MPI is not enough to persuade them to use it.
I believe that your comments are based on a misunderstanding. The goal of the library design was to actually make it efficient and useful for high performance applications. The library allows the use of custom MPI_Datatypes and actually creates them automatically whenever possible, using the serialization library. For those data types for which this is not possible (e.g. the pointers in a linked list) the serialization library is used to serialize the data structure, which is then packed into a message using MPI_Pack. The skeleton&content mechanism is essentially a simple way to create a custom MPI datatype. The "content" is just an MPI_Datatype created for the data members of the object you want to send. Another case is when sending, e.g. a std::vector or other array-like data structure of a type for which a custom MPI_Datatype can be constructed. In that case the serialization library is called once to construct the needed MPI_Datatype for one element in the array, and then communication is done using that MPI_Datatype. You might be worried that the use of the serialization library leads to inefficiencies since in the released versions each element of an array was serialized independently. The recent changes, which are in the CVS HEAD, actually address this issue by providing optimizations for array-like data structures. I hope these answers address the issues you had in mind. I can elaborate if you want.
Maybe it would be possible to use MPI_Pack or create custom MPI_Datatypes?
Yes, both are implemented and used automatically.
- What is your evaluation of the potential usefulness of the library?
At the current state it will probably not be used by people doing High Performance Computing due to the lack of efficiency stated above. If people do not need best performance then the library is useful to them as it is by far easier to use as MPI for a C++ programmer.
I believe, as stated above, that this is just a misunderstanding.
I don't think I am in the position to argue about that as I do not have enough background knowledge about boost, but: If you want to adress people who are into High Performance Computing, you should address the issues above before you incorporate it into boost.
This was one of the main goals, and a large fraction of the design and implementation time was spent on optimizing the array serialization so that the serialization library could be used to create MPI_Datatypes as well as to allow efficient message passing using MPI_Pack.
If you just want people, to get started with parallel programming more easily, go for it. But unless effiency is granted by Boost.MPI those people have to hand code and deal with MPI directly sooner or later to get the efficiency they need.
I hope I could help in the evaluation process.
Thank you very much for your comments! Matthias