[mpi/serialization] Potential bug or just library misunderstanding?

I'm not sure why the following test-class produces this error: "MPI_Recv:
MPI_ERR_TRUNCATE: message truncated"
If the std::map has only a single element (ie I remove one of the
test[].push_back() lines) the error does not occur. Can someone verify that
this is indeed a bug and not just novice misuse of the library?
Thanks,
Tim
#include

Hi Tim,
This is not a bug, the MPI_ERR_TRUNCATE is related to the underlying MPI
implementation and occurs when the receive buffer is lesser than the message
size. For to use the Boost.MPI skeleton and content approach you must
guarantee a fixed "shape" object. So, for example, you should not use this
approach with boost::optional<> unless you know it's state beforehand
because it's "shape" depends on it. (as a discriminated union, it could be
seen as a one element or a empty container)
In other words, Boost.MPI can deal with all Boost.Serialization types, but
not all of them are adequate to do "skeleton and content" in common tasks.
Regards,
Júlio.
2011/9/30 Tim Jacobs
I'm not sure why the following test-class produces this error: "MPI_Recv: MPI_ERR_TRUNCATE: message truncated"
If the std::map has only a single element (ie I remove one of the test[].push_back() lines) the error does not occur. Can someone verify that this is indeed a bug and not just novice misuse of the library?
Thanks, Tim
#include
#include #include #include int main( int argc, char * argv [] ) { boost::mpi::environment env(argc, argv); boost::mpi::communicator world;
if(world.rank()==0){ std::map
std::string > test; test[1].push_back("FOO"); test[50].push_back("BAR"); world.send(1, 1, boost::mpi::skeleton(test)); world.send(1, 1, boost::mpi::get_content(test)); std::cout << "sent" << std::endl; } else{ std::map std::string > test; world.recv(0,1,boost::mpi::skeleton(test)); world.recv(0,1,boost::mpi::get_content(test)); } } _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

Thanks for the reply Julio. I thought that the purpose of using the
skeleton/content approach was precisely to deal with situations such as
this. The object (a map in this case) I am trying to send is fully populated
when the skeleton is sent and thus it's "shape" does not change and the
buffer should be of the appropriate size. If this is not the case then how
does one go about sending a std::map at all? Also, I'm failing to see how
this situation differs from sending a std::vector, which works fine.
Thanks again,
Tim
2011/9/30 Júlio Hoffimann
Hi Tim,
This is not a bug, the MPI_ERR_TRUNCATE is related to the underlying MPI implementation and occurs when the receive buffer is lesser than the message size. For to use the Boost.MPI skeleton and content approach you must guarantee a fixed "shape" object. So, for example, you should not use this approach with boost::optional<> unless you know it's state beforehand because it's "shape" depends on it. (as a discriminated union, it could be seen as a one element or a empty container)
In other words, Boost.MPI can deal with all Boost.Serialization types, but not all of them are adequate to do "skeleton and content" in common tasks.
Regards, Júlio.
2011/9/30 Tim Jacobs
I'm not sure why the following test-class produces this error: "MPI_Recv: MPI_ERR_TRUNCATE: message truncated"
If the std::map has only a single element (ie I remove one of the test[].push_back() lines) the error does not occur. Can someone verify that this is indeed a bug and not just novice misuse of the library?
Thanks, Tim
#include
#include #include #include int main( int argc, char * argv [] ) { boost::mpi::environment env(argc, argv); boost::mpi::communicator world;
if(world.rank()==0){ std::map
std::string > test; test[1].push_back("FOO"); test[50].push_back("BAR"); world.send(1, 1, boost::mpi::skeleton(test)); world.send(1, 1, boost::mpi::get_content(test)); std::cout << "sent" << std::endl; } else{ std::map std::string > test; world.recv(0,1,boost::mpi::skeleton(test)); world.recv(0,1,boost::mpi::get_content(test)); } } _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

Hi Tim,
I don't know, maybe is the way stdlib containers requests memory? As far as
i know, push_back() member functions sometimes allocates more than the
strictly necessary amount to avoid doing this many times for each element
inserted. Maybe the constructors in a nested stdlib type like
std::map
Thanks for the reply Julio. I thought that the purpose of using the skeleton/content approach was precisely to deal with situations such as this. The object (a map in this case) I am trying to send is fully populated when the skeleton is sent and thus it's "shape" does not change and the buffer should be of the appropriate size. If this is not the case then how does one go about sending a std::map at all? Also, I'm failing to see how this situation differs from sending a std::vector, which works fine.
Thanks again, Tim
2011/9/30 Júlio Hoffimann
Hi Tim,
This is not a bug, the MPI_ERR_TRUNCATE is related to the underlying MPI implementation and occurs when the receive buffer is lesser than the message size. For to use the Boost.MPI skeleton and content approach you must guarantee a fixed "shape" object. So, for example, you should not use this approach with boost::optional<> unless you know it's state beforehand because it's "shape" depends on it. (as a discriminated union, it could be seen as a one element or a empty container)
In other words, Boost.MPI can deal with all Boost.Serialization types, but not all of them are adequate to do "skeleton and content" in common tasks.
Regards, Júlio.
2011/9/30 Tim Jacobs
I'm not sure why the following test-class produces this error: "MPI_Recv: MPI_ERR_TRUNCATE: message truncated"
If the std::map has only a single element (ie I remove one of the test[].push_back() lines) the error does not occur. Can someone verify that this is indeed a bug and not just novice misuse of the library?
Thanks, Tim
#include
#include #include #include int main( int argc, char * argv [] ) { boost::mpi::environment env(argc, argv); boost::mpi::communicator world;
if(world.rank()==0){ std::map
std::string > test; test[1].push_back("FOO"); test[50].push_back("BAR"); world.send(1, 1, boost::mpi::skeleton(test)); world.send(1, 1, boost::mpi::get_content(test)); std::cout << "sent" << std::endl; } else{ std::map std::string > test; world.recv(0,1,boost::mpi::skeleton(test)); world.recv(0,1,boost::mpi::get_content(test)); } } _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

On Sep 30, 2011, at 5:18 PM, Tim Jacobs wrote:
I'm not sure why the following test-class produces this error: "MPI_Recv: MPI_ERR_TRUNCATE: message truncated"
If the std::map has only a single element (ie I remove one of the test[].push_back() lines) the error does not occur. Can someone verify that this is indeed a bug and not just novice misuse of the library?
Thanks, Tim
#include
#include #include #include int main( int argc, char * argv [] ) { boost::mpi::environment env(argc, argv); boost::mpi::communicator world;
if(world.rank()==0){ std::map
std::string > test; test[1].push_back("FOO"); test[50].push_back("BAR"); world.send(1, 1, boost::mpi::skeleton(test)); world.send(1, 1, boost::mpi::get_content(test)); std::cout << "sent" << std::endl; } else{ std::map std::string > test; world.recv(0,1,boost::mpi::skeleton(test)); world.recv(0,1,boost::mpi::get_content(test)); } }
The problem is that a std::map cannot be sent using the skeleton&content approach, since the keys need to be known to build a map. Matthias

on Wed Oct 19 2011, Matthias Troyer
On Sep 30, 2011, at 5:18 PM, Tim Jacobs wrote:
I'm not sure why the following test-class produces this error: "MPI_Recv: MPI_ERR_TRUNCATE: message truncated"
If the std::map has only a single element (ie I remove one of the test[].push_back() lines) the error does not occur. Can someone verify that this is indeed a bug and not just novice misuse of the library?
Thanks, Tim
#include
#include #include #include int main( int argc, char * argv [] ) { boost::mpi::environment env(argc, argv); boost::mpi::communicator world;
if(world.rank()==0){ std::map
std::string > test; test[1].push_back("FOO"); test[50].push_back("BAR"); world.send(1, 1, boost::mpi::skeleton(test)); world.send(1, 1, boost::mpi::get_content(test)); std::cout << "sent" << std::endl; } else{ std::map std::string > test; world.recv(0,1,boost::mpi::skeleton(test)); world.recv(0,1,boost::mpi::get_content(test)); } } The problem is that a std::map cannot be sent using the skeleton&content approach, since the keys need to be known to build a map.
I think, at least in principle, it should be possible to send a map that way *if* the keys never change... right? -- Dave Abrahams BoostPro Computing http://www.boostpro.com

On Oct 20, 2011, at 4:36 PM, Dave Abrahams wrote:
on Wed Oct 19 2011, Matthias Troyer
wrote: On Sep 30, 2011, at 5:18 PM, Tim Jacobs wrote:
I'm not sure why the following test-class produces this error: "MPI_Recv: MPI_ERR_TRUNCATE: message truncated"
If the std::map has only a single element (ie I remove one of the test[].push_back() lines) the error does not occur. Can someone verify that this is indeed a bug and not just novice misuse of the library?
Thanks, Tim
#include
#include #include #include int main( int argc, char * argv [] ) { boost::mpi::environment env(argc, argv); boost::mpi::communicator world;
if(world.rank()==0){ std::map
std::string > test; test[1].push_back("FOO"); test[50].push_back("BAR"); world.send(1, 1, boost::mpi::skeleton(test)); world.send(1, 1, boost::mpi::get_content(test)); std::cout << "sent" << std::endl; } else{ std::map std::string > test; world.recv(0,1,boost::mpi::skeleton(test)); world.recv(0,1,boost::mpi::get_content(test)); } } The problem is that a std::map cannot be sent using the skeleton&content approach, since the keys need to be known to build a map.
I think, at least in principle, it should be possible to send a map that way *if* the keys never change... right?
Yes, in principle such a support could be added, but it would need a special serializition function for std::map Matthias
participants (4)
-
Dave Abrahams
-
Júlio Hoffimann
-
Matthias Troyer
-
Tim Jacobs