[mpi/serialization] Potential bug or just library misunderstanding?
I'm not sure why the following test-class produces this error: "MPI_Recv: MPI_ERR_TRUNCATE: message truncated" If the std::map has only a single element (ie I remove one of the test[].push_back() lines) the error does not occur. Can someone verify that this is indeed a bug and not just novice misuse of the library? Thanks, Tim #include <boost/mpi.hpp> #include <boost/serialization/string.hpp> #include <boost/serialization/vector.hpp> #include <boost/serialization/map.hpp> int main( int argc, char * argv [] ) { boost::mpi::environment env(argc, argv); boost::mpi::communicator world; if(world.rank()==0){ std::map<int, std::vector<std::string> > test; test[1].push_back("FOO"); test[50].push_back("BAR"); world.send(1, 1, boost::mpi::skeleton(test)); world.send(1, 1, boost::mpi::get_content(test)); std::cout << "sent" << std::endl; } else{ std::map<int, std::vector<std::string> > test; world.recv(0,1,boost::mpi::skeleton(test)); world.recv(0,1,boost::mpi::get_content(test)); } }
Hi Tim, This is not a bug, the MPI_ERR_TRUNCATE is related to the underlying MPI implementation and occurs when the receive buffer is lesser than the message size. For to use the Boost.MPI skeleton and content approach you must guarantee a fixed "shape" object. So, for example, you should not use this approach with boost::optional<> unless you know it's state beforehand because it's "shape" depends on it. (as a discriminated union, it could be seen as a one element or a empty container) In other words, Boost.MPI can deal with all Boost.Serialization types, but not all of them are adequate to do "skeleton and content" in common tasks. Regards, Júlio. 2011/9/30 Tim Jacobs <tjacobs2@email.unc.edu>
I'm not sure why the following test-class produces this error: "MPI_Recv: MPI_ERR_TRUNCATE: message truncated"
If the std::map has only a single element (ie I remove one of the test[].push_back() lines) the error does not occur. Can someone verify that this is indeed a bug and not just novice misuse of the library?
Thanks, Tim
#include <boost/mpi.hpp> #include <boost/serialization/string.hpp> #include <boost/serialization/vector.hpp> #include <boost/serialization/map.hpp>
int main( int argc, char * argv [] ) { boost::mpi::environment env(argc, argv); boost::mpi::communicator world;
if(world.rank()==0){ std::map<int, std::vector<std::string> > test; test[1].push_back("FOO"); test[50].push_back("BAR"); world.send(1, 1, boost::mpi::skeleton(test)); world.send(1, 1, boost::mpi::get_content(test)); std::cout << "sent" << std::endl; } else{ std::map<int, std::vector<std::string> > test; world.recv(0,1,boost::mpi::skeleton(test)); world.recv(0,1,boost::mpi::get_content(test)); } }
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Thanks for the reply Julio. I thought that the purpose of using the skeleton/content approach was precisely to deal with situations such as this. The object (a map in this case) I am trying to send is fully populated when the skeleton is sent and thus it's "shape" does not change and the buffer should be of the appropriate size. If this is not the case then how does one go about sending a std::map at all? Also, I'm failing to see how this situation differs from sending a std::vector, which works fine. Thanks again, Tim 2011/9/30 Júlio Hoffimann <julio.hoffimann@gmail.com>
Hi Tim,
This is not a bug, the MPI_ERR_TRUNCATE is related to the underlying MPI implementation and occurs when the receive buffer is lesser than the message size. For to use the Boost.MPI skeleton and content approach you must guarantee a fixed "shape" object. So, for example, you should not use this approach with boost::optional<> unless you know it's state beforehand because it's "shape" depends on it. (as a discriminated union, it could be seen as a one element or a empty container)
In other words, Boost.MPI can deal with all Boost.Serialization types, but not all of them are adequate to do "skeleton and content" in common tasks.
Regards, Júlio.
2011/9/30 Tim Jacobs <tjacobs2@email.unc.edu>
I'm not sure why the following test-class produces this error: "MPI_Recv: MPI_ERR_TRUNCATE: message truncated"
If the std::map has only a single element (ie I remove one of the test[].push_back() lines) the error does not occur. Can someone verify that this is indeed a bug and not just novice misuse of the library?
Thanks, Tim
#include <boost/mpi.hpp> #include <boost/serialization/string.hpp> #include <boost/serialization/vector.hpp> #include <boost/serialization/map.hpp>
int main( int argc, char * argv [] ) { boost::mpi::environment env(argc, argv); boost::mpi::communicator world;
if(world.rank()==0){ std::map<int, std::vector<std::string> > test; test[1].push_back("FOO"); test[50].push_back("BAR"); world.send(1, 1, boost::mpi::skeleton(test)); world.send(1, 1, boost::mpi::get_content(test)); std::cout << "sent" << std::endl; } else{ std::map<int, std::vector<std::string> > test; world.recv(0,1,boost::mpi::skeleton(test)); world.recv(0,1,boost::mpi::get_content(test)); } }
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Hi Tim, I don't know, maybe is the way stdlib containers requests memory? As far as i know, push_back() member functions sometimes allocates more than the strictly necessary amount to avoid doing this many times for each element inserted. Maybe the constructors in a nested stdlib type like std::map<int,std::vector<> > are doing something that compromises the approach? As a user of the library, i never get into the details. Would be great if some maintainer of Boost.MPI could explain what is happening. I never see any of them answering questions on this mailing list tough. Yes, this is sad. :-( Good luck, Júlio. 2011/9/30 Tim Jacobs <tjacobs2@email.unc.edu>
Thanks for the reply Julio. I thought that the purpose of using the skeleton/content approach was precisely to deal with situations such as this. The object (a map in this case) I am trying to send is fully populated when the skeleton is sent and thus it's "shape" does not change and the buffer should be of the appropriate size. If this is not the case then how does one go about sending a std::map at all? Also, I'm failing to see how this situation differs from sending a std::vector, which works fine.
Thanks again, Tim
2011/9/30 Júlio Hoffimann <julio.hoffimann@gmail.com>
Hi Tim,
This is not a bug, the MPI_ERR_TRUNCATE is related to the underlying MPI implementation and occurs when the receive buffer is lesser than the message size. For to use the Boost.MPI skeleton and content approach you must guarantee a fixed "shape" object. So, for example, you should not use this approach with boost::optional<> unless you know it's state beforehand because it's "shape" depends on it. (as a discriminated union, it could be seen as a one element or a empty container)
In other words, Boost.MPI can deal with all Boost.Serialization types, but not all of them are adequate to do "skeleton and content" in common tasks.
Regards, Júlio.
2011/9/30 Tim Jacobs <tjacobs2@email.unc.edu>
I'm not sure why the following test-class produces this error: "MPI_Recv: MPI_ERR_TRUNCATE: message truncated"
If the std::map has only a single element (ie I remove one of the test[].push_back() lines) the error does not occur. Can someone verify that this is indeed a bug and not just novice misuse of the library?
Thanks, Tim
#include <boost/mpi.hpp> #include <boost/serialization/string.hpp> #include <boost/serialization/vector.hpp> #include <boost/serialization/map.hpp>
int main( int argc, char * argv [] ) { boost::mpi::environment env(argc, argv); boost::mpi::communicator world;
if(world.rank()==0){ std::map<int, std::vector<std::string> > test; test[1].push_back("FOO"); test[50].push_back("BAR"); world.send(1, 1, boost::mpi::skeleton(test)); world.send(1, 1, boost::mpi::get_content(test)); std::cout << "sent" << std::endl; } else{ std::map<int, std::vector<std::string> > test; world.recv(0,1,boost::mpi::skeleton(test)); world.recv(0,1,boost::mpi::get_content(test)); } }
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
On Sep 30, 2011, at 5:18 PM, Tim Jacobs wrote:
I'm not sure why the following test-class produces this error: "MPI_Recv: MPI_ERR_TRUNCATE: message truncated"
If the std::map has only a single element (ie I remove one of the test[].push_back() lines) the error does not occur. Can someone verify that this is indeed a bug and not just novice misuse of the library?
Thanks, Tim
#include <boost/mpi.hpp> #include <boost/serialization/string.hpp> #include <boost/serialization/vector.hpp> #include <boost/serialization/map.hpp>
int main( int argc, char * argv [] ) { boost::mpi::environment env(argc, argv); boost::mpi::communicator world;
if(world.rank()==0){ std::map<int, std::vector<std::string> > test; test[1].push_back("FOO"); test[50].push_back("BAR"); world.send(1, 1, boost::mpi::skeleton(test)); world.send(1, 1, boost::mpi::get_content(test)); std::cout << "sent" << std::endl; } else{ std::map<int, std::vector<std::string> > test; world.recv(0,1,boost::mpi::skeleton(test)); world.recv(0,1,boost::mpi::get_content(test)); } }
The problem is that a std::map cannot be sent using the skeleton&content approach, since the keys need to be known to build a map. Matthias
on Wed Oct 19 2011, Matthias Troyer <troyer-AT-phys.ethz.ch> wrote:
On Sep 30, 2011, at 5:18 PM, Tim Jacobs wrote:
I'm not sure why the following test-class produces this error: "MPI_Recv: MPI_ERR_TRUNCATE: message truncated"
If the std::map has only a single element (ie I remove one of the test[].push_back() lines) the error does not occur. Can someone verify that this is indeed a bug and not just novice misuse of the library?
Thanks, Tim
#include <boost/mpi.hpp> #include <boost/serialization/string.hpp> #include <boost/serialization/vector.hpp> #include <boost/serialization/map.hpp>
int main( int argc, char * argv [] ) { boost::mpi::environment env(argc, argv); boost::mpi::communicator world;
if(world.rank()==0){ std::map<int, std::vector<std::string> > test; test[1].push_back("FOO"); test[50].push_back("BAR"); world.send(1, 1, boost::mpi::skeleton(test)); world.send(1, 1, boost::mpi::get_content(test)); std::cout << "sent" << std::endl; } else{ std::map<int, std::vector<std::string> > test; world.recv(0,1,boost::mpi::skeleton(test)); world.recv(0,1,boost::mpi::get_content(test)); } }
The problem is that a std::map cannot be sent using the skeleton&content approach, since the keys need to be known to build a map.
I think, at least in principle, it should be possible to send a map that way *if* the keys never change... right? -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On Oct 20, 2011, at 4:36 PM, Dave Abrahams wrote:
on Wed Oct 19 2011, Matthias Troyer <troyer-AT-phys.ethz.ch> wrote:
On Sep 30, 2011, at 5:18 PM, Tim Jacobs wrote:
I'm not sure why the following test-class produces this error: "MPI_Recv: MPI_ERR_TRUNCATE: message truncated"
If the std::map has only a single element (ie I remove one of the test[].push_back() lines) the error does not occur. Can someone verify that this is indeed a bug and not just novice misuse of the library?
Thanks, Tim
#include <boost/mpi.hpp> #include <boost/serialization/string.hpp> #include <boost/serialization/vector.hpp> #include <boost/serialization/map.hpp>
int main( int argc, char * argv [] ) { boost::mpi::environment env(argc, argv); boost::mpi::communicator world;
if(world.rank()==0){ std::map<int, std::vector<std::string> > test; test[1].push_back("FOO"); test[50].push_back("BAR"); world.send(1, 1, boost::mpi::skeleton(test)); world.send(1, 1, boost::mpi::get_content(test)); std::cout << "sent" << std::endl; } else{ std::map<int, std::vector<std::string> > test; world.recv(0,1,boost::mpi::skeleton(test)); world.recv(0,1,boost::mpi::get_content(test)); } }
The problem is that a std::map cannot be sent using the skeleton&content approach, since the keys need to be known to build a map.
I think, at least in principle, it should be possible to send a map that way *if* the keys never change... right?
Yes, in principle such a support could be added, but it would need a special serializition function for std::map Matthias
participants (4)
-
Dave Abrahams -
Júlio Hoffimann -
Matthias Troyer -
Tim Jacobs