[mpi] many undefined references to serialize<boost::mpi::detail::mpi_datatype_oarchive>
Hi all - I've got a class hierarchy that can already be serialized into text or binary archives. I'm explicitly instantiating the template functions for serialize and/or load/save. I'm trying to add the capability to have boost mpi perform the serialization through send/recv/broadcast. I've added explicit instantiations for the boost::mpi::packed_iarchive and packed_oarchive. My code compiles, but at the end I get a bunch of undefined references for serialize<boost::mpi::detail::mpi_datatype_oarchive>. I can add the explicit instantiations for this oarchive, and this seems to fix my linking issue, but this seems broken, i.e. I should not have to deal with anything in a boost::detail namespace? Is there another suggestion for how to get this to work? Thanks, Brian
Brian Budge wrote:
Hi all -
I've got a class hierarchy that can already be serialized into text or binary archives. I'm explicitly instantiating the template functions for serialize and/or load/save.
I'm trying to add the capability to have boost mpi perform the serialization through send/recv/broadcast. I've added explicit instantiations for the boost::mpi::packed_iarchive and packed_oarchive. My code compiles, but at the end I get a bunch of undefined references for serialize<boost::mpi::detail::mpi_datatype_oarchive>.
I can add the explicit instantiations for this oarchive, and this seems to fix my linking issue, but this seems broken, i.e. I should not have to deal with anything in a boost::detail namespace?
I agree. Note that the main boost archive classses - text_oarchive etc. do have *.cpp files with expicit explicit instantiations. Maybe these are missing from the mpi archives. Feel free to investigate and suggest a patch. Robert Ramey
Is there another suggestion for how to get this to work?
Thanks, Brian
Hello, On Tue, Sep 11, 2012 at 12:51 AM, Brian Budge <brian.budge@gmail.com> wrote:
I've got a class hierarchy that can already be serialized into text or binary archives. I'm explicitly instantiating the template functions for serialize and/or load/save.
I'm trying to add the capability to have boost mpi perform the serialization through send/recv/broadcast.
If your class has serialization support, that's enough. You should not provide specialization for mpi::detail classes. (Of course, YMMV - since you didn't provide any code, my comment above could be completely off ...) Cheers, Riccardo
On Tue, Sep 11, 2012 at 8:50 AM, Riccardo Murri <riccardo.murri@gmail.com> wrote:
Hello,
On Tue, Sep 11, 2012 at 12:51 AM, Brian Budge <brian.budge@gmail.com> wrote:
I've got a class hierarchy that can already be serialized into text or binary archives. I'm explicitly instantiating the template functions for serialize and/or load/save.
I'm trying to add the capability to have boost mpi perform the serialization through send/recv/broadcast.
If your class has serialization support, that's enough. You should not provide specialization for mpi::detail classes.
(Of course, YMMV - since you didn't provide any code, my comment above could be completely off ...)
Cheers, Riccardo
Thanks for the replies. Here are the macros I made to perform the explicit instantiation: #define INSTANTIATE_FUNCTION_FOR_INPUT_ARCHIVES(FUNC) \ template void FUNC(boost::archive::binary_iarchive &archive, \ const unsigned int version); \ template void FUNC(boost::mpi::packed_iarchive &archive, \ const unsigned int version); \ template void FUNC(boost::archive::text_iarchive &archive, \ const unsigned int version) #define INSTANTIATE_FUNCTION_FOR_OUTPUT_ARCHIVES(FUNC) \ template void FUNC(boost::archive::binary_oarchive &archive, \ const unsigned int version); \ template void FUNC(boost::mpi::packed_oarchive &archive, \ const unsigned int version); \ template void FUNC(boost::mpi::detail::mpi_datatype_oarchive &archive, \ const unsigned int version); \ template void FUNC(boost::archive::text_oarchive &archive, \ const unsigned int version) #define MAKE_SERIALIZE_FUNCTIONS(CLASS) \ INSTANTIATE_FUNCTION_FOR_INPUT_ARCHIVES(CLASS::serialize); \ INSTANTIATE_FUNCTION_FOR_OUTPUT_ARCHIVES(CLASS::serialize) Then, along with implementing the templatized versions of serialize for each class, I will call the MAKE_SERIALIZE_FUNCTIONS(myclass) in each myclass.cpp. So to Robert's question, I don't think this is something that could be baked into the boost_mpi.so library, since they rely on my own classes. I was just surprised to find that I needed to explicitly instantiate with something from detail rather than just the packed archives. Thanks, Brian
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Hi, Is there a reason you ar doing it this way? Because your question now boils down to: "Why does it hurt, when I turn automatic template generation off?". As an user of an serializable class, I would never expect that my choice of archive would impact whether my code compiles or not. Especially since the interface states that i can (through template<class Archive>). Instead I would assume that i could safely add my own archives without checking weird macros in theird party libraries. Greetings, Oswin because it cna assume it to be there. On 2012-09-11 18:22, Brian Budge wrote:
On Tue, Sep 11, 2012 at 8:50 AM, Riccardo Murri <riccardo.murri@gmail.com> wrote:
Hello,
On Tue, Sep 11, 2012 at 12:51 AM, Brian Budge <brian.budge@gmail.com> wrote:
I've got a class hierarchy that can already be serialized into text or binary archives. I'm explicitly instantiating the template functions for serialize and/or load/save.
I'm trying to add the capability to have boost mpi perform the serialization through send/recv/broadcast.
If your class has serialization support, that's enough. You should not provide specialization for mpi::detail classes.
(Of course, YMMV - since you didn't provide any code, my comment above could be completely off ...)
Cheers, Riccardo
Thanks for the replies. Here are the macros I made to perform the explicit instantiation:
#define INSTANTIATE_FUNCTION_FOR_INPUT_ARCHIVES(FUNC) \ template void FUNC(boost::archive::binary_iarchive &archive, \ const unsigned int version); \ template void FUNC(boost::mpi::packed_iarchive &archive, \ const unsigned int version); \ template void FUNC(boost::archive::text_iarchive &archive, \ const unsigned int version)
#define INSTANTIATE_FUNCTION_FOR_OUTPUT_ARCHIVES(FUNC) \ template void FUNC(boost::archive::binary_oarchive &archive, \ const unsigned int version); \ template void FUNC(boost::mpi::packed_oarchive &archive, \ const unsigned int version); \ template void FUNC(boost::mpi::detail::mpi_datatype_oarchive &archive, \ const unsigned int version); \ template void FUNC(boost::archive::text_oarchive &archive, \ const unsigned int version)
#define MAKE_SERIALIZE_FUNCTIONS(CLASS) \ INSTANTIATE_FUNCTION_FOR_INPUT_ARCHIVES(CLASS::serialize); \ INSTANTIATE_FUNCTION_FOR_OUTPUT_ARCHIVES(CLASS::serialize)
Then, along with implementing the templatized versions of serialize for each class, I will call the MAKE_SERIALIZE_FUNCTIONS(myclass) in each myclass.cpp.
So to Robert's question, I don't think this is something that could be baked into the boost_mpi.so library, since they rely on my own classes. I was just surprised to find that I needed to explicitly instantiate with something from detail rather than just the packed archives.
Thanks, Brian
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
On Tue, Sep 11, 2012 at 10:11 AM, Oswin Krause <Oswin.Krause@ruhr-uni-bochum.de> wrote:
Hi,
Is there a reason you ar doing it this way? Because your question now boils down to: "Why does it hurt, when I turn automatic template generation off?".
As an user of an serializable class, I would never expect that my choice of archive would impact whether my code compiles or not. Especially since the interface states that i can (through template<class Archive>). Instead I would assume that i could safely add my own archives without checking weird macros in theird party libraries.
Greetings, Oswin
Hi Oswin - We are serializing an abstract hierarchy. Because we pass only the base pointer type(s) into the serialization code, we need to explicitly instantiate the functions so that the functions are available at runtime. I could be missing something, but that's the reasoning. If there's a better way, I'm definitely up for that. Thanks, Brian
Hi Brian, Does this help? http://www.boost.org/doc/libs/1_51_0/libs/serialization/doc/serialization.ht... Greetings, Oswin On 2012-09-11 19:27, Brian Budge wrote:
On Tue, Sep 11, 2012 at 10:11 AM, Oswin Krause <Oswin.Krause@ruhr-uni-bochum.de> wrote:
Hi,
Is there a reason you ar doing it this way? Because your question now boils down to: "Why does it hurt, when I turn automatic template generation off?".
As an user of an serializable class, I would never expect that my choice of archive would impact whether my code compiles or not. Especially since the interface states that i can (through template<class Archive>). Instead I would assume that i could safely add my own archives without checking weird macros in theird party libraries.
Greetings, Oswin
Hi Oswin -
We are serializing an abstract hierarchy. Because we pass only the base pointer type(s) into the serialization code, we need to explicitly instantiate the functions so that the functions are available at runtime. I could be missing something, but that's the reasoning. If there's a better way, I'm definitely up for that.
Thanks, Brian _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
On Tue, Sep 11, 2012 at 11:03 AM, Oswin Krause <Oswin.Krause@ruhr-uni-bochum.de> wrote:
Hi Brian,
Does this help?
http://www.boost.org/doc/libs/1_51_0/libs/serialization/doc/serialization.ht...
Greetings, Oswin
I think I see. I believe my co-worker did this so that only the header for the base object was required for serialization into the archive. Changing this requires all relevant headers to be included everywhere an archive is used where that type could be encountered during serialization. I will have to see if we can make this change so we can remove our dependence on the detail archive in boost::mpi. Thanks, Brian
Brian Budge wrote:
So to Robert's question, I don't think this is something that could be baked into the boost_mpi.so library, since they rely on my own classes. ...
look at .../libs/serialization/src/binary_oarchive.cpp This file contains the explict instantiations that are added to the library. I would expect that something like this could be made for mpi archives. Robert Ramey
Thanks, Brian
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
On Tue, Sep 11, 2012 at 11:29 AM, Robert Ramey <ramey@rrsd.com> wrote:
Brian Budge wrote:
So to Robert's question, I don't think this is something that could be baked into the boost_mpi.so library, since they rely on my own classes. ...
look at .../libs/serialization/src/binary_oarchive.cpp
This file contains the explict instantiations that are added to the library. I would expect that something like this could be made for mpi archives.
Robert Ramey
I think you're right Robert. I can get my types to serialize fine with the boost::binary_*archive archives, but not with the mpi ones. It's strange because I can get it to work for send, but not recv. I get an exception related to unregistered type. Indeed if I go and manually serialize every message pointer to a char buffer vis binary_oarchive, and send that via boost mpi, and manually deserialize on the other side from char buffer via binary_iarchive, my program works. I'm unsure what exactly might need to be instantiated in boost::mpi though. Thanks, Brian
Brian Budge wrote:
On Tue, Sep 11, 2012 at 11:29 AM, Robert Ramey <ramey@rrsd.com> wrote:
Brian Budge wrote:
So to Robert's question, I don't think this is something that could be baked into the boost_mpi.so library, since they rely on my own classes. ...
look at .../libs/serialization/src/binary_oarchive.cpp
This file contains the explict instantiations that are added to the library. I would expect that something like this could be made for mpi archives.
Robert Ramey
I think you're right Robert. I can get my types to serialize fine with the boost::binary_*archive archives, but not with the mpi ones. It's strange because I can get it to work for send, but not recv. I get an exception related to unregistered type.
Indeed if I go and manually serialize every message pointer to a char buffer vis binary_oarchive, and send that via boost mpi, and manually deserialize on the other side from char buffer via binary_iarchive, my program works.
I'm unsure what exactly might need to be instantiated in boost::mpi though.
just use binary_iarchive as a guide for mpi_archive.
Thanks, Brian
PLease send me some test code that shows the problem. Are you linking against the Boost.MPI libtrary and to you correctly register your derived types? Matthias On Sep 12, 2012, at 12:30 AM, Brian Budge <brian.budge@gmail.com> wrote:
On Tue, Sep 11, 2012 at 11:29 AM, Robert Ramey <ramey@rrsd.com> wrote:
Brian Budge wrote:
So to Robert's question, I don't think this is something that could be baked into the boost_mpi.so library, since they rely on my own classes. ...
look at .../libs/serialization/src/binary_oarchive.cpp
This file contains the explict instantiations that are added to the library. I would expect that something like this could be made for mpi archives.
Robert Ramey
I think you're right Robert. I can get my types to serialize fine with the boost::binary_*archive archives, but not with the mpi ones. It's strange because I can get it to work for send, but not recv. I get an exception related to unregistered type.
Indeed if I go and manually serialize every message pointer to a char buffer vis binary_oarchive, and send that via boost mpi, and manually deserialize on the other side from char buffer via binary_iarchive, my program works.
I'm unsure what exactly might need to be instantiated in boost::mpi though.
Thanks, Brian _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
participants (5)
-
Brian Budge -
Matthias Troyer -
Oswin Krause -
Riccardo Murri -
Robert Ramey