[mpi] Boost.MPI broken
Almost all of the regression tests for Boost.MPI on the Boost trunk are failing, and this is leading to failures in Boost.Graph.Parallel as well. Is this a known problem? Could someone please work on fixing it? Thank you. -- Jeremiah Willcock
At Tue, 22 Jun 2010 15:34:13 -0400 (EDT), Jeremiah Willcock wrote:
Almost all of the regression tests for Boost.MPI on the Boost trunk are failing, and this is leading to failures in Boost.Graph.Parallel as well. Is this a known problem? Could someone please work on fixing it? Thank you.
It would help if you could go one or two steps further in diagnosing the issue so we know whether it's a Boost.Build problem, a Boost.Test problem, or something else. I.e. who should be paying attention? -- Dave Abrahams BoostPro Computing http://www.boostpro.com
AMDG David Abrahams wrote:
At Tue, 22 Jun 2010 15:34:13 -0400 (EDT), Jeremiah Willcock wrote:
Almost all of the regression tests for Boost.MPI on the Boost trunk are failing, and this is leading to failures in Boost.Graph.Parallel as well. Is this a known problem? Could someone please work on fixing it? Thank you.
It would help if you could go one or two steps further in diagnosing the issue so we know whether it's a Boost.Build problem, a Boost.Test problem, or something else. I.e. who should be paying attention?
It looks like it was broken by a serialization change: ../boost/mpi/datatype.hpp:184: error: no matching function for call to 'assertion_failed(mpl_::failed************ boost::mpi::is_mpi_datatype<boost::archive::version_type>::************)' In Christ, Steven Watanabe
At Tue, 22 Jun 2010 12:49:50 -0700, Steven Watanabe wrote:
David Abrahams wrote:
At Tue, 22 Jun 2010 15:34:13 -0400 (EDT), Jeremiah Willcock wrote:
Almost all of the regression tests for Boost.MPI on the Boost trunk are failing, and this is leading to failures in Boost.Graph.Parallel as well. Is this a known problem? Could someone please work on fixing it? Thank you.
It would help if you could go one or two steps further in diagnosing the issue so we know whether it's a Boost.Build problem, a Boost.Test problem, or something else. I.e. who should be paying attention?
It looks like it was broken by a serialization change:
../boost/mpi/datatype.hpp:184: error: no matching function for call to 'assertion_failed(mpl_::failed************ boost::mpi::is_mpi_datatype<boost::archive::version_type>::************)'
Looks like a job for the Boost.MPI maintainer, whoever that may be. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
Steven Watanabe wrote:
AMDG
David Abrahams wrote:
At Tue, 22 Jun 2010 15:34:13 -0400 (EDT), Jeremiah Willcock wrote:
Almost all of the regression tests for Boost.MPI on the Boost trunk are failing, and this is leading to failures in Boost.Graph.Parallel as well. Is this a known problem? Could someone please work on fixing it? Thank you.
It would help if you could go one or two steps further in diagnosing the issue so we know whether it's a Boost.Build problem, a Boost.Test problem, or something else. I.e. who should be paying attention?
It looks like it was broken by a serialization change:
../boost/mpi/datatype.hpp:184: error: no matching function for call to 'assertion_failed(mpl_::failed************ boost::mpi::is_mpi_datatype<boost::archive::version_type>::************)'
I suspected as much. But I didn't know where to look. I suspect ../boost/mpi/datatype.hpp: needs some code like: /// specialization of is_mpi_datatype serialization types template <> struct is_mpi_datatype<boost::archive::version_type > : public boost::mpl::true_ {}; template <> struct is_mpi_datatype<boost::serialization::item_version_type > : public boost::mpl::true_ {}; template <> struct is_mpi_datatype<boost::serialization::collection_size_type > : public boost::mpl::true_ {}; I would prefer that someone who knows what they're doing check this and make the changes. Robert Ramey
In Christ, Steven Watanabe
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On 22/06/2010 20:46, David Abrahams wrote:
At Tue, 22 Jun 2010 15:34:13 -0400 (EDT), Jeremiah Willcock wrote:
Almost all of the regression tests for Boost.MPI on the Boost trunk are failing, and this is leading to failures in Boost.Graph.Parallel as well. Is this a known problem? Could someone please work on fixing it? Thank you.
It would help if you could go one or two steps further in diagnosing the issue so we know whether it's a Boost.Build problem, a Boost.Test problem, or something else. I.e. who should be paying attention?
Well, it's changes made somewhere around the start of the bug sprint that broke it. I'm sure someone actually mentioned it back then as well.... KTC
On Wed, 23 Jun 2010, KTC wrote:
On 22/06/2010 20:46, David Abrahams wrote:
At Tue, 22 Jun 2010 15:34:13 -0400 (EDT), Jeremiah Willcock wrote:
Almost all of the regression tests for Boost.MPI on the Boost trunk are failing, and this is leading to failures in Boost.Graph.Parallel as well. Is this a known problem? Could someone please work on fixing it? Thank you.
It would help if you could go one or two steps further in diagnosing the issue so we know whether it's a Boost.Build problem, a Boost.Test problem, or something else. I.e. who should be paying attention?
Well, it's changes made somewhere around the start of the bug sprint that broke it. I'm sure someone actually mentioned it back then as well....
It is still broken in the regression tests as of July 1. Is there someone I should email directly to ask about having it fixed? -- Jeremiah Willcock
On Jul 1, 2010, at 7:08 PM, Jeremiah Willcock wrote:
On Wed, 23 Jun 2010, KTC wrote:
On 22/06/2010 20:46, David Abrahams wrote:
At Tue, 22 Jun 2010 15:34:13 -0400 (EDT), Jeremiah Willcock wrote:
Almost all of the regression tests for Boost.MPI on the Boost trunk are failing, and this is leading to failures in Boost.Graph.Parallel as well. Is this a known problem? Could someone please work on fixing it? Thank you.
It would help if you could go one or two steps further in diagnosing the issue so we know whether it's a Boost.Build problem, a Boost.Test problem, or something else. I.e. who should be paying attention?
Well, it's changes made somewhere around the start of the bug sprint that broke it. I'm sure someone actually mentioned it back then as well....
It is still broken in the regression tests as of July 1. Is there someone I should email directly to ask about having it fixed?
Can we invert the dependencies and make Serialization depend on MPI, instead of how it's currently structured? -- Noel
On Jul 1, 2010, at 9:08 PM, Jeremiah Willcock wrote:
On Wed, 23 Jun 2010, KTC wrote:
On 22/06/2010 20:46, David Abrahams wrote:
Almost all of the regression tests for Boost.MPI on the Boost trunk are failing, and this is leading to failures in Boost.Graph.Parallel as well. Is this a known problem? Could someone please work on fixing it? Thank you. It would help if you could go one or two steps further in diagnosing
At Tue, 22 Jun 2010 15:34:13 -0400 (EDT), Jeremiah Willcock wrote: the issue so we know whether it's a Boost.Build problem, a Boost.Test problem, or something else. I.e. who should be paying attention?
Well, it's changes made somewhere around the start of the bug sprint that broke it. I'm sure someone actually mentioned it back then as well....
It is still broken in the regression tests as of July 1. Is there someone I should email directly to ask about having it fixed?
Sorry, I can't help myself this time: http://tinyurl.com/2as875g -- David Abrahams BoostPro Computing http://boostpro.com
Jeremiah Willcock wrote:
It is still broken in the regression tests as of July 1. Is there someone I should email directly to ask about having it fixed?
Looking at the boost users list, I would have guessed that Matthias Troyer is the one that does most of the maintenance for Boost.MPI. However, according to the link Dave send, Doug Gregor is the maintainer. So I took a look at the svn logs, and came to the conclusion that both do some maintenance for Boost.MPI. I think neither Doug nor Matthias will be enthusiastic when they are requested to find and fix something that somebody else broke, but perhaps it's better than having Boost.MPI broken with nobody fixing it. Regards, Thomas
Thomas Klimpel wrote:
Jeremiah Willcock wrote:
It is still broken in the regression tests as of July 1. Is there someone I should email directly to ask about having it fixed?
Looking at the boost users list, I would have guessed that Matthias Troyer is the one that does most of the maintenance for Boost.MPI. However, according to the link Dave send, Doug Gregor is the maintainer. So I took a look at the svn logs, and came to the conclusion that both do some maintenance for Boost.MPI.
I think neither Doug nor Matthias will be enthusiastic when they are requested to find and fix something that somebody else broke,
I think you're drawing a conclusion here without the requisite understanding of the issues. I think it would likely be better characterised a MPI depending on an implementation detail of the serialization library which had to be changed to address some other problem.
but perhaps it's better than having Boost.MPI broken with nobody fixing it.
Matthias has agreed to look at it.
Regards, Thomas _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On 2 Jul 2010, at 07:03, Robert Ramey wrote:
Thomas Klimpel wrote:
Jeremiah Willcock wrote:
It is still broken in the regression tests as of July 1. Is there someone I should email directly to ask about having it fixed?
Looking at the boost users list, I would have guessed that Matthias Troyer is the one that does most of the maintenance for Boost.MPI. However, according to the link Dave send, Doug Gregor is the maintainer. So I took a look at the svn logs, and came to the conclusion that both do some maintenance for Boost.MPI.
Officially Doug has been the maintainer, but he is too busy with clang at the moment so that I took over most maintenance.
I think neither Doug nor Matthias will be enthusiastic when they are requested to find and fix something that somebody else broke,
I think you're drawing a conclusion here without the requisite understanding of the issues. I think it would likely be better characterised a MPI depending on an implementation detail of the serialization library which had to be changed to address some other problem.
I disagree. I have taken a first look and it seems that this is caused by a breaking change in the serialization library. Until now an archive was supposed to handle only "primitive" types, which were all fundamental integer and floating point types and std::string. In the new version in addition types like boost::archive::version_type, tracking_type, .... all have to be handled as primitive types by the archive. In the past those types had a serialize function that took care of serialization. The list of primitive types that an archive has to support is, in my opinion, not an "implementation detail of the serialization library which had to be changed to address some other problem" but part of the public archive concept.
but perhaps it's better than having Boost.MPI broken with nobody fixing it.
Matthias has agreed to look at it.
My conclusion from a look at it is above. I'm in touch with Robert and hope that he can again use the serialize() function to serialize those types, as otherwise we'll have to support a larger bunch of primitive types in all archives. Matthias
Hi, The attached patch fixes the compilation problem on MacOS X - could someone please check whether this also works on other platforms? The fundamental problem is the definition of the set of "primitive" types that any archive class needs to support to be compatible with Boost.Serialization. In the past those had been the fundamental builtin types and std::string. This has now changed to include a list of 10 additional types that were previously serialized using their serialize() function. This change in the list of primitive types that every archive needs to support has been a serious breaking change and will most likely also impact other archives. Robert, it would be good if you could add an explicit list of all primitive types that every archive needs to support to the documentation and archive concept, and announce any upcoming changes? Ideally though it would be great if those types could get their serialize() function back so that they do not need to be treated ad primitive types. Matthias Index: boost/mpi/datatype.hpp =================================================================== --- boost/mpi/datatype.hpp (revision 63522) +++ boost/mpi/datatype.hpp (working copy) @@ -26,6 +26,8 @@ #include <boost/mpl/and.hpp> #include <boost/mpi/detail/mpi_datatype_cache.hpp> #include <boost/mpl/assert.hpp> +#include <boost/archive/basic_archive.hpp> +#include <boost/serialization/item_version_type.hpp> #include <utility> // for std::pair namespace boost { namespace mpi { @@ -298,6 +300,7 @@ BOOST_MPI_DATATYPE(signed char, MPI_SIGNED_CHAR, builtin); #endif + #endif // Doxygen namespace detail { @@ -325,6 +328,23 @@ : boost::mpl::bool_<true> {}; + +#ifndef BOOST_MPI_DOXYGEN +// direct support for special primitive data types of the serialization library +BOOST_MPI_DATATYPE(boost::archive::library_version_type, get_mpi_datatype(uint_least16_t()), integer); +BOOST_MPI_DATATYPE(boost::archive::version_type, get_mpi_datatype(uint_least8_t()), integer); +BOOST_MPI_DATATYPE(boost::archive::class_id_type, get_mpi_datatype(int_least16_t()), integer); +BOOST_MPI_DATATYPE(boost::archive::class_id_reference_type, get_mpi_datatype(int_least16_t()), integer); +BOOST_MPI_DATATYPE(boost::archive::class_id_optional_type, get_mpi_datatype(int_least16_t()), integer); +BOOST_MPI_DATATYPE(boost::archive::object_id_type, get_mpi_datatype(uint_least32_t()), integer); +BOOST_MPI_DATATYPE(boost::archive::object_reference_type, get_mpi_datatype(uint_least32_t()), integer); +BOOST_MPI_DATATYPE(boost::archive::tracking_type, get_mpi_datatype(bool()), builtin); +BOOST_MPI_DATATYPE(boost::serialization::collection_size_type, get_mpi_datatype(std::size_t()), builtin); +BOOST_MPI_DATATYPE(boost::serialization::item_version_type, get_mpi_datatype(uint_least8_t()), builtin); + +#endif // Doxygen + + } } // end namespace boost::mpi // define a macro to make explicit designation of this more transparent Index: boost/mpi/detail/ignore_skeleton_oarchive.hpp =================================================================== --- boost/mpi/detail/ignore_skeleton_oarchive.hpp (revision 63522) +++ boost/mpi/detail/ignore_skeleton_oarchive.hpp (working copy) @@ -17,6 +17,7 @@ #include <boost/archive/detail/oserializer.hpp> #include <boost/serialization/collection_size_type.hpp> #include <boost/serialization/array.hpp> +#include <boost/serialization/item_version_type.hpp> namespace boost { namespace mpi { namespace detail { @@ -53,6 +54,7 @@ BOOST_ARCHIVE_IGNORE_IMPLEMENTATION(archive::class_id_optional_type) BOOST_ARCHIVE_IGNORE_IMPLEMENTATION(archive::version_type) +BOOST_ARCHIVE_IGNORE_IMPLEMENTATION(archive::library_version_type) BOOST_ARCHIVE_IGNORE_IMPLEMENTATION(archive::class_id_type) BOOST_ARCHIVE_IGNORE_IMPLEMENTATION(archive::class_id_reference_type) BOOST_ARCHIVE_IGNORE_IMPLEMENTATION(archive::object_id_type) @@ -60,6 +62,7 @@ BOOST_ARCHIVE_IGNORE_IMPLEMENTATION(archive::tracking_type) BOOST_ARCHIVE_IGNORE_IMPLEMENTATION(archive::class_name_type) BOOST_ARCHIVE_IGNORE_IMPLEMENTATION(serialization::collection_size_type) +BOOST_ARCHIVE_IGNORE_IMPLEMENTATION(serialization::item_version_type) void save_override(std::string const & s, int) { Index: boost/mpi/detail/ignore_iprimitive.hpp =================================================================== --- boost/mpi/detail/ignore_iprimitive.hpp (revision 63522) +++ boost/mpi/detail/ignore_iprimitive.hpp (working copy) @@ -42,13 +42,6 @@ typedef is_mpi_datatype<mpl::_1> use_array_optimization; -#ifndef BOOST_NO_MEMBER_TEMPLATE_FRIENDS - friend class archive::load_access; -protected: -#else -public: -#endif - /// don't do anything when loading primitive types template<class T> void load(T & t)
On Fri, 2 Jul 2010, Matthias Troyer wrote:
Hi,
The attached patch fixes the compilation problem on MacOS X - could someone please check whether this also works on other platforms?
The fundamental problem is the definition of the set of "primitive" types that any archive class needs to support to be compatible with Boost.Serialization. In the past those had been the fundamental builtin types and std::string. This has now changed to include a list of 10 additional types that were previously serialized using their serialize() function.
This change in the list of primitive types that every archive needs to support has been a serious breaking change and will most likely also impact other archives.
Robert, it would be good if you could add an explicit list of all primitive types that every archive needs to support to the documentation and archive concept, and announce any upcoming changes?
Ideally though it would be great if those types could get their serialize() function back so that they do not need to be treated ad primitive types.
It works for me on Linux and Open MPI; I checked both the mpi and graph_parallel test directories. -- Jeremiah Willcock
Matthias Troyer wrote:
Hi,
The attached patch fixes the compilation problem on MacOS X - could someone please check whether this also works on other platforms?
The fundamental problem is the definition of the set of "primitive" types that any archive class needs to support to be compatible with Boost.Serialization. In the past those had been the fundamental builtin types and std::string.
This has now changed to include a list of 10 additional types that were previously serialized using their serialize() function.
I'm not seeing this.
This change in the list of primitive types that every archive needs to support has been a serious breaking change and will most likely also impact other archives.
I'm not seeing this either.
Robert, it would be good if you could add an explicit list of all primitive types that every archive needs to support to the documentation and archive concept, and announce any upcoming changes?
all archives need to support all primitive types. But that doesnt' mean that they all have to be explicitly listed. The archives included in the system only include a few special ones. binary archives only include std::string, version_type (and maybe collection_size_type) because they have changed. All the rest are just written out in binary. text archives only a few special ones like string, char and wchar. All the other primitive types - those convertable to int or unsigned int, don't have to be explicitly specified. A number of types like class_id, .... are implicitly convertable to ints and so don't need special handling - it's done automatically. The are and always have been marked primitive. The only really new case is item_version_type. I tried to handlle it as non-primitive but that created other issues that I'd prefer not to go into here. So if +BOOST_MPI_DATATYPE(boost::archive::class_id_reference_type, get_mpi_datatype(int_least16_t()), integer); wasn't needed before, it shouldn't be needed now nor vice-versa. I think you've over looked the implicit conversion of these types to "true" primitives like int. On the other hand - something like +BOOST_MPI_DATATYPE(boost::archive::version_type, get_mpi_datatype(uint_least8_t()), integer); merit's scrutiny since an 8 bit object in some i/o systems - e.g. text streams will be rendered as an ascii character rather than an integer - which would create problems. On the other, other hand. I believe that MPI is a type of binary archive, so the above issue, doesn't show up as a problem. Also the only reason for special handling of version_type and item_verion_type is to maintain historical compatibility. It's likely that MPI doesn't need to consider this. When I looked at the error message. It seemed that all that was needed would be: BOOST_MPI_DATATYPE( for 3 new types (at most). But since I have no idea what the function of that macro is, I didn't feel comfortable looking into it. Thanks for working on it. Robert Ramey
Ideally though it would be great if those types could get their serialize() function back so that they do not need to be treated ad primitive types.
Matthias
participants (8)
-
Belcourt, Kenneth -
David Abrahams -
Jeremiah Willcock -
KTC -
Matthias Troyer -
Robert Ramey -
Steven Watanabe -
Thomas Klimpel