[serialization] portable serialization archive
Hi, I have seen several mentions on the mailing list of a portable binary serialization archive, what is the place to get the latest and greatest version? Thanks, Chris
Robert Ramey wrote:
Chris Weed wrote:
Hi, I have seen several mentions on the mailing list of a portable binary serialization archive, what is the place to get the latest and greatest version? Thanks, Chris The trunk - but it's still being scrutinized.
Robert Ramey
Any idea how this differs from what's in the vault? Can it be back ported to 1.34.1 and/or 1.35.0? I've just posted a problem I'm having on MAC on the devel list. Thanks, Jeff
the portible binary archive was really made to illustrate how to create a new archive class from an existing one. It wasn't really the best example of this but it was written up and added to the documentation. It wasn't tested and had some bugs. When changes were made to binary archive- the portable binary archive broke. In the meantime, a user of serialization - Cadence Design of Israel - offered to fund the fixing up of this class and permit the updated version to be included in boost. This happened. They also tested it on a wider variety of platforms. So the one in the trunk is the one to use. It should compile with any version of boost. There is a tiny difference between the tested one and the one checked into the trunk. So I've got a niggling doubt about it. If someone really want's to make portable_binary_archive an "officially supported archive". Then some way of testing is required. This would require generating output on one platform and reading on another. This is something that I couldn't figure out how to make BJAM do. Robert Ramey Jeff Flinn wrote:
Robert Ramey wrote:
Chris Weed wrote:
Hi, I have seen several mentions on the mailing list of a portable binary serialization archive, what is the place to get the latest and greatest version? Thanks, Chris The trunk - but it's still being scrutinized.
Robert Ramey
Any idea how this differs from what's in the vault? Can it be back ported to 1.34.1 and/or 1.35.0? I've just posted a problem I'm having on MAC on the devel list.
Thanks, Jeff
Hi! Robert Ramey schrieb:
If someone really want's to make portable_binary_archive an "officially supported archive". Then some way of testing is required. This would require generating output on one platform and reading on another. This is something that I couldn't figure out how to make BJAM do.
Well, you could just instead check whether the produced output is the same everywhere. I thought of calculating the md5 sum and checking against that. That would make a good unit test. Or am I missing something? Frank
Frank Birbacher wrote:
Hi!
Robert Ramey schrieb:
If someone really want's to make portable_binary_archive an "officially supported archive". Then some way of testing is required. This would require generating output on one platform and reading on another. This is something that I couldn't figure out how to make BJAM do.
Well, you could just instead check whether the produced output is the same everywhere. I thought of calculating the md5 sum and checking against that. That would make a good unit test. Or am I missing something?
That's a great idea. We've been generating a test file with representative data for each of our platforms which are checked into source control. A unit test then checks all (3) files on each platform. The Md5 approach would negate the management of archive files. Thanks, Jeff Flinn
Frank Birbacher wrote:
Hi!
Robert Ramey schrieb:
If someone really want's to make portable_binary_archive an "officially supported archive". Then some way of testing is required. This would require generating output on one platform and reading on another. This is something that I couldn't figure out how to make BJAM do.
Well, you could just instead check whether the produced output is the same everywhere. I thought of calculating the md5 sum and checking against that. That would make a good unit test. Or am I missing something?
LOL, This is way too simple for me to have thought of. We'll look into it. Robert Ramey
Frank
Frank Birbacher:
Robert Ramey schrieb:
If someone really want's to make portable_binary_archive an "officially supported archive". Then some way of testing is required. This would require generating output on one platform and reading on another. This is something that I couldn't figure out how to make BJAM do.
Well, you could just instead check whether the produced output is the same everywhere. I thought of calculating the md5 sum and checking against that. That would make a good unit test. Or am I missing something?
This is a good way to test the output part; the input can be tested by trying to read a pre-written archive (possibly embedded in the test source). I made a similar test once when I was playing with shared_ptr serialization: #include <boost/serialization/shared_ptr_132.hpp> #include <boost/serialization/shared_ptr.hpp> #include <boost/serialization/weak_ptr.hpp> #include <boost/serialization/nvp.hpp> #include <boost/serialization/export.hpp> #include <boost/archive/text_oarchive.hpp> #include <boost/archive/xml_oarchive.hpp> #include <iostream> struct A { int x_; template<class Archive> void serialize( Archive & ar, unsigned ) { ar & BOOST_SERIALIZATION_NVP( x_ ); } static int instances; explicit A( int x = 0 ): x_( x ) { ++instances; } virtual ~A() { --instances; } bool operator==( const A & rhs ) const { return x_ == rhs.x_; } }; int A::instances; struct B: public A { int y_; template<class Archive> void serialize( Archive & ar, unsigned ) { ar & BOOST_SERIALIZATION_BASE_OBJECT_NVP( A ); ar & BOOST_SERIALIZATION_NVP( y_ ); } static int instances; explicit B( int x = 0, int y = 0 ): A( x ), y_( y ) { ++instances; } ~B() { --instances; } bool operator==( const B & rhs ) const { return x_ == rhs.x_ && y_ == rhs.y_; } }; int B::instances; BOOST_SHARED_POINTER_EXPORT( B ) template<class Ar> void write_132( Ar & ar ) { boost_132::shared_ptr<B> p1( new B( 701, 801 ) ); boost_132::shared_ptr<A> p2( p1 ); boost_132::shared_ptr<A> p3( new B( 703, 803 ) ); boost_132::shared_ptr<A> p4( new A( 704 ) ); boost_132::shared_ptr<A> p5( p4 ); ar << BOOST_SERIALIZATION_NVP( p1 ); ar << BOOST_SERIALIZATION_NVP( p2 ); ar << BOOST_SERIALIZATION_NVP( p3 ); ar << BOOST_SERIALIZATION_NVP( p4 ); ar << BOOST_SERIALIZATION_NVP( p5 ); } template<class Ar> void write_135( Ar & ar ) { boost::shared_ptr<B> p1( new B( 701, 801 ) ); boost::shared_ptr<A> p2( p1 ); boost::shared_ptr<A> p3( new B( 703, 803 ) ); boost::shared_ptr<A> p4( new A( 704 ) ); boost::shared_ptr<A> p5( p4 ); ar << BOOST_SERIALIZATION_NVP( p1 ); ar << BOOST_SERIALIZATION_NVP( p2 ); ar << BOOST_SERIALIZATION_NVP( p3 ); ar << BOOST_SERIALIZATION_NVP( p4 ); ar << BOOST_SERIALIZATION_NVP( p5 ); } #include <boost/detail/lightweight_test.hpp> template<class Ar> void read( Ar & ar ) { boost::shared_ptr<B> p1; // ( new B( 701, 801 ) ); boost::shared_ptr<A> p2; // ( p1 ); boost::shared_ptr<A> p3; // ( new B( 703, 803 ) ); boost::shared_ptr<A> p4; // ( new A( 704 ) ); boost::shared_ptr<A> p5; // ( p4 ); ar >> BOOST_SERIALIZATION_NVP( p1 ); BOOST_TEST( p1 && *p1 == B( 701, 801 ) ); ar >> BOOST_SERIALIZATION_NVP( p2 ); BOOST_TEST( p1 == p2 ); BOOST_TEST( !( p1 < p2 ) && !( p2 < p1 ) ); ar >> BOOST_SERIALIZATION_NVP( p3 ); BOOST_TEST( p3 && dynamic_cast<B*>( p3.get() ) && dynamic_cast<B&>( *p3 ) == B( 703, 803 ) ); ar >> BOOST_SERIALIZATION_NVP( p4 ); BOOST_TEST( p4 && *p4 == A( 704 ) ); ar >> BOOST_SERIALIZATION_NVP( p5 ); BOOST_TEST( p4 == p5 ); BOOST_TEST( !( p4 < p5 ) && !( p5 < p4 ) ); } char const * data_132 = "22 serialization::archive 4 0 0 2 1 0 0 1 0 1 701 801 0 0 1 1 0 2 2 0 0 0 2 0 1 2 2 3 4 703 803 1 5 2 3 3 6 704 6 1 0 7 3 6 3 6 6 7"; char const * data_135 = "22 serialization::archive 4 0 1 1 1 0 0 1 0 1 701 801 0 1 1 0 1 2 3 703 803 2 4 704 2 4"; #include <boost/archive/text_iarchive.hpp> #include <sstream> int main() { //{ // boost::archive::text_oarchive ar( std::cout ); // write_132( ar ); // std::cout << "\n--\n\n"; //} //{ // boost::archive::text_oarchive ar( std::cout ); // write_135( ar ); // std::cout << "\n--\n\n"; //} { std::istringstream is( data_132 ); boost::archive::text_iarchive ar( is ); read( ar ); } { std::istringstream is( data_135 ); boost::archive::text_iarchive ar( is ); read( ar ); } return boost::report_errors(); }
Peter Dimov wrote:
Frank Birbacher:
Robert Ramey schrieb:
If someone really want's to make portable_binary_archive an "officially supported archive". Then some way of testing is required. This would require generating output on one platform and reading on another. This is something that I couldn't figure out how to make BJAM do.
Well, you could just instead check whether the produced output is the same everywhere. I thought of calculating the md5 sum and checking against that. That would make a good unit test. Or am I missing something?
(possibly embedded in the test source).
Damn - another good idea. This would resolve the problem I had before when I tried this which was related to maintaining other files, etc. Its not clear how easy this would work for binary files. Its clear that it would be a good thing to add to test a group of files serialized with every version of boost so that one could include tests for backward compatibility automatically. However, looks like a fair bit of effort to setup - but still - would be very useful. Robert Ramey
Robert Ramey wrote:
the portible binary archive was really made to illustrate how to create a new archive class from an existing one. It wasn't really the best example of this but it was written up and added to the documentation. It wasn't tested and had some bugs. When changes were made to binary archive- the portable binary archive broke. In the meantime, a user of serialization - Cadence Design of Israel - offered to fund the fixing up of this class and permit the updated version to be included in boost. This happened. They also tested it on a wider variety of platforms. So the one in the trunk is the one to use. It should compile with any version of boost. There is a tiny
Thanks Robert, we got the trunk version to compile/link/run on MSVC8 and Mac XCode3 Intel & PowerPC with 1.34.1. Doing so required copying boost_1_35_0 files: .../archive/shared_ptr.hpp and .../archive/detail/register_archive.hpp. We placed the pb i/o archive cpp's in our project code, which required adding a #include <boost/version.hpp>. It'd be nice to see this added as a standard archive. Jeff Flinn
Jeff Flinn on Tuesday, June 03, 2008 10:22 PM:
Any idea how this differs from what's in the vault? Can it be back ported to 1.34.1 and/or 1.35.0? I've just posted a problem I'm having on MAC on the devel list.
Hi Jeff, I looked into the developer list (which I am not subscribed to) and found your posting:
Using the latest portable_binary_archive from the vault with boost 1.34.1 on Mac XCode 3 I'm getting a linker duplicate symbol error:
ld: duplicate symbol
boost::archive::basic_binary_oarchive<portable_binary_oarchive> ::save_override(boost::archive::class_id_optional_type const&, int)
in .../Persistence.build/Debug/CxxTest.build/Objects-normal/ppc/AAA.o
and .../Persistence.build/Debug/CxxTest.build/Objects-normal/ppc/BBB.o
Any help in resolving this issue would be appreciated. This succesfully compiles/links and runs when built on MSVC 8.
It might be that the mac compiler is a bit more strict on multiple template instatiations or else. I suspect line 203 to cause your error, so you could try to move the (entire) instatiation part into a separate file if you include portable_binary_oarchive.hpp multiple times. Also be sure that your compiler understands #pragma once or change to the traditional #ifndef guard. // explicitly instantiate for this type of binary stream template class basic_binary_oarchive<portable_binary_oarchive> ; We have had no user on the mac side of life so you're lucky to be the first ;) If you do not succeed trying the suggestions above or need further information feel free to contact me directly or post to the user list. It is somewhat confusing to have two portable_binary_archives around (vault and trunk), so always specify which one you are referring to. Regards, -- Christian Pfligersdorffer Software Engineering http://www.eos.info
Pfligersdorffer, Christian wrote:
Jeff Flinn on Tuesday, June 03, 2008 10:22 PM:
Hi Jeff, I looked into the developer list (which I am not subscribed to) and found your posting:
Using the latest portable_binary_archive from the vault with boost 1.34.1 on Mac XCode 3 I'm getting a linker duplicate symbol error: ...
It might be that the mac compiler is a bit more strict on multiple template instatiations or else. I suspect line 203 to cause your error,
From my (limited) experience instantiations seem to occur earlier as well.
so you could try to move the (entire) instatiation part into a separate file if you include portable_binary_oarchive.hpp multiple times. Also be sure that your compiler understands #pragma once or change to the traditional #ifndef guard.
// explicitly instantiate for this type of binary stream template class basic_binary_oarchive<portable_binary_oarchive> ;
Thanks for the tips. If we run into problems with the trunk version we'll look at this separation. For now we're up and running.
We have had no user on the mac side of life so you're lucky to be the first ;) If you do not succeed trying the suggestions above or need further information feel free to contact me directly or post to the user list.
The trunk version now works with Mac XCode 3 intel/ppc(boost_1_34_1) as I mentioned in my reply to Robert! I'll be moving us to 1.35 soon.
It is somewhat confusing to have two portable_binary_archives around (vault and trunk), so always specify which one you are referring to.
I thought I did: "Using the latest portable_binary_archive from the ***vault*** with boost 1.34.1 on Mac XCode 3 I'm getting a linker duplicate symbol error:" Thanks again for the help, Jeff
The trunk version of portable_binary_iarchive.hpp has the following at line 98: // default fall through for any types not specified here template<class T> void load(T & t){ boost::intmax_t l; load_impl(l, sizeof(T)); // use cast to avoid compile time warning t = static_cast<T>(l); } On msvc 8 I get the following warning: F:\Gene_Codes\External_Libraries\boost_1_34_1\boost/archive/portable_binary_iarchive.hpp(103) : warning C4800: 'boost::intmax_t' : forcing value to bool 'true' or 'false' (performance warning) The comment on this function implies that there are specializations somewhere. Would it make sense to have a void load<bool> specialization with: t = l? true : false; Thanks, Jeff
participants (6)
-
Chris Weed -
Frank Birbacher -
Jeff Flinn -
Peter Dimov -
Pfligersdorffer, Christian -
Robert Ramey