
Here's a use case that has been discussed before, but to which I couldn't seem to find any solid resolution the list archives: class A; some_oarchive oa; void mutate(shared_ptr<A>& aptr_) { aptr_ = shared_ptr<A>(new A); }; shared_ptr<A> aptr; for (int i=0; i<LARGE_NUMBER; i++) { mutate(aptr); oa << aptr; } The problem is that your oarchive ends up with somewhere between two and LARGE_NUMBER distinct A's in it. Usually two. Presumably the "two" is because in this simple example only A's are getting allocated and deallocated, and therefore there are often free A-sized blocks conveniently laying around for reuse. aptr gets assigned A's at two different alternating addresses. If it was just a plain A, you could turn off tracking by not serializing through a pointer. You'd then get the right number of A's in your archive. But if elsewhere you want to serialize A through a pointer you can't do that just for this call to operator<<(). Also you don't have a good way to see if A actually is serialized via pointer elsewhere or not. (My use case is one where lots of inexperienced programmers are modifying the data structures contained in A, and they will surely screw up if the rules are more complicated than "use BOOST_CLASS_EXPORT in your header file, and never use bald pointers, always use shared_ptr".) I looked through the archives a bunch and didn't come across anything conclusive. It seemed that some thought this kind of use case was pathological, but I'm not sure why. Of course, I'm not all that experienced with serialization. What I'd like to be able to do is to tell the archive, "The previous calls to operator<<() represent a 'snapshot' of the state of some group of objects, and now I want you to forget about existent objects because I am going to rearrange them all. Continue to track object types, but forget about the addresses." I realize that this creates the possiblity for memory leaks, but if the serialization is done through one toplevel call to operator<< on a shared_ptr whose pointee contains pointers to a whole universe of home-cooked pointer spaghetti, I don't see a better way to do this, and I don't see how to clearly express what I intend via the export and tracking macro mechanisms. You can't close and reopen the archive in the top loop, you get duplicate headers. The list archives mention the use case of serializing the state of some memory pool that is very likely to get reused: I think the little example above is probably the simplest case of this. So without asking for a sanity-check, I implemented basic_oarchive::flush(), and some tests. The changes to basic_oarchive and basic_oarchive_impl are very small. basic_oarchive has an internal object_set, which tracks object_ids and addresses. I add a num_flushed_objects member, and flush() clears out the set and adds the number of objects flushed to this counter. New tracked objects are assigned object_ids starting at num_flushed_objects + object_set.size(). In this way the class_id's are reused post-flush, but object id's are not. The interface is simply this: class A; some_oarchive oa; void mutate(shared_ptr<A>& aptr_) { aptr_ = shared_ptr<A>(new A); }; shared_ptr<A> aptr; for (int i=0; i<LARGE_NUMBER; i++) { mutate(aptr); oa << aptr; oa.flush(); } No modifications are required to the iarchive, but the user code of course has to be written in such a way that objects aren't leaked. though "oa << archive::endl;" would be a cute way to flush, too. Dunno. Is this unnecessary because I'm missing something obvious? At any rate the modified files and two tests are attached. All tests pass with this change... troy d. straszheim /////////1/////////2/////////3/////////4/////////5/////////6/////////7/////////8 // test_flush_simple.cpp // // copyright (c) 2005 // troy d. straszheim <troy@resophonic.com> // http://www.resophonic.com // // Use, modification and distribution is subject to the Boost Software // License, Version 1.0. (See accompanying file LICENSE_1_0.txt or copy at // http://www.boost.org/LICENSE_1_0.txt) // should pass compilation and execution // test verifies that basic_oarchive::flush() "resets" all pointers to objects #include <iostream> #include <fstream> #include <cstdio> // remove #include <boost/config.hpp> #if defined(BOOST_NO_STDC_NAMESPACE) namespace std{ using ::remove; } #endif #include <boost/archive/archive_exception.hpp> #include "test_tools.hpp" #include <boost/preprocessor/stringize.hpp> #include BOOST_PP_STRINGIZE(BOOST_ARCHIVE_TEST) #include <boost/serialization/base_object.hpp> #include <boost/serialization/export.hpp> struct A { unsigned i; template <class Archive> void serialize(Archive &ar, unsigned) { ar & BOOST_SERIALIZATION_NVP(i); } static std::size_t num_objects_; public: virtual ~A(){ num_objects_--; } A() { num_objects_++; } static std::size_t num_objects() { return num_objects_; } }; std::size_t A::num_objects_ = 0; const unsigned NUMOBJS = 4; void do_save(const char *testfile) { test_ostream os(testfile, TEST_STREAM_FLAGS); test_oarchive oa(os); oa.flush(); // be sure you can flush an empty archive A a; for (unsigned int i=0; i<NUMOBJS; i++) { a.i = i; oa << BOOST_SERIALIZATION_NVP(a); oa.flush(); BOOST_CHECK(A::num_objects() == 1); } // do some more flushes for the sake of thoroughness; oa.flush(); oa.flush(); oa.flush(); A* ap = &a; for (unsigned int i=0; i<NUMOBJS; i++) { ap->i = i; oa << BOOST_SERIALIZATION_NVP(ap); oa.flush(); BOOST_CHECK(A::num_objects() == 1); } oa.flush(); oa.flush(); oa.flush(); } // save exported polymorphic class void do_load(const char *testfile) { test_istream is(testfile, TEST_STREAM_FLAGS); test_iarchive ia(is); { A a; for (unsigned int i=0; i<NUMOBJS; i++) { ia >> BOOST_SERIALIZATION_NVP(a); BOOST_CHECK(a.i == i); BOOST_CHECK(A::num_objects() == 1); } } // a goes out of scope: no objects BOOST_CHECK(A::num_objects() == 0); A* ap; for (unsigned int i=0; i<NUMOBJS; i++) { ia >> BOOST_SERIALIZATION_NVP(ap); std::cout << "numobjs=" << A::num_objects() << std::endl; BOOST_CHECK(ap->i == i); BOOST_CHECK(A::num_objects() == (i+1)); // we leak 'em on purpose, we need to count em. } } int test_main( int /* argc */, char* /* argv */[] ) { const char * testfile = boost::archive::tmpnam(NULL); BOOST_REQUIRE(NULL != testfile); do_save(testfile); do_load(testfile); std::remove(testfile); return EXIT_SUCCESS; } // EOF /////////1/////////2/////////3/////////4/////////5/////////6/////////7/////////8 // test_flush.cpp // // copyright (c) 2005 // troy d. straszheim <troy@resophonic.com> // http://www.resophonic.com // // Use, modification and distribution is subject to the Boost Software // License, Version 1.0. (See accompanying file LICENSE_1_0.txt or copy at // http://www.boost.org/LICENSE_1_0.txt) // should pass compilation and execution // test verifies that basic_oarchive::flush() "resets" all pointers to objects #include <iostream> #include <fstream> #include <cstdio> // remove #include <boost/config.hpp> #if defined(BOOST_NO_STDC_NAMESPACE) namespace std{ using ::remove; } #endif #include <boost/archive/archive_exception.hpp> #include "test_tools.hpp" #include <boost/preprocessor/stringize.hpp> #include BOOST_PP_STRINGIZE(BOOST_ARCHIVE_TEST) #include <boost/serialization/base_object.hpp> #include <boost/serialization/export.hpp> class polymorphic_base { friend class boost::serialization::access; template<class Archive> void serialize(Archive & /* ar */, const unsigned int /* file_version */){ } static std::size_t num_objects_; public: virtual ~polymorphic_base(){ num_objects_--; } polymorphic_base() { num_objects_++; } static std::size_t num_objects() { return num_objects_; } }; std::size_t polymorphic_base::num_objects_ = 0; //BOOST_IS_ABSTRACT(polymorphic_base) class polymorphic_derived1 : public polymorphic_base { friend class boost::serialization::access; template<class Archive> void serialize(Archive &ar, const unsigned int /* file_version */){ ar & BOOST_SERIALIZATION_BASE_OBJECT_NVP(polymorphic_base); } public: virtual ~polymorphic_derived1(){} }; BOOST_CLASS_EXPORT(polymorphic_derived1) class polymorphic_derived2 : public polymorphic_base { friend class boost::serialization::access; template<class Archive> void serialize(Archive &ar, const unsigned int /* file_version */){ ar & BOOST_SERIALIZATION_BASE_OBJECT_NVP(polymorphic_base); } public: virtual ~polymorphic_derived2(){} }; BOOST_CLASS_EXPORT(polymorphic_derived2) // save exported polymorphic class void save_exported(const char *testfile) { test_ostream os(testfile, TEST_STREAM_FLAGS); test_oarchive oa(os); polymorphic_base *rb1 = new polymorphic_derived1; polymorphic_base *rb2a = new polymorphic_derived2; polymorphic_base *rb2b = new polymorphic_derived2; oa << BOOST_SERIALIZATION_NVP(rb1); oa << BOOST_SERIALIZATION_NVP(rb1); oa << BOOST_SERIALIZATION_NVP(rb2a); oa << BOOST_SERIALIZATION_NVP(rb2b); BOOST_CHECK(polymorphic_base::num_objects()==3); oa.flush(); oa << BOOST_SERIALIZATION_NVP(rb2b); oa << BOOST_SERIALIZATION_NVP(rb2a); oa << BOOST_SERIALIZATION_NVP(rb1); oa << BOOST_SERIALIZATION_NVP(rb1); BOOST_CHECK(polymorphic_base::num_objects()==3); oa.flush(); oa << BOOST_SERIALIZATION_NVP(rb1); oa << BOOST_SERIALIZATION_NVP(rb1); oa << BOOST_SERIALIZATION_NVP(rb2a); oa << BOOST_SERIALIZATION_NVP(rb2b); BOOST_CHECK(polymorphic_base::num_objects()==3); oa.flush(); delete rb1; delete rb2a; delete rb2b; BOOST_CHECK(polymorphic_base::num_objects()==0); } #define RETRIEVE_AND_CHECK(ARCHIVE, PTR, TYPE, N) \ ARCHIVE >> BOOST_SERIALIZATION_NVP(PTR); \ BOOST_CHECK_MESSAGE( \ boost::serialization::type_info_implementation<TYPE> \ ::type::get_instance() \ == boost::serialization::type_info_implementation<polymorphic_base> \ ::type::get_derived_extended_type_info(*PTR), \ "restored pointer " BOOST_PP_STRINGIZE(PTR) " not of correct type" ); \ BOOST_CHECK(polymorphic_base::num_objects()==N); // save exported polymorphic class void load_exported(const char *testfile) { test_istream is(testfile, TEST_STREAM_FLAGS); test_iarchive ia(is); polymorphic_base *rb1a = NULL, *rb1b = NULL; polymorphic_base *rb2a = NULL, *rb2b = NULL; RETRIEVE_AND_CHECK(ia, rb1a, polymorphic_derived1, 1); RETRIEVE_AND_CHECK(ia, rb1b, polymorphic_derived1, 1); RETRIEVE_AND_CHECK(ia, rb2a, polymorphic_derived2, 2); RETRIEVE_AND_CHECK(ia, rb2b, polymorphic_derived2, 3); RETRIEVE_AND_CHECK(ia, rb2b, polymorphic_derived2, 4); RETRIEVE_AND_CHECK(ia, rb2a, polymorphic_derived2, 5); RETRIEVE_AND_CHECK(ia, rb1a, polymorphic_derived1, 6); RETRIEVE_AND_CHECK(ia, rb1b, polymorphic_derived1, 6); RETRIEVE_AND_CHECK(ia, rb1a, polymorphic_derived1, 7); RETRIEVE_AND_CHECK(ia, rb1b, polymorphic_derived1, 7); RETRIEVE_AND_CHECK(ia, rb2a, polymorphic_derived2, 8); RETRIEVE_AND_CHECK(ia, rb2b, polymorphic_derived2, 9); } int test_main( int /* argc */, char* /* argv */[] ) { const char * testfile = boost::archive::tmpnam(NULL); BOOST_REQUIRE(NULL != testfile); save_exported(testfile); load_exported(testfile); std::remove(testfile); return EXIT_SUCCESS; } // EOF /////////1/////////2/////////3/////////4/////////5/////////6/////////7/////////8 // basic_oarchive.cpp: // (C) Copyright 2002 Robert Ramey - http://www.rrsd.com . // Use, modification and distribution is subject to the Boost Software // License, Version 1.0. (See accompanying file LICENSE_1_0.txt or copy at // http://www.boost.org/LICENSE_1_0.txt) // See http://www.boost.org for updates, documentation, and revision history. #include <boost/config.hpp> // msvc 6.0 needs this for warning suppression #include <cassert> #include <set> #include <boost/limits.hpp> #include <boost/state_saver.hpp> #include <boost/throw_exception.hpp> // including this here to work around an ICC in intel 7.0 // normally this would be part of basic_oarchive.hpp below. #define BOOST_ARCHIVE #include <boost/archive/basic_archive.hpp> #include <boost/archive/detail/basic_oserializer.hpp> #include <boost/archive/detail/basic_pointer_oserializer.hpp> #include <boost/archive/detail/basic_oarchive.hpp> #include <boost/archive/archive_exception.hpp> #ifdef BOOST_MSVC # pragma warning(push) # pragma warning(disable : 4251 4231 4660 4275) #endif using namespace boost::serialization; namespace boost { namespace serialization { class extended_type_info; } namespace archive { namespace detail { class basic_oserializer; class basic_pointer_oserializer; class basic_oarchive_impl { friend class basic_oarchive; ////////////////////////////////////////////////////////////////////// // information about each serialized object saved // keyed on address, class_id struct aobject { const void * address; class_id_type class_id; object_id_type object_id; bool operator<(const aobject &rhs) const { assert(NULL != address); assert(NULL != rhs.address); if( address < rhs.address ) return true; if( address > rhs.address ) return false; return class_id < rhs.class_id; } aobject & operator=(const aobject & rhs) { address = rhs.address; class_id = rhs.class_id; object_id = rhs.object_id; return *this; } aobject( const void *a, class_id_type class_id_, object_id_type object_id_ ) : address(a), class_id(class_id_), object_id(object_id_) {} aobject() : address(NULL){} }; // keyed on class_id, address typedef std::set<aobject> object_set_type; object_set_type object_set; std::size_t num_flushed_objects; ////////////////////////////////////////////////////////////////////// // information about each serialized class saved // keyed on type_info struct cobject_type { const basic_oserializer * bos_ptr; const class_id_type class_id; bool initialized; cobject_type( std::size_t class_id_, const basic_oserializer & bos_ ) : bos_ptr(& bos_), class_id(class_id_), initialized(false) {} cobject_type(const basic_oserializer & bos_) : bos_ptr(& bos_) {} cobject_type( const cobject_type & rhs ) : bos_ptr(rhs.bos_ptr), class_id(rhs.class_id), initialized(rhs.initialized) {} // the following cannot be defined because of the const // member. This will generate a link error if an attempt // is made to assign. This should never be necessary // use this only for lookup argument cobject_type & operator=(const cobject_type &rhs); bool operator<(const cobject_type &rhs) const { return *bos_ptr < *(rhs.bos_ptr); } }; // keyed on type_info typedef std::set<cobject_type> cobject_info_set_type; cobject_info_set_type cobject_info_set; // list of objects initially stored as pointers - used to detect errors // keyed on object id std::set<object_id_type> stored_pointers; // address of the most recent object serialized as a poiner // whose data itself is now pending serialization const void * pending_object; const basic_oserializer * pending_bos; basic_oarchive_impl() : num_flushed_objects(0), pending_object(NULL), pending_bos(NULL) {} const cobject_type & find(const basic_oserializer & bos); const basic_oserializer * find(const serialization::extended_type_info &ti) const; public: const cobject_type & register_type(const basic_oserializer & bos); void save_object( basic_oarchive & ar, const void *t, const basic_oserializer & bos ); void save_pointer( basic_oarchive & ar, const void * t, const basic_pointer_oserializer * bpos ); void flush(); }; ////////////////////////////////////////////////////////////////////// // implementation of basic_oarchive implementation functions // given a type_info - find its bos // return NULL if not found inline const basic_oserializer * basic_oarchive_impl::find(const serialization::extended_type_info & ti) const { class bosarg : public basic_oserializer { bool class_info() const { assert(false); return false; } // returns true if objects should be tracked bool tracking() const { assert(false); return false; } // returns class version unsigned int version() const { assert(false); return 0; } // returns true if this class is polymorphic bool is_polymorphic() const{ assert(false); return false; } void save_object_data( basic_oarchive & ar, const void * x ) const { assert(false); } public: bosarg(const serialization::extended_type_info & type_) : boost::archive::detail::basic_oserializer(type_) {} }; bosarg bos(ti); cobject_info_set_type::const_iterator cit = cobject_info_set.find(cobject_type(bos)); // it should already have been "registered" - see below if(cit == cobject_info_set.end()){ // if an entry is not found in the table it is because a pointer // of a derived class has been serialized through its base class // but the derived class hasn't been "registered" return NULL; } // return pointer to the real class return cit->bos_ptr; } inline const basic_oarchive_impl::cobject_type & basic_oarchive_impl::find(const basic_oserializer & bos) { std::pair<cobject_info_set_type::iterator, bool> cresult = cobject_info_set.insert(cobject_type(cobject_info_set.size(), bos)); return *(cresult.first); } inline const basic_oarchive_impl::cobject_type & basic_oarchive_impl::register_type( const basic_oserializer & bos ){ cobject_type co(cobject_info_set.size(), bos); std::pair<cobject_info_set_type::const_iterator, bool> result = cobject_info_set.insert(co); return *(result.first); } inline void basic_oarchive_impl::save_object( basic_oarchive & ar, const void *t, const basic_oserializer & bos ){ // if its been serialized through a pointer and the preamble's been done if(t == pending_object && pending_bos == & bos){ // just save the object data ar.end_preamble(); (bos.save_object_data)(ar, t); return; } // get class information for this object const cobject_type & co = register_type(bos); if(bos.class_info()){ if( ! co.initialized){ ar.vsave(class_id_optional_type(co.class_id)); ar.vsave(tracking_type(bos.tracking())); ar.vsave(version_type(bos.version())); (const_cast<cobject_type &>(co)).initialized = true; } } // we're not tracking this type of object if(! bos.tracking()){ // just windup the preamble - no object id to write ar.end_preamble(); // and save the data (bos.save_object_data)(ar, t); return; } // look for an existing object id object_id_type oid(object_set.size() + num_flushed_objects); // lookup to see if this object has already been written to the archive basic_oarchive_impl::aobject ao(t, co.class_id, oid); std::pair<basic_oarchive_impl::object_set_type::const_iterator, bool> aresult = object_set.insert(ao); oid = aresult.first->object_id; // if its a new object if(aresult.second){ // write out the object id ar.vsave(oid); ar.end_preamble(); // and data (bos.save_object_data)(ar, t); return; } // check that it wasn't originally stored through a pointer if(stored_pointers.end() != stored_pointers.find(oid)){ // this has to be a user error. loading such an archive // would create duplicate objects boost::throw_exception( archive_exception(archive_exception::pointer_conflict) ); } // just save the object id ar.vsave(object_reference_type(oid)); ar.end_preamble(); return; } // save a pointer to an object instance inline void basic_oarchive_impl::save_pointer( basic_oarchive & ar, const void * t, const basic_pointer_oserializer * bpos_ptr ){ const basic_oserializer & bos = bpos_ptr->get_basic_serializer(); std::size_t original_count = cobject_info_set.size(); const cobject_type & co = register_type(bos); if(! co.initialized){ ar.vsave(co.class_id); // if its a previously unregistered class if((cobject_info_set.size() > original_count)){ if(bos.is_polymorphic()){ const serialization::extended_type_info *eti = & bos.type; const char * key = NULL; if(NULL != eti) key = eti->key; if(NULL != key){ // the following is required by IBM C++ compiler which // makes a copy when passing a non-const to a const. This // is permitted by the standard but rarely seen in practice const class_name_type cn(key); // write out the external class identifier ar.vsave(cn); } else // without an external class name // we won't be able to de-serialize it so bail now boost::throw_exception( archive_exception(archive_exception::unregistered_class) ); } } if(bos.class_info()){ ar.vsave(tracking_type(bos.tracking())); ar.vsave(version_type(bos.version())); } (const_cast<cobject_type &>(co)).initialized = true; } else{ ar.vsave(class_id_reference_type(co.class_id)); } // if we're not tracking if(! bos.tracking()){ // just save the data itself ar.end_preamble(); state_saver<const void *> x(pending_object); state_saver<const basic_oserializer *> y(pending_bos); pending_object = t; pending_bos = & bpos_ptr->get_basic_serializer(); bpos_ptr->save_object_ptr(ar, t); return; } object_id_type oid(object_set.size() + num_flushed_objects); // lookup to see if this object has already been written to the archive basic_oarchive_impl::aobject ao(t, co.class_id, oid); std::pair<basic_oarchive_impl::object_set_type::const_iterator, bool> aresult = object_set.insert(ao); oid = aresult.first->object_id; // if the saved object already exists if(! aresult.second){ // append the object id to he preamble ar.vsave(object_reference_type(oid)); // and windup. ar.end_preamble(); return; } // append id of this object to preamble ar.vsave(oid); ar.end_preamble(); // and save the object itself state_saver<const void *> x(pending_object); state_saver<const basic_oserializer *> y(pending_bos); pending_object = t; pending_bos = & bpos_ptr->get_basic_serializer(); bpos_ptr->save_object_ptr(ar, t); // add to the set of object initially stored through pointers stored_pointers.insert(oid); } // replaces addresses of objects in the object_set with NULL so that // future objects will be duplicated, but object_id's will be // preserved. inline void basic_oarchive_impl::flush() { num_flushed_objects += object_set.size(); object_set.clear(); } ////////////////////////////////////////////////////////////////////// // implementation of basic_oarchive functions BOOST_DECL_ARCHIVE basic_oarchive::basic_oarchive() : pimpl(new basic_oarchive_impl) {} BOOST_DECL_ARCHIVE basic_oarchive::~basic_oarchive() { delete pimpl; } void BOOST_DECL_ARCHIVE basic_oarchive::save_object( const void *x, const basic_oserializer & bos ){ pimpl->save_object(*this, x, bos); } void BOOST_DECL_ARCHIVE basic_oarchive::save_pointer( const void * t, const basic_pointer_oserializer * bpos_ptr ){ pimpl->save_pointer(*this, t, bpos_ptr); } void BOOST_DECL_ARCHIVE basic_oarchive::register_basic_serializer(const basic_oserializer & bos){ pimpl->register_type(bos); } unsigned int BOOST_DECL_ARCHIVE basic_oarchive::library_version() const{ return ARCHIVE_VERSION(); } void BOOST_DECL_ARCHIVE basic_oarchive::end_preamble(){ } void BOOST_DECL_ARCHIVE basic_oarchive::flush(){ pimpl->flush(); } } // namespace detail } // namespace archive } // namespace boost #ifdef BOOST_MSVC #pragma warning(pop) #endif #ifndef BOOST_ARCHIVE_BASIC_OARCHIVE_HPP #define BOOST_ARCHIVE_BASIC_OARCHIVE_HPP // MS compatible compilers support #pragma once #if defined(_MSC_VER) && (_MSC_VER >= 1020) # pragma once #endif /////////1/////////2/////////3/////////4/////////5/////////6/////////7/////////8 // basic_oarchive.hpp: // (C) Copyright 2002 Robert Ramey - http://www.rrsd.com . // Use, modification and distribution is subject to the Boost Software // License, Version 1.0. (See accompanying file LICENSE_1_0.txt or copy at // http://www.boost.org/LICENSE_1_0.txt) // See http://www.boost.org for updates, documentation, and revision history. #include <boost/config.hpp> #include <boost/detail/workaround.hpp> // can't use this - much as I'd like to as borland doesn't support it // #include <boost/scoped_ptr.hpp> #include <boost/archive/basic_archive.hpp> #include <boost/serialization/tracking.hpp> #include <boost/archive/detail/abi_prefix.hpp> // must be the last header namespace boost { namespace archive { namespace detail { class BOOST_DECL_ARCHIVE basic_oarchive_impl; class BOOST_DECL_ARCHIVE basic_oserializer; class BOOST_DECL_ARCHIVE basic_pointer_oserializer; ////////////////////////////////////////////////////////////////////// // class basic_oarchive - write serialized objects to an output stream class BOOST_DECL_ARCHIVE basic_oarchive { friend class basic_oarchive_impl; // hide implementation of this class to minimize header conclusion // in client code. note: borland can't use scoped_ptr //boost::scoped_ptr<basic_oarchive_impl> pimpl; basic_oarchive_impl * pimpl; // overload these to bracket object attributes. Used to implement // xml archives virtual void vsave(const version_type t) = 0; virtual void vsave(const object_id_type t) = 0; virtual void vsave(const object_reference_type t) = 0; virtual void vsave(const class_id_type t) = 0; virtual void vsave(const class_id_optional_type t) = 0; virtual void vsave(const class_id_reference_type t) = 0; virtual void vsave(const class_name_type & t) = 0; virtual void vsave(const tracking_type t) = 0; protected: basic_oarchive(); virtual ~basic_oarchive(); public: unsigned int library_version() const; void save_object( const void *x, const basic_oserializer & bos ); void save_pointer( const void * t, const basic_pointer_oserializer * bpos_ptr ); void register_basic_serializer(const basic_oserializer & bos); void save_null_pointer(){ vsave(NULL_POINTER_TAG); } void end_preamble(); // default implementation does nothing void flush(); }; } // namespace detail } // namespace archive } // namespace boost // required by smart_cast for compilers not implementing // partial template specialization BOOST_BROKEN_COMPILER_TYPE_TRAITS_SPECIALIZATION( boost::archive::detail::basic_oarchive ) #include <boost/archive/detail/abi_suffix.hpp> // pops abi_suffix.hpp pragmas #endif //BOOST_ARCHIVE_BASIC_OARCHIVE_HPP

troy d. straszheim wrote:
Here's a use case that has been discussed before, but to which I couldn't seem to find any solid resolution the list archives:
class A; some_oarchive oa;
void mutate(shared_ptr<A>& aptr_) { aptr_ = shared_ptr<A>(new A); };
shared_ptr<A> aptr;
for (int i=0; i<LARGE_NUMBER; i++) { mutate(aptr); oa << aptr; }
The problem is that your oarchive ends up with somewhere between two and LARGE_NUMBER distinct A's in it. Usually two. Presumably the "two" is because in this simple example only A's are getting allocated and deallocated, and therefore there are often free A-sized blocks conveniently laying around for reuse. aptr gets assigned A's at two different alternating addresses.
For what its worth, I believe there is an warning in the document against this kind of thng. In fact, the documentation says that this will invoke a compile time error. I just checked - it doesn't produce a compile time error as I expected. I found the code that does this commented out - Now I don't remember why I commented it out. That is, the following are not recommended: a) changing he state of data while its in the process of be saved. b) serializating data of the stack. This will break the tracking as different objects will have the same address and be mis-identified as being different. If I had nothing else to do and could figure out how to do it, I would like implement a warning so that if one used a << operator with a non-const argument one would get a warning or maybe I am in the process of adding two more flags to archives: a) no_tracking - which will suppress tracking for all objects regardless of the setting of their serialization traits. My motivation for doing this was to permit the usage of serialization for things such as debug and transaction logs which would generate cases such as yours above. b) no_object_creation - which will simple reload pointers rather than re-create them. My motivation is to permit serialization to be used to implement the memento patter as described in GoF Patterns book. I currently have these changes in my local code base. And I've run all my old tests and they still work. I'm still struggliing with some small issues regarding loading to stl collections with no_object_creation. I'm also struggling with some issues related to these flags being runtime rather than compile time (i.e. template instantiation) options. I'm missing writing tests, demos and tutorial , and documentation. I'm not sure, but I think these new facilities may address the use cases raised here.
I looked through the archives a bunch and didn't come across anything conclusive. It seemed that some thought this kind of use case was pathological, but I'm not sure why.
My view has been that changing the state of an object while it is in the process of being serialized will inevitably lead to program that are not provably/demonstrably correct. The same goes for archive classes whose behavior can be changed during the course of serialization. Now by supporting the usage of serialization for logging - This concept will be broken. I'm still struggling with this. a) the idea of serialization of mutable objects does have application on logging type applications. its appealing to use if for this purpose. b) It wll break the original concept and lead to cases where errors are introduced which are almost impossible to track down without tracing into the implementation of the serialization library itself. This defeats the whole purpose of having a library in the first place.
What I'd like to be able to do is to tell the archive, "The previous calls to operator<<() represent a 'snapshot' of the state of some group of objects, and now I want you to forget about existent objects because I am going to rearrange them all. Continue to track object types, but forget about the addresses." I realize that this creates the possiblity for memory leaks, but if the serialization is done through one toplevel call to operator<< on a shared_ptr whose pointee contains pointers to a whole universe of home-cooked pointer spaghetti, I don't see a better way to do this, and I don't see how to clearly express what I intend via the export and tracking macro mechanisms.
You can't close and reopen the archive in the top loop, you get duplicate headers.
What about "no_header" ? and what would be wrong with duplicate headers anyway? The stream is still open and could just as well contain multiple archives.
The list archives mention the use case of serializing the state of some memory pool that is very likely to get reused: I think the little example above is probably the simplest case of this.
I'm not sure what this means.
So without asking for a sanity-check, I implemented basic_oarchive::flush(), and some tests. The changes to basic_oarchive
sanity is sometimes overrated.
and basic_oarchive_impl are very small. basic_oarchive has an internal object_set, which tracks object_ids and addresses. I add a num_flushed_objects member, and flush() clears out the set and adds the number of objects flushed to this counter. New tracked objects are assigned object_ids starting at num_flushed_objects + object_set.size(). In this way the class_id's are reused post-flush, but object id's are not. The interface is simply this:
My comments above should make it clear I wouldn't be enthusiastic about this approach. Having said that, I find it personally gratifying to find that some people are so enamored with this library to spend this kind of effort. Certain people have taken the library in "experimental" directions and I have worked in their results into the library. Persons who have made significant contributions are: Pavel Vozenilek - borland compilers and documentation Martin Ecker - DLL versions of Serialization and serialization of classes implemented in DLLS (plug-ins) troy d. straszheim(you) - serialization of variant. At the same time I endeavor to keep it from breaking under the weight of its own success. Its a fine line. Key "fixed points" in my requirements were and still are: a) boost acceptance - i need this for my resume as I'm currently looking for work. b) support of all compilers on which tests are run. c) idiot proof user interface and documentation. Ah - maybe i better say, user interface and documentation such that one can use the library without having to delve into its implementation. Also its important to me that one be able to use the library with a very short learning curve - say 1 hour to get started. Personally, I don't have much more patience than that. Robert Ramey
participants (2)
-
Robert Ramey
-
troy d. straszheim