[serialization][1.42.0] gcc(4.01)/xcode(3.1.2) failure to register type
I've followed the doc's 'Static Libraries and Serialization' approach. To move the serialize implementation from the header to the cpp file for my classes in my hierarchy which are serialized through base class pointer. Which runs on MSVC8, after linking with /OPT:NOREF. Unfortunately the BOOST_CLASS_EXPORTxxx facilities appear to be optimized away on gcc/xcode, even though all of the 'strip' settings are off. What approach are others taking to resolve this? Robert's reply to Emil Dotchevski on Sep 25, 2008 suggests that explicitly calling ar.register(static_cast<myType *>(0)); is the only portable way to ensure registration, is this still the case? Should explicit registration be done in combination with or en lieu of BOOST_CLASS_EXPORTxxx? Thanks, Jeff
Jeff Flinn wrote:
I've followed the doc's 'Static Libraries and Serialization' approach. To move the serialize implementation from the header to the cpp file for my classes in my hierarchy which are serialized through base class pointer. Which runs on MSVC8, after linking with /OPT:NOREF.
Unfortunately the BOOST_CLASS_EXPORTxxx facilities appear to be optimized away on gcc/xcode, even though all of the 'strip' settings are off. What approach are others taking to resolve this?
Hmmm I'm assuming BOOST_CLASS_EXPORTxxx refers to BOOST_CLASS_EXPORT_KEY and the other one. This is my latest refinement to resolve the issues related to DLLS. It would seem to me the same and/or similar treatment would also apply to static libraries. Note that in a static library you'll have to explicitly instantiate the serialization functions for the archive classes you use. I'll presume you're past this.
Robert's reply to Emil Dotchevski on Sep 25, 2008 suggests that explicitly calling ar.register(static_cast<myType *>(0)); is the only portable way to ensure registration, is this still the case?
It is still the case and I see no change in the future. EXPORT depends upon non-portable facilities which it seems all our compilers implement (albeit with different syntaxes !! Maybe the C++ standard commitees might want to look in to issues related to dynamic linking?)
Should explicit registration be done in combination with or en lieu of BOOST_CLASS_EXPORTxxx?
That would be one way. Another way would be to create a "pseudo module" in your program which explicitly refers to the serialization functions. (for example by taking their address?) You're kind of on your own. Note that the current tests for the serialization library on both VC and gcc include tests of DLLS where the implemention is in a DLL. One way would be to use a DLL /shared library rather than a static library. The compiler can't discard the code in these cases. Robert Ramey
Robert Ramey wrote:
Jeff Flinn wrote:
I've followed the doc's 'Static Libraries and Serialization' approach. To move the serialize implementation from the header to the cpp file for my classes in my hierarchy which are serialized through base class pointer. Which runs on MSVC8, after linking with /OPT:NOREF.
Unfortunately the BOOST_CLASS_EXPORTxxx facilities appear to be optimized away on gcc/xcode, even though all of the 'strip' settings are off. What approach are others taking to resolve this?
Hmmm I'm assuming BOOST_CLASS_EXPORTxxx refers to BOOST_CLASS_EXPORT_KEY and the other one. This is my latest refinement to resolve the issues related to DLLS. It would seem to me the same and/or similar treatment would also apply to static libraries.
Yes, ..._KEY in the header and _IMPLEMENT in the cpp. By the way, it appears that BOOST_CLASS_EXPORT_KEY requires #include <boost/serialization/extended_type_info.hpp> for my headers to compile in the absence of any archive header. Is that intended?
Note that in a static library you'll have to explicitly instantiate the serialization functions for the archive classes you use. I'll presume you're past this.
Yes I've done this.
Robert's reply to Emil Dotchevski on Sep 25, 2008 suggests that explicitly calling ar.register(static_cast<myType *>(0)); is the only portable way to ensure registration, is this still the case?
It is still the case and I see no change in the future. EXPORT depends upon non-portable facilities which it seems all our compilers implement (albeit with different syntaxes !! Maybe the C++ standard commitees might want to look in to issues related to dynamic linking?)
Should explicit registration be done in combination with or en lieu of BOOST_CLASS_EXPORTxxx?
That would be one way.
Hmm, seems like I asked about two mutually exclusive ways. ;-)
Another way would be to create a "pseudo module" in your program which explicitly refers to the serialization functions. (for example by taking their address?) You're kind of on your own.
Thant's what I've been thinking. It's not so bad if I can then get rid of the BOOST_CLASS_EXPORT_XXX macros. Also I was expecting archive::register_type to merely need a class forward declaration, but appears that it requires a complete type. Is that intended? That adds a lot of coupling.
Note that the current tests for the serialization library on both VC and gcc include tests of DLLS where the implemention is in a DLL.
One way would be to use a DLL /shared library rather than a static library. The compiler can't discard the code in these cases.
Sigh, we're trying to minimize our dll/dylib usage. Thanks, Jeff
Jeff Flinn wrote:
Should explicit registration be done in combination with or en lieu of BOOST_CLASS_EXPORTxxx?
That would be one way.
Hmm, seems like I asked about two mutually exclusive ways. ;-)
Another way would be to create a "pseudo module" in your program which explicitly refers to the serialization functions. (for example by taking their address?) You're kind of on your own.
Thant's what I've been thinking. It's not so bad if I can then get rid of the BOOST_CLASS_EXPORT_XXX macros. Also I was expecting archive::register_type to merely need a class forward declaration, but appears that it requires a complete type. Is that intended? That adds a lot of coupling.
The basic problem is: application pulls from the static library only those functions explicitly referred to. call a function through a base class pointer does explicitly refer to the derived class. Of course these two ideas conflict and there can be no resolution without violating the original motivation for these concepts in the first place. Another way of saying this is that we want to couple our application to some type/function which we haven't explicitly referenced. The only way to resolve this is to explicitly reference these types/functions. The register_type facility does this on an archive by archive level. EXPORT does this for the whole program so it propagates this facility to the all archives. So the simplest would be to use register type. It might be possible to enhance EXPORT in some way for static libraries. The problem is that it would start sucking in all the library code whether need or not - thereby defeating the whole purpose of a static library in the first place. Maybe a nice way would be to craft something like #include <boost/serialization/extended_type_info_typeid.hpp> using boost::serialization; void register_all_modules(){ const extended_type_info & eti = extended_type_info_typeid<my_type>::get_const_instance const extended_type_info & eti = extended_type_info_typeid<my_type>::get_const_instance();
Robert Ramey wrote:
Jeff Flinn wrote:
Should explicit registration be done in combination with or en lieu of BOOST_CLASS_EXPORTxxx? That would be one way. Hmm, seems like I asked about two mutually exclusive ways. ;-)
Another way would be to create a "pseudo module" in your program which explicitly refers to the serialization functions. (for example by taking their address?) You're kind of on your own. Thant's what I've been thinking. It's not so bad if I can then get rid of the BOOST_CLASS_EXPORT_XXX macros. Also I was expecting archive::register_type to merely need a class forward declaration, but appears that it requires a complete type. Is that intended? That adds a lot of coupling.
The basic problem is:
application pulls from the static library only those functions explicitly referred to.
call a function through a base class pointer does explicitly refer to the derived class.
Of course these two ideas conflict and there can be no resolution without violating the original motivation for these concepts in the first place. Another way of saying this is that we want to couple our application to some type/function which we haven't explicitly referenced.
The only way to resolve this is to explicitly reference these types/functions. The register_type facility does this on an archive by archive level. EXPORT does this for the whole program so it propagates this facility to the all archives.
So the simplest would be to use register type. It might be possible to enhance EXPORT in some way for static libraries. The problem is that it would start sucking in all the library code whether need or not - thereby defeating the whole purpose of a static library in the first place.
Maybe a nice way would be to craft something like
#include <boost/serialization/extended_type_info_typeid.hpp>
using boost::serialization;
void register_all_modules(){ const extended_type_info & eti = extended_type_info_typeid<my_type>::get_const_instance const extended_type_info & eti = extended_type_info_typeid<my_type>::get_const_instance();
I'll have to study what you've presented here. Just as another data point, on MSVC I was able to link with /OPT:REF to optimize away unref'd functions and data by declaring my serialize methods as: template<class Archive> BOOST_DLLEXPORT void serialize(Archive&, const unsigned int) BOOST_USED; (the macros being defined in your force_include.hpp) and all links and runs perfectly, reducing my release exe size from 3.6MB to 2.6MB. I'm guessing you have a #pragma somewhere that is responsible for generating .exp and .lib files for my executable as I have no explicit MSVC settings to generate these. Jeff
Jeff Flinn wrote:
void register_all_modules(){ const extended_type_info & eti = extended_type_info_typeid<my_type>::get_const_instance const extended_type_info & eti = extended_type_info_typeid<my_type>::get_const_instance();
I'll have to study what you've presented here.
LOL, I accidently hit the send key while creating my thought. Actually having thought about this some, I think it might be possible, but I think it will take more thought.
Just as another data point, on MSVC I was able to link with /OPT:REF to optimize away unref'd functions and data by declaring my serialize methods as:
template<class Archive> BOOST_DLLEXPORT void serialize(Archive&, const unsigned int) BOOST_USED;
(the macros being defined in your force_include.hpp) and all links and runs perfectly, reducing my release exe size from 3.6MB to 2.6MB. I'm guessing you have a #pragma somewhere that is responsible for generating .exp and .lib files for my executable as I have no explicit MSVC settings to generate these.
This is on the right track. seems to me it should work for gcc as well. It still wouldn't address serialization of objects through a base class pointer. Failure to do this would result in "not registered". For this, one will need similary magic for extended_type_info records. Robert Ramey
Jeff
Robert Ramey wrote: I woke up last night and remembered how I dealt with this in one of the examples. Suppose you've compiled everything into a static library X. There is a header b.hpp which looks like: class derived : public base { ... template<class Archive> void serialize(Archive & ar, const unsigned int version); }; and X will contain a b.cpp file which looks like #include <boost/archive/text_oarchive.hpp> #include <boost/archive/text_oarchive.hpp> ... // other archives template<class Archive> void serialize(Archive & ar, const unsigned int version){ ar & ... } // explicit instantiation template void derived::serialize(boost::archive::text_oarchive & ar, const unsigned int version); template void derived::serialize(boost::archive::text_iarchive & ar, const unsigned int version); ... Now we have program A linked to library X. If A only serializes through a pointer to the class "base" then the code in b.cpp isn't linked in. The easiest way to handle this is to NOT put b.cpp into library X but rather link it in explicitly into program A. So the make for program A looks like A.exe : A.obj b.obj X.lib In effect, the link specification becomes the equivalent "export registration". The only way this will break down is if the linker is too smart for our own good. If you want a finer granularity, you can slightly reorganize code so that you have b_text.cpp, b_binary, etc Or use a compile time switch to indicate which archives you want generated like c++ b.cpp -DARCHIVE=text_iarchive ... or just use the polymorphic archive for everything there are tested demos for doing this. To summarize, I believe that explicitly including modules with instantiating code will be the equivalent to "export registration" since these modules include the static singletons whose pre-main construction makes the required types accessible to the system. I think this is the best way to handle this. In fact, I think this is soo good it should be added as a "case study" to the serialization library documentation. Robert Ramey
participants (2)
-
Jeff Flinn -
Robert Ramey