Hi everyone, Thanks for the reply, The exception I got was a boost::archive::archive_exception, please note that I never get this exception if the derived class + export macro is in the same executable as the base class + main. Francois Mauger wrote:

i think you should read: it works perfectly if *BOOST_CLASS_EXPORT_GUID* is in the main execution. you can still put your derived class wherever you want !

You may be right but if the derived class is in the DLL there is just no way to call the macro EXPORT in the executable (you need the definition of the exported class for the macro to work), so it turns out to be the same. Francois Mauger wrote:

It looks like an issue I've met a few weeks ago. After investigation (note may be I'm wrong but I can only report of my understanding which is far to be good !) I now believe it is NOT possible to scatter such serialization export bits in different places (DLL vs executable, DLL vs DLL). It is because, as far as I understand the concept, the "export" mechanism you mentioned is implemented in the DLL object scope (probably using some static 'local' dictionary instantiated in the DLL scope or the executable scope, depending of the user). You can imagine such case: your base class is registered in the DLL but your Daughter class is registered at the level of the executable object scope : thus there is no way to make them communicate about their relationship. Both dictionaries (one per instantiation unit) acts in different words and know only one subset of the serializable classes to be used in your executable.

Thanks for the explanation, I still have one question through : if I mark a class as exported with the __declspec(dllexport) in the DLL, the class is "exported" to the outside world, so we have parent class and child class registered in the same object scope right ? And how about DLL that is loaded implicitely ? In this case the definition of its class is knowned by the executable from the beginning, will it help solving this issue ? Francois Mauger wrote:

In your case, I think you could step back to the previous approach where all serialization stuff is instantiated in the executable. The alternative is to probably to move your xml_archive ser/deser code in some resource implemented in the DLL and promote some reader and writer classes for XML archives as an element of your DLL, the later being the unique place to safely embed the problematic bits. I'm afraid there is not intermediate.

Well my boss want this feature and I have no power over the decision made. I myself don't think using those numerous DLLs is a good idea either. Once again, thank you for your anwser, if anyone knows a way to get around this issue, please continue the discussion. -- View this message in context: http://boost.2283326.n4.nabble.com/Boost-Serialization-Problem-serializing-d... Sent from the Boost - Users mailing list archive at Nabble.com.

Hi, I use serialization of derived class in DLLs, actually several DLLs, so I guess it works. In BASE.DLL I have the class Base, and in DERIVED.DLL i have the class Derived. The following shows the necessary contents of Base.h/cpp and Derived.h/cpp. There are some issues that are mostly due to the fact that serialization depends on singleton to know how to serialize a given type. Multiple singletons in DLLs are the main problem, but setting "track always" do help. Hope it helps. Guy =================== Base.h class DLLSPEC Base { [...] // only declaration template<class Archive> void serialize(Archive & ar, const unsigned int version); } BOOST_CLASS_TRACKING(Base, boost::serialization::track_always) BOOST_CLASS_EXPORT_KEY2(Base, "Base") BOOST_CLASS_VERSION(Base, 2) ===================== Base.cpp template<class Archive> void Base::serialize(Archive & ar, const unsigned int version) { [...] // the implementation } // force the template instantiation for each archive types you use (typically xml_iarchive and xml_oarchive template DLLSPEC void Base::serialize(A& ar, const unsigned int version); BOOST_CLASS_EXPORT_IMPLEMENT(Base) =================== Derived.h class DLLSPEC Derived { [...] // only declaration template<class Archive> void serialize(Archive & ar, const unsigned int version); } BOOST_CLASS_TRACKING(Derived, boost::serialization::track_always) // maybe needed BOOST_CLASS_EXPORT_KEY2(Derived, "Derived") BOOST_CLASS_VERSION(Derived, 28) ===================== Derived.cpp template<class Archive> void Derived::serialize(Archive & ar, const unsigned int version) { ar & BOOST_SERIALIZATION_BASE_OBJECT_NVP(Base); [...] // the implementation } // force the template instantiation for each archive types you use (typically xml_iarchive and xml_oarchive template DLLSPEC void Derived::serialize(A& ar, const unsigned int version); BOOST_CLASS_EXPORT_IMPLEMENT(Derived) ========================= -- Guy Prémont, D.Sc. Architecte logiciel senior / Senior software architect CM Labs Simulations Inc. http://www.cm-labs.com/ Tel. 514-287-1166 ext. 237

-----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users- bounces@lists.boost.org] On Behalf Of Kolb, Jeremy Sent: Wednesday, April 06, 2011 9:20 AM To: boost-users@lists.boost.org Subject: Re: [Boost-users] [Boost Serialization] Problem serializing derived class declared in DLL

I was never able to get this working properly either. I really like the serialization library but it's impossible to use if you have multiple derived classes across DLLs.

Jeremy

...

-----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users- bounces@lists.boost.org] On Behalf Of François Mauger Sent: Tuesday, April 05, 2011 6:13 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] [Boost Serialization] Problem serializing derived class declared in DLL

Hi,

...

...

...

kmdarkmaster a écrit : Dear Boost community,

My problem is simple : I need to serialize a derived class (named Plugin) of a c++ class (named Base) in my main program.

The derived class Plugin is declared in a DLL which is load explicitely during execution.

I tried to use the macro BOOST_CLASS_EXPORT_GUID (as in the doc), this works perfectly if the derived class is in the main execution, i think you should read: it works perfectly if *BOOST_CLASS_EXPORT_GUID* is in the main execution. you can still put your derived class wherever you want !

...

but will give me an exception what exception ?

...

if the derived class is in the plugin DLL.

It looks like an issue I've met a few weeks ago. After investigation (note may be I'm wrong but I can only report of my understanding which is far to be good !) I now believe it is NOT possible to scatter such serialization export bits in different places (DLL vs executable, DLL vs DLL). It is because, as far as I understand the concept, the "export" mechanism you mentioned is implemented in the DLL object scope (probably using some static 'local' dictionary instantiated in the DLL scope or the executable scope, depending of the user). You can imagine such case: your base class is registered in the DLL but your Daughter class is registered at the level of the executable object scope : thus there is no way to make them communicate about their relationship. Both dictionaries (one per instantiation unit) acts in different words and know only one subset of the serializable classes to be used in your executable.

The solution for me was to make sure that ALL my serialization code, together with the associated export stuff mechanism ,is instantiated in one and only one place: the executable. At least, I do not pollute my DLLs with some serialization related instantiations that may be I won't use in some of my other executables.

In your case, I think you could step back to the previous approach where all serialization stuff is instantiated in the executable. The alternative is to probably to move your xml_archive ser/deser code in some resource implemented in the DLL and promote some reader and writer classes for XML archives as an element of your DLL, the later being the unique place to safely embed the problematic bits. I'm afraid there is not intermediate.

You may find this "feature" annoying or even disappointing. However, it is not a bug ! It is a conceptual choice made by Robert for the Serialization lib. It is not as bad as you could think at a first glance. IMHO it is a good design not to have a central export registration 'manager'. Otherwise, such a beast would encourage developpers to add large amount of instantiated serialization/export code in any DLL for large collection of serializable classes, with some consequence to possibly end up with lots of unused serialization code being linked for nothing. I also think this would break the "template" spirit and probably could lead to performance issues if thousands of inherited classes from several independant DLLs were registered in one single central place.

With the current implementation of the lib, users are forced to do a careful analysis of what they need from the executable in term of base-pointer oriented serialization. They have to manage to instantiate the only serialization bits strictly necessary for their I/O. The corresponding template code that will be generated is thus minimal.

I recommend you not to use the *EXPORT* macro in your class header file, as well as instantiation code of the template "serialize" methods of your class. Use some kind of .ipp files for that to be included from your main program when it makes sense, and not by default in your DLL. I find this technique rather sane (of course it needs extra care, but that does not hurt, isn't it ?). Note also that if you don't want to use some "my_serialization.ipp" file, you can provide an alternative one, with different implementation of the template serialization code.

I hope you find find these comments useful. Note I'm not completely sure they make sense in your situation. In the absence of more details, it is only a guess, probably not very understandable... sorry for that. Also I'm not sure to understand well the pb. I probably have a rather partial view on it.

Hope some gurus will be able to raise the tone of this technical debate. I would enjoy to read more about this topic.

cheers

frc --

...

Forgive me if the problem is too obvious, this is my first time using Boost and I'm still a big noob in c++.

Here is my code :

IN THE MAIN EXECUTION :

Class Base (parent class) :

Base.h : #pragma once; #include "boost\serialization\access.hpp" #include <boost\serialization\nvp.hpp>

class Base { private: friend class boost::serialization::access; template void serialize(Archive & ar, const unsigned int version){} public: int a, b; Base(); virtual ~Base() { } virtual void func()= 0; // this is an abstract base class }

Base.cpp:

#include "StdAfx.h" #include "Base.h"

Base::Base(){ a=0; b=0; }

IN THE DLL (loaded explicitely)

Class Plugin (Child Class) :

Plugin.h:

#pragma once #include "boost\serialization\access.hpp" #include <boost\serialization\nvp.hpp> #include "boost\serialization\export.hpp" #include "Base.h"

class Plugin: public Base { private:

friend class boost::serialization::access; template void serialize(Archive & ar, const unsigned int version){ ar & BOOST_SERIALIZATION_BASE_OBJECT_NVP(Base); ar & BOOST_SERIALIZATION_NVP(a); ar & BOOST_SERIALIZATION_NVP(b); }

public: virtual void func(); };

BOOST_CLASS_EXPORT_GUID(Plugin, "Plugin")

Plugin.cpp:

#include "StdAfx.h" #include "Plugin.h"

void Plugin::func(){ a= 5; b= 6; }

THE MAIN FUNCTION (IN THE MAIN EXECUTION):

.......................................

HINSTANCE hinst= LoadLibrary(L"monDLLdynamique.dll"); if (hinst) { CreatePlugin ctor= (CreatePlugin)GetProcAddress(hinst, "CreatePlugin"); Base *test= ctor();

std::ofstream ofs3("FichierPointeursDerivedDLL.xml"); { boost::archive::xml_oarchive oa(ofs3); oa << BOOST_SERIALIZATION_NVP(test); } ofs3.close();

delete test; // calls Plugin's dtor

FreeLibrary(hinst); }

............................................

Any help will be very very appreciated !!! Thanks

-- View this message in context: http://boost.2283326.n4.nabble.com/Boost-Serialization-Problem- serializing-derived-class-declared-in-DLL-tp3428913p3428913.html Sent from the Boost - Users mailing list archive at Nabble.com. _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

-- François Mauger Groupe "Interactions Fondamentales et Nature du Neutrino" NEMO-3/SuperNEMO Collaboration LPC Caen-CNRS/IN2P3-UCBN-ENSICAEN Département de Physique -- Université de Caen Basse-Normandie Adresse/address: Laboratoire de Physique Corpusculaire de Caen (UMR 6534) ENSICAEN 6, Boulevard du Marechal Juin 14050 CAEN Cedex FRANCE Courriel/e-mail: mauger@lpccaen.in2p3.fr Tél./phone: 02 31 45 25 12 / (+33) 2 31 45 25 12 Fax: 02 31 45 25 49 / (+33) 2 31 45 25 49

_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

---

Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

Dear Boosters, The use of Boost/Serialization through pointers to objects of an abstract derived class across several DLLs has been addressed a couple of times so far in this list. Some very useful comments have been added in the doc by Robert, particularly concerning the "export/registration" mechanism. After investigation and careful reading of the docs and using some comments from this list, I now have what I consider as a minimal use case implementing: - a DLL `A' with an abstract 'base' class and some inherited classes, all serializable. - a DLL `B' with more inherited serializable class. The purpose of this exercise is to demonstrate that it is possible to pre-build serialization code across several DLLs and not only in the executable unit (which is already known to work well). I really need such a system for my project(s) but it is clear from users' requests on the list that other people would like to benefit of such an approach. Basically, the bits attached in the zip file work (under Linux/gcc, 4.X and Boost 1.44) BUT there is still one problem: one of the sample executable badly segfaults while terminating, probably at some singleton termination (however it is just a feeling). Here is a GDB dump:

...

...

from /scratch/sw/boost/install-1_44_0-Linux-i686-gcc45/lib/libboost_serialization.so.1.44.0 #6 0x001aac6b in boost::serialization::void_cast_detail::void_caster_primitive::~void_caster_primitive() () from ./lib/libA.so #7 0x001aaced in boost::serialization::detail::singleton_wrapper >::~singleton_wrapper() () from ./lib/libA.so #8 0x00389e14 in __cxa_finalize () from /lib/i386-linux-gnu/libc.so.6 #9 0x001a02b4 in __do_global_dtors_aux () from ./lib/libA.so #10 0x001adab0 in _fini () from ./lib/libA.so #11 0x0011ec3d in ?? () from /lib/ld-linux.so.2 #12 0x00389a6f in ?? () from /lib/i386-linux-gnu/libc.so.6 #13 0x00389acf in exit () from /lib/i386-linux-gnu/libc.so.6 #14 0x00370e3f in __libc_start_main () from /lib/i386-linux-gnu/libc.so.6 #15 0x08048b41 in _start () <<<

The funny thing (however likely relevant) is that this program does NOT use serialization; it is just linked against DLLs with embedded serialization code. It encounters some kind of side-effect. All the executable that use serialization code from both DLLs work fine and we get the proper serialization/deserialization actions. Also this problem does not occur when the inheritance scheme of serializable class has only one level (see README in the zip file). As it is a rather complex problem which turns to be only demonstrated given a full multi-DLLs skeleton package, the attached ZIP provides all the source code and a README file that explains how the DLLs are designed and used. There are build instructions and scripts too (for Linux). I hope some of you will find a little time to explore this use case (the code is very simple) and will be able to propose a solution or at least some hint to fix this problem. Thanks a lot. Regards frc -- François Mauger Groupe "Interactions Fondamentales et Nature du Neutrino" NEMO-3/SuperNEMO Collaboration LPC Caen-CNRS/IN2P3-UCBN-ENSICAEN Département de Physique -- Université de Caen Basse-Normandie Adresse/address: Laboratoire de Physique Corpusculaire de Caen (UMR 6534) ENSICAEN 6, Boulevard du Marechal Juin 14050 CAEN Cedex FRANCE Courriel/e-mail: mauger@lpccaen.in2p3.fr Tél./phone: 02 31 45 25 12 / (+33) 2 31 45 25 12 Fax: 02 31 45 25 49 / (+33) 2 31 45 25 49

Hi, Thank you Robert for the fast reply and the hint. I've just checkout the trunk and built it. Now I link against 1.47 and my use case ran without apparent problem. Of course it needs more tests to validate a production approach for my DLLs but it is very encouraging after weeks/months of struggle to setup this multi DLL approach. It seems the code that you have modified/fixed is the 'void_caster::recursive_unregister()' method in 'void_cast.cpp' where there was some possibly double deletion occurence, depending on how the set was filled. If I copy the fix in the 1.44 source, is it supposed to work, or is there other stuff to be modified ? frc --

...

...

On 13/06/2011 18:42, Robert Ramey wrote: François Mauger wrote: Basically, the bits attached in the zip file work (under Linux/gcc, 4.X and Boost 1.44) BUT there is still one problem: one of the sample executable badly segfaults while terminating, probably at some singleton termination (however it is just a feeling).

Here is a GDB dump:

...

...

...

from /scratch/sw/boost/install-1_44_0-Linux-i686-gcc45/lib/libboost_serialization.so.1.44.0 #6 0x001aac6b in boost::serialization::void_cast_detail::void_caster_primitive::~void_caster_primitive() () from ./lib/libA.so #7 0x001aaced in boost::serialization::detail::singleton_wrapper >::~singleton_wrapper() () from ./lib/libA.so #8 0x00389e14 in __cxa_finalize () from /lib/i386-linux-gnu/libc.so.6 #9 0x001a02b4 in __do_global_dtors_aux () from ./lib/libA.so #10 0x001adab0 in _fini () from ./lib/libA.so #11 0x0011ec3d in ?? () from /lib/ld-linux.so.2 #12 0x00389a6f in ?? () from /lib/i386-linux-gnu/libc.so.6 #13 0x00389acf in exit () from /lib/i386-linux-gnu/libc.so.6 #14 0x00370e3f in __libc_start_main () from /lib/i386-linux-gnu/libc.so.6 #15 0x08048b41 in _start () <<<

We have recently made some some small changes which I believe may address this. The changes have been checked into the trunk and release branches so should appear in 1.47. It would be valueable for you to test the most recent release version to see if I'm correct on this.

...

The funny thing (however likely relevant) is that this program does NOT use serialization; it is just linked against DLLs with embedded serialization code. It encounters some kind of side-effect. All the executable that use serialization code from both DLLs work fine and we get the proper serialization/deserialization actions. Also this problem

Correct. This is an effect of "registering" the derived types which occurs whenever the DLL is loaded. Since the process of loading a DLL is the same regardless of whether the included functions are called or not, this problem would still manifest itself.

...

does not occur when the inheritance scheme of serializable class has only one level (see README in the zip file).

As it is a rather complex problem which turns to be only demonstrated given a full multi-DLLs skeleton package, the attached ZIP provides all the source code and a README file that explains how the DLLs are designed and used. There are build instructions and scripts too (for Linux).

I think we're ahead of you on this. Turns out that this area is a minefield which is much more subtle than it first appears. On the upside, several good people have taken an interest and progress is (slowly) being made.

Robert Ramey

_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

-- François Mauger Groupe "Interactions Fondamentales et Nature du Neutrino" NEMO-3/SuperNEMO Collaboration LPC Caen-CNRS/IN2P3-UCBN-ENSICAEN Département de Physique -- Université de Caen Basse-Normandie Adresse/address: Laboratoire de Physique Corpusculaire de Caen (UMR 6534) ENSICAEN 6, Boulevard du Marechal Juin 14050 CAEN Cedex FRANCE Courriel/e-mail: mauger@lpccaen.in2p3.fr Tél./phone: 02 31 45 25 12 / (+33) 2 31 45 25 12 Fax: 02 31 45 25 49 / (+33) 2 31 45 25 49

On 14/06/2011 01:37, Robert Ramey wrote:

François Mauger wrote:

...

Hi,

Thank you Robert for the fast reply and the hint. I've just checkout the trunk and built it. Now I link against 1.47 and my use case ran without apparent problem. Of course it needs more tests to validate a production approach for my DLLs but it is very encouraging after weeks/months of struggle to setup this multi DLL approach. Was all this effort related to boost serialization or all the "other" stuff which appears when one does this?

Well, I've used Boost/Serialization since 2007 in a small experimental physics project (however it implied ~TB of experimental data). I was really impressed by the elegance of its approach and implementation. Since 2010, I've tried to generalize the use of B/S in a larger software framework which implies a hierarchy of ~50 serializable classes spreaded across ~10 DLLs. Using 6 different archives (official text/xml/portable binary from Christian Pfligersdorffer + Nan/inf float support) and dozens of serializable classes, it turned out that we cannot go on with the 'single unit instantiation' approach. It is too expensive while compiling the executable, particularly during the development cycle when each minute means a rebuilt ! So making the multi-DLL precompiled serialization code is a must ! It really fastens the build and ease of our libs. It should now be possible to add serialization features to any class in a DLL, with a suitable integration protocol in our framework. So really yes I fought against boost serialization to use it in this way. I was quite desesperate because whatever way I used during the last months has led to broken execution at some points. At the very beginning, I must admit my errors and misunderstanding of some "golden rules" (export...). But at the end, even following strictly the "rules" were of no help to suppress the nasty (and last) effect I mentionned.

...

It seems the code that you have modified/fixed is the 'void_caster::recursive_unregister()' method in 'void_cast.cpp' where there was some possibly double deletion occurence, depending on how the set was filled.

If I copy the fix in the 1.44 source, is it supposed to work, or is there other stuff to be modified ?

In this particular instance, it's just the changes in 'void_caster::recursive_unregister()'

BTW, credit for fixing this incredible obscure and difficult to find bug goes to ...Aaron Barany. I was stuck with this until he fixed it.

Had a look on the fix and it is clearly a very subtle effect, difficult to understand and track. I think here we will have a toast for Aaron ASAP ! ;-) Thanks again for your help and work. frc -- François Mauger Groupe "Interactions Fondamentales et Nature du Neutrino" NEMO-3/SuperNEMO Collaboration LPC Caen-CNRS/IN2P3-UCBN-ENSICAEN Département de Physique -- Université de Caen Basse-Normandie Adresse/address: Laboratoire de Physique Corpusculaire de Caen (UMR 6534) ENSICAEN 6, Boulevard du Marechal Juin 14050 CAEN Cedex FRANCE Courriel/e-mail: mauger@lpccaen.in2p3.fr Tél./phone: 02 31 45 25 12 / (+33) 2 31 45 25 12 Fax: 02 31 45 25 49 / (+33) 2 31 45 25 49