[Serialization] Serializing classes defined in DLLs
Hi, How do I register classes that are defined in a dynamically loaded so/dll? Using BOOST_CLASS_EXPORT just causes "terminate called after throwing an instance of 'boost::archive::archive_exception' what(): unregistered void cast". My set-up is as follows: class A { public: A(); virtual ~A() {} private: // polymorphic archive stuff here }; class B : public A {} BOOST_CLASS_EXPORT(B); Serializing instances of B through a A* works fine if B is defined in the main executable. Now, I have tried the following (approximately): // An exported function that returns a void pointer typedef void* (*func)(); int main() { // Load a dynamic library dll lib = load_dll("mylib.dll"); // Get a pointer to the instance creator function func instance_creator = (func)lib.resolve("instance_creator"); // An instance of B is created in the dll and returned as a void*. A* a = (A*)instance_creator(); ... archive & a; //throws unregistered void cast } B is both defined and exported in mylib.dll. So, what is the deal? I don't understand the soul of BOOST_CLASS_EXPORT well enough to solve the problem myself. But since I'm using the dll version of the serialization library, I'd assume that the dynamically loaded dll (mylib.dll) and the main executable use the same address space for the archive classes. I would understand this if I encountered this problem with static linking because I'd need to inlude two copies of the serialization library... Any advice? Regards, Topi
I know people have been doing this. But I also know there have been issues with the dynamic loading and unloading of modules which contain serialization code. I really need a good small test program to be added to my test suite. I would expect the following to work; Your example isn't realy complete enough for me to comment on. Robert Ramey Topi Mäenpää wrote:
Hi,
How do I register classes that are defined in a dynamically loaded so/dll? Using BOOST_CLASS_EXPORT just causes "terminate called after throwing an instance of 'boost::archive::archive_exception' what(): unregistered void cast". My set-up is as follows:
class A { public: A(); virtual ~A() {} private: // polymorphic archive stuff here };
class B : public A {}
BOOST_CLASS_EXPORT(B);
Serializing instances of B through a A* works fine if B is defined in the main executable. Now, I have tried the following (approximately):
// An exported function that returns a void pointer typedef void* (*func)();
int main() { // Load a dynamic library dll lib = load_dll("mylib.dll"); // Get a pointer to the instance creator function func instance_creator = (func)lib.resolve("instance_creator"); // An instance of B is created in the dll and returned as a void*. A* a = (A*)instance_creator();
...
archive & a; //throws unregistered void cast }
B is both defined and exported in mylib.dll.
So, what is the deal? I don't understand the soul of BOOST_CLASS_EXPORT well enough to solve the problem myself. But since I'm using the dll version of the serialization library, I'd assume that the dynamically loaded dll (mylib.dll) and the main executable use the same address space for the archive classes. I would understand this if I encountered this problem with static linking because I'd need to inlude two copies of the serialization library...
Any advice?
Regards, Topi
Had you defined BOOST_SERIALIZATION_DYN_LINK before including serialization
headers? 1.33 version implicitly links (via # pragma comment(lib, )) static
version of serialization library by default. Also, you can try to rename
static version of library to ensure that it is not used in linking. I'd got
similar strange errors before doing so.
"Topi Mäenpää"
Hi,
How do I register classes that are defined in a dynamically loaded so/dll? Using BOOST_CLASS_EXPORT just causes "terminate called after throwing an instance of 'boost::archive::archive_exception' what(): unregistered void cast". My set-up is as follows:
class A { public: A(); virtual ~A() {} private: // polymorphic archive stuff here };
class B : public A {}
BOOST_CLASS_EXPORT(B);
Serializing instances of B through a A* works fine if B is defined in the main executable. Now, I have tried the following (approximately):
// An exported function that returns a void pointer typedef void* (*func)();
int main() { // Load a dynamic library dll lib = load_dll("mylib.dll"); // Get a pointer to the instance creator function func instance_creator = (func)lib.resolve("instance_creator"); // An instance of B is created in the dll and returned as a void*. A* a = (A*)instance_creator();
...
archive & a; //throws unregistered void cast }
B is both defined and exported in mylib.dll.
So, what is the deal? I don't understand the soul of BOOST_CLASS_EXPORT well enough to solve the problem myself. But since I'm using the dll version of the serialization library, I'd assume that the dynamically loaded dll (mylib.dll) and the main executable use the same address space for the archive classes. I would understand this if I encountered this problem with static linking because I'd need to inlude two copies of the serialization library...
Any advice?
Regards, Topi
In your dll, are you explicitly instantiating template code for B? This
looks like the problem to me.
You'll have to use something like:
#include
Hi,
How do I register classes that are defined in a dynamically loaded so/dll? Using BOOST_CLASS_EXPORT just causes "terminate called after throwing an instance of 'boost::archive::archive_exception' what(): unregistered void cast". My set-up is as follows:
class A { public: A(); virtual ~A() {} private: // polymorphic archive stuff here };
class B : public A {}
BOOST_CLASS_EXPORT(B);
Serializing instances of B through a A* works fine if B is defined in the main executable. Now, I have tried the following (approximately):
// An exported function that returns a void pointer typedef void* (*func)();
int main() { // Load a dynamic library dll lib = load_dll("mylib.dll"); // Get a pointer to the instance creator function func instance_creator = (func)lib.resolve("instance_creator"); // An instance of B is created in the dll and returned as a void*. A* a = (A*)instance_creator();
...
archive & a; //throws unregistered void cast }
B is both defined and exported in mylib.dll.
So, what is the deal? I don't understand the soul of BOOST_CLASS_EXPORT well enough to solve the problem myself. But since I'm using the dll version of the serialization library, I'd assume that the dynamically loaded dll (mylib.dll) and the main executable use the same address space for the archive classes. I would understand this if I encountered this problem with static linking because I'd need to inlude two copies of the serialization library...
Any advice?
Regards, Topi
Hi, I think I resolved the issue. The problem is that if the base class A is defined in the main executable, the dynamically loaded DLL apparently uses a different address space for the definition. Even if I return a pointer to A from the DLL, I get an unregistered void cast exception. (I don't know why. Anyone?) If I define the base class in yet another DLL that both the main program and the dynamically loaded DLL link to, serialization works just fine. Regarding explicit instantiation, I am using polymorphic archives and I'm registering the class after including the polymorphic archive headers. By the way, the serialization library has trouble compiling with gcc 3.4.1 on Mandrakelinux 10.1. For some weird reason, namespace resolution within boost::serialization and its sub-namespaces does not fall back to the global scope. For example, the following doesn't compile within the library: namespace boost { namespace serialization { namespace access { boost::serialization::make_nvp(...); }}} I needed to convert all occurrences of "boost::" to "::boost::". -Topi- Robert Ramey wrote:
In your dll, are you explicitly instantiating template code for B? This looks like the problem to me.
You'll have to use something like:
#include
#include ... // all the archives you expect to use #include
// B.hpp includes BOOST_CLASS_EXPORT(B) in order to ensure that the required code is generated for the archives you expect to use.
Now if you don't want to include serialization code in your dll for a specific set of archive types, you can use polymorphic archive in the DLL. This will use one set of precompiled code (w/o templates) with any of the polymorphic_?archive.hpp used in main.
Robert Ramey
Topi Mäenpää wrote:
Hi,
How do I register classes that are defined in a dynamically loaded so/dll? Using BOOST_CLASS_EXPORT just causes "terminate called after throwing an instance of 'boost::archive::archive_exception' what(): unregistered void cast". My set-up is as follows:
class A { public: A(); virtual ~A() {} private: // polymorphic archive stuff here };
class B : public A {}
BOOST_CLASS_EXPORT(B);
Serializing instances of B through a A* works fine if B is defined in the main executable. Now, I have tried the following (approximately):
// An exported function that returns a void pointer typedef void* (*func)();
int main() { // Load a dynamic library dll lib = load_dll("mylib.dll"); // Get a pointer to the instance creator function func instance_creator = (func)lib.resolve("instance_creator"); // An instance of B is created in the dll and returned as a void*. A* a = (A*)instance_creator();
...
archive & a; //throws unregistered void cast }
B is both defined and exported in mylib.dll.
So, what is the deal? I don't understand the soul of BOOST_CLASS_EXPORT well enough to solve the problem myself. But since I'm using the dll version of the serialization library, I'd assume that the dynamically loaded dll (mylib.dll) and the main executable use the same address space for the archive classes. I would understand this if I encountered this problem with static linking because I'd need to inlude two copies of the serialization library...
Any advice?
Regards, Topi
------------------------------------------------------------------------
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
-- Topi Mäenpää CTO, Intopii +358 40 774 7749
Topi Mäenpää wrote:
Hi,
I think I resolved the issue. The problem is that if the base class A is defined in the main executable, the dynamically loaded DLL apparently uses a different address space for the definition. Even if I return a pointer to A from the DLL, I get an unregistered void cast exception. (I don't know why. Anyone?) If I define the base class in yet another DLL that both the main program and the dynamically loaded DLL link to, serialization works just fine.
This is very odd to me - how does B compile in the DLL without This is still a murkey area. I believe there are still unresolved issues here.
If A is in
Regarding explicit instantiation, I am using polymorphic archives and I'm registering the class after including the polymorphic archive headers.
By the way, the serialization library has trouble compiling with gcc 3.4.1 on Mandrakelinux 10.1. For some weird reason, namespace resolution within boost::serialization and its sub-namespaces does not fall back to the global scope. For example, the following doesn't compile within the library:
namespace boost { namespace serialization { namespace access { boost::serialization::make_nvp(...); }}}
I needed to convert all occurrences of "boost::" to "::boost::".
Hmm - maybe it would have been better to use: namespace boost { namespace serialization { namespace access { make_nvp(...); }}} Robert Ramey
I think I resolved the issue. The problem is that if the base class A is defined in the main executable, the dynamically loaded DLL apparently uses a different address space for the definition. Even if I return a pointer to A from the DLL, I get an unregistered void cast exception. (I don't know why. Anyone?) If I define the base class in yet another DLL that both the main program and the dynamically loaded DLL link to, serialization works just fine.
This is very odd to me - how does B compile in the DLL without
This is still a murkey area. I believe there are still unresolved issues here.
I am also encountered the same problem yesterday. The source of evil seems to be in double registration if rtti-enabled class serialized both in dll and in exe. void extended_type_info::self_register() just called twice - from exe and from dll. Later, code such const boost::serialization::extended_type_info * this_type = boost::serialization::type_info_implementation<T>::type::get_instance(); const boost::serialization::extended_type_info * true_type = boost::serialization::type_info_implementation<T>::type::get_derived_extended_type_info(t); if(*this_type == *true_type){... works wrong because extended_type_info::operator=(const extended_type_info &rhs) uses just an address of extended_type_info, which can be different in such situation. I had attached sample files illustrating problem.
This problem has been reported by Martin Ecker. The easy fix is to just permit multiple definitions of the same virtual functions in different DLLs. This has eliminated the symptom for this user. I'm concerned that this won't really work in the general case. I would be prefer to: a) trap attempts at multiple registrations of the same class as errors. b) explain how to avoid these situtations in the first place. I'm kind of stuck on b). It seems to me that if A (base) is in a DLL and B(derived) is in an executable, there should be no multiple definitions of B. But the brief description of this user's case seems to suggest that this isn't the case - though without more information one can't be sure. Robert Ramey Sergey Skorniakov wrote:
I think I resolved the issue. The problem is that if the base class A is defined in the main executable, the dynamically loaded DLL apparently uses a different address space for the definition. Even if I return a pointer to A from the DLL, I get an unregistered void cast exception. (I don't know why. Anyone?) If I define the base class in yet another DLL that both the main program and the dynamically loaded DLL link to, serialization works just fine.
This is very odd to me - how does B compile in the DLL without
This is still a murkey area. I believe there are still unresolved issues here.
I am also encountered the same problem yesterday. The source of evil seems to be in double registration if rtti-enabled class serialized both in dll and in exe. void extended_type_info::self_register() just called twice - from exe and from dll. Later, code such const boost::serialization::extended_type_info * this_type = boost::serialization::type_info_implementation<T>::type::get_instance(); const boost::serialization::extended_type_info * true_type = boost::serialization::type_info_implementation<T>::type::get_derived_extended_type_info(t); if(*this_type == *true_type){...
works wrong because extended_type_info::operator=(const extended_type_info &rhs) uses just an address of extended_type_info, which can be different in such situation. I had attached sample files illustrating problem.
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
I got the dynamic library stuff work for a while. But now I'm getting a segmentation fault with the following setup. As before, I have class A defined in a dll, say A.dll. This is a library I link against in other dlls and in the main executable. I also have two other libraries, B.dll and C.dll that both include definitions for many classes derived from A. Let us call these B1, B2, ... and C1, C2 ..., respectively. I'm registering the classes in the dlls with BOOST_CLASS_EXPORT. B.dll and C.dll use the definition of A in A.dll so that only a single definition of A should be present. The main executable links against A.dll and loads B.dll and C.dll at run time. With this setup, serialization goes fine if I only use classes defined in either of the dynamically loaded libraries: // create_in_dynamic_library_{b,c} are function pointers to dll // functions A* b1 = (A*)create_in_dynamic_library_b("B1"); A* b2 = (A*)create_in_dynamic_library_b("B2"); A* c1 = (A*)create_in_dynamic_library_c("C1"); A* c2 = (A*)create_in_dynamic_library_c("C2"); ar & b1 & b2; //works fine ar & c1 & c2; //works fine ar & b1 & c1; //crashes ar & c2 & b2; //crashes In other words, I cannot serialize classes defined and exported in different dlls. Only the first one works. The segmentation fault is caused by line 213 in extended_type_info.cpp. extended_type_info::type_info_key_cmp(const extended_type_info & rhs) const { if(type_info_key == rhs.type_info_key) // <---- this == 0! return 0; My debugger reports that "this" is zero. This happens just after boost::serialization::base_object<A>(*this) is called in the serialize member of class {B,C}{1...N}. I'm stuck. Any suggestions? -Topi- Robert Ramey wrote:
This problem has been reported by Martin Ecker. The easy fix is to just permit multiple definitions of the same virtual functions in different DLLs. This has eliminated the symptom for this user.
I didn't quite get this. What do you mean by permitting multiple definitions? How is it done?
I'm concerned that this won't really work in the general case.
I would be prefer to: a) trap attempts at multiple registrations of the same class as errors. b) explain how to avoid these situtations in the first place.
I'm kind of stuck on b). It seems to me that if A (base) is in a DLL and B(derived) is in an executable, there should be no multiple definitions of B. But the brief description of this user's case seems to suggest that this isn't the case - though without more information one can't be sure.
Robert Ramey
It looks to me that A.dll is being unloaded "too soon" or that A.dll is being loaded mulitple times. If you trap on the ctor / dtor code of extended_type_info you might be able to see what's going on. If A.DLL is being loaded automatically on using B.DLL or C.DLL it may well be unloaded automatically "too" soon. Or they maybe multiple copies and the "wrong one" gets dropped from the global tables when one DLL is unloaded. As I've said before - this is a murky area. I may experiment with if I can find the time but I need a good (complete self-contained) test case. Robert Ramey Topi Mäenpää wrote:
I got the dynamic library stuff work for a while. But now I'm getting a segmentation fault with the following setup.
As before, I have class A defined in a dll, say A.dll. This is a library I link against in other dlls and in the main executable. I also have two other libraries, B.dll and C.dll that both include definitions for many classes derived from A. Let us call these B1, B2, ... and C1, C2 ..., respectively. I'm registering the classes in the dlls with BOOST_CLASS_EXPORT. B.dll and C.dll use the definition of A in A.dll so that only a single definition of A should be present. The main executable links against A.dll and loads B.dll and C.dll at run time.
With this setup, serialization goes fine if I only use classes defined in either of the dynamically loaded libraries:
// create_in_dynamic_library_{b,c} are function pointers to dll // functions
A* b1 = (A*)create_in_dynamic_library_b("B1"); A* b2 = (A*)create_in_dynamic_library_b("B2"); A* c1 = (A*)create_in_dynamic_library_c("C1"); A* c2 = (A*)create_in_dynamic_library_c("C2");
ar & b1 & b2; //works fine ar & c1 & c2; //works fine
ar & b1 & c1; //crashes ar & c2 & b2; //crashes
In other words, I cannot serialize classes defined and exported in different dlls. Only the first one works. The segmentation fault is caused by line 213 in extended_type_info.cpp.
extended_type_info::type_info_key_cmp(const extended_type_info & rhs) const { if(type_info_key == rhs.type_info_key) // <---- this == 0! return 0;
My debugger reports that "this" is zero. This happens just after boost::serialization::base_object<A>(*this) is called in the serialize member of class {B,C}{1...N}.
I'm stuck. Any suggestions?
-Topi-
Robert Ramey wrote:
This problem has been reported by Martin Ecker. The easy fix is to just permit multiple definitions of the same virtual functions in different DLLs. This has eliminated the symptom for this user.
I didn't quite get this. What do you mean by permitting multiple definitions? How is it done?
I'm concerned that this won't really work in the general case.
I would be prefer to: a) trap attempts at multiple registrations of the same class as errors. b) explain how to avoid these situtations in the first place.
I'm kind of stuck on b). It seems to me that if A (base) is in a DLL and B(derived) is in an executable, there should be no multiple definitions of B. But the brief description of this user's case seems to suggest that this isn't the case - though without more information one can't be sure.
Robert Ramey
Hi, I have debugged the problem for hours and found a partial solution. Don't know why this works and whether it finally will solve the problem but at least it is one step further. It seems that at least part of the problem is with pure virtual functions. If there is no implementation of a pure virtual function within the dynamic library that defines A and A the pure virtual function is defined in a base class of A, the serialization library crashes. Let's assume A has yet another base class "ABase": class ABase { protected: virtual void purevirtual() = 0; }; class A : public ABase { ... }; If neither A nor any other class between A and ABase in the inheritance hierarchy implements the function, dynamically loaded derived classes won't serialize. If the function is implemented, everything is fine. The funny thing is that if the pure virtual function is declared in A, serialization will also work. I have tried the following set-ups. P denotes a pure virtual function and V a virtual function that has a definition. N means no definition. Class B is defined in a dynamically loaded DLL. Arrows denote inheritance. The problem is in serializing B (* see note). Dynamic library boundary ________ | V A (P) -> B (V) ABase (P) -> A (V) -> B (V) ABase (P) -> ABase2 (P) -> A (V) -> B (V) ABase (P) -> ABase2 (V) -> A (N) -> B (V) These all work. If there is a definition for the virtual function or the inheritance hierarchy is only one level deep, serializing B works. The following set-ups cause a crash: ABase (P) -> A (P) -> B (V) ABase (P) -> ABase2 (P) -> A (P) -> B (V) I kind of hope this helps someone in debugging the problem. I'm still not too familiar with the internals of the serialization library... Oh, and I have tried BOOST_IS_ABSTRACT. It has no effect on my compiler (gcc 3.4.1). * In all the aforementined cases, serializing just B works fine. But once another similar class (say C) is fetched from another DLL, the library crashes as described in a previous posting. I also noticed that it isn't necessary to load both B and C from DLLs. The same problem occurs if B is in the main executable and C is in a DLL. The problem seems to be in crossing a dynamic library boundary in any way. -Topi- Robert Ramey wrote:
It looks to me that A.dll is being unloaded "too soon" or that A.dll is being loaded mulitple times. If you trap on the ctor / dtor code of extended_type_info you might be able to see what's going on.
If A.DLL is being loaded automatically on using B.DLL or C.DLL it may well be unloaded automatically "too" soon. Or they maybe multiple copies and the "wrong one" gets dropped from the global tables when one DLL is unloaded.
As I've said before - this is a murky area. I may experiment with if I can find the time but I need a good (complete self-contained) test case.
Robert Ramey
Another post has me convinced that the problem is that there are multiple copies of a base class extended_type_info record which I believe shouldn't occur. If you trap your debugger the serialization.dll (shared_lib) at line 58 of extened_type_info.cpp you should trap every time an extended_type_info record is "self_registered". Examination of the stack should reveal where this is being called from. I expect that the base class is being self registered from both the DLL as well as the main executable. My current thinking is that this behavior should be prohibited and detected with an exception. Other users have addressed this by tweaking the code in extended_type_info.cpp to permit multiple "self registration" of the same extended_type_info type. This eliminated their symptoms and everything seems to work. But from my understanding of the library, I believe this just hides a deeper problem. Without seeing a test program - which no one has been able to deliver, I'm just speculatiing of course. Good Luck, Robert Ramey
participants (3)
-
Robert Ramey -
Sergey Skorniakov -
Topi Mäenpää