[serialization] Boost.Serialization 1.33.0 Breaks Serialization Code inDLLs
Hello, Boost.Serialization 1.33.0 breaks serialization of derived classes referenced via base class pointers when using DLLs on Windows and VC 7.1. It might also affect shared objects on Unix, but I'm not sure how the various Unix loaders handle multiple instances of the same template member function in multiple shared objects. In particular what happens is that objects that are serialized via a base class pointer are no longer deserialized correctly when deserialized in an application that uses the DLL (and not within the DLL itself). Instead they are deserialized as NULL pointers. The problem is that the virtual equal_to and not_equal_to member functions were removed from the extended_type_info base class. These functions were previously used by extended_type_info::operator == and extended_type_info::operator !=. Now operator == simply performs an equal check on the address of the extended_type_info object, whereas the previous version that used equal_to actually did a full compare of the stored type info keys. If DLLs are used and the DLLs themselves contain serialization code and BOOST_CLASS_EXPORT macros, this will fail because the extended_type_info objects are always static objects allocated on the stack of a member template function, e.g. in extended_type_info_typeid_1::get_instance. In a DLL situation this will cause multiple DLLs that include the header file with that template function to have multiple (i.e. separate) copies of that extended_type_info object. Thus, comparing them only via their addresses leads to disaster. The fix is easy. Simply put the old code for extended_type_info::operator == and extended_type_info::operator != back in. Note that I haven't looked any further if there were other changes made to the code that also rely on the addresses of equal extended_type_info objects to be the same. This will also break serialization code in DLLs. I can only say that after I put the old code back in, our system could deserialize its data properly again. Best Regards, Martin TAB Austria Haiderstraße 40 4052 Ansfelden Austria Phone: +43 7229 78040-218 Fax: +43 7229 78040-209 E-mail: martin.ecker@tab.at http://www.tab.at
As usual a very complete analysis from martin ecker. Its amazing to me that you can understand things at this level of detail. I had concluded that there would never be more than one instance of a extended_type_info object created for the same type. So it would be more efficient to just compare the addresses. Now I see (I think) how this assumption can be wrong. I don't see an easier fix than putting back the old code. I would like to add a couple of new tests to test this kind of thing. You don't have anything suitable for this do you? Robert Ramey martin.ecker@tab.at wrote:
Hello,
Boost.Serialization 1.33.0 breaks serialization of derived classes referenced via base class pointers when using DLLs on Windows and VC 7.1. It might also affect shared objects on Unix, but I'm not sure how the various Unix loaders handle multiple instances of the same template member function in multiple shared objects.
In particular what happens is that objects that are serialized via a base class pointer are no longer deserialized correctly when deserialized in an application that uses the DLL (and not within the DLL itself). Instead they are deserialized as NULL pointers.
The problem is that the virtual equal_to and not_equal_to member functions were removed from the extended_type_info base class. These functions were previously used by extended_type_info::operator == and extended_type_info::operator !=. Now operator == simply performs an equal check on the address of the extended_type_info object, whereas the previous version that used equal_to actually did a full compare of the stored type info keys.
If DLLs are used and the DLLs themselves contain serialization code and BOOST_CLASS_EXPORT macros, this will fail because the extended_type_info objects are always static objects allocated on the stack of a member template function, e.g. in extended_type_info_typeid_1::get_instance.
In a DLL situation this will cause multiple DLLs that include the header file with that template function to have multiple (i.e. separate) copies of that extended_type_info object. Thus, comparing them only via their addresses leads to disaster.
The fix is easy. Simply put the old code for extended_type_info::operator == and extended_type_info::operator != back in.
Note that I haven't looked any further if there were other changes made to the code that also rely on the addresses of equal extended_type_info objects to be the same. This will also break serialization code in DLLs. I can only say that after I put the old code back in, our system could deserialize its data properly again.
Best Regards, Martin
TAB Austria Haiderstraße 40 4052 Ansfelden Austria Phone: +43 7229 78040-218 Fax: +43 7229 78040-209 E-mail: martin.ecker@tab.at http://www.tab.at
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Hello Robert, Robert Ramey wrote:
I would like to add a couple of new tests to test this kind of thing. You don't have anything suitable for this do you?
Unfortunately, I don't. The only "test case" we have is our entire system itself, which is a bit too big I'm afraid. Basically, it would be beneficial to add a DLL to the test suite and then write the following test cases: - Put serialization code in the DLL for a class A. Then serialize an instance of class A first from code in the DLL and then from an executable that links with the DLL. - Derive a class B from class A in the DLL. Serialize an instance of class B via a base class pointer to class A from code in the DLL and from an executable that links with the DLL. Finally, I'd like to bring your attention to the issue of thread safety. Since Boost.Serialization has a number of global objects, access to them should be safe-guarded by a synchronization primitive, such as a mutex. This is particularly important for us when dynamically loading DLLs. For example, we load and unload DLL plugins during runtime, and we have multiple threads that use Boost.Serialization (potentially also while one thread is currently loading a DLL). Before I added mutexes to all global objects in Boost.Serialization, we had random crashes when one thread was dynamically loading a DLL while another thread was reading/writing a Boost.Serialization archive. This is because the global objects in Boost.Serialization are potentially modified while the DLL is being loaded (e.g. if the DLL that is loaded contains BOOST_CLASS_EXPORT macros somewhere). As far as I can tell, the following global objects need to be protected from simultaneous access from multiple threads: - tkmap - ktmap - void_caster_registry - iserializer_map - oserializer_map Best Regards, Martin TAB Austria Haiderstraße 40 4052 Ansfelden Austria Phone: +43 7229 78040-218 Fax: +43 7229 78040-209 E-mail: martin.ecker@tab.at http://www.tab.at
martin.ecker@tab.at wrote:
Hello Robert,
Robert Ramey wrote:
I would like to add a couple of new tests to test this kind of thing. You don't have anything suitable for this do you?
I'm still considering this. I'm a little bit concerned about what happens when there are multiple instances of extended_type_info for a specific type in the global table. The problem is that when a DLL is unloaded, this table entry is deleted - as now the code to serialize this type is not available. The current code doesn't contemplate that the code so serialize a given type might reside in different DLLS. I believe that was my motivation for presuming that there would only be one. Changing the equal_to to value based rather than address based risks removing the "wrong" entry. This that would leave a vtable pointer is pointing to code that has since been unloaded. So this is still an open question as far as I'm concerned.
Unfortunately, I don't. The only "test case" we have is our entire system itself, which is a bit too big I'm afraid.
Well, you could write one using the tests in the library as a basis - its pretty easy - much easier than getting the test to pass on all the compilers!
Basically, it would be beneficial to add a DLL to the test suite and then write the following test cases: ...
When I have some time I want to do this. I also want to make a plug-in demo. This requires adding a little bit to extended_type_info to function as a class factory.
Finally, I'd like to bring your attention to the issue of thread safety. Since Boost.Serialization has a number of global objects, access to them should be safe-guarded by a synchronization primitive, such as a mutex.
This is something that should be left outside the serialization library. Calls to the library should be wrapped in there own locks. I would expect that the top level entry points to serialization would be few in number and that each of these would have its own locks. I see no need to include locks withing the serialization library itself. Robert Ramey
Hi, Robert Ramey wrote:
I'm still considering this. I'm a little bit concerned about what happens when there are multiple instances of extended_type_info for a specific type in the global table.
When unregistering from the global type registry the type info address should be used. But when comparing type info objects for equality, for example when performing void casts, it shouldn't. I agree that it's not the nicest solution and this should be given some more thought.
Finally, I'd like to bring your attention to the issue of thread safety. Since Boost.Serialization has a number of global objects, access to them should be safe-guarded by a synchronization primitive, such as a mutex.
This is something that should be left outside the serialization library.
Calls to the library should be wrapped in there own locks. I would expect that the top level entry points to serialization would be few in number and that each of these would have its own locks. I see no need to include locks withing the serialization library itself.
I strongly disagree. This requires the client to know implementation details of the library. The library should at least guarantee basic thread safety, so that two threads can use two different archive objects at the same time, which currently is not necessarily the case. If we use dynamically loaded DLLs and use multiple threads to do so, for example, we need to lock _every_ call we make to Boost.Serialization. Given that some of these calls are hidden in static global variables, i.e. behind the BOOST_CLASS_EXPORT macro, it is not even possible for us to perform this locking at such a granular level. How about making this an optional feature via a #define? Adding the locks to the code is quite straightforward. I can provide you with a patch, if desired. Best Regards, Martin TAB Austria Haiderstraße 40 4052 Ansfelden Austria Phone: +43 7229 78040-218 Fax: +43 7229 78040-209 E-mail: martin.ecker@tab.at http://www.tab.at
martin.ecker@tab.at wrote:
Hi,
Robert Ramey wrote:
I'm still considering this. I'm a little bit concerned about what happens when there are multiple instances of extended_type_info for a specific type in the global table.
When unregistering from the global type registry the type info address should be used. But when comparing type info objects for equality, for example when performing void casts, it shouldn't. I agree that it's not the nicest solution and this should be given some more thought.
I very agree here. Another idea is to just prohibit more than one DLL to register an extended_type_info object. This could be trapped. In general, I'm wary of overly tricky solutions to address relatively obscure and uncommon cases. (Though I suppose that might not be apparent from looking at the serialization library).
Finally, I'd like to bring your attention to the issue of thread safety. Since Boost.Serialization has a number of global objects, access to them should be safe-guarded by a synchronization primitive, such as a mutex.
This is something that should be left outside the serialization library.
Calls to the library should be wrapped in there own locks. I would expect that the top level entry points to serialization would be few in number and that each of these would have its own locks. I see no need to include locks withing the serialization library itself.
I strongly disagree. This requires the client to know implementation details of the library. The library should at least guarantee basic thread safety, so that two threads can use two different archive objects at the same time, which currently is not necessarily the case.
I believe that this is the case. The only shared global objects are collections of extended_type_info objects and base/derived registrations (void_cast_primitives). These collections are built/modified only during pre-execution or during loading/unloading of DLLs which export types or void_cast_prmitives. So I believe your concern could be addressed by a code which inhibited loading/unloading of DLLS while any archives are open. I don't think this has to be part of the serialization library.
If we use dynamically loaded DLLs and use multiple threads to do so, for example, we need to lock _every_ call we make to Boost.Serialization. Given that some of these calls are hidden in static global variables, i.e. behind the BOOST_CLASS_EXPORT macro, it is not even possible for us to perform this locking at such a granular level.
I disagree - see above. Robert Ramey
Robert Ramey wrote:
Calls to the library should be wrapped in there own locks. I would expect that the top level entry points to serialization would be few in number and that each of these would have its own locks. I see no need to include locks withing the serialization library itself.
I strongly disagree. This requires the client to know implementation details of the library. The library should at least guarantee basic thread safety, so that two threads can use two different archive objects at the same time, which currently is not necessarily the case.
I believe that this is the case. The only shared global objects are collections of extended_type_info objects and base/derived registrations (void_cast_primitives). These collections are built/modified only during pre-execution or during loading/unloading of DLLs which export types or void_cast_prmitives. So I believe your concern could be addressed by a code which inhibited loading/unloading of DLLS while any archives are open. I don't think this has to be part of the serialization library.
I think you're going contrary to the basic threading assumption in the C++. If I explicitly use the same object from two different threads, I must use locking myself. If, for all appearence, I don't use the same object in two threads, but there's some *internal* shared data structure, it should be protected *internally*. In this case, there's some internal serialization table that I know nothing about, so it should be protected by serialization library.
So I believe your concern could be addressed by a code which inhibited loading/unloading of DLLS while any archives are open. I don't think this has to be part of the serialization library.
To begin with, those internal structures are present even when no archives are open, arent't they? Besides, I don't understand how you can "inhibit loading of DLLS". Say, one thread has created an archive and does something with it. Some other thread tries to open DLL. What should happen? Should that other thread get "sorry, some archives are open" error? And what will it do? Wait until the other thread is done with archive? And what if the other thread uses archive to talk over network and will never close the achrive? And even if it will close archive, how will the other thread wait for that? There's no public method that tells that any archive is open. - Volodya
Vladimir Prus wrote:
I think you're going contrary to the basic threading assumption in the C++. If I explicitly use the same object from two different threads, I must use locking myself.
This is what I disagree with. I don't any problem with multiple threads accessing the same data structures as long as the accesses are read-only and the structures are not being modified.
If, for all appearence, I don't use the same object in two threads, but there's some *internal* shared data structure, it should be protected *internally*. In this case, there's some internal serialization table that I know nothing about, so it should be protected by serialization library.
Repeat, I don't think this is necessary.
So I believe your concern could be addressed by a code which inhibited loading/unloading of DLLS while any archives are open. I don't think this has to be part of the serialization library.
To begin with, those internal structures are present even when no archives are open, arent't they?
True
Besides, I don't understand how you can "inhibit loading of DLLS". Say, one thread has created an archive and does something with it. Some other thread tries to open DLL. What should happen? Should that other thread get "sorry, some archives are open" error? And what will it do?
A couple of possibilities. It could somehow check to see that no archives are open. (The serialization library might have to add support for this). It could throw an exception if an attempt is made to change a global structure when when an archive is open.
Wait until the other thread is done with archive? And what if the other thread uses archive to talk over network and will never close the achrive? And even if it will close archive, how will the other thread wait for that? There's no public method that tells that any archive is open.
Correct, but this could be remedied without dragging threads in to it. There's another aspect here. The code for serialization presumes that the global tables are static during the lifetime of the archive. For example, as a particular item is serialized, its class identifier is saved in an archive specific table. Thus the save code used the object identifier to invoke the save method for the specific class. Same goes for the load side. Now, if the global table changes while an archive is open, all bets are off because the save/load code may not be around anymore. So. a) global tables can't be changed while an archive is open an still expect the system to function. b) thus DLLS should be loaded/unloaded only when there are no archives open. c) respecting this rule is the responsability of the calling program. d) The library (with some reaonsable addition) can detect violations of the above rules. While we're at it, I would also like to consider enforcing a rule that there can be at most one instance of serialization code for a specific type existing at one time. a) it can make the system slightly (but measurably?) more efficiant. b) I'm very uncomfortable with the idea that there might (accidently) two different versions of serialization for a particular type floating around. This could happen if multiple DLLS implement serialization for a specific type. I expect this would be an accident. But even if intentional it would create a situation which would he hellish to debug. This rule could be enforced by the serialization library by throwing an exception. It could also be implemented by using only the first loaded DLL and keeping a counter. (or maybe shared pointer). But that would then require a new rule - that DLLS containing serialization code be unloaded in the reverse order of their loading. OK - lets hear it from someone who is going to say the really need to be able to create multiple different simultaneaous serializations. To summarize, I believe this situation should be handled by adding code to the serialization library which will permit one to query whether any archive is open, and throwing an acceptiion if a global table is modified when any archives are open. Fairly simple, 100% effective, and doesn't require dependency on any other library. Robert Ramey
"Robert Ramey" <ramey@rrsd.com> writes:
Vladimir Prus wrote:
I think you're going contrary to the basic threading assumption in the C++. If I explicitly use the same object from two different threads, I must use locking myself.
This is what I disagree with. I don't any problem with multiple threads accessing the same data structures as long as the accesses are read-only and the structures are not being modified.
Yes, that's the part that Vladimir left out. Of course if there's no modification then no locking is needed.
If, for all appearence, I don't use the same object in two threads, but there's some *internal* shared data structure, it should be protected *internally*. In this case, there's some internal serialization table that I know nothing about, so it should be protected by serialization library.
Repeat, I don't think this is necessary.
Why not? Is there no mutation at all?
OK - lets hear it from someone who is going to say the really need to be able to create multiple different simultaneaous serializations.
All I can say is that serialization and concurrency are natural partners, since serializing/deserializing is commonly used for interprocess communication. Whether it will apply to threads, I can't say. -- Dave Abrahams Boost Consulting www.boost-consulting.com
David Abrahams wrote:
If, for all appearence, I don't use the same object in two threads, but there's some *internal* shared data structure, it should be protected *internally*. In this case, there's some internal serialization table that I know nothing about, so it should be protected by serialization library.
Repeat, I don't think this is necessary.
Why not? Is there no mutation at all?
a) Mutation can only occur when a DLL is loaded or unloaded. b) Archive code presumes that the set of known types does not vary between the time an archive is created and the time it is destroyed. c) Ergo, protecting access to these tables with something like a mutex won't be effective in enforcing b) d) So there is no reason to implement it.
All I can say is that serialization and concurrency are natural partners, since serializing/deserializing is commonly used for interprocess communication. Whether it will apply to threads, I can't say.
No problem here. The problem is dynamic loading and unloading of DLLS which contain serialization code. This has nothing to do with the common usage you describe above. Robert Ramey
Robert Ramey wrote:
Vladimir Prus wrote:
I think you're going contrary to the basic threading assumption in the C++. If I explicitly use the same object from two different threads, I must use locking myself.
This is what I disagree with. I don't any problem with multiple threads accessing the same data structures as long as the accesses are read-only and the structures are not being modified.
If, for all appearence, I don't use the same object in two threads, but there's some *internal* shared data structure, it should be protected *internally*. In this case, there's some internal serialization table that I know nothing about, so it should be protected by serialization library.
Repeat, I don't think this is necessary.
Dave has already commented on this point: if the structures are modified by some thread, you need locking.
Besides, I don't understand how you can "inhibit loading of DLLS". Say, one thread has created an archive and does something with it. Some other thread tries to open DLL. What should happen? Should that other thread get "sorry, some archives are open" error? And what will it do?
A couple of possibilities. It could somehow check to see that no archives are open. (The serialization library might have to add support for this). It could throw an exception if an attempt is made to change a global structure when when an archive is open.
Wait until the other thread is done with archive? And what if the other thread uses archive to talk over network and will never close the achrive? And even if it will close archive, how will the other thread wait for that? There's no public method that tells that any archive is open.
Correct, but this could be remedied without dragging threads in to it.
Okay, but then assume the following scenario: 1. A program creates an archive to read from network connection. Network connection is never to be closed. 2. A command arrives via connection that requires loading a plugin. 3. The plugin has BOOST_CLASS_EXPORT, so attempt to load it crashes. We have two possibilities: 3a. Since the exception happens when you modified serialization tables, and that happens in static initializer, you cannot catch the exception anywhere in the plugin code. Since the plugin (on Unix), is opened with C function dlopen, it's not clear if the exception will propagated across that C function. So, the whole application might crash. 3b. Assume we checked if archives are opened before calling dlopen. Actually, we cannot, because we don't know, generally speaking, if the plugin uses BOOST_CLASS_EXPORT or not. But assume we somehow know this and figured that we can't load the plugin. Now: - we can't close the network connection, because it waits for further commands - we can't load plugin because some archive is opened, so we can do the previous command we've received Neither alternative looks particularly attractive to me.
There's another aspect here. The code for serialization presumes that the global tables are static during the lifetime of the archive. For example, as a particular item is serialized, its class identifier is saved in an archive specific table. Thus the save code used the object identifier to invoke the save method for the specific class. Same goes for the load side. Now, if the global table changes while an archive is open, all bets are off because the save/load code may not be around anymore. So.
a) global tables can't be changed while an archive is open an still expect the system to function.
This is over-generalisation. Clearly, new BOOST_CLASS_EXPORT statements won't break anything.
b) thus DLLS should be loaded/unloaded only when there are no archives open.
It's only disallowed to save/load objects that come from DLL and registered in global id table for which the corresponding DLL is unloaded. What this means is that ids should be removed from global table on DLL unload -- I think this can be done by implementing a destructor in the same helper classes that registers id in its constructor.
c) respecting this rule is the responsability of the calling program.
As I've shown above, it's not always possible to respect this rule.
While we're at it, I would also like to consider enforcing a rule that there can be at most one instance of serialization code for a specific type existing at one time.
What do you mean? If there are two DLL defining the same function but with different bodies, that's ODR violation. If they define the same function with the same body, that's no problem. - Volodya
Besides, I don't understand how you can "inhibit loading of DLLS". Say, one thread has created an archive and does something with it. Some other thread tries to open DLL. What should happen? Should that other thread get "sorry, some archives are open" error? And what will it do?
I havn't considered it in detail but its easy to consider using a semaphore (even though I'm sure there are better solutions). Semaphore is incremented each time an archive is opened. DLL Loading/Unloading code blocks until reaches zero. This could be implemented without touching the serialization library at all: a) derive text_archive_with_semaphore (dtaws) from text_archive_impl b) in constructor of dtaws increment global semaphore c) in destructuctor of dtaws-decrement global semaphore d) DLL loader/unloader waits on same semaphore that dtaws uses. Thus - this very rare case whereby the set of types which implement serialization is changing during the course of the program is easily handled without complicating the serialization library.
Okay, but then assume the following scenario:
1. A program creates an archive to read from network connection. Network connection is never to be closed.
2. A command arrives via connection that requires loading a plugin.
3. The plugin has BOOST_CLASS_EXPORT, so attempt to load it crashes. We have two possibilities:
3a. Since the exception happens when you modified serialization tables, and that happens in static initializer, you cannot catch the exception anywhere in the plugin code. Since the plugin (on Unix), is opened with C function dlopen, it's not clear if the exception will propagated across that C function. So, the whole application might crash.
3b. Assume we checked if archives are opened before calling dlopen. Actually, we cannot, because we don't know, generally speaking, if the plugin uses BOOST_CLASS_EXPORT or not. But assume we somehow know this and figured that we can't load the plugin. Now:
- we can't close the network connection, because it waits for further commands - we can't load plugin because some archive is opened, so we can do the previous command we've received
I'm not sure I get all this. In general the recieving machine would have to have the code for doing the loading installed before the archive is opened. It would seem to me you would need to break this in to two transations. One communicate which DLL might be necessary - passing the DLL which contains the right code. Then close the archve, load all desired DLLS and construct a new archive for the "real" data which might be serialized from one of the previous DLLS.
Neither alternative looks particularly attractive to me.
There's another aspect here. The code for serialization presumes that the global tables are static during the lifetime of the archive. For example, as a particular item is serialized, its class identifier is saved in an archive specific table. Thus the save code used the object identifier to invoke the save method for the specific class. Same goes for the load side. Now, if the global table changes while an archive is open, all bets are off because the save/load code may not be around anymore. So.
a) global tables can't be changed while an archive is open an still expect the system to function.
This is over-generalisation. Clearly, new BOOST_CLASS_EXPORT statements won't break anything.
The will if they are included in a DLL that is unload in the middle of the saving/loading of an archive which contains classes of the indicated type.
b) thus DLLS should be loaded/unloaded only when there are no archives open.
It's only disallowed to save/load objects that come from DLL and registered in global id table for which the corresponding DLL is unloaded. What this means is that ids should be removed from global table on DLL unload -- I think this can be done by implementing a destructor in the same helper classes that registers id in its constructor.
This is already implemented. Its not a problem - the problem is doing in the middle of loading/saving an archive.
c) respecting this rule is the responsability of the calling program.
As I've shown above, it's not always possible to respect this rule.
Based on my semaphore example above - I disagree.
While we're at it, I would also like to consider enforcing a rule that there can be at most one instance of serialization code for a specific type existing at one time.
What do you mean? If there are two DLL defining the same function but with different bodies, that's ODR violation. If they define the same function with the same body, that's no problem.
Exactly. But since the code resides in different DLLS which are not visible at compiler time, the ODR violaton cannot be detected by either the compiler or the linker. Actually I'm not sure what happens when one dynamically loads two different DLLS which contain the same function signatures. I suppose its implementation dependent. Either its prohibited by the part of the OS that loads DLLS or it isn't. I very much doubt that it check to see if the code is the same and permits if and only if the code is identical. To summarize a) The only case of a conflict between serialization an mult-threading arises only with very unusual case of dynamically loading/unloading DLLS which contain serialization code. b) In the case where one feels the need to do this, it can be best addressed outside the serialization library itself. Robert Ramey
Robert Ramey wrote:
Besides, I don't understand how you can "inhibit loading of DLLS". Say, one thread has created an archive and does something with it. Some other thread tries to open DLL. What should happen? Should that other thread get "sorry, some archives are open" error? And what will it do?
I havn't considered it in detail but its easy to consider using a semaphore (even though I'm sure there are better solutions). Semaphore is incremented each time an archive is opened. DLL Loading/Unloading code blocks until reaches zero.
All DLL loading/unloading? Remember we can't check if DLL uses serialization until we load it.
1. A program creates an archive to read from network connection. Network connection is never to be closed.
2. A command arrives via connection that requires loading a plugin.
3. The plugin has BOOST_CLASS_EXPORT, so attempt to load it crashes. We have two possibilities:
3a. Since the exception happens when you modified serialization tables, and that happens in static initializer, you cannot catch the exception anywhere in the plugin code. Since the plugin (on Unix), is opened with C function dlopen, it's not clear if the exception will propagated across that C function. So, the whole application might crash.
3b. Assume we checked if archives are opened before calling dlopen. Actually, we cannot, because we don't know, generally speaking, if the plugin uses BOOST_CLASS_EXPORT or not. But assume we somehow know this and figured that we can't load the plugin. Now:
- we can't close the network connection, because it waits for further commands - we can't load plugin because some archive is opened, so we can do the previous command we've received
I'm not sure I get all this. In general the recieving machine would have to have the code for doing the loading installed before the archive is opened. It would seem to me you would need to break this in to two transations. One communicate which DLL might be necessary - passing the DLL which contains the right code. Then close the archve, load all desired DLLS and construct a new archive for the "real" data which might be serialized from one of the previous DLLS.
So, you need to split each request in two parts: first the allows the server to determine which DLLs to load, and another one that gives the other data. You'd need to break connection between those two steps which does not look all the nice. And further: say you have a server that talks with 10 clients at the same time using some archive class. One client requests an operations that needs loading a DLL. Do you expect that: 1. Server will stop accepting new connections 2. Wait till 9 other connections stop 3. After that, load the DLL? or simply that all connection will be dropped?
This is over-generalisation. Clearly, new BOOST_CLASS_EXPORT statements won't break anything.
The will if they are included in a DLL that is unload in the middle of the saving/loading of an archive which contains classes of the indicated type.
I suppose that any use of My_class with simultaneous unloading of DLL that defines that class is so risky, that you should not take special care about this in serialization at all. If users unload DLL while instances of the class are still around, they already have a bug. BTW, on-demand loaded DLLs are not essential to this discussion. BOOST_CLASS_EXPORT becomes static object, right? C++ standard says that static initializer constructor can be executed right before the first function in some module is called. So, you can get unexpected BOOST_CLASS_EXPORT executions even without on-demand loaded DLL. Again -- you read something from archive over network, and take some action. Accidentally, you call a function in new module and BOOST_CLASS_EXPORT is executed. Now, on Unix all static initialisers are run when the program is first loaded, so I probably should not care if some other users run into problems. - Volodya
Vladimir Prus wrote:
I havn't considered it in detail but its easy to consider using a semaphore (even though I'm sure there are better solutions). Semaphore is incremented each time an archive is opened. DLL Loading/Unloading code blocks until reaches zero.
All DLL loading/unloading? Remember we can't check if DLL uses serialization until we load it.
We're loading the DLL because we know apriori that it contains the code that implements serialization for a particular class.
So, you need to split each request in two parts: first the allows the server to determine which DLLs to load, and another one that gives the other data. You'd need to break connection between those two steps which does not look all the nice.
Again, the serialization library doesn't know which DLLS to load. It presumes that code is present for all registered/exported types. If such types are not registered or exported, an exception will be thrown. If you want to load the DLLS on the fly (somehow), youll have to communicate across the space/time which DLLS you want to see loaded before you start to serialize the types they implement. The serialization system has no map which implements (type -> name of DLL which serializes that type). If one feels this is necessary for his application, it would be made outside the serialization system itself.
And further: say you have a server that talks with 10 clients at the same time using some archive class. One client requests an operations that needs loading a DLL. Do you expect that:
1. Server will stop accepting new connections 2. Wait till 9 other connections stop 3. After that, load the DLL?
The serialization system requires that code to save/load types exist in both contexts. This is a fundamental limitation of the system. Its up to the user to decide if this limitation makes it unsuitable for his application. In practice, I would expect that in a system such as the above, all known types would be loaded before any data is transmitted.
or simply that all connection will be dropped?
What ever the library user want's to do.
I suppose that any use of My_class with simultaneous unloading of DLL that defines that class is so risky, that you should not take special care about this in serialization at all. If users unload DLL while instances of the class are still around, they already have a bug.
Agree. My example just showed one way that a library user could avoid such a bug.
BTW, on-demand loaded DLLs are not essential to this discussion. BOOST_CLASS_EXPORT becomes static object, right? C++ standard says that static initializer constructor can be executed right before the first function in some module is called. So, you can get unexpected BOOST_CLASS_EXPORT executions even without on-demand loaded DLL.
This is an interesting point. I believe that the situation is slightly different. In this case: void f(){ static X x; ... } The constructor for x is invoked the first time f() is called. In this case: class X { static X x; ... }; X X::x. The constructor X::x is invoked at pre-execution time. At least that's my understanding and it seems that all the compiler's in our test suite seem to support this behavior.
Again -- you read something from archive over network, and take some action. Accidentally, you call a function in new module and BOOST_CLASS_EXPORT is executed.
Again - I don't believe this could ever happen. The implementation of BOOST_CLASS_EXPORT depends on static constructors being executed at pre-execution time or for the case of DLLS - up DLL loading
Now, on Unix all static initialisers are run when the program is first loaded, so I probably should not care if some other users run into problems.
I don't believe this is a feature of the operating system but rather the compiler. I also believe that the serialization library presumes correctly that they will be invoked a pre-execution time. I never really thought of what happens when a DLL contains BOOST_CLASS_EXPORT, but it turns out only a little bit of extra work was required to remove some table entries when a DLL was unloaded. Robert Ramey
From: "Robert Ramey" <ramey@rrsd.com>
Vladimir Prus wrote:
In this case:
void f(){ static X x; ... }
The constructor for x is invoked the first time f() is called. In this
Yes.
case:
class X { static X x; ... };
X X::x.
The constructor X::x is invoked at pre-execution time. At least that's my understanding and it seems that all the compiler's in our test suite seem to support this behavior.
X::x's static initialization will occur before main(). If X::x requires dynamic intialization, it must occur before the first function following it's definition in that translation unit is invoked. Nothing more is guarranteed. IOW, if the constructor of X requires runtime behavior--opens a file, allocates memory, calculates a number, etc.--then that isn't guarranteed to happen until some function, defined after X::x in the same translation unit, is invoked.
The implementation of BOOST_CLASS_EXPORT depends on static constructors being executed at pre-execution time or for the case of DLLS - up DLL loading
By "pre-execution time," I assume you mean before main() is called. That only applies to static initialization, not dynamic initialization of statics.
Now, on Unix all static initialisers are run when the program is first loaded, so I probably should not care if some other users run into problems.
I don't believe this is a feature of the operating system but rather the compiler. I also believe that the serialization library presumes correctly that they will be invoked a pre-execution time. I never really thought of
Your assumption is ill founded. -- Rob Stewart stewart@sig.com Software Engineer http://www.sig.com Susquehanna International Group, LLP using std::disclaimer;
Robert Ramey wrote:
Vladimir Prus wrote:
I havn't considered it in detail but its easy to consider using a semaphore (even though I'm sure there are better solutions). Semaphore is incremented each time an archive is opened. DLL Loading/Unloading code blocks until reaches zero.
All DLL loading/unloading? Remember we can't check if DLL uses serialization until we load it.
We're loading the DLL because we know apriori that it contains the code that implements serialization for a particular class.
No, you either missed or ignored my one of my points. We might be loading DLL because we want to invoke some service it provides and don't even *care* if that DLL contains any BOOST_CLASS_EXPORT.
And further: say you have a server that talks with 10 clients at the same time using some archive class. One client requests an operations that needs loading a DLL. Do you expect that:
1. Server will stop accepting new connections 2. Wait till 9 other connections stop 3. After that, load the DLL?
The serialization system requires that code to save/load types exist in both contexts. This is a fundamental limitation of the system. Its up to the user to decide if this limitation makes it unsuitable for his application. In practice, I would expect that in a system such as the above, all known types would be loaded before any data is transmitted.
0. Again, DLL loaded by demand might contains BOOST_CLASS_EXPORT that's not even used in the current communication at all. 1. You did not answer my direct question what should happen in this case. Generally speaking, you can expect that all DLLs are loaded before server starts talking over network -- what's the point of on-demand DLLs then?
or simply that all connection will be dropped?
What ever the library user want's to do.
Look, you impose some restriction on library user and says that he can do anything he wants to overcome that restriction. That sounds strange to me -- unless you provide some specific solution, how can be know that a solution exists at all?
Now, on Unix all static initialisers are run when the program is first loaded, so I probably should not care if some other users run into problems.
I don't believe this is a feature of the operating system but rather the compiler. I also believe that the serialization library presumes correctly that they will be invoked a pre-execution time. I never really thought of what happens when a DLL contains BOOST_CLASS_EXPORT, but it turns out only a little bit of extra work was required to remove some table entries when a DLL was unloaded.
As pointed by Rob, this statement is not 100% accurate. - Volodya
Robert Ramey wrote:
While we're at it, I would also like to consider enforcing a rule that there can be at most one instance of serialization code for a specific type existing at one time.
What do you mean? If there are two DLL defining the same function but with different bodies, that's ODR violation. If they define the same function with the same body, that's no problem.
Exactly. But since the code resides in different DLLS which are not visible at compiler time, the ODR violaton cannot be detected by either the compiler or the linker. Actually I'm not sure what happens when one dynamically loads two different DLLS which contain the same function signatures. I suppose its implementation dependent.
If by implementation dependent you mean OS dependent, that is correct. In Windows code called through a dynamically loaded DLL: 1) Must be exported by the DLL 2) Is called using a DLL handle to the particular DLL ie. int (*p)(int); HMODULE h(LOadLibrary("SomeDllName"); // HMODULE is the DLL handle FARPROC fp(GetProcAddress(h,"SomeFunctionName")); // SomeFunctionName must be exported p = reinterpret_cast<p>(fp); int result(p(n...)); // where n is some integer As you say, there is no ODR violation because this does not happen at compile-time but at run-time.
Have you considered this any more? I understand the issue but I'm not convinced that your fix is the best one. This would permit code for serializaton of the same type to reside in different DLLS with out the guarentee that all the code is the same. Also, I'm not conviced that our system of purging global map entries when a DLL containing serialization for a type is unloaded wouldn't create problems. That is I'm not sure that purging all the entries for a given type will be the right thing to do. I'm thinking that the best would be to detect the case when a duplicate extended_type_info record is added to the corresponding global map and throw an exception. I see no other way at this point to guarentee that the system will always work. Robert Ramey martin.ecker@tab.at wrote:
The fix is easy. Simply put the old code for extended_type_info::operator == and extended_type_info::operator != back in.
Note that I haven't looked any further if there were other changes made to the code that also rely on the addresses of equal extended_type_info objects to be the same. This will also break serialization code in DLLs. I can only say that after I put the old code back in, our system could deserialize its data properly again.
Hi, Robert Ramey wrote:
Have you considered this any more? I understand the issue but I'm not convinced that your fix is the best one. This would permit code for serializaton of the same type to reside in different DLLS with out the guarentee that all the code is the same.
I don't see much that can be done about that. But assuming that all DLLs were built with the same version of the header files that contain serialization code (or in other words, assuming the One Definition Rule is not violated), the code should be identical.
Also, I'm not conviced that our system of purging global map entries when a DLL containing serialization for a type is unloaded wouldn't create problems. That is I'm not sure that purging all the entries for a given type will be the right thing to do.
I'm thinking that the best would be to detect the case when a duplicate extended_type_info record is added to the corresponding global map and
Considering that DLLs can be unloaded dynamically, it is probably not necessarily the right thing to do. However, this is a hard problem and requires a bit of redesign in Boost.Serialization. The only way to get rid of the problem that I can see is to make sure that all the objects stored in all global registries (i.e. the type info registry, the pointer serializer maps, and the void caster registry) are allocated only once and on the heap of the Boost.Serialization DLL (preferably using some kind of small heap allocator optimization since the objects stored in the map are all of small sizes). So for example, consider the static pointer_iserializer instances in iserializer.hpp, which can potentially be included by multiple DLLs, thus causing multiple pointer_iserializer objects for the same type T to be contained in multiple DLLs. This must be prevented. So instead of directly having static const pointer_iserializer instance; it would be better to have something like static const pointer_iserializer_reference instance; which is only a smart pointer or smart reference to the actual pointer_iserializer object for a type T. This pointer_iserializer object is only allocated _once_ and on the heap of the Boost.Serialization DLL. Now, if multiple DLLs include iserializer.hpp (by including serialization code for the same type T) they will get multiple instances of pointer_iserializer_reference objects. But that's okay now, since only the first time such a reference object is instantiated, the "real" pointer_iserializer object is created. All subsequent DLLs that get loaded and create a pointer_iserializer_reference object for type T will get back the same pointer_iserializer object that was created previously. pointer_iserializer_reference acts as a smart pointer to a unique pointer_iserializer object for type T. So now we don't have any problems when DLLs that contain serialization code for type T get unloaded. They simply decrease the reference count to the pointer_iserializer object. When the last DLL gets unloaded, the reference count drops to 0 and Boost.Serialization can destroy the pointer_iserializer object for type T. I believe, this approach should work with all types of global objects, i.e. void_caster, pointer_iserializer, pointer_oserializer, and extended_type_info. It should also work in all situations I can think of with DLLs on Windows, i.e. with DLLs loaded at startup and unloaded at shutdown and with DLLs that are loaded/unloaded explicitly during run-time. throw
an exception. I see no other way at this point to guarentee that the system will always work.
This would break all our current code base. When you use DLLs and you have serialization code in one DLL that is then used by another DLL, then - with the current use of (effectively global) static member variables in e.g. pointer_iserializer - you _will_ get multiple objects registered in the global registries and maps. There's nothing you can do about that (at least not on Windows; I believe with the Linux loader this problem does not occur). Throwing an exception if this happens, makes using Boost.Serialization with DLLs impossible. Best Regards, Martin TAB Austria Haiderstraße 40 4052 Ansfelden Austria Phone: +43 7229 78040-218 Fax: +43 7229 78040-209 E-mail: martin.ecker@tab.at http://www.tab.at
martin.ecker@tab.at wrote:
Hi,
Robert Ramey wrote: ... So now we don't have any problems when DLLs that contain serialization code for type T get unloaded. They simply decrease the reference count to the pointer_iserializer object. When the last DLL gets unloaded, the reference count drops to 0 and Boost.Serialization can destroy the pointer_iserializer object for type T.
I believe, this approach should work with all types of global objects, i.e. void_caster, pointer_iserializer, pointer_oserializer, and extended_type_info. It should also work in all situations I can think of with DLLs on Windows, i.e. with DLLs loaded at startup and unloaded at shutdown and with DLLs that are loaded/unloaded explicitly during run-time.
Wouldn't this require that the first DLL loaded be the last one to be unloaded? How would such a requirement be enforced? Wouldn't this put a burden one the library user to load/unload his DLLS in a certain sequence with a catestrophic untraceable error should he make a mistake in this area?
I'm thinking that the best would be to detect the case when a duplicate extended_type_info record is added to the corresponding global map and throw an exception. I see no other way at this point to guarentee that the system will always work.
This would break all our current code base. When you use DLLs and you have serialization code in one DLL that is then used by another DLL,
For the same type I presume.
then -with the current use of (effectively global) static member variables in e.g. pointer_iserializer - you _will_ get multiple objects registered in the global registries and maps. There's nothing you can do about that (at least not on Windows; I believe with the Linux loader this problem does not occur). Throwing an exception if this happens, makes using Boost.Serialization with DLLs impossible.
This would make it an illegal detectable error to include the serialization of a given type in different DLLS and have those DLLS simultaneously loaded. That is, this would enforce the One Definition Rule (at at time) for dynamically loaded DLLS. Although its a user issue, I would think that including the same code in different DLLS which are to be simultaneiously loaded defeats the whole point of DLLS in the first place - to avoid having memory loaded with dead (or sleeping) code. Robert Ramey
Hi, <snip my comments on how to make sure all the global objects, such as type info and void caster objects, are always unique in a process> Robert Ramey wrote:
Wouldn't this require that the first DLL loaded be the last one to be unloaded?
No. The order of loading/unloading DLLs doesn't matter. The first time a DLL is loaded that contains a BOOST_CLASS_EXPORT (and with it a number of global objects, such as type info), the global objects that need to be registered with the global registries and maps are created for the first time. Since they are reference counted, these process- wide unique, global objects only get destroyed when the last reference to them is released - for example, when the last DLL that uses them gets unloaded.
How would such a requirement be enforced? Wouldn't this put a burden one the library user to load/unload his DLLS in a certain sequence with a catestrophic untraceable error should he make a mistake in this area?
No, there are no requirements on user code or DLL loading/unloading order. My proposed scheme should simply "just work".
This would break all our current code base. When you use DLLs and you have serialization code in one DLL that is then used by another DLL,
For the same type I presume.
Yes.
then -with the current use of (effectively global) static member variables in e.g. pointer_iserializer - you _will_ get multiple objects registered in the global registries and maps. There's nothing you can do about that (at least not on Windows; I believe with the Linux loader this problem does not occur). Throwing an exception if this happens, makes using Boost.Serialization with DLLs impossible.
This would make it an illegal detectable error to include the serialization of a given type in different DLLS and have those DLLS simultaneously loaded.
But there's nothing you can do about it. As soon as serialization code, in particular some global objects such as type info, are in a header file and that header file is included in code in multiple DLLs, you _will_ definitely get multiple copies of these objects (on Windows at least). That's why my proposal is to let these global objects only be some kind of smart reference and have the "real" global objects that need to be unique for the running process be created by the Boost.Serialization DLL only once. Currently they are potentially created multiple times in the above scenario (i.e. serialization code in a header file that is included by code in multiple DLLs).
Although its a user issue, I would think that including the same code in different DLLS which are to be simultaneiously loaded defeats the whole
point of DLLS in the first place - to avoid having memory loaded with dead (or sleeping) code.
I agree with this. But because of the way the Windows DLL loader works in combination with global variables declared in C++ header files that are included by code in multiple DLLs, you cannot prevent having multiple instances of global variables. Regards, Martin TAB Austria Haiderstraße 40 4052 Ansfelden Austria Phone: +43 7229 78040-218 Fax: +43 7229 78040-209 E-mail: martin.ecker@tab.at http://www.tab.at
martin.ecker@tab.at wrote:
Hi,
<snip my comments on how to make sure all the global objects, such as type info and void caster objects, are always unique in a process>
Robert Ramey wrote:
Wouldn't this require that the first DLL loaded be the last one to be unloaded?
No. The order of loading/unloading DLLs doesn't matter. The first time a DLL is loaded that contains a BOOST_CLASS_EXPORT (and with it a number of global objects, such as type info), the global objects that need to be registered with the global registries and maps are created for the first time. Since they are reference counted, these process- wide unique, global objects only get destroyed when the last reference to them is released - for example, when the last DLL that uses them gets unloaded.
What I'm worried about are the virtual function calls routed through the base class vtable. The first object created has addresses in its vtable which point to the DLL that loaded it. Multiple entries would have different vtables each pointing to an address in a different DLL. If one DLL gets unloaded but the vtable remains in memory one is going to get a crash (if you're lucky) when makinga call through that base class instance. Hence my view that the first DLL to be loaded would have to be the last unloaded.
then -with the current use of (effectively global) static member variables in e.g. pointer_iserializer - you _will_ get multiple objects registered in the global registries and maps. There's nothing you can do about that (at least not on Windows; I believe with the Linux loader this problem does not occur). Throwing an exception if this happens, makes using Boost.Serialization with DLLs impossible.
This would make it an illegal detectable error to include the
serialization
of a given type in different DLLS and have those DLLS simultaneously loaded.
But there's nothing you can do about it. As soon as serialization code, in particular some global objects such as type info, are in a header file and that header file is included in code in multiple DLLs, you _will_ definitely get multiple copies of these objects (on Windows at least).
Its not clear to me that its an unavoidable problem. Just including the header for a serializable type won't trigger inclusion of the indicated static variables. One has to include archive code and instantiate it for the specific serializable type by invoking and << or >> operator on it. I don't believe the problem would ever arise if archve headers were included in only one DLL.
That's why my proposal is to let these global objects only be some kind of smart reference and have the "real" global objects that need to be unique for the running process be created by the Boost.Serialization DLL only once. Currently they are potentially created multiple times in the above scenario (i.e. serialization code in a header file that is included by code in multiple DLLs).
I understand the proposal. I don't think it would completely address the problem.
Although its a user issue, I would think that including the same code in different DLLS which are to be simultaneiously loaded defeats the whole
point of DLLS in the first place - to avoid having memory loaded with dead (or sleeping) code.
I agree with this. But because of the way the Windows DLL loader works in combination with global variables declared in C++ header files that are included by code in multiple DLLs, you cannot prevent having multiple instances of global variables.
This is what I'm not so sure about. However, avoid the problem would require some attention to code module organization which could be a problem for some people. Robert Ramey
Hello, Robert Ramey wrote:
What I'm worried about are the virtual function calls routed through the
base class vtable.
Can you direct me to where in the Boost.Serialization code this is happening? I'd like to have a look at this. What base class are you referring to?
Its not clear to me that its an unavoidable problem. Just including the
header for a serializable type won't trigger inclusion of the indicated static variables. One has to include archive code and instantiate it for the specific serializable type by invoking and << or >> operator on it. I don't believe the problem would ever arise if archve headers were included in only one DLL.
Of course you're right. Simply including the headers doesn't trigger anything, but for example using BOOST_CLASS_EXPORT in a DLL and then using archives from the main application should trigger the problem for type info objects, if I'm not mistaken. Also using archives from multiple DLLs, which is not an uncommen scenario for us, will trigger multiple instances of static members in header files. Best Regards, Martin TAB Austria Haiderstraße 40 4052 Ansfelden Austria Phone: +43 7229 78040-218 Fax: +43 7229 78040-209 E-mail: martin.ecker@tab.at http://www.tab.at
martin.ecker@tab.at wrote:
Hello,
Robert Ramey wrote:
What I'm worried about are the virtual function calls routed through the
base class vtable.
Can you direct me to where in the Boost.Serialization code this is happening? I'd like to have a look at this. What base class are you referring to?
look at extended_type_info.hpp class extended_type_info ... virtual bool less_than(const extended_type_info &rhs) const = 0; Look at void_cast.hpp struct ... void_caster ... // each derived class must re-implement these; virtual void const * upcast(void const * t) const = 0; virtual void const * downcast(void const * t) const = 0; Look at archive_pointer_iserializer virtual const basic_iserializer & get_basic_serializer() const = 0; and others as well
Of course you're right. Simply including the headers doesn't trigger anything.
I did when we implemented auto-link. But thanks in large part to Joaquins efforts this has been fixed so that things work the way they should. That is - merely including a header from boost/serialization will not trigger linkage to any library or DLL.
but for example using BOOST_CLASS_EXPORT in a DLL and then using archives from the main application should trigger the problem for type info objects.
Only if a header from boost/archive is being included.
if I'm not mistaken. Also using archives from multiple DLLs, which is not an uncommen scenario for us, will trigger multiple instances of static members in header files.
This is the key trigger and source of the problem. My current thinking is that is not addressable from within the library itself. The best we could do is detect when multiple DLLS create duplicates of these static objects and throw an exception. This would be very helpful in creating code that is guarenteed to work. I've described what I don't think I can do and what I can do. I'm not sure what I can offer here, but here's what I can suggest. There are many ways/styles of organizing code in DLLS and libraries. Many of these will create no problem. Consider this: Suppose you have a lot of complicated types in your code. If each DLL had all the code for a type or group of types, there would be no problem. No DLL would contain the same static global object as any other DLL. If you are using polymorphic_archives (as I believe you are) this would create a beautiful system of DLLS that don't intersect. No duplicated code. And your code for a particular type would be all in one place - easing maintainence and upgrades. Note that code in one DLL could easily call code in another DLL without creating problems. Combined with Boost auto-link (and just a little pulling of hair) this creates a great system of orthogonal DLLS which can be called in any combination from any program. This is a system with a lot of the benefits of CORBA or microsoft COM with almost none of the grief. Note that a future test will explicitly test the usage of the serialization library as part of a "plug-in" which is what I've described here. Unfortuntatly, to make this work, extended_type_info has to be enhanced slightly by adding in a "class factory" that will create an object given its export id. (or something like that - still in the conceptual stage). If you have a smaller number of types but want to generate DLLS - one for each type of archive, this can be done by making a small module for each archive similar to text_oarchive.hpp. Then you have one DLL for each archive type. This is basically what I did for the common code shared accross all archives and it has worked very well. If the above is not doable or appropriate, I would be curious to know about it. One writes a library with some sort of anticapted usage in mind. Of course, he can't think of everything. So I'd love to hear about cases where the current system can't be made to work. Robert Ramey
participants (6)
-
David Abrahams -
Edward Diener -
martin.ecker＠tab.at -
Rob Stewart -
Robert Ramey -
Vladimir Prus