[serialization][1.36] ktmap contains extended_type_info with m_key == 0
Hi Robert, I'm hitting a assertion with m_key == 0. With my debugger I can find lots and lots of items with m_key == 0. None are one of my types, there are shared_ptr_1_32 with m_key == 0 and others. Do you have a newer 1.36 version? Am I doing something wrong? I'm using the 1.36 version for use with DLLs (export in a DLL and serialize/deserialize in another). Thanks, -- Felipe Magno de Almeida
I made a workaround for now where I remove the assertions from the key_compare and in key_unregister, if nothing is found with equal_range (equal_range is faster than using upper and lower_bound) then I do a linear search. On Sat, Mar 29, 2008 at 6:43 PM, Felipe Magno de Almeida <felipe.m.almeida@gmail.com> wrote:
Hi Robert,
I'm hitting a assertion with m_key == 0. With my debugger I can find lots and lots of items with m_key == 0. None are one of my types, there are shared_ptr_1_32 with m_key == 0 and others.
Do you have a newer 1.36 version? Am I doing something wrong? I'm using the 1.36 version for use with DLLs (export in a DLL and serialize/deserialize in another).
Thanks, -- Felipe Magno de Almeida
-- Felipe Magno de Almeida
None are one of my types, there are shared_ptr_1_32 with m_key == 0 and others.
The system for type registration has been improved to make the library truely thread-safe and DLL compatible. Sounds like there is an oversight that needs to be fixed. If you can make a small example and indicate which compiler you're using I would like to see it. Robert Ramey Felipe Magno de Almeida wrote:
I made a workaround for now where I remove the assertions from the key_compare and in key_unregister, if nothing is found with equal_range (equal_range is faster than using upper and lower_bound) then I do a linear search.
On Sat, Mar 29, 2008 at 6:43 PM, Felipe Magno de Almeida <felipe.m.almeida@gmail.com> wrote:
Hi Robert,
I'm hitting a assertion with m_key == 0. With my debugger I can find lots and lots of items with m_key == 0. None are one of my types, there are shared_ptr_1_32 with m_key == 0 and others.
Do you have a newer 1.36 version? Am I doing something wrong? I'm using the 1.36 version for use with DLLs (export in a DLL and serialize/deserialize in another).
Thanks, -- Felipe Magno de Almeida
On Sat, Mar 29, 2008 at 11:36 PM, Robert Ramey <ramey@rrsd.com> wrote:
None are one of my types, there are shared_ptr_1_32 with m_key == 0 and others.
The system for type registration has been improved to make the library truely thread-safe and DLL compatible. Sounds like there is an oversight that needs to be fixed. If you can make a small example and indicate which compiler you're using I would like to see it.
Hi Robert, I didn't make an example yet. I'm using msvc-7.1 SP1. I was able to narrow down the issue for when multiple registrations are done for the same type. Is it supported? The multiple registration comes from the same DLL.
Robert Ramey
Thanks, -- Felipe Magno de Almeida
Felipe Magno de Almeida wrote:
On Sat, Mar 29, 2008 at 11:36 PM, Robert Ramey <ramey@rrsd.com> wrote:
None are one of my types, there are shared_ptr_1_32 with m_key == 0 and others.
The system for type registration has been improved to make the library truely thread-safe and DLL compatible. Sounds like there is an oversight that needs to be fixed. If you can make a small example and indicate which compiler you're using I would like to see it.
Hi Robert, I didn't make an example yet. I'm using msvc-7.1 SP1. I was able to narrow down the issue for when multiple registrations are done for the same type. Is it supported? The multiple registration comes from the same DLL.
Hmmm - I don't this should ever occur. That is I would expect one module to invoke no more than one "registration" record as its a singleton. Multiple registration records could occur from having code for the same type in multiple DLLS or in a DLL along with the main module. As each module is unloaded, registrations added the module should be deleted. Perhaps there is something out of whack here Robert Ramey So I would
Robert Ramey
Thanks,
On Sun, Mar 30, 2008 at 12:59 AM, Robert Ramey <ramey@rrsd.com> wrote:
Felipe Magno de Almeida wrote:
[snip]
Hi Robert, I didn't make an example yet. I'm using msvc-7.1 SP1. I was able to narrow down the issue for when multiple registrations are done for the same type. Is it supported? The multiple registration comes from the same DLL.
Hmmm - I don't this should ever occur. That is I would expect one module to invoke no more than one "registration" record as its a singleton.
For each translation file which includes the serialization type and the archive type shows one extended_type_info in tkmap for that serialization type. The serialization type is surely on one DLL only, because they're only defined there. No other project has access to those headers (not in usage-requirements for other projects).
Multiple registration records could occur from having code for the same type in multiple DLLS or in a DLL along with the main module.
I believe the extended_type_info object is inserted for each translation file which includes the archive and serialization type. But it is only called key_register once. That's why there are extended_type_info with m_key == 0 in tkmap.
As each module is unloaded, registrations added the module should be deleted. Perhaps there is something out of whack here
I will try to provide a boost.build-v2 buildable example with the problem when I can in the near time.
Robert Ramey
So I would
Thanks, -- Felipe Magno de Almeida
For each translation file which includes the serialization type and the archive type shows one extended_type_info in tkmap for that serialization type. The serialization type is surely on one DLL only, because they're only defined there. No other project has access to those headers (not in usage-requirements for other projects).
What is "translation file" as opposed to a DLL?. Is the DLL loaded explicitly with "library load" or automatically as an "import library"?
Multiple registration records could occur from having code for the same type in multiple DLLS or in a DLL along with the main module.
I will try to provide a boost.build-v2 buildable example with the problem when I can in the near time.
Good luck with that. I couldn't figure out a way to make bjam do this. Robert Ramey
On Sun, Mar 30, 2008 at 1:59 PM, Robert Ramey <ramey@rrsd.com> wrote:
For each translation file which includes the serialization type and the archive type shows one extended_type_info in tkmap for that serialization type. The serialization type is surely on one DLL only, because they're only defined there. No other project has access to those headers (not in usage-requirements for other projects).
What is "translation file" as opposed to a DLL?
A translation file is a preprocessed .cpp file. A DLL can have lots of source files.
Is the DLL loaded explicitly with "library load"
Yes. ::LoadLibrary through boost.extension.
or automatically as an "import library"?
Multiple registration records could occur from having code for the same type in multiple DLLS or in a DLL along with the main module.
I will try to provide a boost.build-v2 buildable example with the problem when I can in the near time.
Good luck with that. I couldn't figure out a way to make bjam do this.
Lol.
Robert Ramey
Regards, -- Felipe Magno de Almeida
Felipe Magno de Almeida wrote:
On Sun, Mar 30, 2008 at 1:59 PM, Robert Ramey <ramey@rrsd.com> wrote:
A translation file is a preprocessed .cpp file. A DLL can have lots of source files.
Is the DLL loaded explicitly with "library load"
Yes.
LoadLibrary through boost.extension.
you should be able to see (with your debugger) the create of one and only one entry in the kt table when the library is loaded. You should see the deletion of this one entry when the the library is unloaded. I would make a simple program that just loads and unloads the DLL and verify this behavior. This is a spanking new facility so its possible that it has some bugs. Also, I have to test this "by hand" so its not tested as part of the test system. Robert Ramey
On Sun, Mar 30, 2008 at 5:44 PM, Robert Ramey <ramey@rrsd.com> wrote:
Felipe Magno de Almeida wrote:
On Sun, Mar 30, 2008 at 1:59 PM, Robert Ramey <ramey@rrsd.com> wrote:
A translation file is a preprocessed .cpp file. A DLL can have lots of source files.
Is the DLL loaded explicitly with "library load"
Yes.
LoadLibrary through boost.extension.
you should be able to see (with your debugger) the create of one and only one entry in the kt table when the library is loaded.
Ok. I was able to make a reproducible case. Though not *exactly* what I see in my project. It is attached. I hadn't noticed before, but in my project I had two BOOST_CLASS_EXPORT in the DLL project. That did registered the same extended_type_info<> instance to the ktmap. When one was to be destroyed, it sets m_key to zero. In the example I'm sending, the executable breaks in a more mysterious way. Is more than one BOOST_CLASS_EXPORT, in different translation files, suppose to work? Removing the extra one, or not including the archives and export.hpp also worked in this project. It did int-defaulted the return type to the BOOST_CLASS_EXPORT(x); when it was not defined, that's why I it slip through me.
You should see the deletion of this one entry when the the library is unloaded. I would make a simple program that just loads and unloads the DLL and verify this behavior.
This is a spanking new facility so its possible that it has some bugs. Also, I have to test this "by hand" so its not tested as part of the test system.
Yeah, that's very hard to test with boost.test facilities.
Robert Ramey
Thanks, -- Felipe Magno de Almeida
On Mon, Mar 31, 2008 at 3:28 AM, Felipe Magno de Almeida <felipe.m.almeida@gmail.com> wrote:
On Sun, Mar 30, 2008 at 5:44 PM, Robert Ramey <ramey@rrsd.com> wrote:
[snip]
Ok. I was able to make a reproducible case. Though not *exactly* what I see in my project. It is attached. I hadn't noticed before, but in my project I had two BOOST_CLASS_EXPORT in the DLL project. That did registered the same extended_type_info<> instance to the ktmap. When one was to be destroyed, it sets m_key to zero. In the example I'm sending, the executable breaks in a more mysterious way.
Just a PS: mysterious because the memory is reclaimed, and the data is scrambled. But what happens is essentially the same. [snip] -- Felipe Magno de Almeida
On Mon, 31 Mar 2008 03:28:49 -0300, Felipe Magno de Almeida wrote:
Yeah, that's very hard to test with boost.test facilities.
DLL loading? I've done it before with Boost.Test. What do you think is hard to test about it? -- Sohail Somani http://uint32t.blogspot.com
Great, send me a Jamfile. If I create a DLL it ends up in a directory tree bin.v2/serialization/test/... and I create program which uses the DLL it ends up in a different directory. When you run the test, it can't find the DLL Robert Ramey Sohail Somani wrote:
On Mon, 31 Mar 2008 03:28:49 -0300, Felipe Magno de Almeida wrote:
Yeah, that's very hard to test with boost.test facilities.
DLL loading? I've done it before with Boost.Test. What do you think is hard to test about it?
On Mon, 31 Mar 2008 08:02:49 -0800, Robert Ramey wrote:
Great, send me a Jamfile.
You said the magic word. Never used Boost Jam to build anything but Boost :-) Though I can't imagine it would be too hard to set an environment variable for the test run. -- Sohail Somani http://uint32t.blogspot.com
Felipe Magno de Almeida wrote: Is more than one BOOST_CLASS_EXPORT, in different
translation files, suppose to work?
I never considered the possibility that this would ever occur. I see no reason to ever do this and no reason why it should be difficult to avoid doing. I would say is a user error. I would like to trap it as such, but there is no way to distinguish it from the legitimate case where the some type is exported from multiple DLLS. Robert Ramey
On Mon, Mar 31, 2008 at 1:00 PM, Robert Ramey <ramey@rrsd.com> wrote:
Felipe Magno de Almeida wrote:
Is more than one BOOST_CLASS_EXPORT, in different
translation files, suppose to work?
I never considered the possibility that this would ever occur. I see no reason to ever do this and no reason why it should be difficult to avoid doing. I would say is a user error.
What if someone wants to create serialization code for multiple archives in different translation files? I know this is not *very* compelling, but it is a honest use for multiple BOOST_CLASS_EXPORT.
I would like to trap it as such, but there is no way to distinguish it from the legitimate case where the some type is exported from multiple DLLS.
Changing std::multiset to std::set would be enough, wouldn't it? The extended_type_info.cpp file uses a std::multiset<const extended_type_info *, ...> and remove_key tries to handle duplicate insertions in ktmap. But it fails always to do that, because m_key will be set to zero and so ktmap will be in a corrupted state since its key_compare asserts on m_key == 0.
Robert Ramey
Thanks, -- Felipe Magno de Almeida
"Felipe Magno de Almeida" <felipe.m.almeida@gmail.com> wrote in message news:a2b17b60803311200v75904141rbe56c2713ef6fe7d@mail.gmail.com...
On Mon, Mar 31, 2008 at 1:00 PM, Robert Ramey <ramey@rrsd.com> wrote:
Felipe Magno de Almeida wrote:
Is more than one BOOST_CLASS_EXPORT, in different
translation files, suppose to work?
I never considered the possibility that this would ever occur. I see no reason to ever do this and no reason why it should be difficult to avoid doing. I would say is a user error.
What if someone wants to create serialization code for multiple archives in different translation files? I know this is not *very* compelling, but it is a honest use for multiple BOOST_CLASS_EXPORT.
I would like to see BOOST_CLASS_EXPORT in the *.cpp file which contains the class definitions. There should only be one of those. Look at it this way - multiple definitions of the same function will give an error at link time - but here - link time is moved to runtime.
I would like to trap it as such, but there is no way to distinguish it from the legitimate case where the some type is exported from multiple DLLS.
Changing std::multiset to std::set would be enough, wouldn't it?
Nope - that was exactly the problem. If the same type is used in different DLLS, then there will be multiple static objects created each loaded/unloaded as the dlls themselves are loaded/unloaded.
The extended_type_info.cpp file uses a std::multiset<const extended_type_info *, ...> and remove_key tries to handle duplicate insertions in ktmap. But it fails always to do that, because m_key will be set to zero and so ktmap will be in a corrupted state since its key_compare asserts on m_key == 0.
This may be an error. I haven't looked it it yet. Robert Ramey
On Mon, Mar 31, 2008 at 6:35 PM, Robert Ramey <ramey@rrsd.com> wrote:
"Felipe Magno de Almeida" <felipe.m.almeida@gmail.com> wrote in message news:a2b17b60803311200v75904141rbe56c2713ef6fe7d@mail.gmail.com...
On Mon, Mar 31, 2008 at 1:00 PM, Robert Ramey <ramey@rrsd.com> wrote:
[snip]
I never considered the possibility that this would ever occur. I see no reason to ever do this and no reason why it should be difficult to avoid doing. I would say is a user error.
What if someone wants to create serialization code for multiple archives in different translation files? I know this is not *very* compelling, but it is a honest use for multiple BOOST_CLASS_EXPORT.
I would like to see BOOST_CLASS_EXPORT in the *.cpp file which contains the class definitions. There should only be one of those. Look at it this way - multiple definitions of the same function will give an error at link time - but here - link time is moved to runtime.
What if this happens: -- src1.cpp #include "type.hpp" // serialization type #include <boost/archive/xml_iarchive.hpp> BOOST_CLASS_EXPORT(type); // instantiates serialization for iarchive.hpp -- src2.cpp #include "type.hpp" #include <boost/archive/xml_oarchive.hpp> BOOST_CLASS_EXPORT(type); // instantiates serialization for oarchive.hpp -- Why should this code be considered an error?
I would like to trap it as such, but there is no way to distinguish it from the legitimate case where the some type is exported from multiple DLLS.
Changing std::multiset to std::set would be enough, wouldn't it?
Nope - that was exactly the problem. If the same type is used in different DLLS, then there will be multiple static objects created each loaded/unloaded as the dlls themselves are loaded/unloaded.
But they will have different addresses won't they? The problem is with the same instance added twice to ktmap. Changing to std::set will allow the same type to be added, but not the same instance. Also, I would like to suggest changing ktmap and tkmap to a more descriptive name. I know it is an implementation detail, but it is very easy to get confused when debugging this.
Robert Ramey
-- Felipe Magno de Almeida
On Mon, Mar 31, 2008 at 6:35 PM, Robert Ramey <ramey@rrsd.com> wrote:
"Felipe Magno de Almeida" <felipe.m.almeida@gmail.com> wrote in message news:a2b17b60803311200v75904141rbe56c2713ef6fe7d@mail.gmail.com...
[snip]
Changing std::multiset to std::set would be enough, wouldn't it?
Nope - that was exactly the problem. If the same type is used in different DLLS, then there will be multiple static objects created each loaded/unloaded as the dlls themselves are loaded/unloaded.
Sorry, now I understand. I forgot about the key_compare. But it is possible to walk a equal_range and search for this address before adding. I'll patch my version and let you know if it works ok. This should let multiple BOOST_CLASS_EXPORT works.
Robert Ramey
Regards, -- Felipe Magno de Almeida
participants (3)
-
Felipe Magno de Almeida -
Robert Ramey -
Sohail Somani