Serialisation across libraries.
Hello all, I am a bit stumped by a problem I am seeing with serialisation, and wondered whether it may be due to the way I sue some libraries. I have two libraries, one which contains the main code that uses serialisation, and another which contains some small classes used by the main code. The main code serialises a tree like data structure containing some shared pointers, each node of the 'tree' contains member data, and its one of these members that is causing a double free or corruption error on loading. This particular member is a structure from the other library. It contains a serialisation template in the class header which appears to save correctly - the XML output all appears correct. However on loading the double free error occurs. I cannot see anything wrong with the code at all. The only thing I can think of is that there are limitations in doing serialisation across the library boundary. Perhaps I end up with two instantiations of the template for serialisation? One in the main library, and one in the small library? Am I clutching at straws? DO I somehow have to force the instantiation of the serialisation for the class in the sub library to be in there, rather than being created again in the main library? Any thoughts? James This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.
A little extra information would be helpful. a) what compiler/linker etc are you using b) which version of boost serialization are you using. c) what kind of library: static, DLL, or? d) Is this an issue related to shared_ptrs or more general? Robert Ramey Hughes, James wrote:
Hello all,
I am a bit stumped by a problem I am seeing with serialisation, and wondered whether it may be due to the way I sue some libraries.
I have two libraries, one which contains the main code that uses serialisation, and another which contains some small classes used by the main code.
The main code serialises a tree like data structure containing some shared pointers, each node of the 'tree' contains member data, and its one of these members that is causing a double free or corruption error on loading. This particular member is a structure from the other library. It contains a serialisation template in the class header which appears to save correctly - the XML output all appears correct. However on loading the double free error occurs.
I cannot see anything wrong with the code at all. The only thing I can think of is that there are limitations in doing serialisation across the library boundary. Perhaps I end up with two instantiations of the template for serialisation? One in the main library, and one in the small library? Am I clutching at straws? DO I somehow have to force the instantiation of the serialisation for the class in the sub library to be in there, rather than being created again in the main library?
Any thoughts?
James
This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.
Hi Robert, I was worried there might not be enough information, so here goes. See below.
-----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Robert Ramey Sent: 08 May 2007 15:57 To: boost-users@lists.boost.org Subject: Re: [Boost-users] Serialisation across libraries.
A little extra information would be helpful.
a) what compiler/linker etc are you using
gcc 3.4.3 (Cross compiling to Arm)
b) which version of boost serialization are you using.
A completely standard 1.33.1
c) what kind of library: static, DLL, or?
All the libs involved are shared.
d) Is this an issue related to shared_ptrs or more general?
There are shared_ptr's involved. I will endeavour to describe the data
structure in the main library in more detail (limited to the Text case
which is all I am testing currently).
Message - the basic data to be serialised
contains some basic data and a vector of shared_ptrs to Panels (which
is a base class of a number of different panel types)
Panels - Contains some basic member data
TextPanel - Derived from Panels, contains some member data and a
structure, PanelFieldsList
PanelFieldsList contains a vector of shared_ptrs to Fields (which is a
base class to a number of different field types).
Fields
Base class contains a vector of raw pointers back to any panels which
reference them (but not currently serialised - trying to get the basics
working first - may add these in a post process of the loaded data to
save brain ache)
Now....The first area of problem, and the one described in my first
post, is that when I attempt to serialise a data structure that is
contained in an instance of Panels, it saves OK but on loading back in I
get this released twice/corruption error. The data structure is defined
in a header file from a different library (where is implementation is).
The header file for that DS contains the serialisation code, and a CPP
file in this other library contains a BOOST_SERIALIZATION_EXPORT_GUID
statement. Therefor this GUID is NOT available to the first library
until link time. I do wonder if this is where the problem arises. I
would like to keep the serialisation intrusive, in the appropriate
headers in the appropriate library rather than have to move to a
different location.
Now that I have described the DS in more detail, I can also add another
problem. Depending on which bits I attempt to serialise, on loading I
keep getting bus errors at various points, depending on which parts of
the DS I try to serialise. The save appears to work fine - I have
enclosed a dump of the saved DS.
I hope someone has some idea of what's going on, because I have been
staring at this all day to no avail.
Regards
James
Dump :::: Sorry about wrapping.....Is there any way I can reduce the
number of spaces output for indentation???
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE boost_serialization>
Robert Ramey
Hello all,
I am a bit stumped by a problem I am seeing with serialisation, and wondered whether it may be due to the way I sue some libraries.
I have two libraries, one which contains the main code that uses serialisation, and another which contains some small classes used by the main code.
The main code serialises a tree like data structure containing some shared pointers, each node of the 'tree' contains member data, and its one of these members that is causing a double free or corruption error on loading. This particular member is a structure from the other library. It contains a serialisation template in the class header which appears to save correctly - the XML output all appears correct. However on loading the double free error occurs.
I cannot see anything wrong with the code at all. The only
Hughes, James wrote: thing I can
think of is that there are limitations in doing serialisation across the library boundary. Perhaps I end up with two instantiations of the template for serialisation? One in the main library, and one in the small library? Am I clutching at straws? DO I somehow have to force the instantiation of the serialisation for the class in the sub library to be in there, rather than being created again in the main library?
Any thoughts?
James
This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.
I strongly suspect that this is a manifestation of a subtle bug/feature of serialization 133 and maybe 1.34. Its hard to explain in a short email. But the short version is that serialization for the same type is included more than one place - a shared library as well as mainline or another shared library. So you can consider how your code is distributed among the libraries with the idea that serialization for the same type is not spread accross various shared libraries. Robert Ramey Hughes, James wrote:
Hi Robert,
I was worried there might not be enough information, so here goes. See below.
-----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Robert Ramey Sent: 08 May 2007 15:57 To: boost-users@lists.boost.org Subject: Re: [Boost-users] Serialisation across libraries.
A little extra information would be helpful.
a) what compiler/linker etc are you using
gcc 3.4.3 (Cross compiling to Arm)
b) which version of boost serialization are you using.
A completely standard 1.33.1
c) what kind of library: static, DLL, or?
All the libs involved are shared.
d) Is this an issue related to shared_ptrs or more general?
There are shared_ptr's involved. I will endeavour to describe the data structure in the main library in more detail (limited to the Text case which is all I am testing currently).
Message - the basic data to be serialised contains some basic data and a vector of shared_ptrs to Panels (which is a base class of a number of different panel types)
Panels - Contains some basic member data
TextPanel - Derived from Panels, contains some member data and a structure, PanelFieldsList
PanelFieldsList contains a vector of shared_ptrs to Fields (which is a base class to a number of different field types).
Fields Base class contains a vector of raw pointers back to any panels which reference them (but not currently serialised - trying to get the basics working first - may add these in a post process of the loaded data to save brain ache)
Now....The first area of problem, and the one described in my first post, is that when I attempt to serialise a data structure that is contained in an instance of Panels, it saves OK but on loading back in I get this released twice/corruption error. The data structure is defined in a header file from a different library (where is implementation is). The header file for that DS contains the serialisation code, and a CPP file in this other library contains a BOOST_SERIALIZATION_EXPORT_GUID statement. Therefor this GUID is NOT available to the first library until link time. I do wonder if this is where the problem arises. I would like to keep the serialisation intrusive, in the appropriate headers in the appropriate library rather than have to move to a different location.
Now that I have described the DS in more detail, I can also add another problem. Depending on which bits I attempt to serialise, on loading I keep getting bus errors at various points, depending on which parts of the DS I try to serialise. The save appears to work fine - I have enclosed a dump of the saved DS.
I hope someone has some idea of what's going on, because I have been staring at this all day to no avail.
Regards
James
Dump :::: Sorry about wrapping.....Is there any way I can reduce the number of spaces output for indentation???
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?> <!DOCTYPE boost_serialization>
<Message class_id="0" tracking_level="0" version="0"> <Name>Message1</Name> <Filename class_id="1" tracking_level="0" version="0"> <path>/home/jhughes/msg.xml</path> </Filename> <PanelList class_id="2" tracking_level="0" version="0"> <count>1</count> <item class_id="3" tracking_level="0" version="1"> <px class_id="4" class_name="TextPanel" tracking_level="1" version="0" object_id="_0"> <LCPanel class_id="5" tracking_level="0" version="0"> <RasterHeight>7</RasterHeight> <PanelWidth>0</PanelWidth> <PanelHeight>7</PanelHeight> </LCPanel> <TextPanelFields class_id="6" tracking_level="1" version="0" object_id="_1"> <LCPanelFields class_id="7" tracking_level="1" version="0" object_id="_2"> <LCPanelFieldsFacade class_id="8" tracking_level="1" version="0" object_id="_3"></LCPanelFieldsFacade> <Fields class_id="9" tracking_level="0" version="0"> <count>1</count> <item class_id="10" tracking_level="0" version="1"> <px class_id="12" class_name="StaticTextField" tracking_level="1" version="0" object_id="_4">
<LCTextField class_id="13" tracking_level="1" version="0" object_id="_5">
<LCField class_id="11" tracking_level="1" version="0" object_id="_6"></LCField>
<Text></Text>
</LCTextField> </px> </item> </Fields> </LCPanelFields> </TextPanelFields> </px> </item> </PanelList> </Message>
Robert Ramey
Hello all,
I am a bit stumped by a problem I am seeing with serialisation, and wondered whether it may be due to the way I sue some libraries.
I have two libraries, one which contains the main code that uses serialisation, and another which contains some small classes used by the main code.
The main code serialises a tree like data structure containing some shared pointers, each node of the 'tree' contains member data, and its one of these members that is causing a double free or corruption error on loading. This particular member is a structure from the other library. It contains a serialisation template in the class header which appears to save correctly - the XML output all appears correct. However on loading the double free error occurs.
I cannot see anything wrong with the code at all. The only
Hughes, James wrote: thing I can
think of is that there are limitations in doing serialisation across the library boundary. Perhaps I end up with two instantiations of the template for serialisation? One in the main library, and one in the small library? Am I clutching at straws? DO I somehow have to force the instantiation of the serialisation for the class in the sub library to be in there, rather than being created again in the main library?
Any thoughts?
James
This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.
Hi To use boost serialization on vc7.0 , rtti is better to be disabled. One example can be found at boost\boost_1_33_1\libs\serialization\vc6ide\test_no_rtti There are three classes in the example (test_no_rtti) Orgininal code: class polymorphic_base {...} class polymorphic_d1 : public polymorphic_base {...} class polymorphic_d2 : public polymorphic_base {...} I modified the code as following: class polymorphic_base {...} class polymorphic_d1 : public polymorphic_base {...} class polymorphic_d2 : public polymorphic_base { .... void serialize(Archive &ar, const unsigned int /* file_version */) { ar & BOOST_SERIALIZATION_BASE_OBJECT_NVP(polymorphic_base); ar & boost::serialization::make_nvp("Pointer_Int",m_pnInt); ar & boost::serialization::make_nvp("Pointer_D1",m_pD1); } .... public: polymorphic_derived1 * m_pD1; int * m_pnInt; } The problem is occurred when compile the modifed code. I found that the code "ar & boost::serialization::make_nvp("Pointer_Int",m_pnInt);" leads the compile error " \boost\boost_1_33_1\boost\serialization\access.hpp(109): error C2228: left of '.serialize' must have class/struct/union type" If I modified the code from "ar & boost::serialization::make_nvp("Pointer_Int",m_pnInt);" to "ar & boost::serialization::make_nvp("Pointer_Int",*m_pnInt);" It will become ok, but, it leads to another problem when the pointer of m_pnInt is NULL. The above problem doesn't exist for another pointer m_pD1. Both of them are pointer. The difference is that m_pnInt is integer (basic object) and m_pD1 is user-defined class. I wonder what should I do to solve the mentioned problem (serialization of pointer of basic object). Best regards Barr Ng _________________________________________________________________ Catch suspicious messages before you open themwith Windows Live Hotmail. http://imagine-windowslive.com/hotmail/?locale=en-us&ocid=TXT_TAGHM_migration_HM_mini_protection_0507
On May 9, 2007, at 8:40 AM, Ng Pan wrote:
The above problem doesn't exist for another pointer m_pD1. Both of them are pointer. The difference is that m_pnInt is integer (basic object) and m_pD1 is user-defined class.
I wonder what should I do to solve the mentioned problem (serialization of pointer of basic object).
As far as I know, serialization of pointers to primitive types (such as int) is not supported by the serialization library. Just serialize the integer directly. Matthias
Matthias Troyer wrote:
On May 9, 2007, at 8:40 AM, Ng Pan wrote:
The above problem doesn't exist for another pointer m_pD1. Both of them are pointer. The difference is that m_pnInt is integer (basic object) and m_pD1 is user-defined class.
I wonder what should I do to solve the mentioned problem (serialization of pointer of basic object).
As far as I know, serialization of pointers to primitive types (such as int) is not supported by the serialization library. Just serialize the integer directly.
Matthias
pointers to primitive types can only be serialized by including them in a "wrapper" which itself can serialized througha pointer. Robert Ramey
Hi Robert, I have an inkling that I may understand what the feature/bug is, and this seems to coincide with what you say here. I'm wondering whether I can use non-intrusive serialisation for the out of library classes in the main library, and remove the intrusive code from the sub-library to get round the problem. I'll give it a go tomorrow - just had the yearly round of pay rises, so need to go and drown my sorrows. James
-----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Robert Ramey Sent: 09 May 2007 02:57 To: boost-users@lists.boost.org Subject: Re: [Boost-users] Serialisation across libraries.
I strongly suspect that this is a manifestation of a subtle bug/feature of serialization 133 and maybe 1.34.
Its hard to explain in a short email. But the short version is that serialization for the same type is included more than one place - a shared library as well as mainline or another shared library.
So you can consider how your code is distributed among the libraries with the idea that serialization for the same type is not spread accross various shared libraries.
Robert Ramey
Hughes, James wrote:
Hi Robert,
Snipped a load... This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.
participants (4)
-
Hughes, James -
Matthias Troyer -
Ng Pan -
Robert Ramey