Serialization newbie, memory being overwritten?

Hello to you, I wonder if you might be able to solve this conundrum I've been pondering. The boost (1.51) serialization library is giving me some trouble. I've been battling back and forth with the library, and I'm beginning to think I'm missing some key element. At first, I had issues to just getting things written out to a file without crashes. As far as I could tell, this particular problem was compounded by the fact, that I tried to simplify my code by just writing a single primitive referenced by a pointer to an object containing said primitive. As I've since come to learn, serializing pointed-to primitives is apparently not trivial. I have since, gotten the serialization class to properly write out a few simple member primitives from my class, but when I read them back using the exact same function, it seems as though a lot - if not all - of the pre-existing data in the object, which I am not serializing, gets overwritten. As far as I have been able to tell from the tutorials, I am doing everything exactly by the book. I was also concerned that maybe boost doesn't allow partial serialization, but this shouldn't be an issue either as far as I can tell. I must be making some sort of colossal mistake that I just cannot see. My in and out code is fairly straight forward: std::string streamPath = PTMFilePath.string(); std::ifstream ifs(streamPath); boost::archive::xml_iarchive ia(ifs); ia >> BOOST_SERIALIZATION_NVP(aStream); ----- std::string streamPath = PTMFilePath.string(); std::ofstream ofs(streamPath); boost::archive::xml_oarchive oa(ofs); oa << BOOST_SERIALIZATION_NVP(aStream); ------ My serialization function: template<class Archive> void serialize(Archive &ar, const unsigned int file_version) { ar & BOOST_SERIALIZATION_NVP(nrOfSamples); } ------ aStream is a pointer to a custom class, where I have allowed boost access. I am really confused as to what is causing this. Both pointed to data and regular primitives in the aStream class are wiped/altered as soon as the program enteres the serialize function, and even before the actual data seems to be read. I must be overlooking something really basic here. I hope someone can whack me with the solution stick... Thanks, Lasse

Lasse Laursen wrote:
Hello to you,
I wonder if you might be able to solve this conundrum I've been pondering. The boost (1.51) serialization library is giving me some trouble. I've been battling back and forth with the library, and I'm beginning to think I'm missing some key element.
At first, I had issues to just getting things written out to a file without crashes. As far as I could tell, this particular problem was compounded by the fact, that I tried to simplify my code by just writing a single primitive referenced by a pointer to an object containing said primitive. As I've since come to learn, serializing pointed-to primitives is apparently not trivial.
I have since, gotten the serialization class to properly write out a few simple member primitives from my class, but when I read them back using the exact same function, it seems as though a lot - if not all - of the pre-existing data in the object, which I am not serializing, gets overwritten.
As far as I have been able to tell from the tutorials, I am doing everything exactly by the book. I was also concerned that maybe boost doesn't allow partial serialization, but this shouldn't be an issue either as far as I can tell. I must be making some sort of colossal mistake that I just cannot see.
My in and out code is fairly straight forward:
std::string streamPath = PTMFilePath.string(); std::ifstream ifs(streamPath); boost::archive::xml_iarchive ia(ifs);
ia >> BOOST_SERIALIZATION_NVP(aStream);
try ia >> aStream;
-----
std::string streamPath = PTMFilePath.string(); std::ofstream ofs(streamPath); boost::archive::xml_oarchive oa(ofs);
oa << BOOST_SERIALIZATION_NVP(aStream);
oa << aStream;
------
My serialization function:
template<class Archive> void serialize(Archive &ar, const unsigned int file_version) { ar & BOOST_SERIALIZATION_NVP(nrOfSamples); }
------
aStream is a pointer to a custom class, where I have allowed boost access. I am really confused as to what is causing this. Both pointed to data and regular primitives in the aStream class are wiped/altered as soon as the program enteres the serialize function, and even before the actual data seems to be read.
I must be overlooking something really basic here. I hope someone can whack me with the solution stick...
Thanks, Lasse

I'm using the xml archiving classes, which require the boost macro to be used, otherwise compilation fails. Even if applying the changes you recommended would work, that would mean that I can no longer store my serialized data as xml. But, I'll take and try most advice at this point and I can sadly report that the result is the same using the text archiving classes from boost and removing the macro as suggested. As soon as I enter the serialize function declared in my class via the
operator (which I take it boost overloads), the remaining data in aStream gets messed up for reasons that are still beyond me.
On 27-11-2012 16:27, Robert Ramey wrote:
Lasse Laursen wrote:
Hello to you,
I wonder if you might be able to solve this conundrum I've been pondering. The boost (1.51) serialization library is giving me some trouble. I've been battling back and forth with the library, and I'm beginning to think I'm missing some key element.
At first, I had issues to just getting things written out to a file without crashes. As far as I could tell, this particular problem was compounded by the fact, that I tried to simplify my code by just writing a single primitive referenced by a pointer to an object containing said primitive. As I've since come to learn, serializing pointed-to primitives is apparently not trivial.
I have since, gotten the serialization class to properly write out a few simple member primitives from my class, but when I read them back using the exact same function, it seems as though a lot - if not all - of the pre-existing data in the object, which I am not serializing, gets overwritten.
As far as I have been able to tell from the tutorials, I am doing everything exactly by the book. I was also concerned that maybe boost doesn't allow partial serialization, but this shouldn't be an issue either as far as I can tell. I must be making some sort of colossal mistake that I just cannot see.
My in and out code is fairly straight forward:
std::string streamPath = PTMFilePath.string(); std::ifstream ifs(streamPath); boost::archive::xml_iarchive ia(ifs);
ia >> BOOST_SERIALIZATION_NVP(aStream);
try
ia >> aStream;
-----
std::string streamPath = PTMFilePath.string(); std::ofstream ofs(streamPath); boost::archive::xml_oarchive oa(ofs);
oa << BOOST_SERIALIZATION_NVP(aStream);
oa << aStream;
------
My serialization function:
template<class Archive> void serialize(Archive &ar, const unsigned int file_version) { ar & BOOST_SERIALIZATION_NVP(nrOfSamples); }
------
aStream is a pointer to a custom class, where I have allowed boost access. I am really confused as to what is causing this. Both pointed to data and regular primitives in the aStream class are wiped/altered as soon as the program enteres the serialize function, and even before the actual data seems to be read.
I must be overlooking something really basic here. I hope someone can whack me with the solution stick...
Thanks, Lasse

Disclaimer: I haven't used the serialization library, nor have I carefully read your code example. I simply wonder whether your mental model of deserialization agrees with that used by Boost.Serialization. I believe (!) that when deserializing an object through a pointer, Boost.Serialization allocates a new object before filling (some of) its fields from the incoming data stream. You seem to expect Boost.Serialization to reuse an existing object in memory instead. (I hadn't previously encountered the phrase "partial serialization.") I believe that in the typical use case addressed by Boost.Serialization, the deserializing process has no direct access to the memory data used by the serializing process. That might be because the serializing process is shutting down for later resumption, or it might be that the serializing process is running on a different machine than the deserializing process. In either case, the only object fields on which the deserializing process can rely are those that were explicitly serialized and deserialized by your code. Put differently, any fields you fail to de/serialize must be assumed to be uninitialized. You may have an unusual use case. If you can somehow pass some of the fields of your memory object to the receiving process other than through Boost.Serialization -- may we ask why not all of them? Why are you using Boost.Serialization at all? Apologies if I haven't properly understood your issue.

Hey Nat, Thanks for your reply. I think you're right. That is to say, if your expectation of boost allocating a new object - which I'd say is a fair assumption - is correct, then my way of approaching the boost serialization object is the reason for all of this headache. I'm basically using the serialization object as a means to minimize the amount of code (and therefore needed maintenance) needed to save/load data on the HD, used by my application. I've been reading a bit about, and although there seems to be a significant overhead by using boost, I have (so far) judged it to still be worth using, as the data I need to save (from the object) will only increase in complexity. Most of the data contained in my object is meta-data created from an audio source. I'm only passing some of the fields, since I only want to re-create the meta-data, I create while processing the audio source. The actual audio (which the object also contains) is read from a separate file and does not need serialization. I'm all ears for an opinion on the approach. It would just appear to me that serialization is appealing to keep the code simple and still be quite flexible. Regards, Lasse On 28-11-2012 01:23, Nat Linden wrote:
Disclaimer: I haven't used the serialization library, nor have I carefully read your code example.
I simply wonder whether your mental model of deserialization agrees with that used by Boost.Serialization. I believe (!) that when deserializing an object through a pointer, Boost.Serialization allocates a new object before filling (some of) its fields from the incoming data stream. You seem to expect Boost.Serialization to reuse an existing object in memory instead. (I hadn't previously encountered the phrase "partial serialization.")
I believe that in the typical use case addressed by Boost.Serialization, the deserializing process has no direct access to the memory data used by the serializing process. That might be because the serializing process is shutting down for later resumption, or it might be that the serializing process is running on a different machine than the deserializing process. In either case, the only object fields on which the deserializing process can rely are those that were explicitly serialized and deserialized by your code. Put differently, any fields you fail to de/serialize must be assumed to be uninitialized.
You may have an unusual use case. If you can somehow pass some of the fields of your memory object to the receiving process other than through Boost.Serialization -- may we ask why not all of them? Why are you using Boost.Serialization at all?
Apologies if I haven't properly understood your issue.

On Tue, Nov 27, 2012 at 11:54 PM, Lasse Laursen
I'm all ears for an opinion on the approach. It would just appear to me that serialization is appealing to keep the code simple and still be quite flexible.
What about breaking out the metadata you want to serialize as a class, or family of classes, separate from the audio data you don't need to serialize? You could make the audio class point to an instance of the concrete metadata class, or vice-versa, or have a holder class reference both. In any case, the point would be to de/serialize all fields of the metadata class.

I had thought along the same line. I think that's a good idea Nat. For now, I'm just using serialize to instantiate the object and then filling in the remaining fields. A bit hacky, but it works. Regardless - I thank you for taking the time to help get my head on straight with regards to how serialization works in the context I was in Nat. I appreciate it! :) On 28-11-2012 22:43, Nat Linden wrote:
On Tue, Nov 27, 2012 at 11:54 PM, Lasse Laursen
wrote: I'm all ears for an opinion on the approach. It would just appear to me that serialization is appealing to keep the code simple and still be quite flexible.
What about breaking out the metadata you want to serialize as a class, or family of classes, separate from the audio data you don't need to serialize? You could make the audio class point to an instance of the concrete metadata class, or vice-versa, or have a holder class reference both. In any case, the point would be to de/serialize all fields of the metadata class.
participants (3)
-
Lasse Laursen
-
Nat Linden
-
Robert Ramey