Boost serialization storage question
Hello, I wondered how the Boost.Serialization library determines how many bytes to read from an input stream to "fill" a class with data. That is, does it read an entire input stream until the end of the stream and use that as the data to fill a class with, or does it perform a sizeof(class) and read that from the stream ? If it is the latter, then out of curiosity, how does it determine the size of an object with different class versions ( so if an object version 1 has been saved and then tries to load version 2 ) ? Does it write some sort of object size at the beginning of the file ? If it reads the entire stream until the end, what would be a good approach to save multiple objects in a single file, and be able to read just a few objects out of the file ? Thanks in advance for any responses. Regards, Leon Mergen
Leon Mergen wrote:
I wondered how the Boost.Serialization library determines how many bytes to read from an input stream to "fill" a class with data.
That is, does it read an entire input stream until the end of the stream and use that as the data to fill a class with, or does it perform a sizeof(class) and read that from the stream ?
Neither. Each class has its serialize function. Within this function it calls the operator << to copy the member data to archive. There are two cases: a) the data type is a primitive - int, etc - in which case the archive has method for storing it and knows the length of an int on the particular platform. b) the datatype is a class - in which case that class's serialize function is called. So eventually the serialization resolves to a string of calls to primitive types - all of which are supported by the archive class.
If it is the latter, then out of curiosity, how does it determine the size of an object with different class versions ( so if an object version 1 has been saved and then tries to load version
The first piece of data stored with the class is a version number. On loading, this is available to be used in conditional ar >> operations. When one adds a new member to the class - the version number should be updated so that the loading of new member can be skipped if the archive is an older version. 2 ) ? Does it write some sort of object size at the beginning of the file ? no.
If it reads the entire stream until the end,
None of the currently implmented archive classes do this.
what would be a good approach to save multiple objects in a single file, and be able to read just a few objects out of the file ?
Current archive classes don't support a good way to do this. Actually this is an iteresting application - but its not supported by current archives. One might be able make a new archive class or a new derivation from one of the existing ones to support this, but I have no idea how difficult it would be. At a minium it would require a pretty thorough understanding of the library and archive classes.
Thanks in advance for any responses.
you're welcome. Robert Ramey
On Sun, 2005-09-04 at 11:33 -0700, Robert Ramey wrote:
I wondered how the Boost.Serialization library determines how many bytes to read from an input stream to "fill" a class with data.
That is, does it read an entire input stream until the end of the stream and use that as the data to fill a class with, or does it perform a sizeof(class) and read that from the stream ?
Neither. Each class has its serialize function. Within this function it calls the operator << to copy the member data to archive. There are two cases:
a) the data type is a primitive - int, etc - in which case the archive has method for storing it and knows the length of an int on the particular platform.
b) the datatype is a class - in which case that class's serialize function is called.
So eventually the serialization resolves to a string of calls to primitive types - all of which are supported by the archive class.
Ok, sounds logical. What about objects from the stdlib, such as std::string ? I assume this function does not have a serialize() function, but in some prototypes I created I noticed that this worked perfectly.
If it is the latter, then out of curiosity, how does it determine the size of an object with different class versions ( so if an object version 1 has been saved and then tries to load version
The first piece of data stored with the class is a version number. On loading, this is available to be used in conditional ar >> operations. When one adds a new member to the class - the version number should be updated so that the loading of new member can be skipped if the archive is an older version.
Yes, I saw that in an example on the documentation. Looks like a great feature!
2 ) ? Does it write some sort of object size at the beginning of the file ?
no.
If it reads the entire stream until the end,
None of the currently implmented archive classes do this.
Ok, but then, out of curiosity, how does it know when to stop reading ?
what would be a good approach to save multiple objects in a single file, and be able to read just a few objects out of the file ?
Current archive classes don't support a good way to do this. Actually this is an iteresting application - but its not supported by current archives. One might be able make a new archive class or a new derivation from one of the existing ones to support this, but I have no idea how difficult it would be. At a minium it would require a pretty thorough understanding of the library and archive classes.
Yes, and I assume this will be a very hard task making it portable. Ok, guess I'll look further away from serializations for a solution... Thanks a lot for your lengthy reply, at least I'm all out of hopes for the boost serialization library now. :) Regards, Leon Mergen
Leon Mergen wrote:
So eventually the serialization resolves to a string of calls to primitive types - all of which are supported by the archive class.
Ok, sounds logical. What about objects from the stdlib, such as std::string ? I assume this function does not have a serialize() function, but in some prototypes I created I noticed that this worked perfectly.
I gave you the short version - the long version can be found in the manual which describes how to make a type serializable in a non-intrusive way.
Yes, I saw that in an example on the documentation. Looks like a great feature!
Indispensible in my opinion - otherwise there would be no way t read old archives if a class definition changed.
Ok, but then, out of curiosity, how does it know when to stop reading
Reading stops when no more data is required. . That is, data is only read when required, and it should be required if and only i \t was saved. Saved data and loaded must be kept exactly the same length. So each save function must exactly match it load counter part in length. This is usually easily addressed by using the & operator for most serialized classes.
Yes, and I assume this will be a very hard task making it portable.
I'm not sure how hard it would be to do it for a specific application. But it would require a detailed understanding of the library
Ok, guess I'll look further away from serializations for a solution...
God Luck Robert Ramey
participants (2)
-
Leon Mergen -
Robert Ramey