Re: [boost] [serialization] Must the objects being serialized outlive the archive?
Jeff Flinn wrote:
IIUC, tracked objects have their addresses place in a map, so that subsequent serialized pointers of the same value are serialized as a reference to the original object at that address, rather than serializing another copy of the object.
That was spot on! I did a couple of tests and I think I've confirmed this is indeed the behavior and the reason why data gets corrupt. On my platform (MSVC), new data gets allocated to the last free pointer's address and that confuses boost::serialization.
From your understanding, doesn't that basically means the "objects being serialized must outlive the archive"? Since there's no way to tell "new" to not allocate objects at an older address unless I keep objects there. So basically I must keep all data in memory until the serialization process is complete.
I tried to think hard about this. Do you see any workaround to this issue so that I don't have to effectively double the memory usage? (Maybe have boost::serialization write things out immediately and clear the internal cached pointers)? Maybe boost::serialization should provide that command or some kind of "flush" command? Best regards, Chris Yuen
Chris Yuen wrote:
Jeff Flinn wrote:
IIUC, tracked objects have their addresses place in a map, so that subsequent serialized pointers of the same value are serialized as a reference to the original object at that address, rather than serializing another copy of the object.
That was spot on! I did a couple of tests and I think I've confirmed this is indeed the behavior
It's also described in the documentation under "tracking"
and the reason why data gets corrupt. On my platform (MSVC), new data gets allocated to the last free pointer's address and that confuses boost::serialization.
This is only a problem because you delete the data while it's being serialized.
From your understanding, doesn't that basically means the "objects being serialized must outlive the archive"? Since there's no way to tell "new" to not allocate objects at an older address unless I keep objects there. So basically I must keep all data in memory until the serialization process is complete.
I tried to think hard about this. Do you see any workaround to this issue so that I don't have to effectively double the memory usage?
I don't see any doubling of memory usage here.
(Maybe have boost::serialization write things out immediately and clear the internal cached pointers)? Maybe boost::serialization should provide that command or some kind of "flush" command?
Maybe you don't want to serialize via a pointer. Just serialize the data itself. Then there is nothing to allocate/deallocate, etc. If you're having to go to such trouble, you must be doing something fundamentally wrong. Robert Ramey
Robert Ramey wrote:
Chris Yuen wrote:
Jeff Flinn wrote:
IIUC, tracked objects have their addresses place in a map, so that subsequent serialized pointers of the same value are serialized as a reference to the original object at that address, rather than serializing another copy of the object. That was spot on! I did a couple of tests and I think I've confirmed this is indeed the behavior
It's also described in the documentation under "tracking"
Perhaps it could be described in a clearer fashion? What text are you explicitly referring too? I just scanned Reference/Serializable Concept/Pointers/Pointers to Objects of Derived Classes. I don't see any explicit description of the requirements imposed by tracking.
and the reason why data gets corrupt. On my platform (MSVC), new data gets allocated to the last free pointer's address and that confuses boost::serialization.
This is only a problem because you delete the data while it's being serialized.
From your understanding, doesn't that basically means the "objects being serialized must outlive the archive"? Since there's no way to tell "new" to not allocate objects at an older address unless I keep objects there. So basically I must keep all data in memory until the serialization process is complete.
This statement is correct for tracked objects serialized by pointer.
I tried to think hard about this. Do you see any workaround to this issue so that I don't have to effectively double the memory usage?
I don't see any doubling of memory usage here.
Paraphrasing Chris: when properly serializing N Base instances to a single archive, all N Base instances must reside in memory simultaneously. What Chris's trying to do is serialize N Base instances only allocating memory for one Base instance at a time.
(Maybe have boost::serialization write things out immediately and clear the internal cached pointers)? Maybe boost::serialization should provide that command or some kind of "flush" command?
Maybe you don't want to serialize via a pointer. Just serialize the data itself. Then there is nothing to allocate/deallocate, etc.
If you're having to go to such trouble, you must be doing something fundamentally wrong.
I haven't tried [de]serializing through a ref to base class, can that be done? Jeff
participants (3)
-
Chris Yuen -
Jeff Flinn -
Robert Ramey