Hello again,
I have started using your serialization library. It is working pretty well, but I have run into a few small problems:
1) No serialization for bitset: No big deal, because I can just convert to ulong, but I was surprised it wasn't there.
Feel free to add it in along with a test. I would like to see people adding serialization bit by bit to the existing data structures as the need becomes apparent. We're coming to the view that these things should be added to the data structures themselves rather than the serialization library. I'm trying to get myself out of the business of serializing things.
2) I have a text archive with a bunch of pairs of strings. I want to append another pair. There doesn't seem to a way to append items to an archive.
LOL - an interesting question that only recently - (in the last couple of months) occurred to me. There is no reason that you can't create an archive and just append to it as many times as you see fit. Oa << t; ... Oa << t; EXCEPT that the t (and its constituents) should have its serialization trait set so that memory tracking is turned off. Otherwise, copies subsequent to the first one won't copy any data - just a pointer to the original. I have considered making and testing an example of such a thing, but I've been busy. To support this and make a good example what is really needed is: a) another archve construction flag - "no_tracking" to suppress memory tracking for the archive regardless of the setting for each individual type. b) serialization added to boost::any and/or boost::variant. Probably doable but maybe a little tricky. In this way one could just save willy-nilly anything to the archive and load later. This would be very interesting for things like database transaction logs which want to implement rollback/rollforward and things like that. It would also make a dynamite log for debugging. Again, feel free to explore this at your leisure.
3) similarly, if I'm reading items from an archive, there is no simple way to keep reading until the end of the file. I have to catch the exception, which seems wrong for such an unexceptional circumstance. Setting a fail bit as with streams would be nicer.
Checking the fail bit on the stream before reading the next archived item should work. It's intended to anyway. If it doesn't work it's a bug. There might be a little tweaking required as I didn't test for this specifically. If you try this and it doesn't work and you find the oversight let me know and I'll add the fix. Better yet, make and test a small test and submit it to me and I'll add it into the test suite.
However, you could take the view that you should always know what is in the archive, and to have a method of failure that does not throw an exception would invite errors. That suggests that a constructor flag might work (e.g. throw_on_eof default to true).
I recently discovered that one can set a flag on an input stream so that an attempt to read past eof will throw an exception. This would leave all options open. After learning ths I'm considering removing the code that checks the fail bit as I would like to leave the flexibility for the user and make the serialization library smaller. BTW the fail bit means - "the next operation is expected to fail", NOT "the last operation failed". This confused me for quite a while until I actually carefully read the documentation for stream rather than presuming what I thought was obvious.
4) A minor point: It would be nice to be able to construct an archive with a boost::filesystem::path, instead of creating an fstream and then using that to create the archive. It is certainly nice that I can attach an existing stream to an archive, but directly using a path would reduce some clutter. It would also make it a little safer, because I couldn't accidently insert something into the stream that was not serialized.
Feel free to make your own - it should be trivial exercise. It should be easy to do deriving from any of the current archives. Polymorhphic archive works this way. Its one template that "adapts" any archive it's applied to. Also the current setup is specifically designed to permit serialized data to be embedded within some other data stream if the user requires it. I haven't specifically tested this either.
I have read large parts of the discussions when you were submitting your library, but not all. If I have missed something, you can just point me to it.
Seems like you've pretty much got it. The thing that takes special care is the serialization of pointers through abstract base pointers. This works fine but there are a couple of non-intuitive aspects that can trip you up. The manual is pretty complete. This is mostly because I kept adding to it as questions kept coming up on the list. So I would hope that one can understand and use the package without going back and trolling through the list. If you don't mind, I'm posting this to the list. Your questions are interesting, good grist for the list, and help diminish the frequency of the same questions getting asked multiple times. Robert Ramey
"Robert Ramey" <ramey@rrsd.com> wrote:
Hello again,
I have started using your serialization library. It is working pretty well, but I have run into a few small problems:
1) No serialization for bitset: No big deal, because I can just convert to ulong, but I was surprised it wasn't there.
Feel free to add it in along with a test. I would like to see people adding serialization bit by bit to the existing data structures as the need becomes apparent. We're coming to the view that these things should be added to the data structures themselves rather than the serialization library. I'm trying to get myself out of the business of serializing things.
Ok. I just wanted to make sure that you hadn't missed it by accident. Since I've got my stuff working, I probably won't get around to implementing bitset, though.
2) I have a text archive with a bunch of pairs of strings. I want to append another pair. There doesn't seem to a way to append items to an archive.
LOL - an interesting question that only recently - (in the last couple of months) occurred to me.
<snip>
Again, feel free to explore this at your leisure.
Hmm. It looks like more work than I really want to put into this project. I think I will content myself with the simple hack I have been using (open the file in append mode, write a space, then open the archive with no header).
3) similarly, if I'm reading items from an archive, there is no simple way to keep reading until the end of the file. I have to catch the exception, which seems wrong for such an unexceptional circumstance. Setting a fail bit as with streams would be nicer.
Checking the fail bit on the stream before reading the next archived item should work. It's intended to anyway. If it doesn't work it's a bug. There might be a little tweaking required as I didn't test for this specifically. If you try this and it doesn't work and you find the oversight let me know and I'll add the fix. Better yet, make and test a small test and submit it to me and I'll add it into the test suite.
The text deserialization for string doesn't check for whether the deserialization of the size of the string succeeded. Since the size is undefined, that causes read for the string itself to fail. I have attached a patch which seems to fix that for strings only, but it needs to be replicated for all of the other items that have a length argument as well.
However, you could take the view that you should always know what is in the archive, and to have a method of failure that does not throw an exception would invite errors. That suggests that a constructor flag might work (e.g. throw_on_eof default to true).
I recently discovered that one can set a flag on an input stream so that an attempt to read past eof will throw an exception. This would leave all options open. After learning ths I'm considering removing the code that checks the fail bit as I would like to leave the flexibility for the user and make the serialization library smaller. BTW the fail bit means - "the next operation is expected to fail", NOT "the last operation failed". This confused me for quite a while until I actually carefully read the documentation for stream rather than presuming what I thought was obvious.
From near the top of page 598 of "The C++ Standard Library" by Nicolai Josuttis
After reading the last character, the flag eofbit is not yet set. The next attempt to read a character sets eofbit and failbit, because the read fails.
From the C++ standard, I see that when you're doing something like reading formatted input, it might set the eofbit without setting the failbit. But I can't see anything that says that the failbit or eofbit must be set preemptively. I don't see how it could, because it has to read all of the whitespace before figuring out whether the next read will fail.
4) A minor point: It would be nice to be able to construct an archive with a boost::filesystem::path, instead of creating an fstream and then using that to create the archive. It is certainly nice that I can attach an existing stream to an archive, but directly using a path would reduce some clutter. It would also make it a little safer, because I couldn't accidently insert something into the stream that was not serialized.
Feel free to make your own - it should be trivial exercise. It should be easy to do deriving from any of the current archives. Polymorhphic archive works this way. Its one template that "adapts" any archive it's applied to.
Ok. This I probably do have time to do.
Also the current setup is specifically designed to permit serialized data to be embedded within some other data stream if the user requires it. I haven't specifically tested this either.
I think I might have cause to do that. If so, I'll holler if I find any bugs.
I have read large parts of the discussions when you were submitting your library, but not all. If I have missed something, you can just point me to it.
Seems like you've pretty much got it. The thing that takes special care is the serialization of pointers through abstract base pointers. This works fine but there are a couple of non-intuitive aspects that can trip you up. The manual is pretty complete. This is mostly because I kept adding to it as questions kept coming up on the list. So I would hope that one can understand and use the package without going back and trolling through the list.
If you don't mind, I'm posting this to the list. Your questions are interesting, good grist for the list, and help diminish the frequency of the same questions getting asked multiple times.
No problem. Walter Landry wlandry@ucsd.edu --- /home/boo/arx/arx/src/boost/{arch}/++pristine-trees/unlocked/boost/boost--arx/boost--arx--1.31/wlandry@ucsd.edu--arx/boost--arx--1.31--patch-2/boost/archive/impl/text_iarchive_impl.ipp 2004-09-01 13:06:33.000000000 -0400 +++ /home/boo/arx/arx/src/boost/boost/archive/impl/text_iarchive_impl.ipp 2004-09-01 13:12:52.000000000 -0400 @@ -39,13 +39,17 @@ size_t size; * this->This() >> size; // skip separating space - is.get(); - // borland de-allocator fixup - #if BOOST_WORKAROUND(_RWSTD_VER, BOOST_TESTED_AT(20101)) - if(NULL != s.data()) - #endif + if(is) + { + is.get(); + // borland de-allocator fixup + #if BOOST_WORKAROUND(_RWSTD_VER, BOOST_TESTED_AT(20101)) + if(NULL != s.data()) + #endif s.resize(size); - is.read(const_cast<char *>(s.data()), size); + if(is) + is.read(const_cast<char *>(s.data()), size); + } } #ifndef BOOST_NO_INTRINSIC_WCHAR_T
participants (2)
-
Robert Ramey -
Walter Landry