[serialization] New version number in 1.34
The serialization::archive in Boost 1.34 has been bumped to 4. That isn't mentioned in the documentation (actually, there isn't a "new in 1.34" section in the changes docs, or any docs on changes since 2004). Secondly, is there any way to get this version to output 3? I'm pretty sure the answer is no, but the differences in my file are not important, and it is causing some problems with old code, so I was hoping there might be something I missed. Otherwise, has anyone tried using the old Serialization library in Boost 1.33? Thanks, Jared
The serialization library in 1.34 should be upward compatible with previous versions. The version number has been incremented to support this. It is only for this reason that the number has been incremented. So rolling it back would be expected to cause problems. Jared McIntyre wrote:
it is causing some problems with old code
I would not expect this. What kind of problems.
Otherwise, has anyone tried using the old Serialization library in Boost 1.33?
I would guess that is what most people use now. Robert Ramey
Robert Ramey <ramey <at> rrsd.com> writes:
The serialization library in 1.34 should be upward compatible with previous versions.
The version number has been incremented to support this. It is only for this reason that the number has been incremented. So rolling it back would be expected to cause problems.
Jared McIntyre wrote:
it is causing some problems with old code
I would not expect this. What kind of problems.
It is an issue of my own making. We still have quite a few released libraries built on boost 1.33. Our newer prototype version is 1.34 based. It is now generating files that can't be read by those older binaries. In my case, I am lucky enough that the format is actually compatibly, except for the version number and a couple previously non-existend lines (if I change the file to 3, and remove two lines, it all works). So, my problem is that we need to move to Boost 1.34, but I need a serialization library that interoperates with 1.33 serialization. I could hack my code to post-process the file, but it would be ugly.
Otherwise, has anyone tried using the old Serialization library in Boost 1.33?
I would guess that is what most people use now.
I phrased that horribly. I meant, has anyone tried to use the Boost 1.33 version in 1.34 (instead of the one that ships with it).
Jared McIntyre wrote:
So, my problem is that we need to move to Boost 1.34, but I need a serialization library that interoperates with 1.33 serialization. I could hack my code to post-process the file, but it would be ugly.
Hmmm - If you're going to generate new data files and distribute them to be processed with the old binaries - why can't you distribute the new binaries at the same time? Robert Ramey
Robert Ramey <ramey <at> rrsd.com> writes:
Jared McIntyre wrote:
So, my problem is that we need to move to Boost 1.34, but I need a serialization library that interoperates with 1.33 serialization. I could hack my code to post-process the file, but it would be ugly.
Hmmm - If you're going to generate new data files and distribute them to be processed with the old binaries - why can't you distribute the new binaries at the same time?
Robert Ramey
I can't get into the special way that our software is release/delivered, so you'll have to trust me when I say that would add 2-3 months of testing time to do it. I won't be able to get management to okay that for this change. We will probably do it in about a year, but we would like to be using Boost 1.34 between now and then, and the only reason that won't work is that some of the serialized files that are shared between sub-releases are serialization version 3 (binaries are not reused across sub-releases, so for a sub-release that uses 1.33, it is all 1.33, and so forth -- no risk of binary incompatibilities). So I'm trying to come up with a workaround so that these two sets of sub-releases can use the one serialization file shared between them. Jared
Jared McIntyre wrote:
Robert Ramey <ramey <at> rrsd.com> writes:
Jared McIntyre wrote:
So, my problem is that we need to move to Boost 1.34, but I need a serialization library that interoperates with 1.33 serialization. I could hack my code to post-process the file, but it would be ugly.
Hmmm - If you're going to generate new data files and distribute them to be processed with the old binaries - why can't you distribute the new binaries at the same time?
Robert Ramey
I can't get into the special way that our software is release/delivered, so you'll have to trust me when I say that would add 2-3 months of testing time to do it. I won't be able to get management to okay that for this change.
Well, its pretty hard to swim in a straightjacket. But, if that's what you feel you have to do, try the following: a) grep the code for everywhere that the serialization library version is used. There are only a couple of places - mostly in the stl class serialization I think. b) avoid using those types. or c) re-implement the serialization for those types by reverting to the 1.33 version for those types. Good Luck. Robert Ramey
Robert Ramey <ramey <at> rrsd.com> writes:
Well, its pretty hard to swim in a straightjacket.
:) Actually, I think my example makes it possible to even contemplate delivering the changes. If I were delivering these files directly to end-users where I couldn't control what version of the software they were on, it would be really hard to deal with (like if these files were posted on a server somewhere and all the customers software regularly fetched that file). I have a project that I was thinking of using serialization for, and I may have to rethink that now, or at least cover myself more carefully than I expected.
a) grep the code for everywhere that the serialization library version is used. There are only a couple of places - mostly in the stl class serialization I think.
I'll probably give this a shot. Asside from the version number, there are a couple additional tags that mark that it is using version 0 of a class. Before it didn't post anything for version 0. I'll check to see if that even needs modifying or if the 1.33 code can digest version 0 tags for objects. Either way, since there were no major changes to the desrialization format of the objects I'm using, I can probably get this to work.
Good Luck.
Thanks. And thanks for the help. P.S. One of these days I'm also going to get around to implementing those xml forward-compatibility deserialization we talked about ages ago which would probably have helped me here if I'd gotten it done when we talked about it. Jared
Jared McIntyre <jmcintyre <at> dfsoftware.com> writes:
Robert Ramey <ramey <at> rrsd.com> writes:
a) grep the code for everywhere that the serialization library version is used. There are only a couple of places - mostly in the stl class serialization I think.
I'll probably give this a shot. Asside from the version number, there are a couple additional tags that mark that it is using version 0 of a class. Before it didn't post anything for version 0. I'll check to see if that even needs modifying or if the 1.33 code can digest version 0 tags for objects. Either way, since there were no major changes to the desrialization format of the objects I'm using, I can probably get this to work.
I changed ARCHIVE_VERSION in basic_archive.cpp to 3 (from 4) and this worked for me. Luckily, I'm not using any collections whose serialization format has changed. Jared
On Thu, 2007-21-06 at 17:56 +0000, Jared McIntyre wrote:
I changed ARCHIVE_VERSION in basic_archive.cpp to 3 (from 4) and this worked for me. Luckily, I'm not using any collections whose serialization format has changed.
Even if you are, shouldn't it be class_version that matters *not* the serialization library version? Sohail
Sohail Somani wrote:
On Thu, 2007-21-06 at 17:56 +0000, Jared McIntyre wrote:
I changed ARCHIVE_VERSION in basic_archive.cpp to 3 (from 4) and this worked for me. Luckily, I'm not using any collections whose serialization format has changed.
Even if you are, shouldn't it be class_version that matters *not* the serialization library version?
a subtle but important distinction. versions numbers are generally attributes of the class definition/declaration. Including version information with class serialzation can include a small performance penalty and I thought it reasonable for STL serialization to never change (silly me). So I left the STL serializations unversioned. When it came time to change the STL serializations - I had to bump the "global" library version and use that number to branch code. The funny thing about this is that I branched the "save" code as well as the "load" code. (except in optional.hpp). I know I did this without thinking about it, I just implemented symetry out of habit. SO I see now that with relatively small effort - adding a function to the output archive interfac - set_library_version(int) - one could generate backward compatible archives. Except for one thing - the classes which store thier versions along with the class. In order to implement this one would do the following: a) implement "version awareness" for the classes "save" functions. I would guess the difficulty of this would depend upon the features of the already implemented serialization. b) replace the BOOST_CLASS_VERSION(<class name>, <latest class version>) with BOOST_CLASS_VERSION(<class name>, <older class version>). I presume that those who do this would make them selves some pre-processor code which invokes the correct version macro. I suspect this might not be doable with some of more complex serializatons such as shared_ptr - but anyway. All this actually turns out to be kind of amazing to me and totally unexpected. If someone thinks this is interesting and/or important enough to make demo/test along with an addition to the documentation (see - case studies), I would be willing to make the minor changes in the code to support this. Robert Ramey
Sohail
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Robert Ramey <ramey <at> rrsd.com> writes:
If someone thinks this is interesting and/or important enough to make demo/test along with an addition to the documentation (see - case studies), I would be willing to make the minor changes in the code to support this.
I think this could be very useful. I think the reasoning for my situation is rare, but that the situation itself isn't. The scenario Joaquín described earlier is something I think would be a common problem. I'd hate to be in a position where I couldn't upgrade boost, or had to try to use new boost with old serialization, because I need an older serialization format. If you do start to work on it, please let me know. I'd like to take a look at it. Perhaps I could provide you some useful feedback. Jared
On Thu, 21 Jun 2007 11:55:30 -0700, Robert Ramey <ramey@rrsd.com> wrote:
Sohail Somani wrote:
On Thu, 2007-21-06 at 17:56 +0000, Jared McIntyre wrote:
I changed ARCHIVE_VERSION in basic_archive.cpp to 3 (from 4) and this worked for me. Luckily, I'm not using any collections whose serialization format has changed.
Even if you are, shouldn't it be class_version that matters *not* the serialization library version?
a subtle but important distinction.
versions numbers are generally attributes of the class definition/declaration.
Including version information with class serialzation can include a small performance penalty and I thought it reasonable for STL serialization to never change (silly me). So I left the STL serializations unversioned.
When it came time to change the STL serializations - I had to bump the "global" library version and use that number to branch code.
It might be worth sharing my experience with this change in version number. Our product has its own saved files, which are themselves versioned. The mechanism for doing this is as old as C++ and is very intrusive. I came up with a mechanism to embed boost::serialization output in our saved files. Very very slick. Good job, by the way. When I upgraded to 1.34.0, some of our tests started to silently fail. Surprisingly, there was no crash, and we attributed the differences to something else entirely. It took a bit of effort to realize the values were not restoring the same as before. When we determined it was a serialization issue, I immediately looked in the documentation for any notes about breaking changes. I searched the mailing list for references. Nothing helped. I then stepped through the code to determine what changed, which revealed the item_version (if I remember correctly) change which was bracketed by a boost archive version check. The fix was fairly straight-forward. Since we have our own version numbers, I merely created a mapping of our versions to boost archive versions. I also added some code that will immediately crash the application as soon as the boost version is incremented again so the developer knows to update the mapping and bump our version number. All's well that ends well, but some more information up front would have helped me track this down faster. Jeff Faust
All's well that ends well, but some more information up front would have helped me track this down faster.
I never anticipated that anyone would need to do this. If I had, I don't think I could have figured out how to do it. I did address by accident - but that's all it was.
Jeff Faust _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Sohail Somani <s.somani <at> fincad.com> writes:
On Thu, 2007-21-06 at 17:56 +0000, Jared McIntyre wrote:
I changed ARCHIVE_VERSION in basic_archive.cpp to 3 (from 4) and this worked for me. Luckily, I'm not using any collections whose serialization format has changed.
Even if you are, shouldn't it be class_version that matters *not* the serialization library version?
If I had been using any objects whose individual serialization version/format had changed, I wouldn't have been able to trick the 1.33 using binaries into reading the format output by my new 1.34 using binaries. So, I think I got lucky and this hack works. I don't know how likely others will be in running into problems with individual class version increments. Jared
Jared McIntyre wrote:
Even if you are, shouldn't it be class_version that matters *not* the serialization library version?
As I said before, the class serialization attributes for STL collections and types are set to NOT versioned. So I had to bump the whole library version. So normally you'd be right - this is an unusual case Robert Ramey _______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Robert Ramey ha escrito:
Jared McIntyre wrote:
So, my problem is that we need to move to Boost 1.34, but I need a serialization library that interoperates with 1.33 serialization. I could hack my code to post-process the file, but it would be ugly.
Hmmm - If you're going to generate new data files and distribute them to be processed with the old binaries - why can't you distribute the new binaries at the same time?
Because there can be a deployed SW base whose users you cannot force to upgrade --think WWW, servers are under your control and can be cheaply updated, while the deployed browsers will span several versions and be in some cases older than you'd like. Wouldn't it be wise if Boost.Serialization provided backwards compatibility except when doing internal improvements or using new features? Are there really such breaking changes in the 1.33.1-->1.34 transition, apart from the number bump? The decision of automatically making Boost 1.n+1 savers backwards incompatible with Boost 1.n loaders is IMHO a bit controversial and can become a barrier to upgrading (like the case we're discussing right now.) This problem is akin to the ABI issues presented by C++ compilers: breaking it should only be done if strictly necessary. If breaking changes are actually present in Boost 1.34, how hard would it be to provide a 1.33.1 compatibility mode to cope with these situations and so facilitate users' upgrade from 1.33.1 to 1.34? Thank you, Joaquín M López Muñoz Telefónica, Investigación y Desarrollo
Joaquín Mª López Muñoz wrote:
how hard would it be to provide a 1.33.1 compatibility mode to cope with these situations and so facilitate users' upgrade from 1.33.1 to 1.34?
I did a quick grep through my current code and found something interesting. It turns out that "get_library_version" is only used in a couple of places. And what's more, they are in pairs (except for the one in optional.hpp). This fact is amazing to me. I suggests to me that I was depending on symetry to avoid using more that 30% of my available brain power. So that suggests to me that all one would have to do is to augment the library flags used at construction - (with an alternate constructor to avoid breaking interface) to permit specification of a previous archive version - right now its a constant built in at compile time. So, I'll consider this for 1.35. One potential "gotcha". Certain types have version numbers (different that the serialization library version number). So to generate backwards compatibility for those types, more machinery would have to be generated. That might be handled with a custome VERSION macro and augmentation of the serialization of those types. Good Luck with this. Robert Ramey
participants (5)
-
Jared McIntyre -
Jeffrey Faust -
Joaquín Mª López Muñoz -
Robert Ramey -
Sohail Somani