[serialization] Support for multiple class name strings
I did something stupid with my serialization code. I blithely changed the names of several polymorphic classes everywhere they appeared in the code, since they had moved out of a class into that class's namespace (e.g. SomeNS::View::Primitive became SomeNS::Primitive). Several months later, I discovered that I could no longer read old binary archives, because I was using Boost.Serialization's BOOST_CLASS_EXPORT() macro, and this macro was now registering the Primtive type with the name-string "Primitive", instead of "View::Primitive", which is how it was saved in the oldest archives. Since there is a 1-1 mapping between name-strings and types, and since most of the Serialization code uses static/global data, I cannot currently support both the archives saved with and the archives saved without the "View::" in the same program. Since I suspect this is not the first time someone has done this, nor the last time someone will do so, is there a way to fix this using the current library? If not, would it be technically possible to add a BOOST_CLASS_EXPORT_ALIAS() macro, or some such? If it is possible, I'm willing to supply the implementation. Zach Laine
Zach Laine wrote:
I did something stupid with my serialization code. I blithely changed the names of several polymorphic classes everywhere they appeared in the code, since they had moved out of a class into that class's namespace (e.g. SomeNS::View::Primitive became SomeNS::Primitive). Several months later, I discovered that I could no longer read old binary archives, because I was using Boost.Serialization's BOOST_CLASS_EXPORT() macro, and this macro was now registering the Primtive type with the name-string "Primitive", instead of "View::Primitive", which is how it was saved in the oldest archives.
Ouch.
would it be technically possible to add a BOOST_CLASS_EXPORT_ALIAS() macro, or some such? If it is possible, I'm willing to supply the implementation.
I might be possible by hacking your own copy of extended_type_info.hpp. I don't think we'd want to include such thing in the library. So I would suggest: a) make the hack b) make small program which loads your old archives and creates new ones. This should be very easy. c) Forget this ever happened. Robert Ramey
Robert Ramey said: (by the date of Thu, 15 Mar 2007 18:56:25 -0800)
Zach Laine wrote:
I did something stupid with my serialization code. I blithely changed the names of several polymorphic classes everywhere they appeared in the code, since they had moved out of a class into that class's namespace (e.g. SomeNS::View::Primitive became SomeNS::Primitive). Several months later, I discovered that I could no longer read old binary archives, because I was using Boost.Serialization's BOOST_CLASS_EXPORT() macro, and this macro was now registering the Primtive type with the name-string "Primitive", instead of "View::Primitive", which is how it was saved in the oldest archives.
Ouch.
IMHO this is the best example demonstrating that sometimes using a class name as a GUID is not the best idea. If Zach from the very beginning used a custom string (different that the class name) as a GUID, this thing would have never happened. Am I right? -- Janek Kozicki |
On 3/16/07, Janek Kozicki <janek_listy@wp.pl> wrote:
Robert Ramey said: (by the date of Thu, 15 Mar 2007 18:56:25 -0800)
Zach Laine wrote:
I did something stupid with my serialization code. I blithely changed the names of several polymorphic classes everywhere they appeared in the code, since they had moved out of a class into that class's namespace (e.g. SomeNS::View::Primitive became SomeNS::Primitive). Several months later, I discovered that I could no longer read old binary archives, because I was using Boost.Serialization's BOOST_CLASS_EXPORT() macro, and this macro was now registering the Primtive type with the name-string "Primitive", instead of "View::Primitive", which is how it was saved in the oldest archives.
Ouch.
IMHO this is the best example demonstrating that sometimes using a class name as a GUID is not the best idea. If Zach from the very beginning used a custom string (different that the class name) as a GUID, this thing would have never happened. Am I right?
This was my thought as well. In fact, I'm going to switch all our BOOST_CLASS_EXPORT(T) macros to BOOST_CLASS_EXPORT_GUID(T, "[T's canonical name]") macros, and leave a giant note for others to do the same ("Here there be dragons", etc.). In light of this problem, I would say that while use of BOOST_CLASS_EXPORT() is convenient, it is ultimately a maintenance burden. Zach Laine
Zach Laine said: (by the date of Fri, 16 Mar 2007 09:02:04 -0500)
IMHO this is the best example demonstrating that sometimes using a class name as a GUID is not the best idea. If Zach from the very beginning used a custom string (different that the class name) as a GUID, this thing would have never happened. Am I right?
This was my thought as well. In fact, I'm going to switch all our BOOST_CLASS_EXPORT(T) macros to BOOST_CLASS_EXPORT_GUID(T, "[T's canonical name]") macros, and leave a giant note for others to do the same ("Here there be dragons", etc.). In light of this problem, I would say that while use of BOOST_CLASS_EXPORT() is convenient, it is ultimately a maintenance burden.
Robert, can you please put this example in the documentation? It would demonstrate clearly to new users why they should use BOOST_CLASS_EXPORT_GUID instead. Anyway, IMHO adding a support for multiple GUIDs is the best route to better support archives compatibility. -- Janek Kozicki |
This was my thought as well. In fact, I'm going to switch all our BOOST_CLASS_EXPORT(T) macros to BOOST_CLASS_EXPORT_GUID(T, "[T's canonical name]") macros, and leave a giant note for others to do the same ("Here there be dragons", etc.). In light of this problem, I would say that while use of BOOST_CLASS_EXPORT() is convenient, it is ultimately a maintenance burden.
In my opinion the convenience of BOOST_CLASS_EXPORT is offset by the problem Zach is facing, as well as by the physical coupling it introduces between user classes and boost::serialization. Because serialize/load/save are function templates, and because all serialize/load/save calls made from inside another serialize/load/save function template depend on a template parameter (the Archive type), in principle the serialization of a given class foo can be defined in foo.hpp without including any boost serialization headers, however this is not possible if BOOST_CLASS_EXPORT is used, because its purpose is to trigger the registration automatically just because foo.hpp is #included. I would suggest separating all registration in a single compilation unit called explicitly by the main initialization code (having everything in one place makes it much easier to keep track of the IDs for each class.) That compilation unit would include the headers of all classes that need to be serialized, as well as the registration facilities of boost serialization, and register each combination of class and archive that the program uses (each dynamically linked library would need its own similar initialization as well.) Emil Dotchevski
Emil Dotchevski wrote:
Because serialize/load/save are function templates, and because all serialize/load/save calls made from inside another serialize/load/save function template depend on a template parameter (the Archive type), in principle the serialization of a given class foo can be defined in foo.hpp without including any boost serialization headers,
agreed.
however this is not possible if BOOST_CLASS_EXPORT is used, because its purpose is to trigger the registration automatically just because foo.hpp is #included.
I don't think that's true. It was that code was only instantiated when an archive class header was also included. But this implied a header ordering requirement when some found difficult to live with. This was addressed in the HEAD so that there is no restriction on header order. My understanding was that this wouldn't change anything other than fixing the header ordering restriction. Robert Ramey
Emil Dotchevski wrote:
Because serialize/load/save are function templates, and because all serialize/load/save calls made from inside another serialize/load/save function template depend on a template parameter (the Archive type), in principle the serialization of a given class foo can be defined in foo.hpp without including any boost serialization headers,
agreed.
however this is not possible if BOOST_CLASS_EXPORT is used, because its purpose is to trigger the registration automatically just because foo.hpp is #included.
I don't think that's true. It was that code was only instantiated when an archive class header was also included. But this implied a header ordering requirement when some found difficult to live with. This was addressed in the HEAD so that there is no restriction on header order. My understanding was that this wouldn't change anything other than fixing the header ordering restriction.
I wasn't pointing at any implementation difficulties. All I am saying is that if I have class foo defined in foo.hpp, the use of BOOST_CLASS_EXPORT in foo.hpp is undesirable because it introduces physical coupling between the boost serialization library, class foo, and all code that's using class foo (even code that does not serialize objects of class foo), because such code would at least "see" the type registration facilities of boost serialization. I realize that this note is slightly off-topic to this thread, but I believe it is still relevant. I would rather use a system where the class IDs for all classes are specified "manually" in a single compilation unit where all registrations take place in a function called explicitly, when it makes sense to initialize the serialization system. This would have avoided some of the difficulties Zach experiences now. Emil Dotchevski
Emil Dotchevski wrote:
Emil Dotchevski wrote:
I wasn't pointing at any implementation difficulties. All I am saying is that if I have class foo defined in foo.hpp, the use of BOOST_CLASS_EXPORT in foo.hpp is undesirable because it introduces physical coupling between the boost serialization library, class foo, and all code that's using class foo (even code that does not serialize objects of class foo), because such code would at least "see" the type registration facilities of boost serialization.
I realize that this note is slightly off-topic to this thread, but I believe it is still relevant. I would rather use a system where the class IDs for all classes are specified "manually" in a single compilation unit where all registrations take place in a function called explicitly, when it makes sense to initialize the serialization system. This would have avoided some of the difficulties Zach experiences now.
I see BOOST_CLASS_EXPORT a little differently. The main function is to assign an externalizable string to the class to permit reference to it outside of any execution context. Currently part of the serialization library implementation is mixed into this. In hindsight I would have separated out things better. And maybe it will happen someday. But the macro itself is pretty opaque and its function beyond assigning an external name isn't convenienttly available to users. So from a user's point it assigns this name attribute to a class. So to me it seems natural to included in the header along with the whole class declaration. That way when that header gets used by other programs, they all share the same external name - as they should since they're sharing the same header. Of course when you change the external name of something, you're going to have problems connected with users of the old name - in this case existing archives. But, as you pointed out, that is a different problem. Robert Ramey
So to me it seems natural to included in the header along with the whole class declaration.
Consider a library that provides foo.h, which defines class foo, and a program that #includes "foo.h" because it needs to use class foo, but it doesn't need to serialize it. To me, it isn't natural for such a program to require boost serialization headers. I suppose you could separate serialize/load/save overloads in a different header file, but because typically they need to be friends, foo.h would have to declare them in order to make them friends, which is a hassle. Instead, you could simply define serialize/load/save directly in foo.h without including any serialization stuff -- except that a program that does serialize foo objects would have to register class foo "manually".
That way when that header gets used by other programs, they all share the same external name - as they should since they're sharing the same header.
If I want to make a particular class to have the same external name across multiple programs, this is easy to achieve by sharing the code that registers that class with boost serialization. I just don't think that the class' header file is the best place for this registration. Emil Dotchevski
Well, this went a little off-topic, but I still would like to know why Robert thinks that adding aliases is a bad idea. It seems to me that my mistake is a pretty likely one in the long term, and so the Serialization library needs to provide a way to recover from such mistakes, or reduce the chances of their introduction altogether (by removing BOOST_CLASS_EXPORT()). At the very least, if BOOST_CLASS_EXPORT() is to stay, an explicit note regarding this pitfall should be added to the docs. Again, I am happy to write a patch for the code and/or docs. Zach Laine On 3/16/07, Emil Dotchevski <emildotchevski@hotmail.com> wrote:
So to me it seems natural to included in the header along with the whole class declaration.
Consider a library that provides foo.h, which defines class foo, and a program that #includes "foo.h" because it needs to use class foo, but it doesn't need to serialize it. To me, it isn't natural for such a program to require boost serialization headers.
I suppose you could separate serialize/load/save overloads in a different header file, but because typically they need to be friends, foo.h would have to declare them in order to make them friends, which is a hassle. Instead, you could simply define serialize/load/save directly in foo.h without including any serialization stuff -- except that a program that does serialize foo objects would have to register class foo "manually".
That way when that header gets used by other programs, they all share the same external name - as they should since they're sharing the same header.
If I want to make a particular class to have the same external name across multiple programs, this is easy to achieve by sharing the code that registers that class with boost serialization. I just don't think that the class' header file is the best place for this registration.
Emil Dotchevski
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Zach Laine wrote:
Well, this went a little off-topic, but I still would like to know why Robert thinks that adding aliases is a bad idea. It seems to me that my mistake is a pretty likely one in the long term, and so the Serialization library needs to provide a way to recover from such mistakes, or reduce the chances of their introduction altogether (by removing BOOST_CLASS_EXPORT()). At the very least, if BOOST_CLASS_EXPORT() is to stay, an explicit note regarding this pitfall should be added to the docs. Again, I am happy to write a patch for the code and/or docs.
My first reaction is that its a bad idea. But I'm open to having my mind changed. What I'm concerned about is the possibility that adding such a feature would ripple down into the library making it (even) more complex, harder to maintain, perhaps hurting performance, etc. just to address a case which shouldn't have occurred in the first place. So why don't we proceed as follows: a) make the changes you need to make to address your current problem. b) When we get a look at the changes, we can decide whether they're simple and don't complicate the library or whether it starts a whole chain reaction. More interesting to me would be a way to somehow trap this situation before it gets to this point. Something like: An automatic serialization regression tester. Every time the version number is incremented, a (automatically generated?) test file is saved under the name ... version-1. And the all the version test files version-0 through version-n are read back to verify that backward compatibility has been maintained. Another idea would be that the exernal name set with BOOST_CLASS_EXPORT be associated with the current version number. I don't know how that would effect things, but it might be a long/deep ripple effect. So its not that I'm really opposed in principle, I just don't want to start mucking things up to deal with one unusual case which is the result of programmer error. Robert Ramey
On Mon, Mar 19, 2007 at 08:51:12AM -0800, Robert Ramey wrote:
Zach Laine wrote:
Well, this went a little off-topic, but I still would like to know why Robert thinks that adding aliases is a bad idea. It seems to me that my mistake is a pretty likely one in the long term, and so the Serialization library needs to provide a way to recover from such mistakes, or reduce the chances of their introduction altogether (by removing BOOST_CLASS_EXPORT()). At the very least, if BOOST_CLASS_EXPORT() is to stay, an explicit note regarding this pitfall should be added to the docs. Again, I am happy to write a patch for the code and/or docs.
My first reaction is that its a bad idea. But I'm open to having my mind changed. What I'm concerned about is the possibility that adding such a feature would ripple down into the library making it (even) more complex, harder to maintain, perhaps hurting performance, etc. just to address a case which shouldn't have occurred in the first place. So why don't we proceed as follows:
a) make the changes you need to make to address your current problem. b) When we get a look at the changes, we can decide whether they're simple and don't complicate the library or whether it starts a whole chain reaction.
I'm watching with interest too, we also have too much data (~5T and counting) to reprocess. In addition to the namespace-change situation, if has come up that two developers registered the same class under different typedeffed names... I'll have to dig around to recreate that one and supply all the details. -t
On 3/19/07, Robert Ramey <ramey@rrsd.com> wrote:
Zach Laine wrote:
Well, this went a little off-topic, but I still would like to know why Robert thinks that adding aliases is a bad idea. It seems to me that my mistake is a pretty likely one in the long term, and so the Serialization library needs to provide a way to recover from such mistakes, or reduce the chances of their introduction altogether (by removing BOOST_CLASS_EXPORT()). At the very least, if BOOST_CLASS_EXPORT() is to stay, an explicit note regarding this pitfall should be added to the docs. Again, I am happy to write a patch for the code and/or docs.
My first reaction is that its a bad idea. But I'm open to having my mind changed. What I'm concerned about is the possibility that adding such a feature would ripple down into the library making it (even) more complex, harder to maintain, perhaps hurting performance, etc. just to address a case which shouldn't have occurred in the first place. So why don't we proceed as follows:
a) make the changes you need to make to address your current problem. b) When we get a look at the changes, we can decide whether they're simple and don't complicate the library or whether it starts a whole chain reaction.
Fair enough. I'll have a patch ready soon and post it to the list. Unless I hear otherwise, I'll work against version 1.33.1.
More interesting to me would be a way to somehow trap this situation before it gets to this point. Something like:
An automatic serialization regression tester.
[snip] This sounds like a great idea to me. Zach Laine
Zach Laine said: (by the date of Mon, 19 Mar 2007 17:17:29 -0500)
Fair enough. I'll have a patch ready soon and post it to the list. Unless I hear otherwise, I'll work against version 1.33.1.
I suggest that you should work against CVS-HEAD, becasue Robert has already made significant changes in the library implementation in the HEAD. And if your work is to be added to boost, it will be added to HEAD. Converting your work from 1.33 to HEAD will be an extra unnecessary work. -- Janek Kozicki |
Robert Ramey said: (by the date of Mon, 19 Mar 2007 08:51:12 -0800)
So its not that I'm really opposed in principle, I just don't want to start mucking things up to deal with one unusual case which is the result of programmer error.
Robert, think about code refactoring, not about programmer's error. My biggest application (yade, you heard about it already) is constantly evolving, changing the class names is an important part of that evolution. Of course alternative solution, as Emil suggests, is not to change boost::serialization, but to register the classes manually (with manually chosen identifier string). But then we at least need a BIG FAT warning in the documentation regarding the use of BOOST_CLASS_EXPORT, and a possible pitfall: code refactoring loses archive compatibility. Anyway, we will see if changes proposed by Zach will produce that ripple effect of small modifications, and the decide on what to do :) -- Janek Kozicki |
On 3/20/07, Janek Kozicki <janek_listy@wp.pl> wrote:
Robert Ramey said: (by the date of Mon, 19 Mar 2007 08:51:12 -0800)
So its not that I'm really opposed in principle, I just don't want to start mucking things up to deal with one unusual case which is the result of programmer error.
Robert, think about code refactoring, not about programmer's error.
My biggest application (yade, you heard about it already) is constantly evolving, changing the class names is an important part of that evolution.
Of course alternative solution, as Emil suggests, is not to change boost::serialization, but to register the classes manually (with manually chosen identifier string). But then we at least need a BIG FAT warning in the documentation regarding the use of BOOST_CLASS_EXPORT, and a possible pitfall: code refactoring loses archive compatibility.
Anyway, we will see if changes proposed by Zach will produce that ripple effect of small modifications, and the decide on what to do :)
I've attached a patchfile against 1.33.1 that introduces a BOOST_CLASS_EXPORT_ALIAS() macro and a test program that shows that it works. In the test program, initially leave the #define SAVING 1 in place, compile, and execute to create a binary archive. Then #define SAVING 0, compile and execute to read the same archive in using the archived classes in a different namespace. This should blow up, as expected, since it doesn't use the new macro. Finally, #define EXPORT_ALIASES 1, and the archive should load using the aliases that include the old namespace. I'll wait to add anything further to the discussion until Robert has seen the changes and decides whether they are acceptible or need to be adjusted. Zach Laine
Of course alternative solution, as Emil suggests, is not to change boost::serialization, but to register the classes manually (with manually chosen identifier string). But then we at least need a BIG FAT warning in the documentation regarding the use of BOOST_CLASS_EXPORT, and a possible pitfall: code refactoring loses archive compatibility.
There is another warning which is missing from the documentation: portable code can not rely on BOOST_CLASS_EXPORT to register a class unless the user explicitly calls a function from a compilation unit that "sees" that particular BOOST_CLASS_EXPORT call. This is because it relies on a global object's constructor to initiate the registration process, but a good compiler will deadstrip all such global objects unless the user explicitly calls a function from the compilation unit that defines them. In other words BOOST_CLASS_EXPORT isn't automatic because a portable program is required to (manually) call a (possibly empty) function from the registered class' cpp file. This is contradictory to BOOST_CLASS_EXPORT's intended use to register derived classes automatically, just by linking their cpp files. Emil Dotchevski
Zajo wrote:
There is another warning which is missing from the documentation: portable code can not rely on BOOST_CLASS_EXPORT to register a class unless the user explicitly calls a function from a compilation unit that "sees" that particular BOOST_CLASS_EXPORT call. This is because it relies on a global object's constructor to initiate the registration process, but a good compiler will deadstrip all such global objects unless the user explicitly calls a function from the compilation unit that defines them. In other words BOOST_CLASS_EXPORT isn't automatic because a portable program is required to (manually) call a (possibly empty) function from the registered class' cpp file. This is contradictory to BOOST_CLASS_EXPORT's intended use to register derived classes automatically, just by linking their cpp files.
I don't think that's currently true - if it ever was. The code specifically addresses this issue with a number of compiler specific adjustments to insure that code is not stripped. These adjustments have proved effective for all compilers/linkers that boost supports (with the possible exception of CodeWarror). So maybe its true were the system compiled on a strickly comformant compiler which has no extensions to prevent code stripping, but as far as I know, there is no such compiler being used. I also believe that its unlikely that any such compiler/combination will ever see the light of day since it would be useless for building DLLs (shared libraries). Robert Ramey
Emil Dotchevski
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
There is another warning which is missing from the documentation: portable code can not rely on BOOST_CLASS_EXPORT to register a class unless the user explicitly calls a function from a compilation unit that "sees" that particular BOOST_CLASS_EXPORT call. This is because it relies on a global object's constructor to initiate the registration process, but a good compiler will deadstrip all such global objects unless the user explicitly calls a function from the compilation unit that defines them. In other words BOOST_CLASS_EXPORT isn't automatic because a portable program is required to (manually) call a (possibly empty) function from the registered class' cpp file. This is contradictory to BOOST_CLASS_EXPORT's intended use to register derived classes automatically, just by linking their cpp files.
I don't think that's currently true - if it ever was. The code specifically addresses this issue with a number of compiler specific adjustments to insure that code is not stripped. These adjustments have proved effective for all compilers/linkers that boost supports (with the possible exception of CodeWarror). So maybe its true were the system compiled on a strickly comformant compiler which has no extensions to prevent code stripping, but as far as I know, there is no such compiler being used. I also believe that its unlikely that any such compiler/combination will ever see the light of day since it would be useless for building DLLs (shared libraries).
The key word in my statement is portable. I know there are non-portable workarounds on most compilers, but next versions of these same compilers could require a new workaround, and the standard tells you that such workaround may not be possible. Preventing a global object from being deadstripped still does not guarantee that it will be initialized before you call a function from its compilation unit. Suppose you have a global object that allocates a lot of memory, this provision in the standard allows the compiler to postpone this allocation until it is actually needed. In several large code bases I have spent many hours fighting with this same issue of various registration schemes being deadstripped because they rely on a global object to boot. In fact what prompted my post is that I'm currently having the same problem with a popular 3D engine. I have to dig through all of their macro trickery, static class members and whatnot, to figure out what needs to be initialized -- It's very frustrating, and by far outweights any benefits of their "automatic" registration. I think that no harm would be done if the documentation states that BOOST_CLASS_EXPORT relies on compiler-specific tricks and is not portable. This is especially important for a Boost library. Emil Dotchevski
Emil Dotchevski wrote:
I think that no harm would be done if the documentation states that BOOST_CLASS_EXPORT relies on compiler-specific tricks and is not portable. This is especially important for a Boost library.
That's certainly reasonable. I'll update the documentation in the HEAD. Robert Ramey
Emil Dotchevski
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On 3/15/07, Robert Ramey <ramey@rrsd.com> wrote:
Zach Laine wrote:
I did something stupid with my serialization code. I blithely changed the names of several polymorphic classes everywhere they appeared in the code, since they had moved out of a class into that class's namespace (e.g. SomeNS::View::Primitive became SomeNS::Primitive). Several months later, I discovered that I could no longer read old binary archives, because I was using Boost.Serialization's BOOST_CLASS_EXPORT() macro, and this macro was now registering the Primtive type with the name-string "Primitive", instead of "View::Primitive", which is how it was saved in the oldest archives.
Ouch.
would it be technically possible to add a BOOST_CLASS_EXPORT_ALIAS() macro, or some such? If it is possible, I'm willing to supply the implementation.
I might be possible by hacking your own copy of extended_type_info.hpp. I don't think we'd want to include such thing in the library. So I would suggest:
a) make the hack b) make small program which loads your old archives and creates new ones. This should be very easy. c) Forget this ever happened.
Why don't you want to include such a feature in the library? I'll probably go ahead and create the alias-aware patch I was talking about before, and start using the patched version of Boost until I no longer need to support the old data format. Unfortunately, I'm not in a position to reprocess all the old-format data. Zach Laine
On 3/15/07, Zach Laine <whatwasthataddress@gmail.com> wrote:
I did something stupid with my serialization code. I blithely changed the names of several polymorphic classes everywhere they appeared in the code, since they had moved out of a class into that class's namespace (e.g. SomeNS::View::Primitive became SomeNS::Primitive). Several months later, I discovered that I could no longer read old binary archives, because I was using Boost.Serialization's BOOST_CLASS_EXPORT() macro, and this macro was now registering the Primtive type with the name-string "Primitive", instead of "View::Primitive", which is how it was saved in the oldest archives.
Since there is a 1-1 mapping between name-strings and types, and since most of the Serialization code uses static/global data, I cannot currently support both the archives saved with and the archives saved without the "View::" in the same program.
Since I suspect this is not the first time someone has done this, nor the last time someone will do so, is there a way to fix this using the current library? If not, would it be technically possible to add a BOOST_CLASS_EXPORT_ALIAS() macro, or some such? If it is possible, I'm willing to supply the implementation.
Zach Laine
Robert, is there any kind of verdict on the alias patch? I'd like to finish this topic before the thread gets too stale. Zach Laine
How about sending me the whole file. I don't have 1.33 on my machine and export.hpp is ALOT different in the HEAD branch where any changes would be made. Robert Ramey Zach Laine wrote:
On 3/15/07, Zach Laine <whatwasthataddress@gmail.com> wrote:
I did something stupid with my serialization code. I blithely changed the names of several polymorphic classes everywhere they appeared in the code, since they had moved out of a class into that class's namespace (e.g. SomeNS::View::Primitive became SomeNS::Primitive). Several months later, I discovered that I could no longer read old binary archives, because I was using Boost.Serialization's BOOST_CLASS_EXPORT() macro, and this macro was now registering the Primtive type with the name-string "Primitive", instead of "View::Primitive", which is how it was saved in the oldest archives.
Since there is a 1-1 mapping between name-strings and types, and since most of the Serialization code uses static/global data, I cannot currently support both the archives saved with and the archives saved without the "View::" in the same program.
Since I suspect this is not the first time someone has done this, nor the last time someone will do so, is there a way to fix this using the current library? If not, would it be technically possible to add a BOOST_CLASS_EXPORT_ALIAS() macro, or some such? If it is possible, I'm willing to supply the implementation.
Zach Laine
Robert, is there any kind of verdict on the alias patch? I'd like to finish this topic before the thread gets too stale.
Zach Laine _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On 3/22/07, Robert Ramey <ramey@rrsd.com> wrote:
How about sending me the whole file. I don't have 1.33 on my machine and export.hpp is ALOT different in the HEAD branch where any changes would be made.
Robert Ramey
Zach Laine wrote:
On 3/15/07, Zach Laine <whatwasthataddress@gmail.com> wrote:
I did something stupid with my serialization code. I blithely changed the names of several polymorphic classes everywhere they appeared in the code, since they had moved out of a class into that class's namespace (e.g. SomeNS::View::Primitive became SomeNS::Primitive). Several months later, I discovered that I could no longer read old binary archives, because I was using Boost.Serialization's BOOST_CLASS_EXPORT() macro, and this macro was now registering the Primtive type with the name-string "Primitive", instead of "View::Primitive", which is how it was saved in the oldest archives.
Since there is a 1-1 mapping between name-strings and types, and since most of the Serialization code uses static/global data, I cannot currently support both the archives saved with and the archives saved without the "View::" in the same program.
Since I suspect this is not the first time someone has done this, nor the last time someone will do so, is there a way to fix this using the current library? If not, would it be technically possible to add a BOOST_CLASS_EXPORT_ALIAS() macro, or some such? If it is possible, I'm willing to supply the implementation.
Zach Laine
Robert, is there any kind of verdict on the alias patch? I'd like to finish this topic before the thread gets too stale.
Zach Laine _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
The patch is actually against 5 files, so I'm leaving it as a patch. This time the patch is against HEAD. Find the patch and test program attached. Zach Laine
OK - I've taken a cursory look at this an it seems OK. I did have the expected "ripple" but it didn't end up so messy that I couldn't follow it. However, there has to be an easier way to test it. I needs to be automatable in some way. We have a test "test_exported" - use that as a template. This will make the test compatible with our other tests and also permit it to be run against all future archives. I realize you might have to make two/three test programs - one/two to create the test archives and another to read them. Then we'll make sure they are run in sequence. Also we would need updated documentation for extended_type_info as well as BOOST_CLASS_EXPORT Robert Ramey
On 3/23/07, Robert Ramey <ramey@rrsd.com> wrote:
OK - I've taken a cursory look at this an it seems OK. I did have the expected "ripple" but it didn't end up so messy that I couldn't follow it.
However, there has to be an easier way to test it. I needs to be automatable in some way. We have a test "test_exported" - use that as a template. This will make the test compatible with our other tests and also permit it to be run against all future archives.
I realize you might have to make two/three test programs - one/two to create the test archives and another to read them. Then we'll make sure they are run in sequence.
Also we would need updated documentation for extended_type_info as well as BOOST_CLASS_EXPORT
Robert Ramey
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
I've attached a new patchfile and three new files that belong in the serialization test directory. They currently fail, but so does the test_exported.cpp test, which I did not change, except to fix a typo in a comment. They fail in exactly the same way, with a thrown boost::archive::archive_exception (what() == "unregistered void cast"). I'm at a loss, so I'll leave finding this particular problem to you, Robert. I did not change the documentation (again, except for a couple of typos). I'd be happy to do so later, but now seems to be the appropriate time to reopen the issue of whether BOOST_CLASS_EXPORT() should be given a warning in the docs or removed altogether. I favor removing it altogether. Currently, it only acts as a slightly more convenient form of BOOST_CLASS_EXPORT_GUID(), and as such users can easily get along without it. With it, and even with a suitable warning, over time users are bound to run into problems of classes changing names, which will cause archive incompatabilities. The ability to read and write archives compatably should trump the few saved keystrokes that providing BOOST_CLASS_EXPORT() allows. Zach Laine
participants (6)
-
Emil Dotchevski -
Janek Kozicki -
Robert Ramey -
troy d. straszheim -
Zach Laine -
Zajo