
There is a rather bizarre bug in Boost.Serialization, that was reported on this list three years ago: http://article.gmane.org/gmane.comp.lib.boost.devel/147010/match=serializati... +bizarre+bug To sum up, a piece of code that is never even executed, is causing a serialization error in a different part of the program. Unfortunately it seems that this bug was never fixed - I just tried it on Boost 1.39.0, and the bug is still there. Any chance something can be done about it? For reference, here is the sample program that reproduces the issue: #include <sstream> #include <string> #include <vector> #include <boost/archive/text_iarchive.hpp> #include <boost/archive/text_oarchive.hpp> #include <boost/serialization/vector.hpp> template<typename Archive> struct X { template<typename T> void operator<<(const T &t) { Archive &archive = (*(Archive *) NULL); archive << t; } }; void dummy() { typedef boost::archive::text_oarchive Archive; X<Archive> &x = * (X<Archive> *) NULL; std::vector<char> *pt = NULL; // uncomment this line to cause the test in main() to fail //x << pt; } int main(int argc, char* argv[]) { std::vector<char> v0(25, '%'); std::vector<char> v1; std::ostringstream ostr; boost::archive::text_oarchive(ostr) & v0; std::istringstream istr(ostr.str()); boost::archive::text_iarchive(istr) & v1; bool ok = (v0 == v1); return 0; }

Jarl Lindrud wrote:
There is a rather bizarre bug in Boost.Serialization, that was reported on this list three years ago:
http://article.gmane.org/gmane.comp.lib.boost.devel/147010/match=serializati... +bizarre+bug
To sum up, a piece of code that is never even executed, is causing a serialization error in a different part of the program.
Unfortunately it seems that this bug was never fixed - I just tried it on Boost 1.39.0, and the bug is still there.
Any chance something can be done about it?
For reference, here is the sample program that reproduces the issue:
Looks like UB to me. From the C++ standard: Note: in particular, a null reference cannot exist in a well-defined program, because the only way to create such a reference would be to bind it to the "object" obtained by dereferencing a null pointer, which causes undefined behavior. (§ 8.3.2, ¶ 4) There is plenty of stuff out on the net about this; I find this thread pretty entertaining: http://stackoverflow.com/questions/1110111/what-part-of-dereferencing-null-p... -t

I just compiled and ran this test with no problem on MSVC 7.1 Robert Ramey troy d. straszheim wrote:
Jarl Lindrud wrote:
There is a rather bizarre bug in Boost.Serialization, that was reported on this list three years ago:
http://article.gmane.org/gmane.comp.lib.boost.devel/147010/match=serializati... +bizarre+bug
To sum up, a piece of code that is never even executed, is causing a serialization error in a different part of the program.
Unfortunately it seems that this bug was never fixed - I just tried it on Boost 1.39.0, and the bug is still there.
Any chance something can be done about it?
For reference, here is the sample program that reproduces the issue:
Looks like UB to me. From the C++ standard:
Note: in particular, a null reference cannot exist in a well-defined program, because the only way to create such a reference would be to bind it to the "object" obtained by dereferencing a null pointer, which causes undefined behavior. (§ 8.3.2, ¶ 4)
There is plenty of stuff out on the net about this; I find this thread pretty entertaining:
http://stackoverflow.com/questions/1110111/what-part-of-dereferencing-null-p...
-t _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

whoops, turns out the test does fail. I'll look into it. Note that complaints get better results when posted a track items where they hang around rather than scrolling off the list. Robert Ramey Robert Ramey wrote:
I just compiled and ran this test with no problem on MSVC 7.1
Robert Ramey
troy d. straszheim wrote:
Jarl Lindrud wrote:
There is a rather bizarre bug in Boost.Serialization, that was reported on this list three years ago:
http://article.gmane.org/gmane.comp.lib.boost.devel/147010/match=serializati... +bizarre+bug
To sum up, a piece of code that is never even executed, is causing a serialization error in a different part of the program.
Unfortunately it seems that this bug was never fixed - I just tried it on Boost 1.39.0, and the bug is still there.
Any chance something can be done about it?
For reference, here is the sample program that reproduces the issue:
Looks like UB to me. From the C++ standard:
Note: in particular, a null reference cannot exist in a well-defined program, because the only way to create such a reference would be to bind it to the "object" obtained by dereferencing a null pointer, which causes undefined behavior. (§ 8.3.2, ¶ 4)
There is plenty of stuff out on the net about this; I find this thread pretty entertaining:
http://stackoverflow.com/questions/1110111/what-part-of-dereferencing-null-p...
-t _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

troy d. straszheim <troy <at> resophonic.com> writes:
Looks like UB to me. From the C++ standard:
Note: in particular, a null reference cannot exist in a well-defined program, because the only way to create such a reference would be to bind it to the "object" obtained by dereferencing a null pointer, which causes undefined behavior. (§ 8.3.2, ¶ 4)
The dummy() function never even gets executed. The point is that the mere compile-time presence of a particular line in it, causes a deserialization error in main().

I looked into this a little bit. Looks like a bug to me. The problem is that the presence of ar << ptr is use to trigger the instantication of code for saving pointers. This code enables tracking. Also ar >> ptr is used to trigger instantiation of code for loading pointers. The motivation is to avoid instantiation of code that is never called. This system is broken for the case where ar << ptr is invoked but ar >> ptr isn't. I'll look into it. For now you could work around this by a) including more dummy code. b) removing the dummy code c) explicitly setting tracking for the type to track_alwyas or track_never. Robert Ramey Jarl Lindrud wrote:
troy d. straszheim <troy <at> resophonic.com> writes:
Looks like UB to me. From the C++ standard:
Note: in particular, a null reference cannot exist in a well-defined program, because the only way to create such a reference would be to bind it to the "object" obtained by dereferencing a null pointer, which causes undefined behavior. (§ 8.3.2, ¶ 4)
The dummy() function never even gets executed. The point is that the mere compile-time presence of a particular line in it, causes a deserialization error in main().
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Robert Ramey <ramey <at> rrsd.com> writes:
ar >> ptr isn't. I'll look into it. For now you could work around this by a) including more dummy code. b) removing the dummy code c) explicitly setting tracking for the type to track_alwyas or track_never.
Robert Ramey
Thanks Robert. I've entered the issue into trac: https://svn.boost.org/trac/boost/attachment/ticket/3315/

Jarl Lindrud <jarl.lindrud <at> gmail.com> writes:
Robert Ramey <ramey <at> rrsd.com> writes:
ar >> ptr isn't. I'll look into it. For now you could work around this by a) including more dummy code. b) removing the dummy code c) explicitly setting tracking for the type to track_alwyas or track_never.
Robert Ramey
Thanks Robert.
I've entered the issue into trac:
https://svn.boost.org/trac/boost/attachment/ticket/3315/
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
I see this issue has now been closed, without a fix. In my view it's a pretty serious bug, that this code: std::vector<char> v0(25, 'A'); std::vector<char> v1; std::ostringstream ostr; boost::archive::text_oarchive(ostr) & v0; std::istringstream istr(ostr.str()); boost::archive::text_iarchive(istr) & v1; bool ok = (v0 == v1); , will fail, because of implicit compile-time instantiation in a completely different part of the program. For the typical user, this is next to impossible to diagnose, let alone understand. I guess I don't see how this can be thought of as a design feature. I would have thought it would be possible to include sufficient metadata in the output archive, so that B.Ser. is able to correctly deserialize its own output archives, without depending on implicit compile-time instantiation. Sure, that would change the archive format, but it would hardly be the first such change, so I would think whatever backwards compatibility scheme is already in place, could deal with this as well. Hope that makes some sense - I don't know very much about B.Ser. internals. Thanks, Jarl.

Jarl Lindrud wrote:
I see this issue has now been closed, without a fix.
In my view it's a pretty serious bug, that this code:
std::vector<char> v0(25, 'A'); std::vector<char> v1;
std::ostringstream ostr; boost::archive::text_oarchive(ostr) & v0;
std::istringstream istr(ostr.str()); boost::archive::text_iarchive(istr) & v1;
bool ok = (v0 == v1);
, will fail, because of implicit compile-time instantiation in a completely different part of the program. For the typical user, this is next to impossible to diagnose, let alone understand. I guess I don't see how this can be thought of as a design feature.
I would have thought it would be possible to include sufficient metadata in the output archive, so that B.Ser. is able to correctly deserialize its own output archives, without depending on implicit compile-time instantiation.
For collections of primitive types like vector<char> the implementation level was set to "object_serializable". The suppresses inclusion of class information in the archive itself which leads to the the problem shown by your example. Perhaps it was a mistake to do this. But for every person who would conclude that, there is at least one person who would conclude that in this case efficiency would be an overriding concern in this instance. So its a no-win situation for me. If you think it's obvious that the wrong decision was made, consider the following: a) In order for a situation described in your example to arise in real life code, a program which saves data would have to include something like ar << ptr while the program which loads data could not include something like ar >> ptr. Were this the case, serialization has been broken in any case so I don't see how the situation in your example could ever arise in real code. FWIW, you're the first person ever to report this type problem. Of course, now I'm a little bit intrigued how you came upon this in the first place. b) the setting of "object_serializable" is the default. One make a derived class override it.
Sure, that would change the archive format, but it would hardly be the first such change, so I would think whatever backwards compatibility scheme is already in place, could deal with this as well.
If we were convinced that this had to change, it could be dealt with. Even if I were convinced that the original decision was not the optimal one, (which I'm not - though I could be), I would still have to be convinced that the situtuation were so non-optimal that it would be worth the trouble to address it. On a regular basis, I get "complaints" that the archive contains "too much" superfluous information. Basically, I can't please everyone. I tried to make reasonable defaults and provide a mechanism for overriding these defaults. That's the best I can do. Robert Ramey

Robert Ramey <ramey <at> rrsd.com> writes:
If we were convinced that this had to change, it could be dealt with. Even if I were convinced that the original decision was not the optimal one, (which I'm not - though I could be), I would still have to be convinced that the situtuation were so non-optimal that it would be worth the trouble to address it.
On a regular basis, I get "complaints" that the archive contains "too much" superfluous information. Basically, I can't please everyone. I tried to make reasonable defaults and provide a mechanism for overriding these defaults. That's the best I can do.
Robert Ramey
Fair enough. I've no problem with the defaults being set one way or another, but perhaps it's a bit problematic to let compile-time properties of a program influence the runtime archive formats, to the point where they can even become unreadable. FWIW this bug was reported to me by users of RCF ( http://www.codeproject.com/ KB/threads/Rcf_Ipc_For_Cpp.aspx ), a C++ IPC framework that supports several serialization frameworks, one of them being Boost Serialization. Regards, Jarl.

Jarl Lindrud wrote:
Robert Ramey <ramey <at> rrsd.com> writes: Fair enough. I've no problem with the defaults being set one way or another, but perhaps it's a bit problematic to let compile-time properties of a program influence the runtime archive formats, to the point where they can even become unreadable.
FWIW this bug was reported to me by users of RCF ( http://www.codeproject.com/ KB/threads/Rcf_Ipc_For_Cpp.aspx ), a C++ IPC framework that supports several serialization frameworks, one of them being Boost Serialization.
I'm still intrigued as to how such a situation could come up where something is saved through a pointer but in another program it isn't. I looked at the above link and didn't find this. Basically, my view is that anyone tripped up by this error can't be writing anything that makes sense anyway. I'm looking for a counter example to this. Robert Ramey

Robert Ramey <ramey <at> rrsd.com> writes:
I'm still intrigued as to how such a situation could come up where something is saved through a pointer but in another program it isn't. I looked at the above link and didn't find this. Basically, my view is that anyone tripped up by this error can't be writing anything that makes sense anyway. I'm looking for a counter example to this.
Robert Ramey
I don't know this arose originally - there may well have been a situation where the vector<char> was serialized through a pointer in one program, and deserialized as a value, in another program. That is indeed an error, although in this case it does not seem that any exception was thrown to signify the error (the vector just silently failed to deserialize properly). On a more fundamental level, I don't understand how this code can *ever* fail: (1) std::ostringstream ostr; boost::archive::text_oarchive(ostr) & v0; std::istringstream istr(ostr.str()); boost::archive::text_iarchive(istr) & v1; bool ok = (v0 == v1); Here the vector is serialized and deserialized as a value. How is it that "tracking" or "implementation" settings for a particular type can be affected by the mere presence of this code (2) ar << pVec; // pVec is a vector<char> * , or this code (3) ar >> pVec; // pVec is a vector<char> * , in a completely different part of the program? Snippet (2), if it appears in a program without snippet (3) also occurring, makes the program incorrect, if I understand you correctly. But how is it that its mere compile-time presence causes the technically correct code in (1) to fail? And even if snippet (3) is also present somewhere in the program, thus making the program correct in your view, IIUC the archives produced in (1) will then have a different format than those produced, when snippets (2) and (3) were absent. That would imply that archive formats can change silently, through mere addition of source code, anywhere else in the program. And the format change wouldn't be noticed until some *other* program tries to read the archives and fails to do so. Have I understood that correctly? Regards, Jarl.

Jarl Lindrud wrote:
Robert Ramey <ramey <at> rrsd.com> writes:
I'm still intrigued as to how such a situation could come up where something is saved through a pointer but in another program it isn't. I looked at the above link and didn't find this. Basically, my view is that anyone tripped up by this error can't be writing anything that makes sense anyway. I'm looking for a counter example to this.
Robert Ramey
I don't know this arose originally - there may well have been a situation where the vector<char> was serialized through a pointer in one program, and deserialized as a value, in another program. That is indeed an error, although in this case it does not seem that any exception was thrown to signify the error (the vector just silently failed to deserialize properly).
a) don't include class information in the archive advantage: faster performance since class info doesn't have to be looked up or each instance disadvantage: no way verify that the type being loaded is the same as that being stored b) include information in the archive advantage: slower performance since class info doesn't have to be looked up or each instance disadvantage: library can verify that the type being loaded is the same as that being stored I tried to choose reasonable defaults- it's the best I can do.
On a more fundamental level, I don't understand how this code can *ever* fail:
(1) std::ostringstream ostr; boost::archive::text_oarchive(ostr) & v0;
std::istringstream istr(ostr.str()); boost::archive::text_iarchive(istr) & v1;
bool ok = (v0 == v1);
This code can only fail if the saving template for V is different from that for V. Such a program can never work. Without class information inside the archive, I can't detected this type of error.
Here the vector is serialized and deserialized as a value. How is it that "tracking" or "implementation" settings for a particular type can be affected by the mere presence of this code.
I minor correction - "implementation_level" is not effected. tracking is effected if it has been set to "track_selectively" Do you mean how is something like this implemented? The short answer is that T *t ... ar << t instanciates code which refers to a special static object which "remembers" that this code has been invoked. This static object is initialized at pre-main time. So if anywhere else in the program one invokes T t ... ar << t Tracking is turned on for this operation. Note that this has the counter ituitive effect of seeming to "look ahead" in the code.
(2) ar << pVec; // pVec is a vector<char> *
, or this code
(3) ar >> pVec; // pVec is a vector<char> *
, in a completely different part of the program?
Snippet (2), if it appears in a program without snippet (3) also occurring, makes the program incorrect,
No it doesn't. The program is still correct. A valid archive is created. This program or any other can read this archive without problem by including (3). What cannot be done is
ar >> Vec; // Vec is a vector<char> (note missing *)
Since saving and loading types must match.
And even if snippet (3) is also present somewhere in the program, thus making the program correct in your view, IIUC the archives produced in (1) will then have a different format than those produced, when snippets (2) and (3) were absent. That would imply that archive formats can change silently, through mere addition of source code, anywhere else in the program. And the format change wouldn't be noticed until some *other* program tries to read the archives and fails to do so. Have I understood that correctly?
sort of, The format change will be notices when some tries to load a type different than the type saved. In this case, loading a non-pointer type where a pointer type was saved. Just remember this: If each load must use the exact same type as each corresponding save. If you follow this rule, you will never have a problem of this nature. Robert Ramey

Robert Ramey <ramey <at> rrsd.com> writes:
I tried to choose reasonable defaults- it's the best I can do.
I've got no problem with the default setting - it's obviously got to be one way or the other. What I think is unfortunate is that the settings change, based on compile-time instantiation *anywhere* in the program.
On a more fundamental level, I don't understand how this code can *ever* fail:
(1) std::ostringstream ostr; boost::archive::text_oarchive(ostr) & v0;
std::istringstream istr(ostr.str()); boost::archive::text_iarchive(istr) & v1;
bool ok = (v0 == v1);
This code can only fail if the saving template for V is different from that for V. Such a program can never work. Without class information inside the archive, I can't detected this type of error.
But V is vector<char>, and its serialization definition is known - it is given by the library itself, in vector.hpp . I haven't touched it.
Here the vector is serialized and deserialized as a value. How is it that "tracking" or "implementation" settings for a particular type can be affected by the mere presence of this code.
I minor correction - "implementation_level" is not effected. tracking is effected if it has been set to "track_selectively"
Do you mean how is something like this implemented? The short answer is that
T *t ... ar << t
instanciates code which refers to a special static object which "remembers" that this code has been invoked. This static object is initialized at pre-main time. So if anywhere else in the program one invokes
T t ... ar << t
Tracking is turned on for this operation.
Hmmm. This implies that the meaning of the code "ar << t" , depends on the content of the entire program.
Note that this has the counter ituitive effect of seeming to "look ahead" in the code.
What's worse is that it also seems to "look across" :)
Just remember this: If each load must use the exact same type as each corresponding save.
If you follow this rule, you will never have a problem of this nature.
Let me sum up what it is that worries me. Say I have program A, with this kind of code: std::ostringstream ostr; boost::archive::text_oarchive(ostr) & v0; std::istringstream istr(ostr.str()); boost::archive::text_iarchive(istr) & v1; bool ok = (v0 == v1); , and nothing else. Say that I also have another program B, with the same kind of code, that reads archives produced by program A. Everthing is working well. And then in program A, at a later stage, some more code is added, containing these lines: ar << pVec; // pVec is a vector<char> * , and ar >> pVec; // pVec is a vector<char> * Perhaps the code is not even executed. Now, program A still functions. It can read the archives that it has produced *but* the archives now, IIUC, have a different format. So when program B tries to read the archives, it will fail, and may well fail silently. So essentially addition or modification of code, *anywhere* in program A, and regardless of whether it is executed, can cause a runtime failure in program B! You can imagine how difficult this would be to debug. Wouldn't it be easier if serialization settings like tracking_level were set by the user, explicitly at runtime? That way one could set it exactly as one wants it, rather than having to guess at the side effects of compile-time instantiations throughout the entire program. Regards, Jarl.

Jarl Lindrud wrote:
Robert Ramey <ramey <at> rrsd.com> writes: Hmmm. This implies that the meaning of the code "ar << t" , depends on the content of the entire program.
I would agree that the implementation of ar << t depends upon the content of the entire program.
Note that this has the counter ituitive effect of seeming to "look ahead" in the code.
What's worse is that it also seems to "look across" :)
Just remember this: If each load must use the exact same type as each corresponding save.
If you follow this rule, you will never have a problem of this nature.
Let me sum up what it is that worries me. Say I have program A, with this kind of code:
std::ostringstream ostr; boost::archive::text_oarchive(ostr) & v0;
std::istringstream istr(ostr.str()); boost::archive::text_iarchive(istr) & v1;
bool ok = (v0 == v1);
, and nothing else. Say that I also have another program B, with the same kind of code, that reads archives produced by program A.
Everthing is working well. And then in program A, at a later stage, some more code is added, containing these lines:
ar << pVec; // pVec is a vector<char> *
, and
ar >> pVec; // pVec is a vector<char> *
Perhaps the code is not even executed. Now, program A still functions. It can read the archives that it has produced *but* the archives now, IIUC, have a different format. So when program B tries to read the archives, it will fail, and may well fail silently.
True. But note that this is not the case which brought up the discussion.
So essentially addition or modification of code, *anywhere* in program A, and regardless of whether it is executed, can cause a runtime failure in program B! You can imagine how difficult this would be to debug.
Default tracking is "track_selectively" and default implementation level is "? - include class information in the archive. You might want to change the implementation level for your classes to "track_always".
Wouldn't it be easier if serialization settings like tracking_level were set by the user, explicitly at runtime? That way one could set it exactly as one wants it, rather than having to guess at the side effects of compile-time instantiations throughout the entire program.
Actually, I see a lot of merit in this argument. In fact the whole concept of "serialization_traits" is amounts to a global table keyed on data type. So it does have the same problem - ie side effects that any global table does. It's not as bad a normal because it doesn't change dynamically but it can still be a problem. On the other hand, having to explicitly specify these traits for each invocation of ar << ? and ar >> ? would be very tedious and I dare say unpopular. And likely prone to other kinds of errors. You might make a more convincing case that tracking traits should not include "track_selectively". Or maybe the ALL types should have their class information stored in the archive. I'm sure that many people would then complain about efficiency. It would be possible for someone who had nothing else to do to re-implement and enhance the Archive templates with a policy to explicitly use a specific tracking or implemention level regardless of the "traits setting". I've actually considered using such a method to suppress All tracking for the whole archive. This would permit using the serialization system for things Error logging, transacction processing, etc. Note that doing this would not require changing any of the archive or serialization concepts so it would be applicable "for free" to an program which already had serialization implemented. Serialization turns out to be a far richer subject than first meets the eye. Robert Ramey
Regards, Jarl.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Robert Ramey <ramey <at> rrsd.com> writes:
Jarl Lindrud wrote:
Robert Ramey <ramey <at> rrsd.com> writes: Hmmm. This implies that the meaning of the code "ar << t" , depends on the content of the entire program.
I would agree that the implementation of ar << t depends upon the content of the entire program.
But it is actually meaning, not implementation... Depending on the contents of the entire program, the archives will have one binary format or another, and will either be readable by other pre-existing programs, or not. That's really part of the specification of "ar << t" , not the implementation.
Everthing is working well. And then in program A, at a later stage, some more code is added, containing these lines:
ar << pVec; // pVec is a vector<char> *
, and
ar >> pVec; // pVec is a vector<char> *
Perhaps the code is not even executed. Now, program A still functions. It can read the archives that it has produced *but* the archives now, IIUC, have a different format. So when program B tries to read the archives, it will fail, and may well fail silently.
True. But note that this is not the case which brought up the discussion.
Sure. But this is IMO a much more serious manifestation of the same issue, and worries me a lot more. :)
So essentially addition or modification of code, *anywhere* in program A, and regardless of whether it is executed, can cause a runtime failure in program B! You can imagine how difficult this would be to debug.
Default tracking is "track_selectively" and default implementation level is "? - include class information in the archive. You might want to change the implementation level for your classes to "track_always".
But perhaps I do want tracking turned on, in certain situations, and turned off in other situations. What's important is that I know for sure, for any given serialization operation, what the tracking setting actually is, and that it doesn't change implicitly later on, when somebody adds seemingly innocent code to the program.
Wouldn't it be easier if serialization settings like tracking_level were set by the user, explicitly at runtime? That way one could set it exactly as one wants it, rather than having to guess at the side effects of compile-time instantiations throughout the entire program.
Actually, I see a lot of merit in this argument. In fact the whole concept of "serialization_traits" is amounts to a global table keyed on data type. So it does have the same problem - ie side effects that any global table does. It's not as bad a normal because it doesn't change dynamically but it can still be a problem. On the other hand, having to explicitly specify these traits for each invocation of ar << ? and ar >> ? would be very tedious and I dare say unpopular. And likely prone to other kinds of errors.
<somewhat OT> It could be made simpler than that - the traits would have a default compile- time setting, and then users could override those settings through a runtime call. They would presumably do that at program startup (if at all), and then they would still have the option of adjusting the traits again, for any particular serialization operation, if and when its needed. If they want the default compile-time traits, they wouldn't have to do anything. I do think the whole traits issue needs to be dealt with at runtime. What if I need to read archives produced by another program, which in an older release had tracking disabled, and then in a later release had it enabled. If the tracking trait is set at compile-time then I will only be able to read one or the other. </somewhat OT> But getting back to the original problem - really the most important thing here is that the default tracking level should be unaffected by compile-time instantiation. As long as I know what the default is, and that it won't change under my feet, then I would be OK with that. Would that be feasible to implement? Anyway, thanks Robert for bearing with me... I know you get a lot of requests from a lot of people. Regards, Jarl.

Jarl Lindrud wrote:
Robert Ramey <ramey <at> rrsd.com> writes:
Jarl Lindrud wrote:
Robert Ramey <ramey <at> rrsd.com> writes: Hmmm. This implies that the meaning of the code "ar << t" , depends on the content of the entire program.
I would agree that the implementation of ar << t depends upon the content of the entire program.
But it is actually meaning, not implementation... Depending on the contents of the entire program, the archives will have one binary format or another, and will either be readable by other pre-existing programs, or not. That's really part of the specification of "ar << t" , not the implementation.
The specification of the semantics of ar << t supports either interpretation. The actual semantic is dependent on the selected implemenation level and tracking behavior.
Default tracking is "track_selectively" and default implementation level is "? - include class information in the archive. You might want to change the implementation level for your classes to "track_always".
But perhaps I do want tracking turned on, in certain situations, and turned off in other situations. What's important is that I know for sure, for any given serialization operation, what the tracking setting actually is, and that it doesn't change implicitly later on, when somebody adds seemingly innocent code to the program.
LOL - then just do that! That is, set the implementation level and tracking behavior for each of your classes. That way they would never change - until you change the class definitions in which case you've go class versioning to help you out. This is supported by the library.
Actually, I see a lot of merit in this argument. In fact the whole concept of "serialization_traits" is amounts to a global table keyed on data type. So it does have the same problem - ie side effects that any global table does. It's not as bad a normal because it doesn't change dynamically but it can still be a problem. On the other hand, having to explicitly specify these traits for each invocation of ar << ? and ar >> ? would be very tedious and I dare say unpopular. And likely prone to other kinds of errors.
<somewhat OT>
It could be made simpler than that - the traits would have a default compile- time setting, and then users could override those settings through a runtime call. They would presumably do that at program startup (if at all), and then they would still have the option of adjusting the traits again, for any particular serialization operation, if and when its needed.
If you want to do it a startup - you might as well do it a compile time by setting your own traits - see above. if you want to override it for a particular operation - you can make a serialization wrapper. (Currently this can't be done for all traits - this has been noted, but it has never been addressed due to lack of interest)
If they want the default compile-time traits, they wouldn't have to do anything.
"They" don't.
I do think the whole traits issue needs to be dealt with at runtime. What if I need to read archives produced by another program, which in an older release had tracking disabled, and then in a later release had it enabled. If the tracking trait is set at compile-time then I will only be able to read one or the other.
I think you need to study the current library some more. The default implementation level for user classes is to store class version, and tracking behavior in the archve. When the archive is read, the tracking which was set when the archive was created is that which is used when the archive is read. The "current" tracking setting is only used if the class information wasn't stored in the archive. For some commonly used types - in this case vector<char> it was decided that the version would "never change" and that efficiency considerations argued for using "track_selectively" so the default traits were set accordingly. Of course one can differ on what the default should be in any particular case - but that's a different topic. To summarize, I don't see what's missing in order to do what you want. Don't like my selection of default traits settings - don't use them. Don't like the concept of "side effects" - just explicitly assign the traits you want to use for each of your classes. Want to tweak them for each invocation of the << or >> operator, make a wrapper. I don't see that the library inhibits you from doing what you want.
</somewhat OT>
But getting back to the original problem - really the most important thing here is that the default tracking level should be unaffected by compile-time instantiation. As long as I know what the default is, and that it won't change under my feet, then I would be OK with that.
Would that be feasible to implement?
Just specify your own setting with BOOST_CLASS_TRACKING(mytype, track_always) or BOOST_CLASS_TRACKING(mytype, track_never) for your own classes. If this is too much - then you could tweak the "tracking.hpp" file so that the default is "track_always" or "track_never" or whatever.
Anyway, thanks Robert for bearing with me... I know you get a lot of requests from a lot of people.
Actually, it's very odd to me that I get so few requests and questions. The perception is that the library is widely used. But, if that's the case I'm very surprised how little traffic I get. And 80 percent of the questions are just people who don't want to read the manual. Robert Ramey

Robert Ramey <ramey <at> rrsd.com> writes:
But getting back to the original problem - really the most important thing here is that the default tracking level should be unaffected by compile-time instantiation. As long as I know what the default is, and that it won't change under my feet, then I would be OK with that.
Would that be feasible to implement?
Just specify your own setting with BOOST_CLASS_TRACKING(mytype, track_always) or BOOST_CLASS_TRACKING(mytype, track_never) for your own classes.
If this is too much - then you could tweak the "tracking.hpp" file so that the default is "track_always" or "track_never" or whatever.
OK, I think we can wrap this up then... My understanding now is : ----------------------------------------- * If one program writes archives like this: ar & vec; // vec is a std::vector<char> , it is then impossible to reliably load that archive in another program, unless one is aware of the entire compile-time content of the first program. * When one writes code like this: ar << pVec; // pVec is a vector<char> * , it has the compile-time side effect of silently changing the archive format for *all* archives produced by that program, that happen to include a vector<char>. Making matters worse, the archive format change is compatibility- breaking. ----------------------------------------- IMHO, this is a rather fragile state of affairs, but it's your call and I won't argue it further. I'll go with the workaround, and document to users of my library, that if they use B.Ser. to serialize vector<char> (and presumably a number of other types?), they must always write BOOST_CLASS_TRACKING(vector<char>, track_never) , and that failure to do so is likely to trigger silent deserialization errors in other programs. Regards, Jarl.

Jarl Lindrud wrote:
Robert Ramey <ramey <at> rrsd.com> writes:
OK, I think we can wrap this up then... My understanding now is :
-----------------------------------------
* If one program writes archives like this:
ar & vec; // vec is a std::vector<char>
, it is then impossible to reliably load that archive in another program, unless one is aware of the entire compile-time content of the first program.
Nope. The only time a problem can occur if one does ar & vec; // vec is a std::vector<char> In the saving program and ar & p_vec; // vec is a * std::vector<char> in the loading progam. But this wouldn't work anyway.
* When one writes code like this:
ar << pVec; // pVec is a vector<char> *
, it has the compile-time side effect of silently changing the archive format for *all* archives produced by that program, that happen to include a vector<char>.
As it must. If one is going to serialize through a pointer in one part of the program, you have to include tracking information in the archive for ALL ar << t to avoid creating multiple instances ] when the objects pointed to are re-created. That is ar << t; // save a t ... // in some other module ar << p_t // where t points to some t saved somewhere else in the program - perhaps even in another source module! In order to avoid created an extra t, you have to track ALL serializations of t. It seems that this could be a problem if the schema is changed so that it was saved without pointers but now it is. But if the schema is changed, you can't load the old archives anyway. That's why the problem never comes up in real code. Besides this, it's a special case because the serialization traits for collections of primitives have tracking set to "track_selectively" AND implemention level set to "object_serializable" which doesn't include the historical tracking behavior inside the archive. The default settings for user classes is to include the historical tracking inside the class information in the archive so even if it's changed in the future, old archives will still be read. Making matters worse, the archive
format change is compatibility- breaking.
Only for collections of primitives.
IMHO, this is a rather fragile state of affairs, but it's your call and I won't argue it further. I'll go with the workaround, and document to users of my library, that if they use B.Ser. to serialize vector<char> (and presumably a number of other types?), they must always write
BOOST_CLASS_TRACKING(vector<char>, track_never)
LOL - what if they want to do ar << pVec // pVec is a pointer to vector<char> and ar >> pVec // pVec without accidently creating pointers to two different objects when they only saved one? That bug is going to a major bitch to find. Maybe you want to use "track_always". Or maybe you want to change the "implementation_level" so the next higher one which stores tracking information in the archive itself. (Caveat, Logically it would make sense, but I don't think the library as it is currently implemented permits "overriding" previously set serialization traits. Maybe someday when someone has nothing else to do)
, and that failure to do so is likely to trigger silent deserialization errors in other programs.
I would use "concievable" rather than "likely". In fact, I can't think of any real program where this could occur without there being some other error that would also prevent this program from functioning. Robert Ramey

Robert Ramey <ramey <at> rrsd.com> writes:
-----------------------------------------
* If one program writes archives like this:
ar & vec; // vec is a std::vector<char>
, it is then impossible to reliably load that archive in another program, unless one is aware of the entire compile-time content of the first program.
Nope. The only time a problem can occur if one does
ar & vec; // vec is a std::vector<char>
In the saving program and
ar & p_vec; // vec is a * std::vector<char>
in the loading progam.
Robert, that's manifestly incorrect. I have appended 3 well-formed programs, A1, A2, and B, to this post. B can read archives from A1, but it *cannot* read archives from A2. If you don't believe me, then please run the programs yourself. The only difference between A1 and A2 is compile-time instantiation, of code that is never executed.
Making matters worse, the archive
format change is compatibility- breaking.
Only for collections of primitives.
So we do agree that compatibility does break, due to compile-time instantiation.
I would use "concievable" rather than "likely". In fact, I can't think of any real program where this could occur without there being some other error that would also prevent this program from functioning.
Do you consider there to be any errors in either A1, A2 or B? If you were to implement a generic IPC marshalling layer, you would end up with the kind of code that A1, A2, and B contain. IPC serialization is a bit different from file-based serialization, in that serialization and deserialization generally take place in different programs. Another wrinkle in IPC serialization, is that one program may contain only serialization code for a type T (and not deserialization), while another program may contain only deserialization code for type T (and not serialization). If that makes the programs ill-formed in your view, then I guess one would have to conclude that B.Ser. is not an appropriate choice for IPC applications. Regards, Jarl ---------------------------------- // A1 #include <fstream> #include <boost/archive/text_iarchive.hpp> #include <boost/archive/text_oarchive.hpp> #include <boost/serialization/vector.hpp> int main() { std::vector<char> vec(25, 'A'); std::ofstream fout("A.txt"); boost::archive::text_oarchive(fout) & vec; return 0; } ---------------------------------- // A2 #include <fstream> #include <boost/archive/text_iarchive.hpp> #include <boost/archive/text_oarchive.hpp> #include <boost/serialization/vector.hpp> template<typename Archive> struct X { template<typename T> void operator<<(const T &t) { Archive &archive = (*(Archive *) NULL); archive << t; } template<typename T> void operator>>(T &t) { Archive &archive = (*(Archive *) NULL); archive >> t; } }; // This function is instantiated by the compiler, but never executed. void dummy() { typedef boost::archive::text_iarchive IArchive; typedef boost::archive::text_oarchive OArchive; std::vector<char> *pt = NULL; X<OArchive> & oar = * (X<OArchive> *) NULL; oar << pt; X<IArchive> & iar = * (X<IArchive> *) NULL; iar >> pt; } int main() { std::vector<char> vec(25, 'A'); std::ofstream fout("A.txt"); boost::archive::text_oarchive(fout) & vec; return 0; } ---------------------------------- // B #include <fstream> #include <boost/archive/text_iarchive.hpp> #include <boost/archive/text_oarchive.hpp> #include <boost/serialization/vector.hpp> int main() { std::vector<char> vec; std::ifstream fin("A.txt"); // If A1 produced the archive, this line succeeds. // If A2 produced the archive, this line will silently fail. boost::archive::text_iarchive(fin) & vec; return 0; } ----------------------------------

Jarl Lindrud wrote:
Robert Ramey <ramey <at> rrsd.com> writes:
-----------------------------------------
* If one program writes archives like this:
ar & vec; // vec is a std::vector<char>
, it is then impossible to reliably load that archive in another program, unless one is aware of the entire compile-time content of the first program.
Nope. The only time a problem can occur if one does
ar & vec; // vec is a std::vector<char>
In the saving program and
ar & p_vec; // vec is a * std::vector<char>
in the loading progam.
Robert, that's manifestly incorrect. I have appended 3 well-formed programs, A1, A2, and B, to this post. B can read archives from A1, but it *cannot* read archives from A2. If you don't believe me, then please run the programs yourself. The only difference between A1 and A2 is compile-time instantiation, of code that is never executed.
Here's my version of the three programs. This is the way I would expect most users use the library. A1: struct X { vector<char> m_v; template<class Archive> save(Archive & ar, const unsigned int version) const { ar << m_v; } template<class Archive> load(Archive & ar, const unisgned int version){ ar >> m_v; } }; int main(){ X x(...); std::ofstream os("file.txt") boost::archive::text_oarchive oa(os); oa << x; } A2: // compiled by never invoked void dummy(){ vector<char> * pv = new vector<char>(); std::ofstream os("file.txt") boost::archive::text_oarchive oa(os); oa << x; } int main(){ X x(...); std::ofstream os("file.txt") boost::archive::text_oarchive oa(os); oa << x; } OK - A2 produces an archive that cannot be read. I would argue that including the dummy just to provoke this "bug" is just a pathological case and basically constitutes an error - albiet a subtle one. A much more interesting scenario in this vein is the following: C: Sometime later, the definition of X is changed to struct X { vector<char> * m_pv; template<class Archive> save(Archive & ar, const unsigned int version) const { ar << m_pv; } template<class Archive> load(Archive & ar, const unisgned int version){ // uh - oh need to support old archives ! if(version < 1){ m_pv = new vector<char>() ar >> *m_pv } else ar >> m_pv; } }; BOOST_CLASS_VERSION(X, 1) This WOULD be a problem and would fail to read old archives. I'm willing to consider this a legitimate concern. Still, it's a very infrequent cases. So in short, I believe that the only way to have such a problem is include dummy code which I don't consider a huge problem.
Do you consider there to be any errors in either A1, A2 or B?
A1 serializes vector<char> directly rather than as a member of a class. Not illegal - be extremely unusual. A2 includes dummy code solely to provoke this "bug". I can't think of a situation where inclusion of such code would be legitimate. And it's not easy to do either as in the majority of cases one uses ar & rather than ar << in which cast, the saving and loading are always in sync. B - I didn't look at it - I presume its' correct.
If you were to implement a generic IPC marshalling layer, you would end up with the kind of code that A1, A2, and B contain.
I'm not seeing that.
IPC serialization is a bit different from file-based serialization, in that serialization and deserialization generally take place in different programs.
Agreed - but I don't think it's relevant. The relevant issue is that serialization and deserialization use the same data schema - and this must be true for the system to work at all.
Another wrinkle in IPC serialization, is that one program may contain only serialization code for a type T (and not deserialization), while another program may contain only deserialization code for type T (and not serialization). If that makes the programs ill-formed in your view,
It doesn't
then I guess one would have to conclude that B.Ser. is not an appropriate choice for IPC applications.
Hence, I would not draw such a conclusion. The real issue is my choice of implementation_level "object_serializable" in combination with "track_selectively" for a few specific types. Did I make the wrong choice? Maybe - but this is a very narrow issue related to these types - collections of primitives. For these types serialization implementation is not expected to change and for which considerations of efficiency are greater than usual. In any case, this has got me thinking about how to avoid this. As I write this, I'm think maybe the best would be to emit a compile time warning when ever anyone tries to serialize a pointer to a type whose serialization traits are both "object_serializable" and "track_selectively". I'll think about this. Robert Ramey
Regards, Jarl

On Aug 10, 2009, at 12:06 PM, Robert Ramey wrote:
So in short, I believe that the only way to have such a problem is include dummy code which I don't consider a huge problem.
The "dummy" code could be from serialization of different classes to a different archive in another part of a big program. Consider for example our ALPS library for which we will use Boost.Serialization in the future, and already use it via Boost.MPI. If a user extends our framework with a class that serializes a vector<double> through a pointer, all the files written by the framework that serialized vector<double> will become unreadable! The only solution I see is to use archive wrappers that remove all tracking information for the files written by the framework. Matthias

Matthias Troyer wrote:
On Aug 10, 2009, at 12:06 PM, Robert Ramey wrote:
So in short, I believe that the only way to have such a problem is include dummy code which I don't consider a huge problem.
The "dummy" code could be from serialization of different classes to a different archive in another part of a big program. Consider for example our ALPS library for which we will use Boost.Serialization in the future, and already use it via Boost.MPI. If a user extends our framework with a class that serializes a vector<double> through a pointer, all the files written by the framework that serialized vector<double> will become unreadable!
I think my last post acknowledged this situation. There is code in the trunk which traps with BOOST_STATIC_WARNING attempts to serialize these types through a pointer but currently it's disabled. I'm going to look at this more carefully.
The only solution I see is to use archive wrappers that remove all tracking information for the files written by the framework.
This is one of a few circumstances where it would be convienent to temporarily override serialization traits with a wrapper. Unfortunately, no one has figured out how to do that yet. Robert Ramey

Robert Ramey wrote:
OK - A2 produces an archive that cannot be read. I would argue that including the dummy just to provoke this "bug" is just a pathological case and basically constitutes an error - albiet a subtle one.
A much more interesting scenario in this vein is the following:
C: Sometime later, the definition of X is changed to
struct X { vector<char> * m_pv; template<class Archive> save(Archive & ar, const unsigned int version) const { ar << m_pv; } template<class Archive> load(Archive & ar, const unisgned int version){ // uh - oh need to support old archives ! if(version < 1){ m_pv = new vector<char>() ar >> *m_pv } else ar >> m_pv; } };
BOOST_CLASS_VERSION(X, 1)
This WOULD be a problem and would fail to read old archives. I'm willing to consider this a legitimate concern.
Still, it's a very infrequent cases.
So in short, I believe that the only way to have such a problem is include dummy code which I don't consider a huge problem.
Both cases alltought maybe not a most common are nevertheless legitimate. We use extensively serialization framework in our code and case when someone changes a class data member to be handled with a shared pointer would not surprise me at all. However the user would be unaware of the subtility of a such change and a fact that would make old archives unreadable. Similar situation would be I beleive adding shared_from_this<T> as a base class - no versioning could help restore the old archives...

Robert Ramey <ramey <at> rrsd.com> writes:
OK - A2 produces an archive that cannot be read. I would argue that including the dummy just to provoke this "bug" is just a pathological case and basically constitutes an error - albiet a subtle one.
What you call dummy code, was whittled down from the original (real) code that triggered the error. It's not pathological in any way. It's nothing more than a serialization instantiation in a different and unrelated part of the program. What's pathological is the runtime effect that the instantiation has.
So in short, I believe that the only way to have such a problem is include dummy code which I don't consider a huge problem.
Really now... Could we please drop the assertion that it is "dummy code". It's a minimal reproduction of a real isue. It's been stripped down for your own convenience, to demonstrate the root cause of the problem.
If you were to implement a generic IPC marshalling layer, you would end up with the kind of code that A1, A2, and B contain.
I'm not seeing that.
I guess you could always take my word for it. In any case, we can conclude that all 3 programs make correct use of B.Ser. , and yet the deserialization error occurs.
Another wrinkle in IPC serialization, is that one program may contain only serialization code for a type T (and not deserialization), while another program may contain only deserialization code for type T (and not serialization). If that makes the programs ill-formed in your view,
It doesn't
then I guess one would have to conclude that B.Ser. is not an appropriate choice for IPC applications.
Hence, I would not draw such a conclusion.
I think I must profess some skepticism here. In your post on Aug 4 you wrote "This system is broken for the case where ar << ptr is invoked but ar >> ptr isn't", regarding instantiation of pointer serialization code. And yet that is exactly the situation that would arise, when you have request and response objects in an IPC system, each containing different sets of user-defined types. Clients serialize requests, but they don't deserialize them. And conversely, servers deserialize requests, but don't serialize them. Regards, Jarl.

Jarl Lindrud wrote:
Robert Ramey <ramey <at> rrsd.com> writes:
Another wrinkle in IPC serialization, is that one program may contain only serialization code for a type T (and not deserialization), while another program may contain only deserialization code for type T (and not serialization). If that makes the programs ill-formed in your view,
It doesn't
then I guess one would have to conclude that B.Ser. is not an appropriate choice for IPC applications.
Hence, I would not draw such a conclusion.
I think I must profess some skepticism here. In your post on Aug 4 you wrote "This system is broken for the case where ar << ptr is invoked but ar >> ptr isn't", regarding instantiation of pointer serialization code.
I never said that the ar >> ptr and ar << ptr have to be in the same program - and in fact they don't. program 1 containing only ar << Tptr program 2 containing only ar >> Tptr presents no problem. The problem only occurs when program 1 contains only ar << T and program 2 contains ar >> T and ar >> Tptr But if only program 2 contains ar >> Tptr - how does the archive it's supposed to read get created? It can be only created by a program which also contains ar << Tptr. That is in order for serialization to make any sense at all both programs have to contain exactly the same sequence of loads and saves. Hence. they will both either serializations for the same set of types at compiler time. The only case where this can occur is where the program is changed after an archive is created. And even then it takes some effort to produce. This is the situation I was referring to and I include the only example that occured to me where this would arise in practice.
And yet that is exactly the situation that would arise, when you have request and response objects in an IPC system, each containing different sets of user-defined types. Clients serialize requests, but they don't deserialize them. And conversely, servers deserialize requests, but don't serialize them.
That is not relevant. The ar >> and ar << don't have to be in the same program. They just have to be consistent with each other. In the IPC case, the program could only occur when the recieving program is a later version of the sending version. That is the same problem - reading an old archive version. Robert Ramey

Robert Ramey <ramey <at> rrsd.com> writes:
That is not relevant. The ar >> and ar << don't have to be in the same program. They just have to be consistent with each other.
Thanks for the clarification. So the program in the OP (let me call it A0 :) ), is a valid test case then, and a simpler one at that, than A1, A2 and B.
In the IPC case, the program could only occur when the recieving program is a later version of the sending version. That is the same problem - reading an old archive version.
Well, the problem also manifests in single programs, as A0 indicates. There's no versioning going on there. Regards, Jarl.

Jarl Lindrud wrote:
Robert Ramey <ramey <at> rrsd.com> writes:
That is not relevant. The ar >> and ar << don't have to be in the same program. They just have to be consistent with each other.
Thanks for the clarification. So the program in the OP (let me call it A0 :) ), is a valid test case then, and a simpler one at that, than A1, A2 and B.
In the IPC case, the program could only occur when the recieving program is a later version of the sending version. That is the same problem - reading an old archive version.
Well, the problem also manifests in single programs, as A0 indicates. There's no versioning going on there.
This program contains a dummy function whose only purpose is to illustrate a problem that could occur under very infrequent circumstances. Whether or not it should be characterised as a valid usage of the library is a question I'll leave unaddressed. Basically there were and are three options here. a) include class information for all types inside of all archives. b) eliminate the option "track_selectively" c) do nothing. There are trade-offs associated with each option. Given the trade-offs, I still believe that c) is the best choice. And I also believe most people with a good understanding of these trade-offs would agree with me. Having said that I'm considering emitting a compile time warning for types to which all the following apply a) use "track_selectively" b) don't store class data in the archive c) serialialize through a pointer. This warning wouldn't come up very often, (though it would in your test case) and almost all the time it would be spurious. It would be legitimate only in my previous versioning example and your test case. So then I would have to explain what this means either in the documentation or repeatedly on this list - probably both. Then someone would (legitimately) want to be able to turn it off. So even this "simple" idea entails a lot of work. I've also considered - as I have in the past - emitting a compiler time error if versioning is used on type which doesn't store class information in the archive. I tried this before but I couldn't get it implemented without creating other problems - but I think I might have method now. This is only peripheraly related to this discussion, I just mention it so that no one will think this is a total waste of time. Robert Ramey
Regards, Jarl.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Robert Ramey <ramey <at> rrsd.com> writes:
Well, the problem also manifests in single programs, as A0 indicates. There's no versioning going on there.
This program contains a dummy function whose only purpose is to illustrate a problem that could occur under very infrequent circumstances. Whether or not it should be characterised as a valid usage of the library is a question I'll leave unaddressed.
Robert, if you are going to hint that A0 is invalid, please specify exactly what would make it so. For the Nth time, what you call dummy code, is code that was originally instantiated by some other, completely different part of the program, for completely different purposes. The fact that such code has program-wide side effects in the form of incompatible archive format changes, can't be seen as anything but a serious problem. That's exactly what Matthias pointed out in his post. Are you going to characterize his code as "dummy code", as well?
Basically there were and are three options here.
a) include class information for all types inside of all archives. b) eliminate the option "track_selectively" c) do nothing.
I'm not versed in the internals of B.Ser. , so I really can't suggest how to fix this. I just want to see it acknowledged as a real issue, and if it's not fixable, to be documented as such. Regards, Jarl.

Jarl Lindrud wrote:
Robert Ramey <ramey <at> rrsd.com> writes:
Well, the problem also manifests in single programs, as A0 indicates. There's no versioning going on there.
This program contains a dummy function whose only purpose is to illustrate a problem that could occur under very infrequent circumstances. Whether or not it should be characterised as a valid usage of the library is a question I'll leave unaddressed.
Robert, if you are going to hint that A0 is invalid, please specify exactly what would make it so.
For the Nth time, what you call dummy code, is code that was originally instantiated by some other, completely different part of the program, for completely different purposes. The fact that such code has program-wide side effects in the form of incompatible archive format changes, can't be seen as anything but a serious problem. That's exactly what Matthias pointed out in his post. Are you going to characterize his code as "dummy code", as well?
Basically there were and are three options here.
a) include class information for all types inside of all archives. b) eliminate the option "track_selectively" c) do nothing.
I'm not versed in the internals of B.Ser. , so I really can't suggest how to fix this. I just want to see it acknowledged as a real issue,
I think I did this in response to Jeffrey Bosboom's post. Robert Ramey
and if it's not fixable, to be documented as such.
Regards, Jarl.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Robert Ramey <ramey <at> rrsd.com> writes:
I'm not versed in the internals of B.Ser. , so I really can't suggest how to fix this. I just want to see it acknowledged as a real issue,
I think I did this in response to Jeffrey Bosboom's post.
Robert Ramey
Indeed... Thanks for your time, I will rest my fingers now. :) Regards, Jarl.

Jarl Lindrud wrote:
Robert Ramey <ramey <at> rrsd.com> writes:
I'm not versed in the internals of B.Ser. , so I really can't suggest how to fix this. I just want to see it acknowledged as a real issue,
I think I did this in response to Jeffrey Bosboom's post.
Robert Ramey
Indeed... Thanks for your time, I will rest my fingers now. :)
Regards, Jarl.
Actualy a solution might be that your customers change default tracking strategy to track_allways. Then all of the discussed problems would vanish. It is not very difficult to do it just grep for track_selectively and apply relevant changes. I had a similar problem when I ran serialization from different DLLs and that change seems to have fixed the problem. On the other hand it would be nice if one can set default strategy for the archive somehow. Doing it for each and single class is not the most convenient thing to do. Also hacking the library the way I proposed above is not a pleasent experience either... Any suggestions Robert?

Robert Ramey wrote:
The problem only occurs when program 1 contains only
ar << T
and program 2 contains
ar >> T and ar >> Tptr
But if only program 2 contains ar >> Tptr - how does the archive it's supposed to read get created? It can be only created by a program which also contains ar << Tptr. That is in order for serialization to make any sense at all both programs have to contain exactly the same sequence of loads and saves. Hence. they will both either serializations for the same set of types at compiler time.
I've been following this discussion, but now I'm a bit confused. If program A contains ar << T and program B contains ar << Tptr, can program C read from files produced by both programs A and B? (C would thus require ar >> T in the code that reads files produced by A and ar
Tptr when reading files produced by B.)
I certainly think it "ought to", but as I understand it from the discussion, only the files produced by program B would be readable by program C. --Jeffrey Bosboom

Jeffrey Bosboom wrote:
Robert Ramey wrote:
The problem only occurs when program 1 contains only
ar << T
and program 2 contains
ar >> T and ar >> Tptr
But if only program 2 contains ar >> Tptr - how does the archive it's supposed to read get created? It can be only created by a program which also contains ar << Tptr. That is in order for serialization to make any sense at all both programs have to contain exactly the same sequence of loads and saves. Hence. they will both either serializations for the same set of types at compiler time.
I've been following this discussion, but now I'm a bit confused.
If program A contains ar << T and program B contains ar << Tptr, can program C read from files produced by both programs A and B? (C would thus require ar >> T in the code that reads files produced by A and ar
Tptr when reading files produced by B.)
The implementation of ar >> Tptr requires the usage of ar >> T by the library. So ar >> T is always there. Program A contains only ar << T Proram B contains only ar << Tptr (which implies ar << T as well) Program C is designed to read both input from program A as well as program B. So it must contain both: ar >> T and ar >> Tptr
I certainly think it "ought to", but as I understand it from the discussion, only the files produced by program B would be readable by program C.
LOL - dammit- looks like your correct. All this time to get a good example. So your example would look like for T = vector<char> or some other collection of primitives struct X { ... ar << T } struct Y { .. ar << Tptr } Program A ar << x; // only x serialized - struct Y not included in program Program B ar << y; Program C ar >> y; // OK ar >> x ;// error ! - can't read archive produced by Program A. // if x was produced by program B - then no problem So a good argument for at least including the compile time warning. Very astute. Robert Ramey

Robert Ramey wrote:
The problem only occurs when program 1 contains only
ar << T
and program 2 contains
ar >> T and ar >> Tptr
But if only program 2 contains ar >> Tptr - how does the archive it's supposed to read get created? It can be only created by a program which also contains ar << Tptr. That is in order for serialization to make any sense at all both programs have to contain exactly the same sequence of loads and saves. Hence. they will both either serializations for the same set of types at compiler time.
The only case where this can occur is where the program is changed after an archive is created. And even then it takes some effort to produce.
The other case in which this occurs is when program 2 needs to read archives from both program 1 and program 3 (containing ar << TPtr).
In the IPC case, the program could only occur when the recieving program is a later version of the sending version. That is the same problem - reading an old archive version.
And you are saying that old archive versions are not supposed to be readable?

Peter Dimov wrote:
Robert Ramey wrote:
In the IPC case, the program could only occur when the recieving program is a later version of the sending version. That is the same problem - reading an old archive version.
And you are saying that old archive versions are not supposed to be readable?
LOL - of course not. In the IPC case, it's even less likely or the archive writing and reading to get "out of sync" - but it's just the same original complaint in a slightly different wrapper. I brought up the issue of old archives as the only realistic scenario that occured to me where the situation could occur. That's all. Robert Ramey
participants (7)
-
Jarl Lindrud
-
Jeffrey Bosboom
-
Matthias Troyer
-
Peter Dimov
-
Robert Ramey
-
troy d. straszheim
-
Zoran Cvetkov