Re: [boost] Re: Using Serialization for binary marshalling
Matthew Vogt wrote:
Yes, I had better explain here. In the case that I'm addressing, the format is not a means to an end, but an end in itself. The goal of this effort is to use the serialization library framework not to produce reversible transformations between arbitrary C++ and a byte stream, but to
take targeted C++ objects and produce known binary representations.
I don't think it is a case of coupling, merely one of limitation. An object that can be serialized into XDR format must use only a limited range of C++ types in it's composition - but having accepted that limitation, it can still be serialized into a less limiting archive type with the same code.
3) I'm curious about the override for the saving of vector. ...
I need to override the serialization of vector, because the vector must be serialized with a known policy to yield a required layout in the archive. The fact that this serialization happens (at this point in time) to be exactly the same as the default implementation is not relevant - that can be changed at any time, but the CDR, XDR and other binary formats must not change.
I'm still not convinced - its seems to me that it shouldn't need to be overridden for XDR and CDR which is what these classes do. What about
OK - I'm still wary, but if you find this doable and useful more power to you. list,
deque, set, etc.
3) I'm curious about the override for the saving of vector. ...
Well, I never thought these were necessary, remembering that I am providing for classes, which are designed to be serialized into a particular binary format. In a binary format, all collections will boil down to a group of repetitions, which are either preceded by a length/count argument, or whose length is a defined property of the format itself.
Correct. That's exactly what the current implementations of STL classes do. So my question is why override vector for you xdr/cdr archives?
For me, vector has always sufficed, in either fixed_length<>, or variable_length<> guise. Perhaps others have differing experience.
For your xdr/cdr archives to appealing to the boost users that have asked for this and for these archives to be tested to the same standards, all STL collections will have to be handled. I do not think the override for vector is necessary. If it is necessary then its probably necessary to provide overrides for all the STL collection classes.
But there are inherent limitations in marshalling (per my usage). Pointers, which are adeptly handled by the general library, are not valid elements of a marshalled data set.
I disagree here, I think the ability to pass "equivalent" pointers across space( marshalling ) is just as valuable as the ability to pass them acrose time ( persistence ). This puts a fine point on my views. I think that in the past, much of the difficulty of the topic was due the fact that different people viewed in terms of marshalling and other in terms of persistence and there is a lot of overlap. (One attempt to define "serialization" when to 200 words !). I believe my definition as stated in the start of the documentation factored out the commonality and I resolved to focus on that. I also believe that this has turned out to be one of the main reasons that the library has been embraced to the extent that it has. This explains my reluctance to consider Marshalling as a special case. I would like to see your xdr/cdr archives finished off at the same level of generality as the other archives. Any special purpose marshalling would leverage off that. Robert Ramey
Robert Ramey <ramey <at> rrsd.com> writes:
Well, I never thought these were necessary, remembering that I am providing for classes, which are designed to be serialized into a particular binary format. In a binary format, all collections will boil down to a group of repetitions, which are either preceded by a length/count argument, or whose length is a defined property of the format itself.
Correct. That's exactly what the current implementations of STL classes do. So my question is why override vector for you xdr/cdr archives?
Primarily, because the implementation you have chosen for default serialization of STL containers is just an implementation detail from your point of view; you could quite reasonably change it in the next version, if someone with alternate requirements requested that, and it didn't conflict with your own purposes. OTOH, XDR and CDR mandate how sequences should appear in a stream, so it is necessary to ensure that the default implementation is not relied upon for conformance. Another reason, is that other formats will vary the implementation in minor ways - by using a preceding length field in 8-bit signed format, for example.
For me, vector has always sufficed, in either fixed_length<>, or variable_length<> guise. Perhaps others have differing experience.
For your xdr/cdr archives to appealing to the boost users that have asked for this and for these archives to be tested to the same standards, all STL collections will have to be handled.
Ok. I'll add this, although I would be mildly surprised to see it in use.
But there are inherent limitations in marshalling (per my usage). Pointers, which are adeptly handled by the general library, are not valid elements of a marshalled data set.
I disagree here, I think the ability to pass "equivalent" pointers across space( marshalling ) is just as valuable as the ability to pass them acrose time ( persistence ).
Yes, but you're describing a higher-level view of marshalling. Such protocols certainly exist, and may well be desirable for boost users. They are, however, implemented in terms of CDR, XDR or equivalent, so I may as well get these right first.
This puts a fine point on my views. I think that in the past, much of the difficulty of the topic was due the fact that different people viewed in terms of marshalling and other in terms of persistence and there is a lot of overlap. (One attempt to define "serialization" when to 200 words !). I believe my definition as stated in the start of the documentation factored out the commonality and I resolved to focus on that. I also believe that this has turned out to be one of the main reasons that the library has been embraced to the extent that it has.
This explains my reluctance to consider Marshalling as a special case.
I think it is appropriate to view marshalling as a degenerate case, utilising the serialization framework, but not fulfilling its functional range.
I would like to see your xdr/cdr archives finished off at the same level of generality as the other archives. Any special purpose marshalling would leverage off that.
I don' think this is possible, since these formats do not specify sufficient types to do so. It would be possible to allow type promotion to cover some missing types - for example, in XDR, it would be possible to allow all C++ integral types less than 32-bits to be promoted to 32-bits during serialization, but I think this would be misleading. More problematically, there is no concept of objects in this situation. It wouldn't make sense to transmit objects via pointers, since 'the other side' will never reconstitute objects. Eliding extraneous copies of already- serialized objects would be an error. Matt
participants (2)
-
Matthew Vogt -
Robert Ramey