Re: [boost] [Serialization] Bizarre bug

10 Aug 2009

      Jarl Lindrud wrote:
...
Robert Ramey <ramey <at> rrsd.com> writes:
...
...
-----------------------------------------
* If one program writes archives like this:
ar & vec; // vec is a std::vector<char>
, it is then impossible to reliably load that archive in another
program, unless one is aware of the entire compile-time content of
the first program.
Nope.  The only time a problem can occur if one does
ar & vec; // vec is a std::vector<char>
In the saving program and
ar & p_vec; // vec is a * std::vector<char>
in the loading progam.
Robert, that's manifestly incorrect. I have appended 3 well-formed
programs, A1, A2, and B, to this post. B can read archives from A1,
but it *cannot* read archives from A2. If you don't believe me, then
please run the programs yourself. The only difference between A1 and
A2 is compile-time instantiation, of code that is never executed.
Here's my version of the three programs.  This is the way I would
expect most users use the library.

A1:
struct X {
    vector<char> m_v;
    template<class Archive>
    save(Archive & ar, const unsigned int version) const {
        ar << m_v;
    }
    template<class Archive>
    load(Archive & ar, const unisgned int version){
        ar >> m_v;
    }
};

int main(){
    X x(...);
    std::ofstream os("file.txt")
    boost::archive::text_oarchive oa(os);
    oa << x;
}

A2:

// compiled by never invoked
void dummy(){
    vector<char> * pv = new vector<char>();
    std::ofstream os("file.txt")
    boost::archive::text_oarchive oa(os);
    oa << x;
}

int main(){
    X x(...);
    std::ofstream os("file.txt")
    boost::archive::text_oarchive oa(os);
    oa << x;
}

OK - A2 produces an archive that cannot be read.
I would argue that including the dummy just to
provoke this "bug" is just a pathological case and
basically constitutes an error - albiet a subtle one.

A much more interesting scenario in this vein is the following:

C: Sometime later, the definition of X is changed to

struct X {
    vector<char> * m_pv;
    template<class Archive>
    save(Archive & ar, const unsigned int version) const {
        ar << m_pv;
    }
    template<class Archive>
    load(Archive & ar, const unisgned int version){
        // uh - oh need to support old archives !
        if(version < 1){
            m_pv = new vector<char>()
            ar >> *m_pv
        }
        else
            ar >> m_pv;
    }
};

BOOST_CLASS_VERSION(X, 1)

This WOULD be a problem and would fail to read
old archives.  I'm willing to consider this a legitimate
concern.

Still, it's a very infrequent cases.

So in short, I believe that the only way to have such
a problem is include dummy code which I don't
consider a huge problem.
...
Do you consider there to be any errors in either A1, A2 or B?
A1 serializes vector<char> directly rather than as a member of
a class.  Not illegal - be extremely unusual.
A2 includes dummy code solely to provoke this "bug".  I can't
think of a situation where inclusion of such code would be
legitimate.  And it's not easy to do either as in the majority of
cases one uses ar & rather than ar << in which cast, the
saving and loading are always in sync.

B - I didn't look at it - I presume its' correct.
...
If you were to implement a generic IPC marshalling layer, you would
end up with the kind of code that A1, A2, and B contain.
I'm not seeing that.
...
IPC serialization is a bit different from file-based serialization, in
that serialization and deserialization generally take place in
different programs.
Agreed - but I don't think it's relevant.  The relevant issue is that
serialization and deserialization use the same data schema - and this
must be true for the system to work at all.
...
Another wrinkle in IPC serialization, is that one program may contain
only serialization code for a type T (and not deserialization), while
another program may contain only deserialization code for type T (and
not serialization). If that makes the programs ill-formed in your
view,
It doesn't
...
then I guess one would have to conclude that B.Ser. is not an
appropriate choice for IPC applications.
Hence, I would not draw such a conclusion.

The real issue is my choice of implementation_level "object_serializable"
in combination with  "track_selectively" for a few specific types.

Did I make the wrong choice? Maybe - but this
is a very narrow issue related to these types
- collections of primitives. For these types serialization
implementation is not expected to change and
for which considerations of efficiency are greater
than usual.

In any case, this has got me thinking about how
to avoid this.  As I write this, I'm think maybe the
best would be to emit a compile time warning when
ever anyone tries to serialize a pointer to a type
whose serialization traits are both "object_serializable"
and "track_selectively". I'll think about this.

Robert Ramey
...
Regards,
Jarl