"Robert Ramey" <ramey@rrsd.com> writes:
David Abrahams wrote:
Note that points not explicitly responded to have just been fixed in accordance with your suggestion.
It's an improvement, but either you missed or intentionally discarded some of my more important remarks. The most important question was: **** Isn't something in an Archive required to call "serialize?" ****
Strictly speaking no.
<snip>
The point is that if one writes a program in accordance with the requirements stated here, the program will be a legal program and will compile.
But that's not enough. I'm not interested in making an archive that just compiles; I want it to have some sensible semantics. What requirements do I have to fulfill in order to make the archive actually *work*? If you look at the iterator requirements, for example, you'll see that there are semantic requirements in place, not just syntactic ones. These requirements are needed so that the algorithms can say something about what they do. The sort() algorithm wouldn't be able to make any claims about its result if writing into a random access iterator were allowed to be a no-op. As I've said before, you will need eventually to describe the relationship between compatible loading and saving archives. It will be something like T x, y; // arbitrary operations on x to set its state sar & x; lar & y; Postcondition: y is equivalent to x [Someone else suggested a complex way to describe this idea without mentioning the word "equivalent" but I've no idea whether it hangs together, and at least this is good enough for the standard] You can't say anything like that unless loading and saving are required to do _something_. out of context:
Of course, I would expect all archive implementations to call "serialize" but maybe someone comes up with an archive implementation that just checks syntax or something like that. Someone might want to make an archive which only stored/loaded primitive types. Or someone might make an archive that was implemented solely in terms of save/load_binary. These would never call serialize.
Do not build in generality that you're not even sure is needed, especially if it costs you your ability to write concepts that tell people how to get sensible results. You can always introduce less-refined concepts that allow these kinds of things and provide weaker semantic guarantees.
If not, how does an Archive work with Serializable types?
That is defined by the particular archive implementation. There are 5 included and maybe more on the way.
You keep missing the point. I don't care about what you've included in the library. I'm implementing my own archive because what you've included doesn't do the job.
There may be one on the way that is only for output - perhaps useful for debug logging. These are not meant to be prohibited by these requirements.
If the requirements aren't strong enough to tell me how to do something useful, they're not much good.
Common Type Definitions in all Archives
This should be a section for the "Archive" concept.
From your later writing in this post you seem to have missed, ignored, or discarded this suggestion. Actually, it's stronger: an assertion. I don't believe it is optional: you need to specifically define an "Archive" concept of which Loading Archive and Saving Archive are refinements.
These are not type definitions but valid expressions.
A::is_saving An integral constant of type boost::mpl::bool_. I'm sorry to be so blunt, but that's nonsense. A::is_saving is a type, not an integral constant, and boost::mpl::bool_ is a class template, not a type. If you mean to refer to the MPL Integral Constant concept, the correct thing to say would be: A boolean MPL Integral Constant with a link to the documentation for that concept.
Value is boost::mpl::bool_<true>
OK what I meant to say is
A boolean MPL Integral Constant for which the following expression is true
boost::mpl::is_equal< A::is_loading, boost::mpl::bool_<true>
<snip>
How would you suggest I phrase this?
I believe that requirement is too strong, and it would be sufficient to say An MPL Integral Constant with a nonzero ::value member
Common Type Definitions in all Archives
Common Member Functions in all Archives
Again, these are not member functions, but valid expressions. Presumably "ar & x" can be implemented by a free function and "ar.template register_type<T>()" invokes a member function template.
I have to confess I plagerized this exact phrasing from my copy of STL Tutorial and Reference Guide - page 263.
In that case I have lost a lot of respect for that book. It was co-written by one of the founders of Generic Programming, but that might just mean he was the PhD adviser for the person who actually did the work. I suggest you try some others, like Josuttis', "The C++ Standard Library" or Austern's, "Generic Programming and the STL"
I felt it was a very close analogy and a good model. I did note that in this reference operators that could or would be implemented as free functions were included under the "wrong" heading but I presume this was done this way on purpose in order to improve clarity
Inaccuracy and inconsistency never improves clarity. It just makes me wonder what the author really meant.
and reflect the fact that most common implmentations would be member functions.
register_type is not a member function but a member function template. A template is not a function.
I could change the heading to "Common Operations for all Archives" or .. your suggestion here.
The heading should be "Archive Concept Requirements" with a note that each entry describes an expression that must be valid.
ar & x Returns a reference to ar. Appends the value of x along with other information to ar.
I think you should swap the two sentences; emphasis should be on effects and only secondarily on return value.
Again I Plagerized this form the same source as above page 264.
Doesn't make it right.
I'm not sure this is a big issue
I am. The phrasing makes it sound like the job of that expression is to return a reference to ar.
But wait. Surely this expression, with the specified semantics, doesn't have to be valid for loading archives? Are you saying that all archives have to be able to save?
You did say that points not explicitly responded to have just been fixed in accordance with your suggestion. but since I didn't make a suggestion here I can only assume that you ignored my questions. Or maybe you assume my questions to refer to the following line instead of the one above? For future reference, what I'm responding to is always above my response unless there is a colon folowing what I write. I meant: Surely the expression "ar & x" , with the semantics, "appends the value of x along with other information to ar", doesn't have to be valid for loading archives? This is the sort of thing that makes me think you're not really paying attention. Also, I just noticed "along with other information." If I'm writing an archive, how am I supposed to fulfill that requirement? Can I write any arbitrary "other information" I want? Suppose I write a zero-valued char? Suppose I don't write _any_ "other information?"
ar.template register_type<T>();
To deal with derived types not otherwise explicitly referred to, any invocation of this function must appear in both the saving AND loading archives in the same place. This is the mechanism by which saving and loading is kept in syncronization.
Sure, that's what I expected.
You should leave out "template." This is a little tricky, but that expression will never be valid in a context where A is not a dependent type. I realize you do need it when A is a dependent type, but the convention is to use the simpler notation.
Hmm - all Archives included with the library are templates.
Irrelevant.
So, removing this will be including a "valid expression" which will fail to compile on all conforming compilers.
No. some_archive_template<Whatever> ar; ar.template register_type<T>(); is ill-formed. some_archive_template<Whatever> ar; ar.register_type<T>(); is well-formed if T is a type and Whatever is a valid template argument to some_archive_template.
I user who looks here to see what the valid syntax is for invoking this operation, and uses it to get a bug will be writing to tell me the manual is wrong. I stll could use ar.register_type(& t) - but that's confusing as well. So what's a good solution here?
Do it the way I'm suggesting. If you like, add a footnote for the uninitiated explaining that when A is a dependent type, the "template" keyword may be required.
The author of an archive isn't going to be calling register_type anyway (especially not in a dependent context), so it won't help prevent him from making errors.
This is extremely curious to me. This whole section started as the "Interface for users of the serialization library". I would expect that this is one section that would be of interest to users.
If you want this to be directed only at users, take the "Concept" label off it and put that elsewhere. Go back to doing whatever you want here to informally explain archives to users, and in some other section, give me real, complete concept requirements that tell me what I need to fulfill in order to create a working archive.
register_type is essential to library users. So it has to be explained correctly right here.
The explanation I've suggested _is_ correct.
. count is an integer that can be converted to std::size_t. ^^^^^^^ no, it's an instance of any type that can be so converted.
OK Your "strong typedef" isn't an integral type. ^ did your mailer kill the newline here?
I don't see how this is related.
You're right; it's not.
Given that all archives present the same public interface, specifcation of serialization is exactly the same for all archives.
I think that should be pretty clear
Yes, it's clear. Not sure what your point is.
Are you telling me, after all this, that there are additional requirements on archives not documented in the section on Archive concepts?
No.
Maybe I've misinterpreted the contents of http://www.boost.org/libs/serialization/doc/archive_reference.html#implement... Is that not describing the _real_ requirements for a working archive?
Archive classes have other members not mentioned here.
For example, in order to use a text_oarchive, a user must invoke the appropriate constructor with the appropriate arguments.
Okay, that's reasonable, but doesn't merit a mention here; it's just confusing to do so and adds nothing.
However they are related to the internal functioning of the library and are not meant to be called by users of an archive.
Hmm - now that I read this I'm not sure what I really meant. I'll think about this.
OK.
I find this rather alarming. Haven't I made it abundantly clear that the concept documentation has to give at least a minimal and complete documentation of what's required of an Archive?
I don't believe anything so far conflicts with that.
Maybe I've been too hasty, but it seemed to me, from the lack of any requirement that the archive do something that interacts with the Serializable concept (and produce sensible semantics) and your reference to a separate section describing how to implement an archive, that you had been trying to make the "Archive Concept" section a "user-only" document, and had put the real requirements elsewhere.
Okay, I don't know what to say at this point. I'm about to give up, because I don't have the time to keep going over this. It sure seems like you want to do this your own way, rather than by following the established practices and norms. If you were just "creative" but still rigorous, it would be one thing, but it doesn't even seem like you're paying close attention to what you yourself are writing.
Actually that's not true.
I think the record on this list and CVS shows multiple versions of trying to get this right.
True, you have responded, but the record also shows that I've needed to ping you several times to keep from being ignored -- or so it seemed -- which has been discouraging.
Other changes you have suggested have been incorporated once the issues have been clarified. I have based my various versions on several known "good" models.
Maybe they're leading you astray. If you stop and think about some of the things I mentioned in this email I think you'll see that some of them just don't make sense except under the loosest, most informal interpretations.
You have said yourself that the latest version is an improvement.
What should I do?
You're doing just great Dave, just hang in there.
Well, thanks for the reassurance. I'll try.
Implementation of new archives is discussed in New Archives - Implementation. Which then goes on to describe some "input archive" idea that hasn't been defined, etc... <snip>
Assuming that "Archive Concept" is more or less resloved,
Not yet.
the question arises regarding the rest of the documentation. Basically there are two "large" sections.
Serializable Concept.
Without having looked at it recently, I'm can assume this section doesn't meet your definition of formal documentation.
I haven't looked at it yet.
I would guess this could be addressed along the lines of Archive Concept, but I do have a couple of reservations even before I look at it. Much of the documentation there explains things in terms of implementation behavior (how serialization of pointers works, etc) which doesn't seem to fit all what well with the definition of formal semantics.
The whole question of how much semantics should be in a formal reference is murky to me. It seems that the idea of formal specification is to emulate something like the specification of and algebra - not a bad place to start. But specification of the rules for an algebra makes no reference at all to its semantics - hence its wide applicability.
No, that's not true. An algebra's semantic rules describe invariants. The meanings of statements in an algebra is constrained their invariant transformations. a = b ==> b = a is semantic. Anyway, I don't think that trying to frame this question in terms of algebra is going to be helpful. Semantic requirements show up in concept definitions for entirely practical reasons.
In the case of library specification we want to role library semantics into this. It seems to me that that might obscure the whole idea in the first place. Your question - when does the "serialize" funcition get called? and my response "what does it matter" illustrate differing understandings about this.
Clearly.
I'm starting to wonder - if its really a formal definition - how can it have sematics? And if doesn't have semantics, how can it be useful?
I hate to be blunt, but I don't really care whether this meets your idea of what "formal" means. Let's not make this a philosophical argument. It's all about practical concerns.
This is the reason I was so intrigued with Joaquin's post. It touches upon what to me is the crux of the issue. I'm not sure it resolves it - but it does address the proper place of semantics in the discussion.
Joaquin's post takes an "innovative" approach to the problem of specifying semantics but it isn't at all clear to me that it holds water. The reason that "equivalent" is a fuzzy term in C++ comes down to the fact that two distinct objects always have detectably distinct addresses, so no two distinct objects can _truly_ be equivalent. Leaving aside that language corner, the idea of equivalence works perfectly well. I suggest you use that, and the established conventions from the literature, to describe semantics. You have, essentially, an emergency on your hands -- this is not the time to try untested approaches. First plug the dyke and then, if you have time, think about a rewrite.
So, I'm still thinking about what needs to be done (if anything) regarding the "Serializable" type concept section.
Don't feel obligated to coment on the above.
Too late ;-)
Achive Implementation
Here is the part of the documentation that I have been most concerned about.
I changed "Archive User Interface" to "Archive Concept" without thinking too much about it.
When was that? Maybe that was the root of the problem.
I don't remember any user having any question about this. So I wasn't too concerned.
This part of the library really describes a "tool kit" for implementing the Achive Concepts. To make an analogy to your iterator library there are two things going.
Definition of iterator (archive) concept - A user of an iterator instance need only look at this part of the documentation to use an iterator which models the concept.
And an implementor of an iterator need only look at the iterator requirements to _implement_ an iterator that models the concept. How do you think we came up with iterator_facade and iterator_adaptor, anyway? They weren't spun out of thin air: we carefully studied the iterator concept requirements.
Definintion of the iterator (archive) CRTP classes for deriving one's own iterators (archive) This is a lot harder to explain and use. I know because I did use it. I should say that I was very pleased with the results as once I got things to compile, I had very few problems.
So I've been planning to re-visit "Archive Implementation" in any case. But I don't think there is any quick fix. I just have to spend more time on it. And there are a couple of other issues. I had to add to the interface a tiny bit in order to support shared_ptr serialization. Its not a big thing but I didn't want to document it because I'm thinking its not quite right (though its good enough for shared_ptr) and I don't want anyone else to use it. There might be a couple of other things that might or might not need to be made public such as stack_construct<T>(Archive &ar).
I'm still thinking about how to handle things BOOST_SERIALIZATION_NVP. This is optional for most archives but required for xml_archives. So does this mean an refinement on the concepts already described. It would look like it.
So - I would like to:
a) resolve the final issues regarding "Archive Concept" I don't think we're at all far apart in this.
Remains to be seen.
b) ship RC_1_33_0 (again). This is important to me as I've fixed the header sequence issue as well as a nasty bug. So I would like to see the new version shipped ASAP. c) Give me some time to go back to the things I mentioned above.
Sounds fine to me.
I believe that the serialization library has proved quite useful to some people inspite of the lack of formality in its documentation and shipping of a corrected version shouldn't be held up for this.
It's not formality that's the problem: it's vagueness, inconsistency, and lack of a practical document that describes what an archive implementor needs to do.
Lastly, I know I'm driving you crazy. I'm sorry about that. Please believe me that its not intentional on my part - its just comes naturally.
:)
I do sometimes think you underestimate the depth of my thinking on some of these subjects. Perhaps I'm confused about some of these things because I think about them too much.
I'm beginning to think that's true. Maybe a little less abstract thought and a little more attention to practical detail would help.
I don't know if its any consolation, but its stressful for me as well.
I'm sure. I know I can be a really tough critic and I'm trying to restrain myself a little at least.
But your input has been very helpful and added a lot to quality of the library and its documentation and that's why I put up with it.
I sure hope you can laugh about this.
If I couldn't, it would be pretty sad. Thanks for your goodwill. -- Dave Abrahams Boost Consulting www.boost-consulting.com