[1.34.0] serialization, spirit, borland and msvc-6.5
Roland and Nicola have being discussing the situation with serialization/spirit interaction with not-so-comforming compilers like borland and msvc-6.5. The conclusion as I understand it seems to be: 1. It's possible to make serialization work, by taking previous release of Spirit 1.6, removing Spirit from Boost RC and unpacking older release on top of the tree. 2. The 'out-of-tree' spirit 1.6 does not seem to work, for several reasons: - There's no V2 mechanism to plug out-of-tree spirit 1.6. Both Roland and Nicola have patches to fix this. - For borland, it requires patching Spirit 1.6 to change include statements from ""-includes to <>-includes. We don't have any regression results with the patches spirit. As Alisdair mentions, it would be bad to just drop serialization support for msvc and borland. At the same time, we don't have any idea if spirit 1.6 is the only problem with those compilers, so I think here's what we should do: 1. Nicola sends Roland and me his patched spirit. 2. Roland sets up regression tests that use the patched spirit 1.6. He changes the name of toolsets to indicate that patched spriit is used, say: msvc-6.5_spirit1.6 and does a test run. 3. If results are good enough, or can be made good enough in reasonable time, we make this patched spirit available from SF -- it can be either new release of the spirit project, or new download from the boost project with prominent warning that it's just for msvc-6.5 and borland users. 4. We also mark serialization library as unusable for msvc-6.5 and borland, and include a link for the procedure to download spirit 1.6 and set it up. This way, we correctly indicate that serialization does not work in RC download, say how it can be made to work, and have some test coverage for RC + spirit 1.6. In case if RC + spirit 1.6 has serious problems for msvc/borland, and there's nobody to fix those problems, we'll declare serialization unusable for the problematic toolsets. Now, two questions. 1. Does the above sounds good to everybody involved? Robert? Nicola? Roland? 2. If yes, Nicola and Roland, can you work on this plan? Thanks, Volodya
Vladimir Prus schrieb:
2. The 'out-of-tree' spirit 1.6 does not seem to work, for several reasons:
- There's no V2 mechanism to plug out-of-tree spirit 1.6. Both Roland and Nicola have patches to fix this.
- For borland, it requires patching Spirit 1.6 to change include statements from ""-includes to <>-includes.
My latest findings let me believe that there are even more issues with the out of tree usage. Having the entire serialization depend on the out-of tree 1.6 does give yet more errors. The good news is that having only the parts of serialization depending on spirit including the out of tree spirit seems to work. At least msvc-6.5 is compilable, and testing results in a much better figure (still not perfect though). I managed this by only having xml_[w]grammer.cpp and xml_[w]iarchive.cpp including the out of tree spirit. I am still struggling with the <>/"" issue for borland. I hope Nicola will be able to provide further input.
1. Nicola sends Roland and me his patched spirit.
I obtained the above results with SPIRIT_RC_1_6_2 branch. Perhaps it is a good idea to apply the patch to this branch? (If boost policy allows this.)
2. Roland sets up regression tests that use the patched spirit 1.6. He changes the name of toolsets to indicate that patched spriit is used, say:
msvc-6.5_spirit1.6
and does a test run.
Ok.
3. If results are good enough, or can be made good enough in reasonable time, we make this patched spirit available from SF -- it can be either new release of the spirit project, or new download from the boost project with prominent warning that it's just for msvc-6.5 and borland users.
See my above comment about patching SPIRIT_RC_1_6_2.
4. We also mark serialization library as unusable for msvc-6.5 and borland, and include a link for the procedure to download spirit 1.6 and set it up.
Hmm, not sure. What about making the tests expecting to fail. A regression run that unexpectedly passes will show up with a description why this happened. (Out of tree spirit.) So the user will see more green lights.
2. If yes, Nicola and Roland, can you work on this plan?
Yes for me. Roland
Roland Schwarz <roland.schwarz <at> chello.at> writes:
Vladimir Prus schrieb: [...]
- For borland, it requires patching Spirit 1.6 to change include statements from ""-includes to <>-includes.
My latest findings let me believe that there are even more issues with the out of tree usage. Having the entire serialization depend on the out-of tree 1.6 does give yet more errors.
The only problems I'm aware of are the need to remove non Spirit portions of Boost from the Spirit branch and the static linking issue I described in my answer to Volodya. [...]
1. Nicola sends Roland and me his patched spirit.
I obtained the above results with SPIRIT_RC_1_6_2 branch. Perhaps it is a good idea to apply the patch to this branch? (If boost policy allows this.)
As I said, the branch to use is SPIRIT_1_6 . This is the one from which Spirit 1.6.3 was obtained, even if unfortunately it wasn't tagged as such. [...]
4. We also mark serialization library as unusable for msvc-6.5 and borland, and include a link for the procedure to download spirit 1.6 and set it up.
Hmm, not sure. What about making the tests expecting to fail. A regression run that unexpectedly passes will show up with a description why this happened. (Out of tree spirit.) So the user will see more green lights.
Yes, I like this option better than Volodya's. Cheers, Nicola Musatti
I'm fine with all of this. In addition to what's been said, here is are a couple of other observations: a) The issue is without spirit 1.6, one file required for xml parsing will not build. The stops the library from being built. which makes the tests show all failure. I copied spirit 1.6 to my machine a long time ago and I've never had a problem testing with it. But then I only test with borland 5.51, 5.64, and msvc 6.1. b) I would think that the "best" solution would be for the Jamfile to detect that spirit 1.6 isn't available on a platform which requires it and skip adding the xml_?archives to the library. So the library would build in such a case without xml_support. c) Similarly, tests in such circumstances would exclude tests for xml_archives. These would then show up as blanks on the test matrix - indicating test not performed - perfectly reasonable to me. I believe that this idea is could be supported with the new V2 in a natural way. I'm not sure how much jamfile gymnastics would be necessary but hopefully it would be pretty straight forward. Robert Ramey
Now, two questions.
1. Does the above sounds good to everybody involved? Robert? Nicola? Roland?
2. If yes, Nicola and Roland, can you work on this plan?
Robert Ramey wrote:
b) I would think that the "best" solution would be for the Jamfile to detect that spirit 1.6 isn't available on a platform which requires it and skip adding the xml_?archives to the library. So the library would build in such a case without xml_support.
Spirit has a "boost/spirit/version.hpp" file which you could check and disable the xml support at the library level. -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo
Actually this kind of approach has interested me for some time. I just haven't found the time to invest in it. In the context of wide character support, I wanted to make the wide character librariy build and tests dependent upon the existence of a boost config macro BOOST_NO_WCHAR (or whatever it is). This would entail making a small test program which fails if the required facility isn't present and making the library builds and tests dependent on that. Actually, I'd like to carry the idea even further and do something like make a test for success with BOOST_STRONG_TYPEDEF and make the higher order tests not even run if the lower order tests fail. Other situations would be something like not even running test of serialization with say - variant, unless all the variant test tests pass, etc. Of course this would be if I had nothing else to do. The whole issue here final tweaks in the testing setup so that serialization passes for the latest borland compiler and msvc 7.1. Note that these tests pass with V1 so the issue isn't with the serialization library itself, its just that that is where some "corner cases" show up. I don't have a problem with just just suppressing the tests in the test matrix and adding a note to the 1.34 documenation that the library is known to work with the inclusion of spirit 1.6x . I would hate to see a very small issue hold up a whole release. Robert Ramey Rene Rivera wrote:
Robert Ramey wrote:
b) I would think that the "best" solution would be for the Jamfile to detect that spirit 1.6 isn't available on a platform which requires it and skip adding the xml_?archives to the library. So the library would build in such a case without xml_support.
Spirit has a "boost/spirit/version.hpp" file which you could check and disable the xml support at the library level.
Robert Ramey wrote:
Actually this kind of approach has interested me for some time.
[cut] That's not what I meant... What I meant is that *you* can change the serialization *code* to disable the serialization library features when they are not available/usable.
Note that these tests pass with V1 so the issue isn't with the serialization library itself, its just that that is where some "corner cases" show up.
BBv1 had kludges to suppress/work-around the errors in the serialization library. Depending on the build system for portable software is never the best solution. -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo
Rene Rivera wrote:
Robert Ramey wrote:
Actually this kind of approach has interested me for some time.
[cut]
That's not what I meant... What I meant is that *you* can change the serialization *code* to disable the serialization library features when they are not available/usable.
But they would still show up as failures in the test system. The test results would look the same. That's why I am attracted to idea of running only those tests which actually test something expected to work. This requires some sort of feedback from config.h to the test system which is why I was thinking of a small test program which would fail if the system can't be built and make the other tests dependent on that test.
Note that these tests pass with V1 so the issue isn't with the serialization library itself, its just that that is where some "corner cases" show up.
BBv1 had kludges to suppress/work-around the errors in the serialization library.
I am not aware of any such errors in the serialization libary. I don't consider it an error that some compilers cannot support conforming code - and I don't think boost does either. I'm not aware that BBv1 had/has any such kludges. V1 jamfiles had some "kludges" to permit testing on these platforms. These reflect weakness in V1 which I'm hoping have been addressed in V2. For me the jury is still out on this as I haven't really re-visited Jamfiles with V2.
Depending on the build system for portable software is never the best solution.
I'm depending on the build system for building and testing the library to the extent that it is supported by the compiler and environment. I believe that is what the build/test system is for. In the case in question, the serialization library depends upon spirit to parse xml input. The boost version of spirit - 1.8 does not support these compilers. So the serialization library doesn't either. The developer's of spirit have kept 1.6 around for such situations the users of the serialization libary with these compilers have the option to use spirit 1.6 or not use xml archives. These are all legitimate options. Ideally the build/test system would support the options preferred by the user. Note that the whole issue is repeated with regard to wide char i/o. It doesn't really show up now that it seems that (almost?) all platforms being tested support wide char i/o. So to me, there are couple of disjoint issues. a) getting 1.34 out in the "best" way. b) how should libraries which support different levels of functionality be handled with V2? I really think this should be handled in the HEAD branch. Robert Ramey
Robert Ramey <ramey <at> rrsd.com> writes: [...]
b) I would think that the "best" solution would be for the Jamfile to detect that spirit 1.6 isn't available on a platform which requires it and skip adding the xml_?archives to the library. So the library would build in such a case without xml_support.
Note that what we're discussing here is a solution for Boost 1.34 . My personal take is that this should be the smallest, simplest change that makes it possible to use serialization with Borland compilers and VC++ 6.5 The solution for 1.35 should be agreed upon with Joel and the other Spirit developers; they are considering making different versions of Spirit available at the same time within the Boost distribution. This would entail a very different approach for future versions.
c) Similarly, tests in such circumstances would exclude tests for xml_archives. These would then show up as blanks on the test matrix - indicating test not performed - perfectly reasonable to me.
I believe that this idea is could be supported with the new V2 in a natural way. I'm not sure how much jamfile gymnastics would be necessary but hopefully it would be pretty straight forward.
I like your approach, but I think it's to late to apply it to Boost 1.34. Cheers, Nicola Musatti
Vladimir Prus <ghost <at> cs.msu.su> writes:
Roland and Nicola have being discussing the situation with serialization/spirit interaction with not-so-comforming compilers like borland and msvc-6.5.
The conclusion as I understand it seems to be:
1. It's possible to make serialization work, by taking previous release of Spirit 1.6, removing Spirit from Boost RC and unpacking older release on top of the tree.
Correct.
2. The 'out-of-tree' spirit 1.6 does not seem to work, for several reasons:
- There's no V2 mechanism to plug out-of-tree spirit 1.6. Both Roland and Nicola have patches to fix this.
Correct.
- For borland, it requires patching Spirit 1.6 to change include statements from ""-includes to <>-includes.
To be more precise, include directives must use angle brackets and be relative to the Boost root directory, i.e. look like #include <boost/spirit/version.hpp>
We don't have any regression results with the patches spirit.
Correct! Unfortunately the issue with truncated test output made it look like tests were running without problems.
As Alisdair mentions, it would be bad to just drop serialization support for msvc and borland. At the same time, we don't have any idea if spirit 1.6 is the only problem with those compilers, so I think here's what we should do:
1. Nicola sends Roland and me his patched spirit.
OK.
2. Roland sets up regression tests that use the patched spirit 1.6. He changes the name of toolsets to indicate that patched spriit is used, say:
msvc-6.5_spirit1.6
and does a test run.
3. If results are good enough, or can be made good enough in reasonable time, we make this patched spirit available from SF -- it can be either new release of the spirit project, or new download from the boost project with prominent warning that it's just for msvc-6.5 and borland users.
Note that once this issue is corrected, we'll be faced with the static linking problem I mentioned on the Boost Build mailing list. This is currently causing almost all the serialization static linking tests to fail.
4. We also mark serialization library as unusable for msvc-6.5 and borland, and include a link for the procedure to download spirit 1.6 and set it up.
This way, we correctly indicate that serialization does not work in RC download, say how it can be made to work, and have some test coverage for RC + spirit 1.6.
No objections.
In case if RC + spirit 1.6 has serious problems for msvc/borland, and there's nobody to fix those problems, we'll declare serialization unusable for the problematic toolsets.
No objections.
Now, two questions.
1. Does the above sounds good to everybody involved? Robert? Nicola? Roland?
Fine by me.
2. If yes, Nicola and Roland, can you work on this plan?
Certainly. Cheers, Nicola Musatti
Nicola Musatti wrote:
Vladimir Prus <ghost <at> cs.msu.su> writes:
1. It's possible to make serialization work, by taking previous release of Spirit 1.6, removing Spirit from Boost RC and unpacking older release on top of the tree.
Correct.
Doesn't look so for me. I tried all of boost SPIRIT_RC_1_6_2 SPIRIT_1_6 from the boost cvs and spirit-1.6.3.zip and spirit-only-1.6.3.zip from http://spirit.sourceforge.net. I am specifically unsure what it means "pack on top of tree". Perhaps this is where I am making a mistake. Please be more explicit. Roland btw.: I am trying for msvc-6.5 first.
Roland Schwarz <roland.schwarz <at> chello.at> writes: [...]
I tried all of boost SPIRIT_RC_1_6_2 SPIRIT_1_6 from the boost cvs and spirit-1.6.3.zip and spirit-only-1.6.3.zip from http://spirit.sourceforge.net.
I am specifically unsure what it means "pack on top of tree".
Perhaps this is where I am making a mistake. Please be more explicit.
I'm going to make a few tests and provide a detailed description of my environment. OK? Cheers, Nicola Musatti
participants (5)
-
Nicola Musatti -
Rene Rivera -
Robert Ramey -
Roland Schwarz -
Vladimir Prus