[Serialization] Major regressions on RC_1_33_0
The last round of changes to the Serialization library have caused major regressions (205, to be precise) on the RC_1_33_0 branch. If these regressions have not been fixed by Monday morning I will revert these changes automatically. Doug
That would be regrettable. The last round of changes address some very obscure issues that mostly appear in applications which serialize types defined in DLLs. Some compilers also trip here on code that has nothing to do with DLLs. None of these errrors are detectible in the tests that boost runs. The new errors can be clasified as a) the VC 7.1 errors are most likely due the fact that bjam cannot be counted upon to detect dependent changes. I had the same problem on my own VC 7.1 system and had to explicitly rebuild the library to get the error to go away. b) demo_load_xml is surely due to the fact that it is run before demo_save_xml - this will disappear as well. c) the new errors (which now don't show) on CW 9_4 will disappear on the next round as I make another tweak. This is related to the fact that it seems this particular version of this compiler handles const template parameters different than other compilers. I'm testing the change now. d) This leaves two tests test_exported, and test_unregistered_pointers in the following compilers. gcc-3.2.3-linux gcc-4.01-linux intel-8.1-linux - windows version passes ok intel-9.0-linux - windows version passes ok I believe that there is something in these compilers that needs to be worked around. The error messages suggest that certain explicitly instantiated template code has been eliminated as it is not explicitly referred to by name. Many other compilers exihibit this quirk. Link and Error messages, are cryptic as usual but seem to support this idea. bugzill for gcc-4.01 lists a bug that seems to be relatd. I would be curious to see what gcc 4.02 has to say. Its odd that intel behavior varies between linux and windows. Perhaps there is a difference in optimization switches in the different cases. None of these compilers are in the CVS-HEAD so I don't know if these errors just recently appeared or are due to the latest changes. (could be either way) So I'm a little bit at a loss here. Anyone who has experience with these compilers is welcome to jump in here. e) There are a couple of new failures in MSVC 6.5. These are caused by making export.hpp even more elaborate than it used to be resulting in some new ICE's. I haven't been able to fix them without backing out the changes that permit headers to be in almost any order - and I don't think we want to do that. So I think we have to live with them - or someone who needs this can invest some time trying to work around the ICE's Robert Ramey Douglas Gregor wrote:
The last round of changes to the Serialization library have caused major regressions (205, to be precise) on the RC_1_33_0 branch. If these regressions have not been fixed by Monday morning I will revert these changes automatically.
Doug _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On Sun, Oct 09, 2005 at 02:36:54PM -0700, Robert Ramey wrote:
a) the VC 7.1 errors are most likely due the fact that bjam cannot be counted upon to detect dependent changes. I had the same problem on my own VC 7.1 system and had to explicitly rebuild the library to get the error to go away.
b) demo_load_xml is surely due to the fact that it is run before demo_save_xml - this will disappear as well.
I was going to ask about this when I thought I'd isolated the problem. I'm working in the tests and have recently noticed that these two tests, [ test-bsl-run_files test_class_info_save ] [ test-bsl-run_files test_class_info_load ] sometimes don't appear to obey these, DEPENDS test_class_info_load_text_archive.run : test_class_info_save_text_archive.run ; It seems that there is different behaviour on different machines. Rebuilding from scratch just now made the problem go away, but of course I'd rather isolate it than let it jump me again later... I'm going to soon be offering up a conversion to unit tests and hooks for portability testing. Is it OK to leave it like that, have I created some unrelated problem, should I chase this down, or what? Thanks in advance, -t
troy d. straszheim wrote:
On Sun, Oct 09, 2005 at 02:36:54PM -0700, Robert Ramey wrote:
a) the VC 7.1 errors are most likely due the fact that bjam cannot be counted upon to detect dependent changes. I had the same problem on my own VC 7.1 system and had to explicitly rebuild the library to get the error to go away.
Grr, Robert please stop saying that. It is reliable, as long as the dependencies are regular '#include <>' and '#include ""'. So if you think it's not handling those cases please point out where so we can fix it. If you think it should handle other cases please also point those out so we can add support for them.
b) demo_load_xml is surely due to the fact that it is run before demo_save_xml - this will disappear as well.
I was going to ask about this when I thought I'd isolated the problem. I'm working in the tests and have recently noticed that these two tests,
[ test-bsl-run_files test_class_info_save ] [ test-bsl-run_files test_class_info_load ]
sometimes don't appear to obey these,
DEPENDS test_class_info_load_text_archive.run : test_class_info_save_text_archive.run ;
It seems that there is different behaviour on different machines.
No. As was pointed out before that DEPENDS has no effect as you need the real target names, which those aren't. -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - Grafik/jabber.org
On Mon, Oct 10, 2005 at 01:22:48AM -0500, Rene Rivera wrote:
No. As was pointed out before that DEPENDS has no effect as you need the real target names, which those aren't.
Glad I asked. I went back and found your detailed post: http://article.gmane.org/gmane.comp.lib.boost.devel/111585 I'll dig in. Thanks, -t
a) the VC 7.1 errors are most likely due the fact that bjam cannot be counted upon to detect dependent changes. I had the same problem on my own VC 7.1 system and had to explicitly rebuild the library to get the error to go away.
Grr, Robert please stop saying that. It is reliable, as long as the dependencies are regular '#include <>' and '#include ""'. So if you think it's not handling those cases please point out where so we can fix it. If you think it should handle other cases please also point those out so we can add support for them.
There is definitely a problem, a clean rebuild does result in a bunch a failures, all but one of these goes away if you then build again. These are clearly related to Rene's explanation (http://article.gmane.org/gmane.comp.lib.boost.devel/111585) which hasn't been applied inspite of a comment in the Jamfile that seems to describe that very fix. I had a grep through the headers and couldn't find any instances of obfuscated header includes that might trigger a problem in the library, however in the test sources there are a lot that contain: #include BOOST_PP_STRINGIZE(BOOST_ARCHIVE_TEST) BOOST_ARCHIVE_TEST gets defined as a macro in the Jamfile so that different archive forms (text, binary xml etc) all get tested. So.... if there is a change in the library the *test* may not get rebuilt. Robert is that a fair characterisation of the problem? I don't see any easy solution to this, unless a DEPENDS clause can be added at the same time as the "run" clause gets added to the tests (inside another rule that is). Rene would that be possible? Any chance you could look at the Jamfile and figure out how at least one of the DEPENDS clauses should look so the rest of us mortals could take it from there? Thanks, John.
John Maddock wrote:
So.... if there is a change in the library the *test* may not get rebuilt. Robert is that a fair characterisation of the problem?
That seems right to me
I don't see any easy solution to this, unless a DEPENDS clause can be added at the same time as the "run" clause gets added to the tests (inside another rule that is). Rene would that be possible? Any chance you could look at the Jamfile and figure out how at least one of the DEPENDS clauses should look so the rest of us mortals could take it from there?
Actually, Rene did this for me sometime ago but it was absolutely un-understandable to me. It depended on internal, low-level implementation behavior of bjam and I couldn't fathom how and why it worked. As part of this effort I augmented the Jamfile to skip tests which were appropriate in the given environment (e.g. lack of support for spirit, wide chars, etc). So I didn't think I could get all this working and have something I could maintain without constantly bugging other people to help. I hadn't dwelled on this issue because it didn't bother me all that much and I've always assumed that bjam v2 was on the horizon. I also hoped that it would address in an obvious and transparent way the problem of sequencing certain tests - perhaps by permitting one to designate the intermediate file as a test target. Robert Ramey
John Maddock wrote:
I had a grep through the headers and couldn't find any instances of obfuscated header includes that might trigger a problem in the library, however in the test sources there are a lot that contain:
#include BOOST_PP_STRINGIZE(BOOST_ARCHIVE_TEST)
BOOST_ARCHIVE_TEST gets defined as a macro in the Jamfile so that different archive forms (text, binary xml etc) all get tested.
So.... if there is a change in the library the *test* may not get rebuilt. Robert is that a fair characterisation of the problem?
I don't see any easy solution to this, unless a DEPENDS clause can be added at the same time as the "run" clause gets added to the tests (inside another rule that is). Rene would that be possible? Any chance you could look at the Jamfile and figure out how at least one of the DEPENDS clauses should look so the rest of us mortals could take it from there?
I'm not sure that's possible. And there is a much easier solution. The tests can be changed to have the needed #include references to all those headers. All it takes is adding this, to each point there is the above PP include: /* #include "text_archive.hpp" #include "text_warchive.hpp" #include "binary_archive.hpp" #include "xml_archive.hpp" #include "xml_warchive.hpp" */ Since it's commented out the tests themselves don't change behavior. And since bjam doesn't care about comments it will find them. If the extra typing is a bummer those, along with the PP include, could be placed in a separate archive_test.hpp which has: #include BOOST_PP_STRINGIZE(BOOST_ARCHIVE_TEST) /* #include "text_archive.hpp" #include "text_warchive.hpp" #include "binary_archive.hpp" #include "xml_archive.hpp" #include "xml_warchive.hpp" */ And then change the PP includes to '#include "archive_test.hpp"'. Drawback is that a library change will cause every type of archive to be tested, but it would do that anyway even if bjam parsed the PP include :-) -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - Grafik/jabber.org
I'm not sure that's possible. And there is a much easier solution. The tests can be changed to have the needed #include references to all those headers. All it takes is adding this, to each point there is the above PP include:
/* #include "text_archive.hpp" #include "text_warchive.hpp" #include "binary_archive.hpp" #include "xml_archive.hpp" #include "xml_warchive.hpp" */
Since it's commented out the tests themselves don't change behavior. And since bjam doesn't care about comments it will find them. If the extra typing is a bummer those, along with the PP include, could be placed in a separate archive_test.hpp which has:
#include BOOST_PP_STRINGIZE(BOOST_ARCHIVE_TEST) /* #include "text_archive.hpp" #include "text_warchive.hpp" #include "binary_archive.hpp" #include "xml_archive.hpp" #include "xml_warchive.hpp" */
And then change the PP includes to '#include "archive_test.hpp"'. Drawback is that a library change will cause every type of archive to be tested, but it would do that anyway even if bjam parsed the PP include :-)
That sounds like a plan, Robert? We just need to reliably solve the "test X run depends on test Y run output" in a way that Robert can maintain, or failing that just hack it so that we can clear the regressions for 1.33.1. John.
John Maddock wrote:
We just need to reliably solve the "test X run depends on test Y run output" in a way that Robert can maintain, or failing that just hack it so that we can clear the regressions for 1.33.1.
I think the changes Troy posted are reasonably maintainable as they are the most direct. -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - Grafik/jabber.org
In principle I have no problem with this. I'm not sure whether we want to do it right now. Its really only a very minor inconvenience that only effects the test matrix which will soon be replaced with HEAD again so my vote is to just not mess with this until later. If you guys give the go ahead I'll do it now - otherwise we can let it slide until 1.33.1 is out. Robert Ramey P.S. sorry I couldn't figure this out myself. John Maddock wrote:
I'm not sure that's possible. And there is a much easier solution. The tests can be changed to have the needed #include references to all those headers. All it takes is adding this, to each point there is the above PP include:
/* #include "text_archive.hpp" #include "text_warchive.hpp" #include "binary_archive.hpp" #include "xml_archive.hpp" #include "xml_warchive.hpp" */
Since it's commented out the tests themselves don't change behavior. And since bjam doesn't care about comments it will find them. If the extra typing is a bummer those, along with the PP include, could be placed in a separate archive_test.hpp which has:
#include BOOST_PP_STRINGIZE(BOOST_ARCHIVE_TEST) /* #include "text_archive.hpp" #include "text_warchive.hpp" #include "binary_archive.hpp" #include "xml_archive.hpp" #include "xml_warchive.hpp" */
And then change the PP includes to '#include "archive_test.hpp"'. Drawback is that a library change will cause every type of archive to be tested, but it would do that anyway even if bjam parsed the PP include :-)
That sounds like a plan, Robert?
We just need to reliably solve the "test X run depends on test Y run output" in a way that Robert can maintain, or failing that just hack it so that we can clear the regressions for 1.33.1.
John.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Robert Ramey wrote:
In principle I have no problem with this. I'm not sure whether we want to do it right now.
I guess this is a question for Doug. Do this now or later?
Its really only a very minor inconvenience that only effects the test matrix which will soon be replaced with HEAD again so my vote is to just not mess with this until later.
What does HEAD vs. RC have to do with this? Is the same issue fixed in HEAD and not RC?
P.S. sorry I couldn't figure this out myself.
If I had a nickel for every obvious problem someone else pointed out to me... OK so I wouldn't be rich, but I'd be closer ;-) -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - Grafik/jabber.org
Rene Rivera wrote:
What does HEAD vs. RC have to do with this? Is the same issue fixed in HEAD and not RC?
what I meant was that RC will be soon be closed and we'll be back to checking into HEAD again.
P.S. sorry I couldn't figure this out myself.
If I had a nickel for every obvious problem someone else pointed out to me... OK so I wouldn't be rich, but I'd be closer ;-)
well it wasn't THAT obvious Robert Ramey
On Oct 10, 2005, at 5:36 PM, Rene Rivera wrote:
Robert Ramey wrote:
In principle I have no problem with this. I'm not sure whether we want to do it right now.
I guess this is a question for Doug. Do this now or later?
Go ahead and do it now, please. Doug
In principle I have no problem with this. I'm not sure whether we want to do it right now. Its really only a very minor inconvenience that only effects the test matrix which will soon be replaced with HEAD again so my vote is to just not mess with this until later. If you guys give the go ahead I'll do it now - otherwise we can let it slide until 1.33.1 is out.
We should check with Doug, but personally I would be in favour of making the change now: the problem we have at present is that we have no way of knowing whether there are regressions in serialization or not, we know there are failures showing up in the tests, but can we be certain that all of them bjam-dependency related? Just my 2c worth, John.
Rene Rivera <grafik.list@redshift-software.com> writes:
troy d. straszheim wrote:
On Sun, Oct 09, 2005 at 02:36:54PM -0700, Robert Ramey wrote:
a) the VC 7.1 errors are most likely due the fact that bjam cannot be counted upon to detect dependent changes. I had the same problem on my own VC 7.1 system and had to explicitly rebuild the library to get the error to go away.
Grr, Robert please stop saying that.
Yes, blaming the tools without evidence is a common rookie error; I'm surprised that someone as experienced as Robert should make such an implication. Normally a few embarrassing incorrect accusations are enough to stop a person from doing that (I learned my big lesson in this when I was still a teenager). -- Dave Abrahams Boost Consulting www.boost-consulting.com
On Sun, Oct 09, 2005 at 02:36:54PM -0700, Robert Ramey wrote:
a) the VC 7.1 errors are most likely due the fact that bjam cannot be counted upon to detect dependent changes. I had the same problem on my own VC 7.1 system and had to explicitly rebuild the library to get the error to go away.
b) demo_load_xml is surely due to the fact that it is run before demo_save_xml - this will disappear as well.
Greets, Things make more sense to me now. If you do: local foo = [ test-bsl-run_files test_class_info_save ] ; Echo $(foo) ; you can see all the tasty gristle on the front of the target generated by invoking test-bsl-run_files. You can transform the suffixes to .run with $(foo:S=.run), and then you've got something you can feed to DEPENDS. Tested it here, it works. So replace DEPENDS test_class_info_load_text_archive.run : test_class_info_save_text_archive.run ; DEPENDS test_class_info_load_text_warchive.run : test_class_info_save_text_warchive.run ; DEPENDS test_class_info_load_xml_archive.run : test_class_info_save_xml_archive.run ; DEPENDS test_class_info_load_xml_archive.run : test_class_info_save_xml_warchive.run ; DEPENDS test_class_info_load_binary_archive.run : test_class_info_save_binary_archive.run ; with local saving-tests = [ test-bsl-run_files test_class_info_save ] ; local loading-tests = [ test-bsl-run_files test_class_info_load ] ; DEPENDS $(loading-tests:S=.run) : $(saving-tests:S=.run) ; and DEPENDS test_demo_xml_load.run : test_demo_xml_save.run ; with local saving-tests = [ test-bsl-run test_demo_xml_save ] ; local loading-tests = [ test-bsl-run test_demo_xml_load ] ; DEPENDS $(loading-tests:S=.run) : $(saving-tests:S=.run) ; -t
participants (6)
-
David Abrahams -
Douglas Gregor -
John Maddock -
Rene Rivera -
Robert Ramey -
troy d. straszheim