Boost regression notification (2006-11-05 [RC_1_34_0])
Boost Regression test failures Report time: 2006-11-05T11:02:13Z This report lists all regression test failures on release platforms. Detailed report: http://engineering.meta-comm.com/boost-regression/CVS-RC_1_34_0/developer/is... 37 failures in 11 libraries graph (1) iostreams (10) mpl (4) program_options (7) random (1) regex (1) serialization (7) spirit (1) statechart (2) utility (1) xpressive (2) |graph| adj_list_cc: qcc-3.3.5_gpp |iostreams| bzip2_test: msvc-7.1 msvc-8.0 example_test: vc-6_5-stlport file_descriptor_test: cw-9.4 finite_state_filter_test: cw-9.4 gzip_test: msvc-7.1 msvc-8.0 mapped_file_test: cw-9.4 zlib_test: msvc-7.1 msvc-8.0 |mpl| multiset: gcc-4.0.3_linux gcc-4.1.0_linux gcc-4.1.0_linux_x86_64 gcc-4.1.1_sunos_i86pc |program_options| cmdline_test_dll: cw-9.4 options_description_test_dll: cw-9.4 parsers_test_dll: cw-9.4 positional_options_test_dll: cw-9.4 unicode_test_dll: cw-9.4 variable_map_test_dll: cw-9.4 winmain_dll: cw-9.4 |random| random_test: intel-linux-9.0 |regex| concept_check: qcc-3.3.5_cpp |serialization| test_class_info_load_text_warchive: vc-6_5 test_reset_object_address: vc-7_0 test_reset_object_address_dll: vc-7_0 test_variant_xml_archive: borland-5_8_2 test_variant_xml_archive_dll: borland-5_8_2 test_variant_xml_warchive: borland-5_8_2 test_variant_xml_warchive_dll: borland-5_8_2 |spirit| scanner_value_type_tests: gcc-4.1.0_linux |statechart| TransitionTestBoth: qcc-3.3.5_gpp TransitionTestRelaxed: qcc-3.3.5_gpp |utility| operators_test: gcc-3.4.5_linux_x86_64 |xpressive| misc1: qcc-3.3.5_cpp qcc-3.3.5_gpp
Douglas Gregor <dgregor@cs.indiana.edu> writes:
This report lists all regression test failures on release platforms.
Detailed report: http://engineering.meta-comm.com/boost-regression/CVS-RC_1_34_0/developer/is...
37 failures in 11 libraries graph (1) iostreams (10) mpl (4) program_options (7) random (1) regex (1) serialization (7) spirit (1) statechart (2) utility (1) xpressive (2)
Do failures like this one not warrant a mention? (bind on vc 7.1) http://engineering.meta-comm.com/boost-regression/CVS-RC_1_34_0/developer/ou... Nothing has changed in CVS since I ran my tests yesterday. Anthony -- Anthony Williams Software Developer Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Anthony Williams wrote:
Do failures like this one not warrant a mention? (bind on vc 7.1)
http://engineering.meta-comm.com/boost-regression/CVS-RC_1_34_0/developer/ou...
This failure is caused by VC's linker folding duplicate identical functions into one (which is why the test only fails in release mode). I don't know what can be done about it. It's probably not a problem in practice.
"Peter Dimov" <pdimov@mmltd.net> writes:
Anthony Williams wrote:
Do failures like this one not warrant a mention? (bind on vc 7.1)
http://engineering.meta-comm.com/boost-regression/CVS-RC_1_34_0/developer/ou...
This failure is caused by VC's linker folding duplicate identical functions into one (which is why the test only fails in release mode). I don't know what can be done about it. It's probably not a problem in practice.
Maybe it should be marked up as "expected failure" or something, then. Currently the RC-1.34 summary at http://engineering.meta-comm.com/boost-regression/CVS-RC_1_34_0/developer/su... shows it as a red "Broken" box, yet it's not on the regression notification report. There's a few others like that, too. http://engineering.meta-comm.com/boost-regression/CVS-RC_1_34_0/developer/ou... http://engineering.meta-comm.com/boost-regression/CVS-RC_1_34_0/developer/ou... http://engineering.meta-comm.com/boost-regression/CVS-RC_1_34_0/developer/ou... http://engineering.meta-comm.com/boost-regression/CVS-RC_1_34_0/developer/ou... All these are from the tests I ran yesterday; I haven't cross-checked to see if there are similar cases from other people's test runs. The important thing from my point of view is that the regression report doesn't replicate the summary page. This means that people won't be investigating potentially broken things. Secondly, the results from my regression runs don't seem to tally with those from RudbekAssociates-V2, despite my having a reasonably standard installation of MSVC7.1 and MSVC8.0Express on Windows 2000. What's up with that. Anthony -- Anthony Williams Software Developer Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Anthony Williams wrote:
"Peter Dimov" <pdimov@mmltd.net> writes:
Anthony Williams wrote:
Do failures like this one not warrant a mention? (bind on vc 7.1)
http://engineering.meta-comm.com/boost-regression/CVS-RC_1_34_0/developer/ou...
This failure is caused by VC's linker folding duplicate identical functions into one (which is why the test only fails in release mode). I don't know what can be done about it. It's probably not a problem in practice.
Maybe it should be marked up as "expected failure" or something, then.
It's not an expected failure if the optimization is not enabled.
"Peter Dimov" <pdimov@mmltd.net> writes:
Anthony Williams wrote:
"Peter Dimov" <pdimov@mmltd.net> writes:
Anthony Williams wrote:
Do failures like this one not warrant a mention? (bind on vc 7.1)
http://engineering.meta-comm.com/boost-regression/CVS-RC_1_34_0/developer/ou...
This failure is caused by VC's linker folding duplicate identical functions into one (which is why the test only fails in release mode). I don't know what can be done about it. It's probably not a problem in practice.
Maybe it should be marked up as "expected failure" or something, then.
It's not an expected failure if the optimization is not enabled.
I wouldn't know that if you hadn't just told me, and the summary report shows it as "Broken". Several of the "unexpected passes" I've seen have comments along the lines of "this seems to vary depending on compiler configuration". Couldn't you add a similar comment for the bind failure? Anthony -- Anthony Williams Software Developer Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Anthony Williams wrote:
"Peter Dimov" <pdimov@mmltd.net> writes:
It's not an expected failure if the optimization is not enabled.
I wouldn't know that if you hadn't just told me, and the summary report shows it as "Broken".
That's because the summary report has no way of knowing that the last known good release has been using a different configuration, as it only looks at the toolset name - AFAIK. It could be made smarter... or you could pick a different toolset name such as vc71-release so it can be marked up appropriately. Or is it already possible to mark failures based on debug/release? I don't know. In general I don't like marking tests that work as expected failures because when they fail for real, nobody notices.
Peter Dimov said: (by the date of Mon, 6 Nov 2006 03:44:00 +0200)
Anthony Williams wrote:
"Peter Dimov" <pdimov@mmltd.net> writes:
It's not an expected failure if the optimization is not enabled.
I wouldn't know that if you hadn't just told me, and the summary report shows it as "Broken".
That's because the summary report has no way of knowing that the last known good release has been using a different configuration, as it only looks at the toolset name - AFAIK. It could be made smarter... or you could pick a different toolset name such as vc71-release so it can be marked up appropriately. Or is it already possible to mark failures based on debug/release? I don't know. In general I don't like marking tests that work as expected failures because when they fail for real, nobody notices.
Is is possible to mark expected failures as "with exactly this error message" ? If the message is different then it is no longer an expected failure but a real failure and people should notice. By that way regression tests will only allow this certain failure where the "linker is folding duplicate identical functions into one". -- Janek Kozicki |
participants (4)
-
Anthony Williams -
Douglas Gregor -
Janek Kozicki -
Peter Dimov