Boost regression notification (2005-12-05 [RC_1_33_0])
Boost regression test failures ------------------------------ Report time: 2005-12-05T13:53:08Z This report lists all regression test failures on release platforms. Detailed report: http://engineering.meta-comm.com/boost-regression/CVS-RC_1_33_0/developer/is... 20 failures in 1 library: graph (20) |graph| layout_test: cw-8_3 cw-9_4 gcc-2.95.3-linux gcc-2.95.3-stlport-4.6.2-linux gcc-3.2.3-linux gcc-3.3.6-linux gcc-3.3.6-linux gcc-3.4.4-linux gcc-3.4.4-linux gcc-3_3-darwin gcc-4.0.1-linux gcc-4.0.2-linux gcc-4_0-darwin intel-8.1-linux intel-9.0-linux intel-win32-8_1 mingw-3_4_2 vc-7_0 vc-7_1 vc-8_0
On 12/5/05 12:00 PM, "Douglas Gregor" <dgregor@cs.indiana.edu> wrote:
Boost regression test failures ------------------------------ Report time: 2005-12-05T13:53:08Z
This report lists all regression test failures on release platforms.
Detailed report: http://engineering.meta-comm.com/boost-regression/CVS-RC_1_33_0/developer/is... es.html
20 failures in 1 library: graph (20)
|graph| layout_test: cw-8_3 cw-9_4 gcc-2.95.3-linux gcc-2.95.3-stlport-4.6.2-linux gcc-3.2.3-linux gcc-3.3.6-linux gcc-3.3.6-linux gcc-3.4.4-linux gcc-3.4.4-linux gcc-3_3-darwin gcc-4.0.1-linux gcc-4.0.2-linux gcc-4_0-darwin intel-8.1-linux intel-9.0-linux intel-win32-8_1 mingw-3_4_2 vc-7_0 vc-7_1 vc-8_0
This could be another case to watch out for: when a library fails on every compiler (or a least a lot of them). Here it would be the library that's possibly broken. -- Daryle Walker Mac, Internet, and Video Game Junkie darylew AT hotmail DOT com
Daryle Walker wrote:
|graph| layout_test: cw-8_3 cw-9_4 gcc-2.95.3-linux gcc-2.95.3-stlport-4.6.2-linux gcc-3.2.3-linux gcc-3.3.6-linux gcc-3.3.6-linux gcc-3.4.4-linux gcc-3.4.4-linux gcc-3_3-darwin gcc-4.0.1-linux gcc-4.0.2-linux gcc-4_0-darwin intel-8.1-linux intel-9.0-linux intel-win32-8_1 mingw-3_4_2 vc-7_0 vc-7_1 vc-8_0
This could be another case to watch out for: when a library fails on every compiler (or a least a lot of them). Here it would be the library that's possibly broken.
Isn't that exactly what these tests are supposed to measure ? Or do you mean that in case *all* tests fail the report could just mention a single failure, on a different granularity scale ? Regards, Stefan
On 12/6/05 6:00 PM, "Stefan Seefeld" <seefeld@sympatico.ca> wrote:
Daryle Walker wrote:
[SNIP a library that failed on every compiler]
This could be another case to watch out for: when a library fails on every compiler (or a least a lot of them). Here it would be the library that's possibly broken.
Isn't that exactly what these tests are supposed to measure ? Or do you mean that in case *all* tests fail the report could just mention a single failure, on a different granularity scale ?
Yes, I meant the latter. If we suddenly see a compiler fail on every library, or a library fail on every compiler, then we probably have a configuration issue. (But how do we implement "suddenly," since the tests don't keep a history of past results? We only want to block blooper runs, not compilers or libraries that legitimately fail everything.) -- Daryle Walker Mac, Internet, and Video Game Junkie darylew AT hotmail DOT com
Daryle Walker wrote:
On 12/6/05 6:00 PM, "Stefan Seefeld" <seefeld@sympatico.ca> wrote:
Daryle Walker wrote:
[SNIP a library that failed on every compiler]
This could be another case to watch out for: when a library fails on every compiler (or a least a lot of them). Here it would be the library that's possibly broken.
Isn't that exactly what these tests are supposed to measure ? Or do you mean that in case *all* tests fail the report could just mention a single failure, on a different granularity scale ?
Yes, I meant the latter. If we suddenly see a compiler fail on every library, or a library fail on every compiler, then we probably have a configuration issue. (But how do we implement "suddenly," since the tests don't keep a history of past results? We only want to block blooper runs, not compilers or libraries that legitimately fail everything.)
Indeed. One thing I already proposed in the past: some dummy tests (akin to autoconf macros) that make sure a sane configuration / environment is available at all. These tests could cover a specific toolchain, or even some lightweight library-specific code. If these are known as dependencies for the rest of the test suite(s), quite a bit of computing power (and mail bandwidth) could be spared whenever these fail. Regards, Stefan
Indeed. One thing I already proposed in the past: some dummy tests (akin to autoconf macros) that make sure a sane configuration / environment is available at all. These tests could cover a specific toolchain, or even some lightweight library-specific code. If these are known as dependencies for the rest of the test suite(s), quite a bit of computing power (and mail bandwidth) could be spared whenever these fail.
Good thinking, all we need is a "hello world" program and make all the tests dependent upon a successful build of that specific program. Adding a hello world program to say Boost.Config is pretty trivial, would someone like to volenteer to hack Boost.Build? John.
On 12/7/05 2:18 PM, "Stefan Seefeld" <seefeld@sympatico.ca> wrote:
Daryle Walker wrote:
On 12/6/05 6:00 PM, "Stefan Seefeld" <seefeld@sympatico.ca> wrote:
Daryle Walker wrote: [SNIP a library that failed on every compiler]
This could be another case to watch out for: when a library fails on every compiler (or a least a lot of them). Here it would be the library that's possibly broken.
Isn't that exactly what these tests are supposed to measure ? Or do you mean that in case *all* tests fail the report could just mention a single failure, on a different granularity scale ?
Yes, I meant the latter. If we suddenly see a compiler fail on every library, or a library fail on every compiler, then we probably have a configuration issue. (But how do we implement "suddenly," since the tests don't keep a history of past results? We only want to block blooper runs, not compilers or libraries that legitimately fail everything.)
Indeed. One thing I already proposed in the past: some dummy tests (akin to autoconf macros) that make sure a sane configuration / environment is available at all. These tests could cover a specific toolchain, or even some lightweight library-specific code. If these are known as dependencies for the rest of the test suite(s), quite a bit of computing power (and mail bandwidth) could be spared whenever these fail.
Maybe we can go by percentages too. If a test fail a super-majority of the compilers, then we could report that test has a configuration problem. We could do a similar test for a library if a super-majority of its tests fail. A compiler would get reported if it appears in a super-majority of the tests as a failure. I don't know which percentage to use (66%, 75%, 90%, etc.). -- Daryle Walker Mac, Internet, and Video Game Junkie darylew AT hotmail DOT com
participants (4)
-
Daryle Walker -
Douglas Gregor -
John Maddock -
Stefan Seefeld