________________________________________ From: Boost [boost-bounces@lists.boost.org] on behalf of Tom Kent [lists@teeks99.com] Sent: 03 January 2015 16:36 To: Running Boost regression tests; Boost Developers List Subject: [boost] Linux regression test runners - What do devs want?
I've been pondering this recently, but a forced upgrade of my azure VMs last night is pressing the issue...and I wanted to solicit feedback before I go and set things back up...
....
Instead of running lots of different versions, I was thinking about running various compatibility options of the two main compilers...I've seen some other test runners with things like libc++, c++11, c++14, etc. My real question, would developers be better served by these options than different versions? Would they prefer both with longer revisit times between each test type? Other thoughts?
Tom
_______________________________________________ Tom First of all, thank you to you and everyone else who runs tests. As a maintainer (of Boost Phoenix) I need your tests and I am grateful for them. Let me answer you by explaining my recent experience. I have been attempting to sort out some rather elusive bugs which are in the Boost Phoenix code (sorry folks, not solved all of them yet). In the course of this several things have become clear from the testing. 1. One bug showed up in recent compilers (gcc 4.9 and later and clang 3.5 and later) but not in earlier compilers. This turned out to be because the more recent compilers have a different policy on function overloading resolution. 2. Some bugs showed up with compilation for C++11, but not for the same compiler with C++03. 3. Some bugs show up for older gcc e.g. 4.4 and 4.5, but not for later compilers. 4. Some bugs show up as running out of memory on some systems, so I have tried to reduce the header range. 5. Some bugs show up on one testers version of a compiler and not someone elses. A real puzzle those. Looking at what you do, you have been covering the range of compilers. Other people are doing the C++11 comparison. It is good if between you there are a range of these things covered. It is also good if tests do run every few days, so that confirmation can be gained of the effectiveness of a fix. This has been a problem over the New Year as I think some test stations are just not running for a while. It is good to be able to distinguish between problems which are mine to sort out, and those caused by a dependency gping wrong, as happened today to Boost Move. BTW, two of your runs on Ubuntu 14.04.arm seem to be failing on everything with this: ../libs/phoenix/test/include/core/actor.cpp:7:13: fatal error: error writing to /tmp/ccaRf02g.s: No space left on device int main() {} I hope this helps in your decision making. Please ask me if you want more information. Happy New Year John