First... Thanks to all those who made it to the testing session at BoostCon, especially Victor for the beer :-) Even though I was dead tired by the end of it, it was a lot of fun. The rest... I'll be posting the slides and notes from the session at some near future (when it's clear where they get posted). Don't worry it's nothing exciting ;-) The immediate result from the session is that the current system is very brittle. In particular Thomas identified getting reliable testing results as the most imminent failure point. Unfortunately we didn't get to discussing longer, and hence conclusions, as to the longer term issues. The plan for the near future is to try out the Dart testing results system <http://www.na-mic.org/Wiki/index.php/Dart2Summary>. We are attacking that task from three sides: * Doug & Troy, are going to do a test case of using CMake+CTest which support posting results to Dart. * Noel and myself, are going to make changes to bjam and Boost.Build to support posting results to Dart directly. * I volunteered to make changes to the regression Python scripts to allow posting current results to Dart in addition to posting them to Meta-Comm. I finished doing those changes last night, and the new scripts are in CVS-HEAD. Now, by default, testers doing HEAD regression runs will also post to the Dart server at <http://beta.boost.org:8081/>. Take this also as a notice to testers...
If you are testing CVS-HEAD, please update your regression.py and/or collect_and_upload_logs.py scripts.
* Everyone else is hereby drafted to look at the Dart results, when available, and comment on the system. There was also discussion about using BuildBot for the front half of the testing problem, controlling testing resources. Sorry if I can't remember the persons name, but there was one person and myself willing to look at setting up and developing a test use case for BuildBot. The future... Since we didn't get to discuss much past the above topics, I'd like it if we continue the discussion in the testing list. Topics to still discuss: * Other possible immediate solutions than the ones above. In particular Stefans mentioned QMTest before the conference, so it would be nice to get someone to work on a test use case for that. * The longer term question of how to get more testing resources and how to manage them is still up in the air. Finally, I'll set up a section in the new wiki to keep track of testing information as needed. PS. I've avoided bringing up the subject of CMake vs. Boost.Build, mostly because it's a subject with personal ramifications, and partly because it's mostly a topic a separate topic. Posts about that topic are forthcoming. -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo