Vladimir Prus wrote:
Doug Gregor wrote:
On Aug 2, 2007, at 12:51 PM, Vladimir Prus wrote:
P.S. Here are some of the many things I could have done that would have been more productive than writing the message above:
Oh, how I have avoided responding in these recent threads, exactly for that reason...
1) Made regression.py work with Subversion, so that we would be performing regression testing on the trunk.
I'll get to it <http://svn.boost.org/trac/boost/ticket/1122> this weekend, unless someone wants to grab that task from me now.
2) Looked at the changes made on the RC_1_34_0 branch to determine which ones can be merged back to the trunk.
Volodya and myself are going to do some of that for Boost.Build next week.
3) Fixed some of the current failures on the trunk. 4) Setup a new nightly regression tester. 5) Studied Dart2 to see how we can make it work for Boost
Noel and I are still trying to figure out how to make Boost.Build output the needed XML files directly. I think we need some help from Volodya at this point since the core support is done in bjam and we are both a bit lost in how to approach the problem in Boost.Build.
6) Investigated the problems with incremental testing.
That's something long overdue, and I can probably fix it.
I've been looking at this as a prelude to #5 above.
The question is -- will process_jam_logs remain?
No. My goal is to make Boost.Build itself generate the XML result fragment files directly. This should now be possible as bjam now supports capturing the output of all actions it runs and calling a rule with all the information about the action, like target, timing, output, etc. I've been hacking at testing.jam to see how to make it generate the XML files. But this is not something that's in my top 5 to do list :-\
If not, I'd rather not spend time doing chances that will have to be redone.
7) Improved the existing test reporting system to track Subversion revisions associated with test runs, and link to those revisions in the Trac. 8) Improved the existing test reporting system to track changes from day to day
Those are precisely what BuildBot was designed for. I've been talking with Stefan on what needs to get set up to start using it ASAP. This is another task I have for myself this weekend. But BuildBot it not a complete solution. It is not a reporting system, as I mentioned at BoostCon. But in combination with Dart2 we might be able to get close.
I'd add
8.1) Implemented a mechanism to record a revision in which a failure first occured, and a previous revision where test passed.
Which is easy if you record *all* past test results and the revision they tested. And Dart2 does some of this, although not as nicely as I'd like.
Before I reply to any messages in this thread, I'll be thinking about that list. Will you?
I think this is a good list -- it's likely to have more direct effect than any process discussion.
:-) -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo