John Maddock wrote:
* I think we could organize the testing more efficiently for faster turnaround and better integration testing, and much to my surprise I'm coming round to Robert Ramey's suggestion that we reorganize testing on a library-by-library basis, with each library tested against the current release/stable branch.
+1 of course,
* I think the release branch is closed for stabilization for too long, and that beta's are too short.
I would hope that implementing the above would make this a non-issue.
Here's a concrete suggestion for how the testing workflow might work:
* Test machine pulls changes for lib X from version control (whatever tool that is). * Iff there are changes (either to lib X or to release), only then run the tests for that library against current release branch. * The testers machine builds it's own test results pages - ideally these should go into some form of version control as well so we can roll back and see what broke when. * When a tester first starts testing they would add a short meta-description to a script, and run the script to generate the test results index pages. ie there would be no need for a separate machine collecting and processing the results. * The test script should run much of the above in parallel if requested.
The aim would be to speed processing of testing by reducing the cycle time (most libraries most of the time don't need re-testing).
+1 to all of the above. Robert Ramey