Beman Dawes wrote:
On Thu, May 21, 2009 at 12:45 PM, Robert Ramey <ramey@rrsd.com> wrote:
This discussion demonstrates what's wrong with the current testing of the trunk.
Here is the way it should be done.
For each library that is changed, the library version on the trunk should be tested against the release branch. In practice this would work something like the following:
On the tester's machine start out with a current image of the release branch. For each library A switch library A's directories to the trunk run tests Restore image to release
Yep, that would often be optimal. Without spending a lot of time thinking it through, that sounds roughly like the test-on-demand scenario we've often talked about, except with test-on-demand it might be any branch, not just trunk, that gets switched to.
Suppose that it was a large loop: for each library switch directories from release to trunk run bjam from within that directory since bjam tracks dependencies if there is no change no tests are run if there are some changes only the dependent tests are run switch library directories back to release on to the next library Testers would run the whole loop occasionally. My guess is that the total amount of time required would be similar to running the current test setup or maybe it would be a lot less.
One issue, however, is when a library that other libraries depend on makes a breaking change.
This should be infrequent and would be a special case.
In that case a set of libraries have to be switched together for testing, and then merged into release at the same time.
I don't think so. Note that in the above regimen, such a breaking change (intentional or not) would appear as a whole slew of failures in dependent libraries as soon as the breaking change is merged into the release. At that point they dependent libraries would have to be fixed. But this doesn't affect the testing in any way. It just slogs on with more failures until they are fixed and the the fixed libraries are merged into the release. Robert Ramey
--Beman _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost