(Cross-posting to boost-testing and boost-devel...) On 1/6/2010 7:32 AM, Robert Ramey wrote:
I realise that I've brought this up before, but it seems that there is an opportunity to get another hearing on it.
Thanks for (re-)raising this issue.
I would like to see trunk testing altered so that for each library it is tested against the release branch of all the other libraries rather than the trunk versions of all the other libraries as happens now.
Boost.Test is a perfect example. The author of boost test tests his stuff and checks it into the trunk. This is what he has to do in order to see his stuff tested on all the other platforms he doesn't have. From time to time this is going to reveal a bug that he didn't know about. So far so good. Except, since all the other libraries are using this component, they show up with bugs also. Now the author of Boost Test is on the hot seat and can't work on his own schedule. The whole boost release is now waiting on him. It's unfair to the author and breaks/suspends trunk testing for all the other libraries which might depend upon Boost Test.
Of course this can occur not only with boost test but any library which is depended upon by others.
Right. Boost.Test and Boost.Build are particularly critical.
Also, the fact that a library passes test against the trunk is no guarentee that it will pass when merged into the release. A library might be dependent upon a change which has yet to be merged into release branch.
I've been bitten by this before.
This situation wouldn't occur if each library in the trunk was tested against release.
I agree that the current situation, where we have several single points of failure -- a bug introduced by one component on trunk gumming up the whole works for everyone -- is unsatisfying. Your suggestion sounds reasonable, but I don't know enough about our test infrastructure to see the possible implications for our test runners and for our developers. One concern I have is that most developers will not be running their own tests in this configuration before committing to trunk. I suppose that's not so bad because at worst, their component ends up broken on trunk, but the rest of the world is unaffected because presumably they haven't merged their broken change to release yet (and everybody else is using the last-known-good version of that component from release). Can someone who knows more about our test infrastructure say a bit more about Robert's suggestion? As a half-measure, we could treat Boost.Build and Boost.Test as special and have the test runners use the versions on the release branch. -- Eric Niebler BoostPro Computing http://www.boostpro.com