troy d. straszheim wrote:
Robert Ramey wrote:
This discussion demonstrates what's wrong with the current testing of the trunk.
Here is the way it should be done.
For each library that is changed, the library version on the trunk should be tested against the release branch. In practice this would work something like the following:
On the tester's machine start out with a current image of the release branch. For each library A switch library A's directories to the trunk run tests Restore image to release
Or (indulge me) more generally:
merge branch B with release branch run tests
I've avoided the word "merge" as (at least in SVN) it does updates the target which changes. switch (in SVN) is a merge in this sense. For purposes of this discussion, consider "switch" equivalent to changing a hard link to another directory.
i.e., "library A on the trunk" is just a specific case of a branch, and "switch library A's directories to the trunk" is just a specific (painfully easy) case of a merge.
Basically we agree - this is an implmentation detail at this point.
This would
a) attribute failures to their proper source b) prevent testing of each library (A) with provisional (trunk) code c) guarentee that when changes are merged into the release no new failures occur. d) reduce testing time since a small expermental change in one library in the trunk wouldn't provoke rebuild and test of all its dependents.
On my own machine, I have a release branch and I've switched the serialization library directories to the trunk. I can verify that all the above mentioned benefits are in fact realized. Boost testing would benefit greatly from adopting the same strategy.
This sounds generally right to me, but I think there are problems. I'm trying to be genericize it, and I'm not sure I've got it yet:
I've found it helpful to do it on my own machine to get used to it. It's very easy to get used to. There are a couple of things to be aware of - like accidently changing something outside one's own directory - but in general, it's very easy to get used to.
There is a 'testing' branch, to which libraries that are known to build with the release branch (with say a couple of compilers) get merged, and these libraries sit there until all-green: when that happens, they get merged to release, i.e. the testing branch essentially becomes the release. This way release *always* builds.
I'm not sure what this refers to. Perhaps CMake. What I currently do is a) test on my own machine according to the above b) upload to trunk and watch the tests. c) when I feel good about it, merge to release. b) would be more useful to me if testing on the trunk were as I describe above.
There are other cases where two libraries are mutually dependent; This should be avoided but will happen. In this case they must go simultaneously to the testing queue.
I'm not seeing that. If they are two libraries that are mutually dependent, that would be a bad thing and contradicts the whole idea of a library in the first place. If its a breaking change, that's a bad thing that will happen but doesn't break the testing regimen.
This implies that the testing branch is changing every so often.
I'm not sure what "testing branch" refers to
Diagram:
serialization mpi parallel_graph release <= testing <= upstream <= upstream
That is, a new release of serialization is in the testing queue getting hammered on by testing drones. Next up is a new version of MPI, which depends on serialization, and will have to build against whatever version of serialization goes green for release. When serialization reaches the testing queue a couple tweaks get made for borland, something majorish gets backed out. The MPI will have to update (merge) these new changes left-to-right through the queue and test privately to be sure he's ready for the testing queue. This merge operation has to be *easy* and quick.
All correct - and OK as far as I'm concerned.
And so on. parallel_graph, in this case, is watching the testing queue and doing periodic merges and builds to be sure he's ready when it is his turn.
I don't see that I have to watch the queue. The only thing I have to do is watch my tests for breaking changes in dependent libraries. Which I have to do now in any case.
So the "switch library A's directories" mechanism seems to break down a bit here; when serialization gets tweaked in the testing queue, MPI has to somehow "switch directories"... and so on backwards up the queue. I'm not seeing it quite yet.
Here's the scenario. a) I make a breaking change to serialization. b) after testing in the trunk - I merge into the release queue c) MPI tests start to break. d) I get an email from the MPI authors which starts Dear $#$#%$%$&$^*%^* e) Haggling involving me, the MPI authors and perhaps the release manager result in one of the following: i) backing out changes in the serializaiton library ii) MPI authors accomodating the change. Robert Ramey