Peter Dimov wrote:
Beman Dawes:
In doing a postmortem of the past couple of releases with Thomas Witt, he made a very strong case that testing is a major bottleneck. If developers and release managers have to wait several days to find out if a fix works, it slows progress to a crawl.
One of the things we can do to eliminate that bottleneck is to cut the number of release criteria compilers down to a more manageable number, and to compilers where testing is very reliably and runs several times a day.
My candidates for the release criteria compilers are:
* Microsoft VC++ 8.0 on Win32 * Intel 10.0 on Win32 * GCC on Linux * GCC on Darwin
If these compilers provide reliable testing, running several times a day, they will deliver their results regardless of whether they are the only ones in the list.
Trimming the list will not increase their reliability, it will just decrease the reliability of the compilers that are left off the list.
In short, I believe that this is a step backwards. It does enable us to get a release out that doesn't work on an important compiler that is not on the list. Is this a feature?
Depends. If we could get a release out on a certain date, regardless of how many compilers are considered in the release criteria, then it would certainly be better to include more compilers. But that isn't the case. Every added compiler delays the release. So the question becomes what compilers can we include and still grind out a release by, say, mid-November? I'm not willing to wait a year for the next release of Boost. So I'm pushing hard to limit our goals to what we can do, rather than what we would like to do. If anyone feels strongly that compiler X on platform Y should be included, they should consider becoming a tester for compiler X on platform Y. And for reliability, we really need more than one tester for compiler X on platform Y, unless past history shows a particular tester is unusually reliable. --Beman