[1.35.0] Release candidate quality assurance?
1.35.0 Release Candidate 1 is about to become available. How can we tell if a release candidate is good enough to ship? While the final decision will come down to a value judgment on the part of the release manager, after consulting with the list, it would be useful to know that obvious showstoppers like missing files aren't an issue. It seems to me that this need breaks down into two categories: * Checks that could be automated. Perhaps someone could write a script that downloads each of the packages (.bz2, .gz, .7z, .zip), unpacks them, and does some basic QA. For example, runs our inspect tool, and makes sure that the number of problems reported isn't greater than expected. Verifies certain files are present. You get the idea. * Checks that require human judgment. Perhaps we could have a check list. It would have to be limited to things that could be checked in a reasonable period of time. We can't expect volunteers to labor for many hours over each release candidate. Thoughts? --Beman
Beman Dawes wrote:
1.35.0 Release Candidate 1 is about to become available.
It would help me in testing the web site integration if those archives followed the names of previous releases (both externally and internally). Hence "boost_1_35_0.[zip|tar.gz|tar.bz2|7z" with the top level dir of "boost_1_35_0". As otherwise I have to manually twiddle the ZIP for use in the web site.
How can we tell if a release candidate is good enough to ship? While the final decision will come down to a value judgment on the part of the release manager, after consulting with the list, it would be useful to know that obvious showstoppers like missing files aren't an issue.
It seems to me that this need breaks down into two categories:
* Checks that could be automated. Perhaps someone could write a script that downloads each of the packages (.bz2, .gz, .7z, .zip), unpacks them, and does some basic QA. For example, runs our inspect tool, and makes sure that the number of problems reported isn't greater than expected. Verifies certain files are present. You get the idea.
* Checks that require human judgment. Perhaps we could have a check list. It would have to be limited to things that could be checked in a reasonable period of time. We can't expect volunteers to labor for many hours over each release candidate.
Not sure which of the those two you would put this into... But we need to run the build/install for the variety of compilers and systems. This is something that could be automated, and I had it automated at one point using Buildbot. But that's something for next time ;-) -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org (msn) - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim,yahoo,skype,efnet,gmail
participants (2)
-
Beman Dawes -
Rene Rivera