[release management] Draft policy for changes between beta and release
In another post, John Maddock asked what sort of QA problems should be fixed between the beta and final releases, and what the criteria and procedures are. Here is a quick draft of a possible policy: * Showstoppers. Criteria: Bugs or other problems so serious that the release is seriously compromised if not fixed. Procedure: After discussion on the developers list, release managers will specify how the problem is to be attacked, and what effect it will have on schedule. * Documentation and other minor fixes not affecting code. Criteria: Changes not requiring regression testing and unlikely to impact other libraries. Procedure: Merges to branches/release are OK, after fix applied to trunk, and do not require a release manager's permission. * Minor code fixes. Criteria: Changes believed to have limited risk and that have already been made to trunk and are stable on trunk regression testing. Procedure: Merges to branches/release after fix stable on trunk tests do not require a release manager's permission. * Questionable code fixes. Criteria: Any code changes not meeting the "Minor code fix" criteria, such as a change with a lot of possible ripple effect on other libraries. Procedure: Ask release managers for permission. * Tools changes and fixes. Code changes to bjam, Boost.Build, Boost.Test, the doc build process, and other tools are discouraged late in release cycles in general and between beta and release in particular. Ask release managers for permission if you really think the change is warranted for this release. * New features, breaking changes, major reworks, new libraries. These are not suitable for merging between beta and release. Wait until the next release. Comments? --Beman
Beman Dawes escribió:
In another post, John Maddock asked what sort of QA problems should be fixed between the beta and final releases, and what the criteria and procedures are.
Here is a quick draft of a possible policy:
[...]
Comments?
I'd make asking for permission mandatory for all code fixes, including minor code fixes: after all, what's minor and what's not is a subjective assesment. Joaquín M López Muñoz Telefónica, Investigación y Desarrollo
I second this !!! Robert Ramey joaquin@tid.es wrote:
Beman Dawes escribió:
In another post, John Maddock asked what sort of QA problems should be fixed between the beta and final releases, and what the criteria and procedures are.
Here is a quick draft of a possible policy:
[...]
Comments?
I'd make asking for permission mandatory for all code fixes, including minor code fixes: after all, what's minor and what's not is a subjective assesment.
Joaquín M López Muñoz Telefónica, Investigación y Desarrollo
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On Thu, Oct 23, 2008 at 10:17 AM, <joaquin@tid.es> wrote:
Beman Dawes escribió:
In another post, John Maddock asked what sort of QA problems should be fixed between the beta and final releases, and what the criteria and procedures are.
Here is a quick draft of a possible policy:
[...]
Comments?
I'd make asking for permission mandatory for all code fixes, including minor code fixes: after all, what's minor and what's not is a subjective assesment.\
Changed, particularly since others agreed with you. Thanks, --Beman
Beman Dawes wrote:
In another post, John Maddock asked what sort of QA problems should be fixed between the beta and final releases, and what the criteria and procedures are.
Here is a quick draft of a possible policy:
* Showstoppers. Criteria: Bugs or other problems so serious that the release is seriously compromised if not fixed. Procedure: After discussion on the developers list, release managers will specify how the problem is to be attacked, and what effect it will have on schedule.
Nod.
* Documentation and other minor fixes not affecting code. Criteria: Changes not requiring regression testing and unlikely to impact other libraries. Procedure: Merges to branches/release are OK, after fix applied to trunk, and do not require a release manager's permission.
OK, but the author should verify that changes don't break the main doc build if that's effected.
* Minor code fixes. Criteria: Changes believed to have limited risk and that have already been made to trunk and are stable on trunk regression testing. Procedure: Merges to branches/release after fix stable on trunk tests do not require a release manager's permission.
Although it's more work, my gut feeling is that the release team should be involved in that.
* Questionable code fixes. Criteria: Any code changes not meeting the "Minor code fix" criteria, such as a change with a lot of possible ripple effect on other libraries. Procedure: Ask release managers for permission.
Probably not likely to be given if they're not showstoppers?
* Tools changes and fixes. Code changes to bjam, Boost.Build, Boost.Test, the doc build process, and other tools are discouraged late in release cycles in general and between beta and release in particular. Ask release managers for permission if you really think the change is warranted for this release.
Nod.
* New features, breaking changes, major reworks, new libraries. These are not suitable for merging between beta and release. Wait until the next release.
Nod. John.
Beman Dawes wrote:
what sort of QA problems should be fixed between the beta and final releases * Minor code fixes. * Questionable code fixes.
One important factor not mentioned in the message is whether the fix is for a regression relative to the last release or is fixing a long-standing problem. From the point of view of an end-user, the difference between the new version "still not working" and "being more broken than the last" is significant. I would suggest that even quite large fixes should be accepted if they are fixing regressions, while even minor fixes that do not fix regressions should only be accepted if very obviously harmless. Phil.
Phil Endecott wrote:
Beman Dawes wrote:
what sort of QA problems should be fixed between the beta and final releases * Minor code fixes. * Questionable code fixes.
One important factor not mentioned in the message is whether the fix is for a regression relative to the last release or is fixing a long-standing problem. From the point of view of an end-user, the difference between the new version "still not working" and "being more broken than the last" is significant. I would suggest that even quite large fixes should be accepted if they are fixing regressions, while even minor fixes that do not fix regressions should only be accepted if very obviously harmless.
Any library which has regressions as shown in trunk testing should not be merged into the release ready branch. If it is discovered that this has occurred, then the merge should be reverted. This policy is necessary for boost releases to guarentee a strictly increasing level of quality. Robert Ramey
On Sun, Oct 26, 2008 at 2:07 PM, Robert Ramey <ramey@rrsd.com> wrote:
Phil Endecott wrote:
Beman Dawes wrote:
what sort of QA problems should be fixed between the beta and final releases * Minor code fixes. * Questionable code fixes.
One important factor not mentioned in the message is whether the fix is for a regression relative to the last release or is fixing a long-standing problem. From the point of view of an end-user, the difference between the new version "still not working" and "being more broken than the last" is significant. I would suggest that even quite large fixes should be accepted if they are fixing regressions, while even minor fixes that do not fix regressions should only be accepted if very obviously harmless.
Any library which has regressions as shown in trunk testing should not be merged into the release ready branch. If it is discovered that this has occurred, then the merge should be reverted.
This policy is necessary for boost releases to guarentee a strictly increasing level of quality.
Added. Thanks, --Beman
On Sun, Oct 26, 2008 at 2:07 PM, Robert Ramey <ramey@rrsd.com> wrote:
Phil Endecott wrote:
Beman Dawes wrote:
what sort of QA problems should be fixed between the beta and final releases * Minor code fixes. * Questionable code fixes.
One important factor not mentioned in the message is whether the fix is for a regression relative to the last release or is fixing a long-standing problem. From the point of view of an end-user, the difference between the new version "still not working" and "being more broken than the last" is significant. I would suggest that even quite large fixes should be accepted if they are fixing regressions, while even minor fixes that do not fix regressions should only be accepted if very obviously harmless.
Any library which has regressions as shown in trunk testing should not be merged into the release ready branch. If it is discovered that this has occurred, then the merge should be reverted.
This policy is necessary for boost releases to guarentee a strictly increasing level of quality.
Added. Thanks, --Beman
participants (5)
-
Beman Dawes -
joaquin＠tid.es -
John Maddock -
Phil Endecott -
Robert Ramey