Release procedure. Where do we go?
Hopefully I am not seen as trying to add fuel to the fire, but I am wondering if anyone tried to perform an analysis of what exactly did not work with the (previous) release procedure. I have seen David Abrahams trying to analyze the current tools, but I have not seen an analysis of the procedure and why it fails. (But I might have missed it in the long thread.) So I dare to ask the question again: What is wrong on 1) working on trunk, the bleeding edge 2) branching out for release when the trunk is release-stable 2.1) feature freeze and stabilizing, merging relevant parts to trunk 3) releasing from release candidate branch, i.e. tagging the branch 3.1) bugfixing on branch, merging to trunk, tagging the branch Is it this procedure that is broken, or is it the tools that are insufficient to support this procedure? In this scenario there is also always a release-ready branch insofar as a branch, once released still is release-ready (e.g. for fixing). And there is nothing that can potentially destabilize a branch that has already been forked out. There is also no requirement that a release branch ever dies. Bug fixes still can go there after it has been tagged. I also see nothing wrong with multiple release candidate branches at the same time. A release branch simply is another name for a certain set of features that are covered. No resources to test everything? Only a reported failure indicates something being wrong, that needs to be fixed. So I think it only fair to ask the reporter to help with testing by providing access to her environment and spend the necessary processor cycles to fix the problem. (Of course we would need the tools that would allow us this kind of on demand remote testing.) (This kind of support is also where developers could earn money if they like.) A potential user just browses through the testing pages and selects the most up to date release that covers the required features. (With the option for bug-fixing if something does not work for his environment.) Of course in this scenario the release manager would have less work to do :-D . Just watching out when a agreed feature set is in the trunk, and branch when there. (Of course developers always try to hold trunk as stable as possible ;-) ) Another potential benefit of this model is that developers already are used to it. So my bottom point is to generalize the current model to what I have said above and concentrate to get better tools that support this. AFAIK this is also the model other big monolithic projects are using. Departing from it would require modularization of the boost library, which at the current state does not look like an easy task to me. A small example: I was trying to single out the asio lib, and discovered that it had quite some implicit (undocumented) dependencies on other header-only/prebuilt libs. Only when I tried to use auto_link on an msvc platform with BOOST_LIB_DIAGNOSTIC turned on I eventually figured out that I also had to provide boost_system boost_date_time and boost_regex to the linker while compiling with gcc. So the recipe I once heard on this list: just take the relevant files and copy to your directory if you don't need all of boost, is hoping at best. Roland aka speedsnail
on Mon Sep 03 2007, Roland Schwarz <roland.schwarz-AT-chello.at> wrote:
Hopefully I am not seen as trying to add fuel to the fire, but I am wondering if anyone tried to perform an analysis of what exactly did not work with the (previous) release procedure.
I have seen David Abrahams trying to analyze the current tools, but I have not seen an analysis of the procedure and why it fails. (But I might have missed it in the long thread.)
So I dare to ask the question again: What is wrong on
1) working on trunk, the bleeding edge
2) branching out for release when the trunk is release-stable
Right there you've got a problem. The problem is that the trunk will never become release-stable on its own if it is allowed to become "the bleeding edge." So the release manager ends up freezing all checkins on the trunk while we try to stabilize it, which takes forever. In the meantime, "nobody" else gets any work done, because "everybody" thinks of the trunk as their own sandbox in which to do active development. Eventually some people discover they can work on a branch while the trunk is frozen, but some people are still scared of branching.
2.1) feature freeze and stabilizing, merging relevant parts to trunk
3) releasing from release candidate branch, i.e. tagging the branch 3.1) bugfixing on branch, merging to trunk, tagging the branch
Is it this procedure that is broken, or is it the tools that are insufficient to support this procedure?
The first problem, as I see it, is that the trunk is allowed to become broken, and there isn't a great deal of immediate pressure on the person who caused the breakage to fix it. If the person doesn't have access to the platform that broke, typically he just throws up his hands and says "somebody who knows this platform will have to help me," when instead he ought to back his changes out immediately. We need our tools to be able to test branches so that won't happen. The second problem is that the list of tested release compilers has shifted around unpredictably, so people often can't find out when they've broken a platform until long after it's too late for an immediate reversion of their changes to the trunk. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com The Astoria Seminar ==> http://www.astoriaseminar.com
David Abrahams wrote:
on Mon Sep 03 2007, Roland Schwarz wrote:
2) branching out for release when the trunk is release-stable
Right there you've got a problem. The problem is that the trunk will never become release-stable on its own if it is allowed to become "the bleeding edge." So the release manager ends up freezing all checkins on the trunk while we try to stabilize it, which takes forever.
I admit that this is indeed bad wording, and not really what I had in mind. What I really meant was: branch if all features planned for the next release are in trunk. Then stabilize the branch, but do not wait with release until all bugs on all platforms are fixed, but allow for a gradual healing of the release branch until it comes to a halt because either no more bugs, or no interest from users. (Or no commits from commercial users that sell bug-fixing/platform-adaption to their customers.)
The first problem, as I see it, is that the trunk is allowed to become broken, and there isn't a great deal of immediate pressure on the person who caused the breakage to fix it.
This is question of socialization of the community. If it is common place to drive too fast, only police can help (and even that turns out not to work well :-( )
If the person doesn't have access to the platform that broke, typically he just throws up his hands and says "somebody who knows this platform will have to help me," when instead he ought to back his changes out immediately. We need our tools to be able to test branches so that won't happen.
Better tools will help here, no doubt.
The second problem is that the list of tested release compilers has shifted around unpredictably, so people often can't find out when they've broken a platform until long after it's too late for an immediate reversion of their changes to the trunk.
Hmm, I guess I was one of those who made it appear as if the list was shifted around. In reality the list was rather fixed, but there simply nobody was testing a couple of toolsets. Starting these tests after a a while made a couple of regressions visible. However this is not a problem with my proposed refinement of the procedure: Allow for gradual healing, i.e. release while stabilizing. (Btw. Only lawyers might think that there is such a thing as bug free software ) But I have to admit that I do not have terribly much experience with management of such a huge code base as boost is. So perhaps arguing with you (the top level committing person) isn't bringing us closer to a solution. May I instead ask some more pragmatic questions: 1) What should developers do until the new procedures are established? Should they try to stabilize branch ? Should they merge new code from trunk to RC_1_34_0 ? 2) Bug Fixes for 1.34.1 Where should they go? RC_1_34_0 ? Btw.: You didn't respond to my comment about modularization (ala. CPAN) vs. monolithic (ala. Linux Kernel). So I guess you rather prefer the current monolithic approach (for good reasons I believe). Roland aka speedsnail
on Mon Sep 03 2007, Roland Schwarz <roland.schwarz-AT-chello.at> wrote:
David Abrahams wrote:
on Mon Sep 03 2007, Roland Schwarz wrote:
2) branching out for release when the trunk is release-stable
Right there you've got a problem. The problem is that the trunk will never become release-stable on its own if it is allowed to become "the bleeding edge." So the release manager ends up freezing all checkins on the trunk while we try to stabilize it, which takes forever.
I admit that this is indeed bad wording, and not really what I had in mind. What I really meant was: branch if all features planned for the next release are in trunk. Then stabilize the branch,
This puts the release schedule completely at the mercy of developers who have checked in unstable or "broken" code. The release branch should not start from a bunch of desired features without regard to the code's stability. It should start with the latest stable code with little regard to what features it contains.
but do not wait with release until all bugs on all platforms are fixed, but allow for a gradual healing of the release branch until it comes to a halt because either no more bugs, or no interest from users. (Or no commits from commercial users that sell bug-fixing/platform-adaption to their customers.)
Sounds like a recipe for never releasing anything to me.
The first problem, as I see it, is that the trunk is allowed to become broken, and there isn't a great deal of immediate pressure on the person who caused the breakage to fix it.
This is question of socialization of the community. If it is common place to drive too fast, only police can help (and even that turns out not to work well :-( )
I do not believe that, given the necessary feedback mechanisms which are currently not in place, Boost lacks the discipline to maintain a clean trunk.
If the person doesn't have access to the platform that broke, typically he just throws up his hands and says "somebody who knows this platform will have to help me," when instead he ought to back his changes out immediately. We need our tools to be able to test branches so that won't happen.
Better tools will help here, no doubt.
The second problem is that the list of tested release compilers has shifted around unpredictably, so people often can't find out when they've broken a platform until long after it's too late for an immediate reversion of their changes to the trunk.
Hmm, I guess I was one of those who made it appear as if the list was shifted around. In reality the list was rather fixed, but there simply nobody was testing a couple of toolsets.
That's why I said the list of *tested* release compilers.
May I instead ask some more pragmatic questions:
1) What should developers do until the new procedures are established? Should they try to stabilize branch ? Should they merge new code from trunk to RC_1_34_0 ?
2) Bug Fixes for 1.34.1 Where should they go? RC_1_34_0 ?
Btw.: You didn't respond to my comment about modularization (ala. CPAN) vs. monolithic (ala. Linux Kernel).
I haven't noticed such a comment, even upon rereading your message. But my initial reply was only responding to the top half of your message. I hadn't read the whole thing at that point.
So I guess you rather prefer the current monolithic approach (for good reasons I believe).
I wouldn't draw any conclusions if I were you. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com The Astoria Seminar ==> http://www.astoriaseminar.com
David Abrahams wrote:
on Mon Sep 03 2007, Roland Schwarz wrote:
Btw.: You didn't respond to my comment about modularization (ala. CPAN) vs. monolithic (ala. Linux Kernel).
I haven't noticed such a comment, even upon rereading your message. But my initial reply was only responding to the top half of your message. I hadn't read the whole thing at that point.
Re-citing the part I was refering to:
AFAIK this is also the model other big monolithic projects are using. Departing from it would require modularization of the boost library, which at the current state does not look like an easy task to me. A small example: I was trying to single out the asio lib, and discovered that it had quite some implicit (undocumented) dependencies on other header-only/prebuilt libs. Only when I tried to use auto_link on an msvc platform with BOOST_LIB_DIAGNOSTIC turned on I eventually figured out that I also had to provide boost_system boost_date_time and boost_regex to the linker while compiling with gcc. So the recipe I once heard on this list: just take the relevant files and copy to your directory if you don't need all of boost, is hoping at best.
Hmm, perhaps I shouldn't have posted at all? Who is in the position to give the answers to my pragmatic questions?
1) What should developers do until the new procedures are established? Should they try to stabilize branch ? Should they merge new code from trunk to RC_1_34_0 ?
2) Bug Fixes for 1.34.1 Where should they go? RC_1_34_0 ?
Roland aka speedsnail
on Tue Sep 04 2007, Roland Schwarz <roland.schwarz-AT-chello.at> wrote:
Who is in the position to give the answers to my pragmatic questions?
Well, I don't know. We need to come to an agreement.
1) What should developers do until the new procedures are established?
In my view, we urgently need to establish the new procedures.
Should they try to stabilize branch ?
I don't know what that means.
Should they merge new code from trunk to RC_1_34_0 ?
No, that is a release branch; it's where any further fixes to 1.34 should go, pending a 1.34.2 release.
2) Bug Fixes for 1.34.1 Where should they go? RC_1_34_0 ?
That sounds right to me. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com The Astoria Seminar ==> http://www.astoriaseminar.com
Roland Schwarz wrote:
A small example: I was trying to single out the asio lib, and discovered that it had quite some implicit (undocumented) dependencies on other header-only/prebuilt libs. Only when I tried to use auto_link on an msvc platform with BOOST_LIB_DIAGNOSTIC turned on I eventually figured out that I also had to provide boost_system boost_date_time and boost_regex to the linker while compiling with gcc.
Erm, actually these dependencies are documented in the "Using Boost.Asio" section in the docs: <http://asio.sourceforge.net/boost_asio_0_3_8/libs/asio/doc/html/boost_asio/using.html> Cheers, Chris
Roland Schwarz <roland.schwarz <at> chello.at> writes: [...]
I admit that this is indeed bad wording, and not really what I had in mind. What I really meant was: branch if all features planned for the next release are in trunk. Then stabilize the branch, but do not wait with release until all bugs on all platforms are fixed, but allow for a gradual healing of the release branch until it comes to a halt because either no more bugs, or no interest from users. (Or no commits from commercial users that sell bug-fixing/platform-adaption to their customers.)
Considering that until now "all bugs fixed" meant really "for each regression test failure either fix it or mark it expected" and bug report statistics didn't enter the picture, I see no reason to slacken the current criteria. Cheers, Nicola Musatti
Hi! Roland Schwarz schrieb:
AFAIK this is also the model other big monolithic projects are using. Departing from it would require modularization of the boost library, which at the current state does not look like an easy task to me.
I'd prefer to have a branch for each "library" of boost. This would be a complete (constant-time) copy of trunk. Using snvmerge we could merge changes back and forth. Merges into trunk are done if features in a library become stable. The trunk should pass all tests and is the source for the release branch. Library authors could merge trunk changes in other libraries (maybe dependant ones) into their own branch if needed. After a library feature was completely developed the library could start with a new branch straight from trunk if needed or could just continue on the "old" branch (because all trunk changes have been merge in already). Bugfixes to releases would be done in a library or on the trunk and then merged into the release branch by the release manager. Doing it in the trunk or library branch provides testing the change. It could then be tested with a working copy of the release branch which has the trunk change merged in. If it still passes tests it can be committed to the release branch. Comments? Frank
participants (5)
-
Christopher Kohlhoff -
David Abrahams -
Frank Birbacher -
Nicola Musatti -
Roland Schwarz