Folks, I would have like to merge Boost.Build state from trunk to release branch. This matter was discussed briefly in http://thread.gmane.org/gmane.comp.lib.boost.devel/180814/ but it didi not lead anywhere, and further it seems to be some confusion between merging Boost.Jam (which Rene wants) and merging Boost.Build. To reiterate, I think Boost.Build should be merged because: - it now supports linux->windows cross compile - it allows to eliminate configuration warning messages from libraries that are not built (which requires corresponding merges for libraries Jamfiles, but it's trivial) - it fixes a bug where include=/foo/bar could not be specified on the command line - it has some other smaller fixes - I don't see any breakage on trunk tests Comments? - Volodya -- Vladimir Prus http://vladimir_prus.blogspot.com Boost.Build V2: http://boost.org/boost-build2
Vladimir Prus wrote:
Folks, I would have like to merge Boost.Build state from trunk to release branch. This matter was discussed briefly in
http://thread.gmane.org/gmane.comp.lib.boost.devel/180814/
but it didi not lead anywhere, and further it seems to be some confusion between merging Boost.Jam (which Rene wants) and merging Boost.Build.
To reiterate, I think Boost.Build should be merged because:
- it now supports linux->windows cross compile - it allows to eliminate configuration warning messages from libraries that are not built (which requires corresponding merges for libraries Jamfiles, but it's trivial) - it fixes a bug where include=/foo/bar could not be specified on the command line - it has some other smaller fixes - I don't see any breakage on trunk tests
Comments?
The development process should work like this: you make a change to trunk, wait for tests to cycle, and if they look good, you merge the change to release. The process should *not* work like this: you make change after change to trunk, and wait until the very last minute to drop them all into the release branch at once. Some of these improvements are desirable, though. Can you send links to the changesets you'll be merging? And describe potential impacts? -- Eric Niebler BoostPro Computing http://www.boostpro.com
Eric Niebler wrote:
Vladimir Prus wrote:
To reiterate, I think Boost.Build should be merged because:
The development process should work like this: you make a change to trunk, wait for tests to cycle, and if they look good, you merge the change to release. The process should *not* work like this: you make change after change to trunk, and wait until the very last minute to drop them all into the release branch at once.
Perhaps there's been a slight misunderstanding. Keep in mind that build tools infrastructure is treated somewhat differently than library code: * The Boost.Build changes always get tested on trunk, and on the release branch. * AFAIK there have been numerous cycles of the Boost tests with the changes Volodya has in mind. * We have traditionally merged Boost.Build to the release at the very last minute to have the release contain the most accurate state of what we test. Although in the last release we froze the tools code shortly before the release to reduce the chances of problems. Hence: * There is no advantage to not merging Boost.Build to release. * It's actually a disservice to users and library authors since it would make for discrepancy between what is tested and what is released. * When the Boost.Build merge to release happens it also means that the Boost.Build code will also need to be frozen on branches/release and on trunk. At which point Volodya, Jurko, myself, and others get to either avoid doing Boost.Build stuff or work in other branches. Consequently I'm not only in favor of merging Boost.Build to release, but I'm insisting it's required. Note, this special arrangement may change in the near future. But for now we've tried to make the Boost.Build development as minimally disruptive and maximally advantageous as we can manage. -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org (msn) - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim,yahoo,skype,efnet,gmail
Rene Rivera wrote:
Eric Niebler wrote:
Vladimir Prus wrote:
To reiterate, I think Boost.Build should be merged because:
The development process should work like this: you make a change to trunk, wait for tests to cycle, and if they look good, you merge the change to release. The process should *not* work like this: you make change after change to trunk, and wait until the very last minute to drop them all into the release branch at once.
Perhaps there's been a slight misunderstanding. Keep in mind that build tools infrastructure is treated somewhat differently than library code:
* The Boost.Build changes always get tested on trunk, and on the release branch.
* AFAIK there have been numerous cycles of the Boost tests with the changes Volodya has in mind.
* We have traditionally merged Boost.Build to the release at the very last minute to have the release contain the most accurate state of what we test. Although in the last release we froze the tools code shortly before the release to reduce the chances of problems.
Hence:
* There is no advantage to not merging Boost.Build to release.
* It's actually a disservice to users and library authors since it would make for discrepancy between what is tested and what is released.
* When the Boost.Build merge to release happens it also means that the Boost.Build code will also need to be frozen on branches/release and on trunk. At which point Volodya, Jurko, myself, and others get to either avoid doing Boost.Build stuff or work in other branches.
Consequently I'm not only in favor of merging Boost.Build to release, but I'm insisting it's required.
Note, this special arrangement may change in the near future. But for now we've tried to make the Boost.Build development as minimally disruptive and maximally advantageous as we can manage.
Thanks for filling me in on the state of affairs. You're saying the release testers are using Boost.Build from trunk? I find that surprising. What prevents instability on trunk from hosing the release tests, which would be really unfortunate? Why is this better than the normal procedure of changes to BB on trunk getting tested there and merged piecemeal over time to release? Then the release branch can be tested with BB from release. I've missed some rationale here. Anyway, I'm all for syncing what we ship with 1.37 with what we've been testing on the release branch. If that means merge, then by all means, do it. But let's please rethink this procedure. The process for the tools should be the same for the libraries, IMO, with the exception that tools get frozen early. And the release branch should be tested with tools from the release branch. And changes are merged to release piecemeal and not all at once. Objections? -- Eric Niebler BoostPro Computing http://www.boostpro.com
On Oct 10, 2008, at 6:45 PM, Eric Niebler wrote:
Rene Rivera wrote:
Eric Niebler wrote:
Vladimir Prus wrote:
To reiterate, I think Boost.Build should be merged because:
The development process should work like this: you make a change to trunk, wait for tests to cycle, and if they look good, you merge the change to release. The process should *not* work like this: you make change after change to trunk, and wait until the very last minute to drop them all into the release branch at once.
Perhaps there's been a slight misunderstanding. Keep in mind that build tools infrastructure is treated somewhat differently than library code:
* The Boost.Build changes always get tested on trunk, and on the release branch.
* AFAIK there have been numerous cycles of the Boost tests with the changes Volodya has in mind.
* We have traditionally merged Boost.Build to the release at the very last minute to have the release contain the most accurate state of what we test. Although in the last release we froze the tools code shortly before the release to reduce the chances of problems.
Hence:
* There is no advantage to not merging Boost.Build to release.
* It's actually a disservice to users and library authors since it would make for discrepancy between what is tested and what is released.
* When the Boost.Build merge to release happens it also means that the Boost.Build code will also need to be frozen on branches/release and on trunk. At which point Volodya, Jurko, myself, and others get to either avoid doing Boost.Build stuff or work in other branches.
Consequently I'm not only in favor of merging Boost.Build to release, but I'm insisting it's required.
Note, this special arrangement may change in the near future. But for now we've tried to make the Boost.Build development as minimally disruptive and maximally advantageous as we can manage.
Thanks for filling me in on the state of affairs. You're saying the release testers are using Boost.Build from trunk? I find that surprising. What prevents instability on trunk from hosing the release tests, which would be really unfortunate? Why is this better than the normal procedure of changes to BB on trunk getting tested there and merged piecemeal over time to release? Then the release branch can be tested with BB from release. I've missed some rationale here.
Anyway, I'm all for syncing what we ship with 1.37 with what we've been testing on the release branch. If that means merge, then by all means, do it.
But let's please rethink this procedure. The process for the tools should be the same for the libraries, IMO, with the exception that tools get frozen early. And the release branch should be tested with tools from the release branch. And changes are merged to release piecemeal and not all at once. Objections?
Hi Eric, I'm personally in favor of the longer testing cycles that you propose. That said, there are changes in Boost Build and Bjam that I'd like to see in 1.37, so I'd support your suggestions starting with the 1.38 release. Given the track record of previous tools releases, I'm quite comfortable with the changes Rene and Volodya want to merge into the release. If it would help, I'd be willing to run additional tests on the release platforms if that will help ensure confidence in the tools changes. -- Noel
Eric Niebler wrote:
Rene Rivera wrote:
Eric Niebler wrote:
Vladimir Prus wrote:
To reiterate, I think Boost.Build should be merged because:
The development process should work like this: you make a change to trunk, wait for tests to cycle, and if they look good, you merge the change to release. The process should *not* work like this: you make change after change to trunk, and wait until the very last minute to drop them all into the release branch at once.
Perhaps there's been a slight misunderstanding. Keep in mind that build tools infrastructure is treated somewhat differently than library code:
* The Boost.Build changes always get tested on trunk, and on the release branch.
* AFAIK there have been numerous cycles of the Boost tests with the changes Volodya has in mind.
* We have traditionally merged Boost.Build to the release at the very last minute to have the release contain the most accurate state of what we test. Although in the last release we froze the tools code shortly before the release to reduce the chances of problems.
Hence:
* There is no advantage to not merging Boost.Build to release.
* It's actually a disservice to users and library authors since it would make for discrepancy between what is tested and what is released.
* When the Boost.Build merge to release happens it also means that the Boost.Build code will also need to be frozen on branches/release and on trunk. At which point Volodya, Jurko, myself, and others get to either avoid doing Boost.Build stuff or work in other branches.
Consequently I'm not only in favor of merging Boost.Build to release, but I'm insisting it's required.
Note, this special arrangement may change in the near future. But for now we've tried to make the Boost.Build development as minimally disruptive and maximally advantageous as we can manage.
Thanks for filling me in on the state of affairs. You're saying the release testers are using Boost.Build from trunk? I find that surprising. What prevents instability on trunk from hosing the release tests, which would be really unfortunate? Why is this better than the normal procedure of changes to BB on trunk getting tested there and merged piecemeal over time to release? Then the release branch can be tested with BB from release. I've missed some rationale here.
Anyway, I'm all for syncing what we ship with 1.37 with what we've been testing on the release branch. If that means merge, then by all means, do it.
But let's please rethink this procedure. The process for the tools should be the same for the libraries, IMO, with the exception that tools get frozen early. And the release branch should be tested with tools from the release branch. And changes are merged to release piecemeal and not all at once. Objections?
The development process for libraries, at least that on http://svn.boost.org/trac/boost/wiki/ImprovingPractices does not require that library changes be changed piecemeal, and as I say in other email, I think it would be a mistake to require piecemeal merge of changes. It only imposes additional overhead. There are cases where splitting a big change into individual chunks is desirable. E.g. this is what happens with GNU projects usually, where folks try to split large work into logical bits, exactly because somebody should sign-off on each patch. But preparing such a clean patch series takes a lot of effort, and also note that even with GNU projects, it's considerable acceptable to only test that middle patch compile, and run tests on full result. But in current case, that example is not even relevant, - we are not talking about any big feature checked in to trunk, but a number of individual unrelated changes that should be on release. Every project I know (including GNU project), creates fresh release branch out of current trunk state. Piecemeal merges are reserved for important changes on trunk *after* the release cut-off date. Speaking of early freezing, I think we should be pragmatic. Freezing tools are the start of release cycle means that if I make any good change at the beginning of release cycle, then it won't be merged until next release cycle (3 months) and won't be released until next release (3 months). So, there's 6 months delay, which seems like disservice to users. I'd agree that some advance merging to tools makes sense given their global impact, but I think that the additional freeze time should correspond with risks. E.g. take release N, and measure the time during which release results were unstable due to tool issues. Let that be U weeks. Require tool freeze for the next release to happen U+1 weeks before the libraries. - Volodya
The development process for libraries, at least that on
http://svn.boost.org/trac/boost/wiki/ImprovingPractices
does not require that library changes be changed piecemeal, and as I say in other email, I think it would be a mistake to require piecemeal merge of changes. It only imposes additional overhead.
I would agree with Validir on this. I see the process where one makes (maybe alot) of changes in the trunk and once in a while gets to a point where he thinks - this is a real improvement over the current release ready branch and then does the merge.
Speaking of early freezing, I think we should be pragmatic. ....
I would like to see all tests run with the boost build and boost test in the release ready branch. This would permit developers of these tools to feel comfortable about commiting changes which might break something without creating a lot of problems for those using these tools. This would permit boost build to progress at its own schedule and pace while maintaining an ever improving product for use by boost and others. This would eliminate the requirement for any kind of "freeze" The same goes for boost documentation which I would really like to use. But I don't feel confident that "it just works" Robert Ramey
Robert Ramey wrote:
The development process for libraries, at least that on
http://svn.boost.org/trac/boost/wiki/ImprovingPractices
does not require that library changes be changed piecemeal, and as I say in other email, I think it would be a mistake to require piecemeal merge of changes. It only imposes additional overhead.
I would agree with Validir on this. I see the process where one makes (maybe alot) of changes in the trunk and once in a while gets to a point where he thinks - this is a real improvement over the current release ready branch and then does the merge.
Yes, agreed. The granularity at which things get merged to release should be at the feature level and bug-fix level, not the per-commit level. Thanks for correcting me. It seemed to me that Volodya was merging many complete and separate features at once. I'm saying a more piecemeal approach would prevent this sort of hand-wringing about destabilization. But since the release branch had been tested with BB from trunk, my hand-wringing was unnecessary.
Speaking of early freezing, I think we should be pragmatic. ....
I would like to see all tests run with the boost build and boost test in the release ready branch. This would permit developers of these tools to feel comfortable about commiting changes which might break something without creating a lot of problems for those using these tools. This would permit boost build to progress at its own schedule and pace while maintaining an ever improving product for use by boost and others.
Agree.
This would eliminate the requirement for any kind of "freeze"
Let me understand ... it would eliminate the need for a tool freeze *on trunk*, not on release. A freeze on the release branch is essential and standard practice. And IMO the freeze on tools should happen earlier ... by some weeks as Volodya suggested ... than the freeze on libraries.
The same goes for boost documentation which I would really like to use. But I don't feel confident that "it just works"
Heh, there's nobody here who would disagree with that. -- Eric Niebler BoostPro Computing http://www.boostpro.com
Eric Niebler wrote:
This would eliminate the requirement for any kind of "freeze"
Let me understand ... it would eliminate the need for a tool freeze *on trunk*, not on release. A freeze on the release branch is essential and standard practice. And IMO the freeze on tools should happen earlier ... by some weeks as Volodya suggested ... than the freeze on libraries.
OK - I guess we're sort of splitting hairs here. as a practical matter, I would be reluctant to merge any changes into the release near to the release target date. It would be on a case by case basis. Actually, its interesting I find myself in exactly this situation right this minute. We have a bug in the serialization library. It only shows up in certain circumstances on certain versions of the gcc compiler and on none of the test platforms. (I don't think anyone will argue that the serialization library doesn't have enough tests). It's ever worse, if one has this bug, there is no way to work around it. Now, after months of trying to track this down, Troy d. straszheim has managed to make a repeatable test that fails in his environment and I think I've finally managed to fix it. It looks like the latest fix hasn't broken anything else and hopefully Troy will be able to verify that his test case now passes. This illustrates the following for the new procedure. a) In spite of one's best efforts, s*** happens. b) This situation isn't holding up the whole release for everyone else c) Even if we don't get the bug fix into this release, the serialization and boost 1.37 as a whole will be strictly better than the previous version. d) These improvements will be be available to users in a timely manner. So, I believe our experience with the current release process demontrates a huge improvement over the previous one. Robert Ramey
Robert Ramey wrote:
Speaking of early freezing, I think we should be pragmatic. ....
I would like to see all tests run with the boost build and boost test in the release ready branch. This would permit developers of these tools to feel comfortable about commiting changes which might break something without creating a lot of problems for those using these tools. This would permit boost build to progress at its own schedule and pace while maintaining an ever improving product for use by boost and others. This would eliminate the requirement for any kind of "freeze"
Let me try to have all "i" dotted and "t" crossed: 1. Build.Build has extensive testsuite. 2. It would be nice to have it run together with library tests, to improve Boost.Build quality on some platforms where it's not actively tested by users. It will help Boost.Build users, as well folks who are both C++ Boost users and Boost.Build users. 3. Any such improvement won't help C++ Boost itself much. The only thing C++ Boost cares is whether C++ Boost libraries and tests work. If a change breaking that on some platform is checked in, the current test framework will surely detect this. Adding Boost.Build tests to the mix will only mean the breakage will be detected somewhat earlier and be somewhat easier to figure. So, adding Boost.Build tests to the test results is not something C++ Boost users or developers should care very much about. - Volodya
Vladimir Prus wrote:
Robert Ramey wrote: Let me try to have all "i" dotted and "t" crossed:
1. Build.Build has extensive testsuite.
good.
2. It would be nice to have it run together with library tests, to improve Boost.Build quality on some platforms where it's not actively tested by users. It will help Boost.Build users, as well folks who are both C++ Boost users and Boost.Build users.
great, then let's do it.
3. Any such improvement won't help C++ Boost itself much. The only thing C++ Boost cares is whether C++ Boost libraries and tests work.
not untrue - but a more robust boost build well permit better testing especially on less frequent platforms.
If a change breaking that on some platform is checked in, the current test framework will surely detect this.
True -
Adding Boost.Build tests to the mix will only mean the breakage will be detected somewhat earlier and be somewhat easier to figure.
which would be quite helpful it seems to me.
So, adding Boost.Build tests to the test results is not something C++ Boost users or developers should care very much about.
lol - well, I care about it. and everything in boost currently depends upon it. So an improvement in robustness with this tool would be very helpful. If we already have an extensive test suite, why not run it? What's the downside here? Robert Ramey
Robert Ramey wrote:
Vladimir Prus wrote:
Robert Ramey wrote: Let me try to have all "i" dotted and "t" crossed:
1. Build.Build has extensive testsuite.
good.
2. It would be nice to have it run together with library tests, to improve Boost.Build quality on some platforms where it's not actively tested by users. It will help Boost.Build users, as well folks who are both C++ Boost users and Boost.Build users.
great, then let's do it.
We've been planning to "do it" for a long time now. But wanting and doing are usually far part ;-)
Adding Boost.Build tests to the mix will only mean the breakage will be detected somewhat earlier and be somewhat easier to figure.
which would be quite helpful it seems to me.
So, adding Boost.Build tests to the test results is not something C++ Boost users or developers should care very much about.
If we already have an extensive test suite, why not run it? What's the downside here?
The downside is that it increases the burden on the test system and testers. What I've wanted to do for more than a year now is to have a separate pool of testers, likely with some overlap with current testers, to do separate tool, documentation, and other testing. Until very recently I didn't have the resources to try and set up such a separate testing cycle as I was devoting all of one of my machines to trunk testing results. And now that Noel has taken over that burden I can set this up. But of course, doing is still not close to wanting, since this is not a high-priority it won't get done in the near future. -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org (msn) - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim,yahoo,skype,efnet,gmail
Rene Rivera wrote:
If we already have an extensive test suite, why not run it? What's the downside here?
The downside is that it increases the burden on the test system and testers.
Why is that - doe it take a huge amount of time to run? or what?
What I've wanted to do for more than a year now is to have a separate pool of testers,
why can't we have it as "one more library" in the same testing setup we have now? To me the the only difference would be that rather in directory ../libs/serializaton it would be in directory ../tools/bjam or? What am I missing here?
likely with some overlap with current testers, to do separate tool, documentation, and other testing.
My view would be to make the directory structure of tool directories similar to those of libraries and just add the tool test suite to that of the libraries. That is, I think the distinction of "tools" and "libraries" should be very much smaller than it is now and that this would simplify a lot of questions: testing, where to put documentation, where to put Jamfiles and how to run them, etc. The library testing model works - lets build on that!
Until very recently I didn't have the resources to try and set up such a separate testing cycle as I was devoting all of one of my machines to trunk testing results.
I would hope that we don't need a separate testing cycle. Why would we need this? We have two now trunk and release which seems fine to me. Robert Ramey
Eric Niebler wrote:
Vladimir Prus wrote:
Folks, I would have like to merge Boost.Build state from trunk to release branch. This matter was discussed briefly in
http://thread.gmane.org/gmane.comp.lib.boost.devel/180814/
but it didi not lead anywhere, and further it seems to be some confusion between merging Boost.Jam (which Rene wants) and merging Boost.Build.
To reiterate, I think Boost.Build should be merged because:
- it now supports linux->windows cross compile - it allows to eliminate configuration warning messages from libraries that are not built (which requires corresponding merges for libraries Jamfiles, but it's trivial) - it fixes a bug where include=/foo/bar could not be specified on the command line - it has some other smaller fixes - I don't see any breakage on trunk tests
Comments?
The development process should work like this: you make a change to trunk, wait for tests to cycle, and if they look good, you merge the change to release. The process should *not* work like this: you make change after change to trunk, and wait until the very last minute to drop them all into the release branch at once.
I disagree. By requiring a test run after each change you impose very high additional overhead and slow down development considerably. No other project has such requirements. If trunk has 30 changes, and the end result has good test results, it is a waste of time to run tests on 29 intermediate states that are not meant to be used.
Some of these improvements are desirable, though. Can you send links to the changesets you'll be merging? And describe potential impacts?
I'll be merging everything, as discussed later in the thread. - Volodya
Vladimir Prus wrote:
Folks, I would have like to merge Boost.Build state from trunk to release branch. This matter was discussed briefly in
http://thread.gmane.org/gmane.comp.lib.boost.devel/180814/
but it didi not lead anywhere, and further it seems to be some confusion between merging Boost.Jam (which Rene wants) and merging Boost.Build.
To reiterate, I think Boost.Build should be merged because:
- it now supports linux->windows cross compile - it allows to eliminate configuration warning messages from libraries that are not built (which requires corresponding merges for libraries Jamfiles, but it's trivial) - it fixes a bug where include=/foo/bar could not be specified on the command line - it has some other smaller fixes - I don't see any breakage on trunk tests
Comments?
So, what is the final conclusion? Rene and Eric agree to merge, but I don't quite understand to gets to say final "yes". Thanks, Volodya
On Mon, Oct 13, 2008 at 2:00 AM, Vladimir Prus <vladimir@codesourcery.com>wrote:
Vladimir Prus wrote:
So, what is the final conclusion? Rene and Eric agree to merge, but I don't quite understand to gets to say final "yes".
Rene and Eric are release managers. If they say it is OK to merge, that's enough. You don't need an OK from me. So please do go ahead. Thanks, --Beman
Beman Dawes wrote:
On Mon, Oct 13, 2008 at 2:00 AM, Vladimir Prus <vladimir@codesourcery.com>wrote:
Vladimir Prus wrote:
So, what is the final conclusion? Rene and Eric agree to merge, but I don't quite understand to gets to say final "yes".
Rene and Eric are release managers. If they say it is OK to merge, that's enough. You don't need an OK from me. So please do go ahead.
This is now done. I've also merged Jamfile.v2 for graph, mpi and regex, to grab the change that stops them from emitting configuration-related warnings if they are not built. Each jamfile had no changes except for that one. Python still emits a configuration warning; I've filed http://svn.boost.org/trac/boost/ticket/2417 for later release. - Volodya
On Thu, Oct 16, 2008 at 5:26 AM, Vladimir Prus <vladimir@codesourcery.com>wrote:
Beman Dawes wrote:
On Mon, Oct 13, 2008 at 2:00 AM, Vladimir Prus < vladimir@codesourcery.com>wrote:
Vladimir Prus wrote:
So, what is the final conclusion? Rene and Eric agree to merge, but I don't quite understand to gets to say final "yes".
Rene and Eric are release managers. If they say it is OK to merge, that's enough. You don't need an OK from me. So please do go ahead.
This is now done. I've also merged Jamfile.v2 for graph, mpi and regex, to grab the change that stops them from emitting configuration-related warnings if they are not built. Each jamfile had no changes except for that one. Python still emits a configuration warning; I've filed
http://svn.boost.org/trac/boost/ticket/2417
for later release.
Thanks, --Beman
participants (7)
-
Beman Dawes -
Eric Niebler -
K. Noel Belcourt -
Rene Rivera -
Robert Ramey -
Vladimir Prus -
Vladimir Prus