RE process (prospective from a retired FreeBSD committer)...
[[ whoops! I double tapped the send keyboard shortcut on a foreground message and sent this before I'd finished cleaning it up. Apologies for the near-double post. The last few paragraphs are different ]] I originally sent this to Eric but am cleaning it up and reposting here per his suggestion. From an outsider's perspective it appears as though the lack of branching/merging tied to release management is the biggest issue for Boost at present. As a retired FreeBSD committer (and a boost-digest lurker since 1.33), I wanted to point out how the FreeBSD community deals with release engineering and see if there are a few nugget policies that could be cherry picked for boost. FreeBSD's process has been in place and refined for the last 15 years and comes with already written documentation as a starting point. It's not perfect, but there's a pound of prevention that comes via policy that no tool can replicate. head/trunk/tip is known as the -CURRENT branch. Released versions for production used (or are in prep for being released) are known as -STABLE branches. When FreeBSD 8 was being developed, people committed to HEAD (aka -CURRENT). When the re@ team called for a freeze for release and -CURRENT was in sufficiently good condition, they branched HEAD in to RELENG_8_STABLE. Everything in RELENG_8_* is expected to be production quality (or nearing it). Upon release of 8.0, RELENG_8 was branched again to RELEASE_8_0. 8.1, 8.2, etc were all created from RELENG_8_STABLE. Fixes for things in stable go to -CURRENT first, then are back patched to the appropriate stable releases, colloquially known as MFC's or Merge From -CURRENT. -CURRENT is the wild west of OS development and repo trash. Use at your own risk. Stability and likelihood of all tests passing is marginal, at best. Towards the end of a given major version -CURRENT becomes more stable, but once -STABLE has been branched it quickly degenerates some as people implement the impetus to checkin new features. -STABLE should always be passing 100% of the regression tests present in the branch at all times. A quick doc on merging between branches: http://wiki.freebsd.org/SVN_Merging I don't know how wide spread it is and despite svk having gone in to maintenance mode, but the use of svk has merit (partial checkouts): http://svk.bestpractical.com/view/HomePage A schedule and handoff of hats/responsibilities for each of the branches is pretty well documented and is now time driven (it used to be feature driven, which didn't work as well as they'd hoped): http://www.freebsd.org/releng/ The way that FreeBSD handles its permissions and merging from one branch to the next is via the commit message, commit hooks and approvals from different people wearing different hats. A few examples: http://lists.freebsd.org/pipermail/svn-src-stable/2011-January/008657.html The above has an MFC line in there to denote the changes included. The commit handler verifies that the diffed files largely match (at least it used to back in the day when I was committing via CVS). http://lists.freebsd.org/pipermail/svn-src-stable/2011-January/008683.html A 'Reviewed by' commit header. This gives greater latitude in terms of what can be included in the commit. http://lists.freebsd.org/pipermail/svn-src-stable/2011-January/008808.html And the 'Approved by' commit header. Once a release is frozen, all commits need to have this tag otherwise the commit will fail. If someone commits with that line and didn't have permission to do so, as a policy, the commit is always reverted 100% of the time as a matter of principle. Frequently it's re-committed, but it's a big slap to the back of the hands to have to go through the process again. Needless to say, the commit mailing list is the most active and widely read list as a result. And tons of code gets reviewed with many eyes viewing it as a result. Unlike boost, FreeBSD uses an abridged commit message that doesn't include the actual diff itself (if you're a committer, then you see the diffs to areas of the tree that you subscribe to). I don't know how many people actually follow boost's commit log on a per commit basis, but given 100% of all commits go to a single mailing list and the diffs can sometimes be massive, it seems like the current list infrastructure makes that kind of review process unsustainable for Joe Reviewer (vs. the shorter abridged commit messages w/ a link to the actual diff). Links to each of the diffs is useful and lets people click through if reviewing the diff is of interest. I can't stress the organizational difference and peer pressure a widely used commit mailing list brings about. There are other bits worth noting, acls apply to different parts of the tree (doc commit bits and a doc-re@ hat vs a src-re@ hat and even ports@), but the mailing list/commit reviews and branching/merging are the biggies that I wanted to point out. Branching trunk in to a boost_1_46_prerelease branch three weeks before the release (or some sane interval) so that people can fix up the code seems like a pretty painless adjustment, then snapping/releasing boost_1_46_0 when the pre-release is ready. Post-1.46 release fixes could go in to 1.46.0 and (heaven forbid), maybe even a micro version with specific fixes for the .46 minor version. PostgreSQL ships with a contrib/ directory of "soon to be" or "possible candidates for being a core component" which serves the same purpose as FreeBSD's ports structure. This gets yet-to-be finalized modules out in the wild and helps garner interest. PostgreSQL's autovacuum went from being a fringe project to a contrib/ module and a core feature in 2 minor releases because of the huge interest in its use/adoption. Boost.Log or Atomic or any of the other "we all really want this but it's not quite finalized" modules seem like ideal candidates for inclusion in such a directory because it could generate additional interest/eyes. A contrib/ or proposed/ would go a long way towards keeping boost lean-ish, too. At present it seems like boost-*.tar.bz2 is on track to including boost/kitchen/sink.hpp and boost/bath/water.hpp and that's something that is a bit concerning to me on the long-term scale. Me personally, I keep running 'svn ls http://svn.boost.org/svn/boost/branches/' to see if a branch pops up for release but I haven't yet. The structure on the server is largely there, but it the svn tree looks pretty disorganized with lots of legacy clutter so it doesn't look like it's being used well. And lastly re: VCSs, lots of people fork FreeBSD to do experimental work out of the tree via git and hg, but the monolithic and serialized commit/review process seems to be working quite well from my perspective. A little bureaucratic but very stable and democratic without reliance on any one person to push the release forward. Anyway, food for thought. Hopefully there's something there that you can pick out of value. If you have questions, feel free to ask on or offline (CC if online since I'm a digest subscriber). Cheers. -Sean -- Sean Chittenden sean@chittenden.org
At Thu, 27 Jan 2011 11:24:30 -0800, Sean Chittenden wrote:
As a retired FreeBSD committer (and a boost-digest lurker since 1.33), I wanted to point out how the FreeBSD community deals with release engineering and see if there are a few nugget policies that could be cherry picked for boost. FreeBSD's process has been in place and refined for the last 15 years and comes with already written documentation as a starting point. It's not perfect, but there's a pound of prevention that comes via policy that no tool can replicate.
FreeBSD is very carefully done, consists of many diverse and interdependent modules, and would make a good model for Boost. I, for one, appreciate your perspective very much.
head/trunk/tip is known as the -CURRENT branch. Released versions for production used (or are in prep for being released) are known as -STABLE branches. When FreeBSD 8 was being developed, people committed to HEAD (aka -CURRENT). When the re@ team called for a freeze for release and -CURRENT was in sufficiently good condition, they branched HEAD in to RELENG_8_STABLE. Everything in RELENG_8_* is expected to be production quality (or nearing it). Upon release of 8.0, RELENG_8 was branched again to RELEASE_8_0. 8.1, 8.2, etc were all created from RELENG_8_STABLE. Fixes for things in stable go to -CURRENT first, then are back patched to the appropriate stable releases, colloquially known as MFC's or Merge From -CURRENT.
-CURRENT is the wild west of OS development and repo trash. Use at -your own risk. Stability and likelihood of all tests passing is -marginal, at best. Towards the end of a given major version --CURRENT becomes more stable, but once -STABLE has been branched it -quickly degenerates some as people implement the impetus to checkin -new features.
-STABLE should always be passing 100% of the regression tests -present in the branch at all times.
This is all pretty standard procedure for many projects, IIUC. However, my vision for Boost is a bit different: * each library has its own Git repo * each library maintainer tests and produces versioned releases of that library on his/her own timetable * the latest release of a library is tagged "STABLE" * When assembling a new Boost release, the release manager marks the "STABLE" revision from each library and begins boost-wide integration/interoperability testing on those revisions * When a library release fails integration testing, the release manager has several options: * Give the library maintainer a time window in which to produce a new STABLE release that passes integration testing * Use an earlier STABLE release of the library * Withold the library from the next Boost release (drastic) * If the failures appear to be due to breakage in another library, the release manager can apply these procedures to the culprit library One idea behind this process is that it allows individual libraries to "officially release" fixes and updates without forcing Boost's release managers to coordinate a Boost-wide point release.
A quick doc on merging between branches: http://wiki.freebsd.org/SVN_Merging
I don't know how wide spread it is and despite svk having gone in to maintenance mode, but the use of svk has merit (partial checkouts):
No offense intended to anyone, but SVK is at best a poor-man's Git :-)
A schedule and handoff of hats/responsibilities for each of the branches is pretty well documented and is now time driven (it used to be feature driven, which didn't work as well as they'd hoped):
Not sure how that page illustrates what you're saying above. Care to illuminate?
The way that FreeBSD handles its permissions and merging from one branch to the next is via the commit message, commit hooks and approvals from different people wearing different hats. A few examples:
http://lists.freebsd.org/pipermail/svn-src-stable/2011-January/008657.html
The above has an MFC line in there to denote the changes included. The commit handler verifies that the diffed files largely match (at least it used to back in the day when I was committing via CVS).
http://lists.freebsd.org/pipermail/svn-src-stable/2011-January/008683.html
A 'Reviewed by' commit header.
Git has a mechanism for signing off on changes with cryptographic security, so that you can have reasonable assurance that the change was actually approved by the person claimed.
This gives greater latitude in terms of what can be included in the commit.
Could you be more specific?
http://lists.freebsd.org/pipermail/svn-src-stable/2011-January/008808.html
And the 'Approved by' commit header. Once a release is frozen, all commits need to have this tag otherwise the commit will fail. If someone commits with that line and didn't have permission to do so, as a policy, the commit is always reverted 100% of the time as a matter of principle. Frequently it's re-committed, but it's a big slap to the back of the hands to have to go through the process again. Needless to say, the commit mailing list is the most active and widely read list as a result.
Sorry, I don't understand. How does that procedure result in many people reading the commit list?
And tons of code gets reviewed with many eyes viewing it as a result. Unlike boost, FreeBSD uses an abridged commit message that doesn't include the actual diff itself
Do you mean that the message sent to the commit mailing list is abridged?
(if you're a committer, then you see the diffs to areas of the tree that you subscribe to).
Nifty; that keeps down the noise.
I don't know how many people actually follow boost's commit log on a per commit basis, but given 100% of all commits go to a single mailing list and the diffs can sometimes be massive, it seems like the current list infrastructure makes that kind of review process unsustainable for Joe Reviewer (vs. the shorter abridged commit messages w/ a link to the actual diff). Links to each of the diffs is useful and lets people click through if reviewing the diff is of interest.
Makes sense.
I can't stress the organizational difference and peer pressure a widely used commit mailing list brings about.
Specifically?
There are other bits worth noting, acls apply to different parts of the tree (doc commit bits and a doc-re@ hat vs a src-re@ hat and even ports@),
Splitting Boost up into separate repositories should handle most access-control issues. On top of that, if we needed it, something like git-o-lite would allow us to control access to paths and branches within a single repo.
but the mailing list/commit reviews and branching/merging are the biggies that I wanted to point out. Branching trunk in to a boost_1_46_prerelease branch three weeks before the release (or some sane interval) so that people can fix up the code seems like a pretty painless adjustment, then snapping/releasing boost_1_46_0 when the pre-release is ready.
Believe it or not, we used to have a process very much like that. Beman has explained here why the current process is an improvement for release managers, but I don't remember the rationale. Beman, maybe you'd like to repeat it?
Post-1.46 release fixes could go in to 1.46.0 and (heaven forbid), maybe even a micro version with specific fixes for the .46 minor version.
Sorry, I don't understand the above at all. How could post-1.46 stuff go into 1.46.0? 1.46 == 1.46.0! And I don't even grok the "micro version..." part enough know what questions to ask you about it. Care to try again?
PostgreSQL ships with a contrib/ directory of "soon to be" or "possible candidates for being a core component" which serves the same purpose as FreeBSD's ports structure. This gets yet-to-be finalized modules out in the wild and helps garner interest.
having a contrib/ directory is an interesting idea for Boost. That'd be very different from the sandbox. But who's responsible for maintaining that code?
PostgreSQL's autovacuum went from being a fringe project to a contrib/ module and a core feature in 2 minor releases because of the huge interest in its use/adoption. Boost.Log or Atomic or any of the other "we all really want this but it's not quite finalized" modules seem like ideal candidates for inclusion in such a directory because it could generate additional interest/eyes. A contrib/ or proposed/ would go a long way towards keeping boost lean-ish, too.
How so? It sounds like it means adding more to each release.
At present it seems like boost-*.tar.bz2 is on track to including boost/kitchen/sink.hpp and boost/bath/water.hpp and that's something that is a bit concerning to me on the long-term scale.
That seems like a semi-random fear that I don't see being addressed by a contrib/ directory.
Me personally, I keep running 'svn ls http://svn.boost.org/svn/boost/branches/' to see if a branch pops up for release but I haven't yet.
Because it isn't done in the standard way here (c.f. Beman's explanation).
The structure on the server is largely there, but it the svn tree looks pretty disorganized with lots of legacy clutter so it doesn't look like it's being used well.
And lastly re: VCSs, lots of people fork FreeBSD to do experimental work out of the tree via git and hg, but the monolithic and serialized commit/review process seems to be working quite well from my perspective. A little bureaucratic but very stable and democratic without reliance on any one person to push the release forward. Anyway, food for thought. Hopefully there's something there that you can pick out of value.
One of the differences between FreeBSD and Boost is (I think) that the vast majority of the actual code in FreeBSD is in the kernel, and thus fairly highly coupled. IMO in Boost, maintainers quite properly have much less interest in what is happening in other libraries. That may make a difference in choosing a suitable procedure. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
Dave Abrahams wrote:
At Thu, 27 Jan 2011 11:24:30 -0800, Sean Chittenden wrote:
However, my vision for Boost is a bit different:
* each library has its own Git repo
right - but if each library has it's own repo - they don't all have to use the same system.
* each library maintainer tests and produces versioned releases of that library on his/her own timetable
yep
* the latest release of a library is tagged "STABLE"
or merged into the "current release"
* When assembling a new Boost release, the release manager marks the "STABLE" revision from each library and begins boost-wide integration/interoperability testing on those revisions
yep.
* When a library release fails integration testing, the release manager has several options:
* Give the library maintainer a time window in which to produce a new STABLE release that passes integration testing
* Use an earlier STABLE release of the library
This is the one!
* Withold the library from the next Boost release (drastic)
with the above option, (use earlier release), this should never be necessary.
* If the failures appear to be due to breakage in another library, the release manager can apply these procedures to the culprit library
This would almost never occur if each library were testes against all the other libraries makred "STABLE" rather than all the other libraries on the trunk as we do now.
One idea behind this process is that it allows individual libraries to "officially release" fixes and updates without forcing Boost's release managers to coordinate a Boost-wide point release.
Halleluhah
having a contrib/ directory is an interesting idea for Boost. That'd be very different from the sandbox. But who's responsible for maintaining that code?
Ahhh - see my "boost candidate proposal" at bootcon 2010
At present it seems like boost-*.tar.bz2 is on track to including boost/kitchen/sink.hpp and boost/bath/water.hpp and that's something that is a bit concerning to me on the long-term scale.
That seems like a semi-random fear that I don't see being addressed by a contrib/ directory.
eventually the whole issue will disappear when I can go to website and check off all the libraries I'm interested in and have the "package" made to order and download it all in one operation. Or better yet update my local system just like Cygwin does now. The only difference between "official and/or certified" boost libraries will be the "official certification" which boost and only boost will be able to confer. Issues of critique, valueation, assessment, maintainence, etc., etc. would be at the library level in the hands of the library developer/maintainer. Unmaintained/obsolete libraries will die a natural death as demand atrophies while users migrate to better replacments. Robert Ramey
This is all pretty standard procedure for many projects, IIUC.
However, my vision for Boost is a bit different:
* each library has its own Git repo
* each library maintainer tests and produces versioned releases of that library on his/her own timetable
* the latest release of a library is tagged "STABLE"
* When assembling a new Boost release, the release manager marks the "STABLE" revision from each library and begins boost-wide integration/interoperability testing on those revisions
* When a library release fails integration testing, the release manager has several options:
* Give the library maintainer a time window in which to produce a new STABLE release that passes integration testing
* Use an earlier STABLE release of the library
* Withold the library from the next Boost release (drastic)
* If the failures appear to be due to breakage in another library, the release manager can apply these procedures to the culprit library
One idea behind this process is that it allows individual libraries to "officially release" fixes and updates without forcing Boost's release managers to coordinate a Boost-wide point release.
This sounds reasonable to me, though what happens to software that works and is now in maintenance mode with an author that has left the boost community? As for modularizing things out of the base, this is what FreeBSD has done with many pieces of software via the ports tree. In fact, many other traditional bits of software that many feel are default parts of an OS have been moved to the ports tree (e.g. perl is not in the base OS install on FreeBSD, and from what I hear gcc will be moved out as well in favor of llvm).
No offense intended to anyone, but SVK is at best a poor-man's Git :-)
None taken as I've never used it myself. :~]
A schedule and handoff of hats/responsibilities for each of the branches is pretty well documented and is now time driven (it used to be feature driven, which didn't work as well as they'd hoped):
Not sure how that page illustrates what you're saying above. Care to illuminate?
There are people who maintain roles and responsibilities for various branches as each branch goes through its life cycle. Each branch roughly moves through the following life cycle: HEAD = Any developer w/ commit access can commit pre-release branch (e.g. BOOST_1_46_prerelease) = Any re@ can approve commits to the branch but developers are required to get re@ permission (re@ = release engineer hat) released branch (e.g. BOOST_1_46_0) = re@ hands the branch off to security-officer@ for security related fixes. Sometimes re@ will push out a micro version release for critical bugs, but security-officer@ takes ownership of the 1_46_0 branch until the branch is EoL'ed (I don't think boost has a concept of an EoL'ed branch at present). So in that scenario, there are three hats that could be worn (some/often the same people, but looking at a problem from different perspectives). The above link is just the time table and indicator for developers for who has the ability to bless a commit to a particular branch. Once a branch is handed off to a particular hat (e.g. re@ or sec@), then you have to go find the right person. One update that I would like to see for boost would be for the hats for each branch to be solidified. Maintainers of the 1.46 branch might be different than 1.45 vs 1.44 and there's a certain amount of institutional knowledge that comes with shepherding a release that results in slightly better judgement calls (i.e. "oh! that release had a funky interoperability bit with XYZ so we did ABC to fix this"). Actually, a real world example of this would probably be what happened with boost::serialize a release or two back. When 1.50 comes around and someone else is doing the re@ process, but there's a fix for 1.40, the 1.40 re@ team will almost certainly remember the oddity, but the 1.50 re@ team may not.
http://lists.freebsd.org/pipermail/svn-src-stable/2011-January/008683.html
A 'Reviewed by' commit header.
Git has a mechanism for signing off on changes with cryptographic security, so that you can have reasonable assurance that the change was actually approved by the person claimed.
The crypto assurances that the handoff was valid is important in a dvcs, though my point was in pushing MFC-like approvals in to the commit messages so that people can identify when a change was approved or why, or spark a discussion on a mailing list.
This gives greater latitude in terms of what can be included in the commit.
Could you be more specific?
An MFC's commit-hook requires that filenames match, that's it. When you use the 'Reviewed By' commit message header, you can include different files that weren' in the original commit to -CURRENT. I don't know that this level of control needs to be in place, but there are specific process definitions for each of the commit headers.
http://lists.freebsd.org/pipermail/svn-src-stable/2011-January/008808.html
And the 'Approved by' commit header. Once a release is frozen, all commits need to have this tag otherwise the commit will fail.
I think this statement stand on its own. To be clear, the commit hook actually examines the contents of the commit message and looks for various elements and will fail a commit if the commit message is not in accordance with the branch's ACL's.
If someone commits with that line and didn't have permission to do so, as a policy, the commit is always reverted 100% of the time as a matter of principle. Frequently it's re-committed, but it's a big slap to the back of the hands to have to go through the process again. Needless to say, the commit mailing list is the most active and widely read list as a result.
Sorry, I don't understand. How does that procedure result in many people reading the commit list?
People read the commit mailing lists because that's the easiest way to quickly digest the changes that are made to the tree. If something catches your eye, you can quickly go from the mailing list to a diff to review the change. Because enough people do this and the attention level has reached critical mass, the commit list is the focal point for most of the technical/mechanical discussions.
And tons of code gets reviewed with many eyes viewing it as a result. Unlike boost, FreeBSD uses an abridged commit message that doesn't include the actual diff itself
Do you mean that the message sent to the commit mailing list is abridged?
Yes.
(if you're a committer, then you see the diffs to areas of the tree that you subscribe to).
Nifty; that keeps down the noise.
Indeed. At several hundred commits per day, managing the signal to noise ratio is an important part of the equation.
I can't stress the organizational difference and peer pressure a widely used commit mailing list brings about.
Specifically?
If many eyes are watching a commit list and a discussion starts, relevant and interested parties are quick to respond to the technical merits/problems with a particular commit. Because of this self-policing and pressure to perform in front of your peers, the quality of commit messages is very high (as well as commits since reverts show up on the commit mailing list as well). My gripe with git and all dcvs's is this frequent lack of a central review process and core community of reviewers. I forgot to mention, but in the case of the BSDs, if you have commit access, you are involuntarily subscribed to the commit mailing list.
Post-1.46 release fixes could go in to 1.46.0 and (heaven forbid), maybe even a micro version with specific fixes for the .46 minor version.
Sorry, I don't understand the above at all. How could post-1.46 stuff go into 1.46.0? 1.46 == 1.46.0! And I don't even grok the "micro version..." part enough know what questions to ask you about it. Care to try again?
No worries, I was in a bit of a hurry after I double tapped send. Here's the significance of the tag/branch naming scheme in a boost hypothetical situation. BOOST_1_CURRENT = trunk for Boost 1.X development (plans/criteria for a 2.0?) BOOST_1_STABLE = the branch that will have the next release (e.g. 1.47.0). There isn't much in the way of API stability that needs to be maintained atm, but if there were, a BOOST_2_CURRENT with incompatible APIs from BOOST_1_CURRENT would probably need a head vs production branch. BOOST_1_46 = re@ has branched BOOST_1_CURRENT to create BOOST_1_46 to get the tree in shape for a release. The only commits allowed to go in at this time are commits that pertain to stabilizing the release and getting tests to pass. BOOST_1_46_0 = re@ has shipped the 1.46.0 release. This branch is now managed by security-officer@ and to a lesser degree the re@ team. BOOST_1_46_1 = A branch/tag that security-officer@ creates once there is a reason to release an updated release of 1.46.0. 1.46.0 to 1.46.1 is required to be API compatible. In the case of the BSDs, all minor and micro versions are required to be ABI compatible (which is why gcc in the base system only gets updated in the BSDs along major release numbers). Is that a better explanation?
PostgreSQL ships with a contrib/ directory of "soon to be" or "possible candidates for being a core component" which serves the same purpose as FreeBSD's ports structure. This gets yet-to-be finalized modules out in the wild and helps garner interest.
having a contrib/ directory is an interesting idea for Boost. That'd be very different from the sandbox. But who's responsible for maintaining that code?
Whoever's the review manager and the review manager's mentee who originally wrote the module.
PostgreSQL's autovacuum went from being a fringe project to a contrib/ module and a core feature in 2 minor releases because of the huge interest in its use/adoption. Boost.Log or Atomic or any of the other "we all really want this but it's not quite finalized" modules seem like ideal candidates for inclusion in such a directory because it could generate additional interest/eyes. A contrib/ or proposed/ would go a long way towards keeping boost lean-ish, too.
How so? It sounds like it means adding more to each release.
My train of thought was starting to drift here and that last sentence was largely a reaction to not being able to checkout the entire boost svn repository because of the sandbox. Let me clean that thought up real quick and rephrase. PostgreSQL's autovacuum went from being a fringe project to a contrib/ module and a core feature in 2 minor releases because of the huge interest in its use/adoption. Boost.Log or Atomic or any of the other "we all really want this but it's not quite finalized" modules seem like ideal candidates for inclusion in such a directory because it could generate additional interest/eyes. Like autovacuum, Boost.Atomic has the potential to be a fantastic piece of boost's infrastructure where many boost modules could make use of it. A contrib/ or proposed/ would go a long way towards keeping boost lean-ish, too. Here's how: Being able to modularize boost modules in to little packages that can be enabled/disabled independently of the main source tree/core will reduce clutter and make interdependencies explicit. Right now it takes ~10min and 1GB of ram on my 2.8Ghz Xeon to get bjam to complete its dependency checks before the first line of anything is compiled. Having smaller modules that can be enabled/disabled quickly because things are well contained in isolation should reduce the overhead for source installs. As time passes and the number of headers or modules grow inside of the main boost source tree, how long will that process take to complete in the future? Reducing the number of things that bjam has to keep track of in the source tree seems like the only way to mitigate this (for me personally, this has been a growing source of frustration and was a sore spot for me with boost since day #1 and why I initially was using cmake instead of bjam).
At present it seems like boost-*.tar.bz2 is on track to including boost/kitchen/sink.hpp and boost/bath/water.hpp and that's something that is a bit concerning to me on the long-term scale.
That seems like a semi-random fear that I don't see being addressed by a contrib/ directory.
See above re: bjam's install time. Patching in various boost libraries to be compiled and installed via bjam is laborious, especially when things are still under development and the compile bombs out.
The structure on the server is largely there, but it the svn tree looks pretty disorganized with lots of legacy clutter so it doesn't look like it's being used well.
And lastly re: VCSs, lots of people fork FreeBSD to do experimental work out of the tree via git and hg, but the monolithic and serialized commit/review process seems to be working quite well from my perspective. A little bureaucratic but very stable and democratic without reliance on any one person to push the release forward. Anyway, food for thought. Hopefully there's something there that you can pick out of value.
One of the differences between FreeBSD and Boost is (I think) that the vast majority of the actual code in FreeBSD is in the kernel, and thus fairly highly coupled. IMO in Boost, maintainers quite properly have much less interest in what is happening in other libraries. That may make a difference in choosing a suitable procedure.
I see boost as much closer to the FreeBSD port system, actually. I think there is a boost "core" but things that are written using boost aren't explicitly core until something depends on them. Anyway, hopefully this has been useful (and a break from the string discussion, yikes!). -sc -- Sean Chittenden sean@chittenden.org
At Fri, 28 Jan 2011 13:30:53 -0800, Sean Chittenden wrote:
This is all pretty standard procedure for many projects, IIUC.
However, my vision for Boost is a bit different:
* each library has its own Git repo
* each library maintainer tests and produces versioned releases of that library on his/her own timetable
* the latest release of a library is tagged "STABLE"
* When assembling a new Boost release, the release manager marks the "STABLE" revision from each library and begins boost-wide integration/interoperability testing on those revisions
* When a library release fails integration testing, the release manager has several options:
* Give the library maintainer a time window in which to produce a new STABLE release that passes integration testing
* Use an earlier STABLE release of the library
* Withold the library from the next Boost release (drastic)
* If the failures appear to be due to breakage in another library, the release manager can apply these procedures to the culprit library
One idea behind this process is that it allows individual libraries to "officially release" fixes and updates without forcing Boost's release managers to coordinate a Boost-wide point release.
This sounds reasonable to me, though what happens to software that works and is now in maintenance mode with an author that has left the boost community?
Note that I didn't mention the author above. :-)
As for modularizing things out of the base, this is what FreeBSD has done with many pieces of software via the ports tree. In fact, many other traditional bits of software that many feel are default parts of an OS have been moved to the ports tree (e.g. perl is not in the base OS install on FreeBSD, and from what I hear gcc will be moved out as well in favor of llvm).
Cool.
A schedule and handoff of hats/responsibilities for each of the branches is pretty well documented and is now time driven (it used to be feature driven, which didn't work as well as they'd hoped):
Not sure how that page illustrates what you're saying above. Care to illuminate?
There are people who maintain roles and responsibilities for various branches as each branch goes through its life cycle. Each branch roughly moves through the following life cycle:
HEAD = Any developer w/ commit access can commit
pre-release branch (e.g. BOOST_1_46_prerelease) = Any re@ can approve commits to the branch but developers are required to get re@ permission (re@ = release engineer hat)
released branch (e.g. BOOST_1_46_0) = re@ hands the branch off to security-officer@ for security related fixes. Sometimes re@ will push out a micro version release for critical bugs, but security-officer@ takes ownership of the 1_46_0 branch until the branch is EoL'ed (I don't think boost has a concept of an EoL'ed branch at present).
So in that scenario, there are three hats that could be worn (some/often the same people, but looking at a problem from different perspectives).
If we can push all or nearly all of Boost into modules, managing these kinds of phases will become the responsibility of individual library maintainers.
The above link is just the time table and indicator for developers for who has the ability to bless a commit to a particular branch. Once a branch is handed off to a particular hat (e.g. re@ or sec@), then you have to go find the right person.
One update that I would like to see for boost would be for the hats for each branch to be solidified. Maintainers of the 1.46 branch might be different than 1.45 vs 1.44 and there's a certain amount of institutional knowledge that comes with shepherding a release that results in slightly better judgement calls (i.e. "oh! that release had a funky interoperability bit with XYZ so we did ABC to fix this").
I'd rather see that kind of knowledge pushed upstream in the form of permanent changes to the individual libraries involved.
Actually, a real world example of this would probably be what happened with boost::serialize a release or two back. When 1.50 comes around and someone else is doing the re@ process, but there's a fix for 1.40, the 1.40 re@ team will almost certainly remember the oddity, but the 1.50 re@ team may not.
And I'd like the maintainer of Boost.Serialization to be the responsible party for that.
my point was in pushing MFC-like approvals in to the commit messages so that people can identify when a change was approved or why, or spark a discussion on a mailing list.
Makes sense.
This gives greater latitude in terms of what can be included in the commit.
Could you be more specific?
An MFC's commit-hook requires that filenames match, that's it. When you use the 'Reviewed By' commit message header, you can include different files that weren' in the original commit to -CURRENT. I don't know that this level of control needs to be in place, but there are specific process definitions for each of the commit headers.
OK. In my proposal it's up to each library maintainer to certify particular states as "releasable," and he or she can choose the process used to get there.
If someone commits with that line and didn't have permission to do so, as a policy, the commit is always reverted 100% of the time as a matter of principle. Frequently it's re-committed, but it's a big slap to the back of the hands to have to go through the process again. Needless to say, the commit mailing list is the most active and widely read list as a result.
Sorry, I don't understand. How does that procedure result in many people reading the commit list?
People read the commit mailing lists because that's the easiest way to quickly digest the changes that are made to the tree. If something catches your eye, you can quickly go from the mailing list to a diff to review the change. Because enough people do this and the attention level has reached critical mass, the commit list is the focal point for most of the technical/mechanical discussions.
Yeah, OK... but what motivates people to actually do that? Why do people want to "quickly digest the changes that are made to the tree?" I think the answer to that question is the key to what makes this process work for FreeBSD.
And tons of code gets reviewed with many eyes viewing it as a result. Unlike boost, FreeBSD uses an abridged commit message that doesn't include the actual diff itself
Do you mean that the message sent to the commit mailing list is abridged?
Yes.
Nice idea.
I can't stress the organizational difference and peer pressure a widely used commit mailing list brings about.
Specifically?
If many eyes are watching a commit list and a discussion starts, relevant and interested parties are quick to respond to the technical merits/problems with a particular commit. Because of this self-policing and pressure to perform in front of your peers, the quality of commit messages is very high (as well as commits since reverts show up on the commit mailing list as well).
That makes a *lot* of sense, thanks. I believe we *could* set this up for Boost even if it were split into separate modules/repositories.
My gripe with git and all dcvs's is this frequent lack of a central review process and core community of reviewers.
I don't see what that has to do with the tools.
I forgot to mention, but in the case of the BSDs, if you have commit access, you are involuntarily subscribed to the commit mailing list.
Didn't you just tell me that you can filter out messages about the parts of the tree you're not interested in?
Post-1.46 release fixes could go in to 1.46.0 and (heaven forbid), maybe even a micro version with specific fixes for the .46 minor version.
Sorry, I don't understand the above at all. How could post-1.46 stuff go into 1.46.0? 1.46 == 1.46.0! And I don't even grok the "micro version..." part enough know what questions to ask you about it. Care to try again?
No worries, I was in a bit of a hurry after I double tapped send.
Here's the significance of the tag/branch naming scheme in a boost hypothetical situation.
BOOST_1_CURRENT = trunk for Boost 1.X development (plans/criteria for a 2.0?)
Let's not start a whole new bag of threads right now, OK? :-)
BOOST_1_STABLE = the branch that will have the next release (e.g. 1.47.0). There isn't much in the way of API stability that needs to be maintained atm, but if there were, a BOOST_2_CURRENT with incompatible APIs from BOOST_1_CURRENT would probably need a head vs production branch.
BOOST_1_46 = re@ has branched BOOST_1_CURRENT to create BOOST_1_46 to get the tree in shape for a release. The only commits allowed to go in at this time are commits that pertain to stabilizing the release and getting tests to pass.
BOOST_1_46_0 = re@ has shipped the 1.46.0 release. This branch is now managed by security-officer@ and to a lesser degree the re@ team.
BOOST_1_46_1 = A branch/tag that security-officer@ creates once there is a reason to release an updated release of 1.46.0. 1.46.0 to 1.46.1 is required to be API compatible. In the case of the BSDs, all minor and micro versions are required to be ABI compatible (which is why gcc in the base system only gets updated in the BSDs along major release numbers).
Is that a better explanation?
Sure, though if you hadn't confused me with the other part, probably unnecessary; something like that is a pretty standard procedure across many projects.
having a contrib/ directory is an interesting idea for Boost. That'd be very different from the sandbox. But who's responsible for maintaining that code?
Whoever's the review manager and the review manager's mentee who originally wrote the module.
/me bites his nails That idea gives me the willies.
PostgreSQL's autovacuum went from being a fringe project to a contrib/ module and a core feature in 2 minor releases because of the huge interest in its use/adoption. Boost.Log or Atomic or any of the other "we all really want this but it's not quite finalized" modules seem like ideal candidates for inclusion in such a directory because it could generate additional interest/eyes.
Like autovacuum, Boost.Atomic has the potential to be a fantastic piece of boost's infrastructure where many boost modules could make use of it.
A contrib/ or proposed/ would go a long way towards keeping boost lean-ish, too. Here's how:
Being able to modularize boost modules in to little packages that can be enabled/disabled independently of the main source tree/core will reduce clutter and make interdependencies explicit. Right now it takes ~10min and 1GB of ram on my 2.8Ghz Xeon to get bjam to complete its dependency checks before the first line of anything is compiled.
Most of that is not actually due to dependency analysis.
Having smaller modules that can be enabled/disabled quickly because things are well contained in isolation should reduce the overhead for source installs. As time passes and the number of headers or modules grow inside of the main boost source tree, how long will that process take to complete in the future? Reducing the number of things that bjam has to keep track of in the source tree seems like the only way to mitigate this (for me personally, this has been a growing source of frustration and was a sore spot for me with boost since day #1 and why I initially was using cmake instead of bjam).
I'm all for modularization, but again, I don't see how *adding* contrib/ to the boost distribution can help make boost leaner. I think it's just a logical impossibility.
At present it seems like boost-*.tar.bz2 is on track to including boost/kitchen/sink.hpp and boost/bath/water.hpp and that's something that is a bit concerning to me on the long-term scale.
That seems like a semi-random fear that I don't see being addressed by a contrib/ directory.
See above re: bjam's install time. Patching in various boost libraries to be compiled and installed via bjam is laborious, especially when things are still under development and the compile bombs out.
Again, I don't see this being addressed by the presence of contrib/
One of the differences between FreeBSD and Boost is (I think) that the vast majority of the actual code in FreeBSD is in the kernel, and thus fairly highly coupled. IMO in Boost, maintainers quite properly have much less interest in what is happening in other libraries. That may make a difference in choosing a suitable procedure.
I see boost as much closer to the FreeBSD port system, actually.
+1
I think there is a boost "core" but things that are written using boost aren't explicitly core until something depends on them.
https://github.com/boost-lib/core
Anyway, hopefully this has been useful (and a break from the string discussion, yikes!). -sc
:-) -- Dave Abrahams BoostPro Computing http://www.boostpro.com
Dave Abrahams wrote:
However, my vision for Boost is a bit different:
* each library has its own Git repo
* each library maintainer tests and produces versioned releases of that library on his/her own timetable
* the latest release of a library is tagged "STABLE"
* When assembling a new Boost release, the release manager marks the "STABLE" revision from each library and begins boost-wide integration/interoperability testing on those revisions
I am rather sceptical about this approach. The goodness of what we have now is that there's a single trunk that holds code that is: - supposedly not worse in test results in all other code, - is the newest one satisfying the above constraint Therefore, if I wish to make sure my code works with new release of Boost, of if I wish to check if some bug is fixed, that is the place to go. Further, this place is jointly updated by every boost developer. What you propose breaks this useful thing, because: - Only release managers "pull" into the unified thing - Release managers only pull "releases", which, together with delays added by release managers, means that a fix to a component will be added to the unified thing after considerable delay - In practice, release managers will only pull a week or two before the release, therefore the unified thing we have now will no longer exists, and two weeks before the release we'll get a frankenstein creature that might technically pass all tests (though I doubt that), but won't be tested by any users in real projects. So, to reiterate, no matter what tools are proposed, I strongly thing that there should always be UNIFIED in-development tree of ENTIRE BOOST to which all library maintainers can add changes without coordination with release managers. Thanks, -- Vladimir Prus Mentor Graphics +7 (812) 677-68-40
On Sat, Jan 29, 2011 at 3:55 PM, Vladimir Prus <vladimir@codesourcery.com> wrote:
Dave Abrahams wrote:
However, my vision for Boost is a bit different:
* each library has its own Git repo
* each library maintainer tests and produces versioned releases of that library on his/her own timetable
* the latest release of a library is tagged "STABLE"
* When assembling a new Boost release, the release manager marks the "STABLE" revision from each library and begins boost-wide integration/interoperability testing on those revisions
I am rather sceptical about this approach. The goodness of what we have now is that there's a single trunk that holds code that is:
- supposedly not worse in test results in all other code, - is the newest one satisfying the above constraint
Therefore, if I wish to make sure my code works with new release of Boost, of if I wish to check if some bug is fixed, that is the place to go. Further, this place is jointly updated by every boost developer.
What you propose breaks this useful thing, because:
- Only release managers "pull" into the unified thing
Not really. Every developer can do the same thing that release managers do. To get a full Boost locally, you pull from multiple repositories, and you develop against a "stable" or "tagged" version of each library in your local repository.
- Release managers only pull "releases", which, together with delays added by release managers, means that a fix to a component will be added to the unified thing after considerable delay
Not really either. Release managers can just pull from libraries that have updates, and even do that incrementally. More to the point, developers willing to work on just getting a stable release out (maybe a stable release team or "support engineers") can contribute to the libraries "upstream" fixes that are found to be required for the "STABLE" versions for integration to happen. Actually I would even think that "STABLE" in each library would be a branch, typically "master" or could easily be just another branch, so changes that are specific to the "STABLE" version stay in that branch.
- In practice, release managers will only pull a week or two before the release, therefore the unified thing we have now will no longer exists, and two weeks before the release we'll get a frankenstein creature that might technically pass all tests (though I doubt that), but won't be tested by any users in real projects.
If everybody had a way of pulling the "latest and greatest" of all the "STABLE" branches of all the libraries they want to use into their own git repo, I don't see what's the difference from having everything in the same place. Am I missing something?
So, to reiterate, no matter what tools are proposed, I strongly thing that there should always be UNIFIED in-development tree of ENTIRE BOOST to which all library maintainers can add changes without coordination with release managers.
Everyone can have that locally as a conglomeration of all the libraries they're interested in. I don't see why it has to be just one server.
Thanks,
HTH -- Dean Michael Berris about.me/deanberris
Dean Michael Berris wrote:
On Sat, Jan 29, 2011 at 3:55 PM, Vladimir Prus <vladimir@codesourcery.com> wrote:
Dave Abrahams wrote:
However, my vision for Boost is a bit different:
* each library has its own Git repo
* each library maintainer tests and produces versioned releases of that library on his/her own timetable
* the latest release of a library is tagged "STABLE"
* When assembling a new Boost release, the release manager marks the "STABLE" revision from each library and begins boost-wide integration/interoperability testing on those revisions
I am rather sceptical about this approach. The goodness of what we have now is that there's a single trunk that holds code that is:
- supposedly not worse in test results in all other code, - is the newest one satisfying the above constraint
Therefore, if I wish to make sure my code works with new release of Boost, of if I wish to check if some bug is fixed, that is the place to go. Further, this place is jointly updated by every boost developer.
What you propose breaks this useful thing, because:
- Only release managers "pull" into the unified thing
Not really. Every developer can do the same thing that release managers do. To get a full Boost locally, you pull from multiple repositories, and you develop against a "stable" or "tagged" version of each library in your local repository.
Just to clarify, by "unified thing" I mean a specific URL that everybody can use to checkout/clone/fork/whatever a version of boost with all the current changes. You suggestion that I manually pull from N different repositories seems like a mess to me, sorry.
- Release managers only pull "releases", which, together with delays added by release managers, means that a fix to a component will be added to the unified thing after considerable delay
Not really either. Release managers can just pull from libraries that have updates, and even do that incrementally.
More to the point, developers willing to work on just getting a stable release out (maybe a stable release team or "support engineers") can contribute to the libraries "upstream" fixes that are found to be required for the "STABLE" versions for integration to happen.
Actually I would even think that "STABLE" in each library would be a branch, typically "master" or could easily be just another branch, so changes that are specific to the "STABLE" version stay in that branch.
I am not sure this addresses my concern in any way. Right now, if I have a one-letter typo in my library, I fix it, do "svn commit" and then everybody who checks out trunk a second later gets that fix. With your workflow, it's necessary that: 1. I commit the fix locally 2. I push to a publicly available repository 3. I mark the current state of publically available repository as good for pull 4. Release managers pull into the official master repository. It's about 99.9% probability that if (4) requires manual action, it means my fix won't appear for a month at least. And while you can probably can script (4), the question arises why we need all this mess, as opposed to direct push into the official master repository. I am sure that most Boost developers will prefer this model, as opposed to being forced to "release" their libraries separately.
- In practice, release managers will only pull a week or two before the release, therefore the unified thing we have now will no longer exists, and two weeks before the release we'll get a frankenstein creature that might technically pass all tests (though I doubt that), but won't be tested by any users in real projects.
If everybody had a way of pulling the "latest and greatest" of all the "STABLE" branches of all the libraries they want to use into their own git repo, I don't see what's the difference from having everything in the same place.
Am I missing something?
I might have missed your proposed way for such pulling of everything. Could you spell it again?
So, to reiterate, no matter what tools are proposed, I strongly thing that there should always be UNIFIED in-development tree of ENTIRE BOOST to which all library maintainers can add changes without coordination with release managers.
Everyone can have that locally as a conglomeration of all the libraries they're interested in. I don't see why it has to be just one server.
Why should it be multiple servers? Apparently, having multiple servers is MORE work, not less, and so far, I don't see Boost developers voting for having their libraries hosted elsewhere. - Volodya -- Vladimir Prus Mentor Graphics +7 (812) 677-68-40
On Sun, Jan 30, 2011 at 9:23 PM, Vladimir Prus <vladimir@codesourcery.com> wrote:
Dean Michael Berris wrote:
On Sat, Jan 29, 2011 at 3:55 PM, Vladimir Prus
- Only release managers "pull" into the unified thing
Not really. Every developer can do the same thing that release managers do. To get a full Boost locally, you pull from multiple repositories, and you develop against a "stable" or "tagged" version of each library in your local repository.
Just to clarify, by "unified thing" I mean a specific URL that everybody can use to checkout/clone/fork/whatever a version of boost with all the current changes. You suggestion that I manually pull from N different repositories seems like a mess to me, sorry.
Nope. I still meant cloning from one URL. git has sub-trees and sub-modules support -- think of it as svn:externals only distributed and generally "better". When you say 'git pull' or 'git submodule update', well it does the correct thing and updates the local repo from different places.
- Release managers only pull "releases", which, together with delays added by release managers, means that a fix to a component will be added to the unified thing after considerable delay
Not really either. Release managers can just pull from libraries that have updates, and even do that incrementally.
More to the point, developers willing to work on just getting a stable release out (maybe a stable release team or "support engineers") can contribute to the libraries "upstream" fixes that are found to be required for the "STABLE" versions for integration to happen.
Actually I would even think that "STABLE" in each library would be a branch, typically "master" or could easily be just another branch, so changes that are specific to the "STABLE" version stay in that branch.
I am not sure this addresses my concern in any way. Right now, if I have a one-letter typo in my library, I fix it, do "svn commit" and then everybody who checks out trunk a second later gets that fix. With your workflow, it's necessary that:
1. I commit the fix locally 2. I push to a publicly available repository
This 1 and 2 is similar to your 'svn commit'.
3. I mark the current state of publically available repository as good for pull
Nope, by pushing to the repository that's already done for you.
4. Release managers pull into the official master repository.
It's about 99.9% probability that if (4) requires manual action, it means my fix won't appear for a month at least. And while you can probably can script (4), the question arises why we need all this mess, as opposed to direct push into the official master repository. I am sure that most Boost developers will prefer this model, as opposed to being forced to "release" their libraries separately.
I obviously don't qualify for "Boost developers" I think so I can't really say. But largely from the library developers' perspective, what can happen is that he can just worry about the things he's doing locally and pulling regularly -- because git pulls are pretty much painless even if you have local modifications that aren't in the remote repo that you're pulling from. Less synchronization means very little need for waiting -- the release managers (or whoever for that matter) can absolutely fix any broken local libraries they're dealing with, submit the patches upstream and not worry about whether those patches make it or not because then they have a repository that works locally that they can deal with. If the original maintainer doesn't introduce the patch to his repository the release managers can choose to incorporate it. Instead of a centralized approach, you solve the problem in a distributed approach. So now from the maintainer's perspective, what would happen is he doesn't merge anything to the 'STABLE' branch until he's sufficiently satisfied that it's OK. Now once he does that though, the release managers can choose to: 1) pull everything and trust the maintainer on it -- assuming that it works with other STABLE release branches of dependency libraries -- or 2) revert their local repo to a previous version until failures are fixed in co-existing libraries. If someone -- doesn't need to be a release manager -- wants to fix the whole mess locally in their repos, and then submit the patches upstream to the offending libraries that break each other, then that's largely possible and *seamless* to the whole process. This is what encourages people to fix things locally and submit patches upstream, mainly because they: 1) don't need permission to do this and 2) because it's so painless to do it.
If everybody had a way of pulling the "latest and greatest" of all the "STABLE" branches of all the libraries they want to use into their own git repo, I don't see what's the difference from having everything in the same place.
Am I missing something?
I might have missed your proposed way for such pulling of everything. Could you spell it again?
Git sub-trees and sub-modules.
So, to reiterate, no matter what tools are proposed, I strongly thing that there should always be UNIFIED in-development tree of ENTIRE BOOST to which all library maintainers can add changes without coordination with release managers.
Everyone can have that locally as a conglomeration of all the libraries they're interested in. I don't see why it has to be just one server.
Why should it be multiple servers? Apparently, having multiple servers is MORE work, not less, and so far, I don't see Boost developers voting for having their libraries hosted elsewhere.
Why would it be more work? I don't see why you think it would be more work. If the server choices are github, gitorious, sourceforge git repos, or your own publicly accessible server, who does it have to be more work for? And to that point, if pulling from multiple sources is just one command, why would that even be more work? -- Dean Michael Berris about.me/deanberris
On Jan 30, 2011, at 12:24 PM, Dean Michael Berris wrote:
Why would it be more work? I don't see why you think it would be more work. If the server choices are github, gitorious, sourceforge git repos, or your own publicly accessible server, who does it have to be more work for? And to that point, if pulling from multiple sources is just one command, why would that even be more work?
Aha. I thought there must be some misunderstanding here. Distributed repositories shouldn't mean that everyone has to keep track of dozens of servers. Dean, or other Git advocates, can you also answer Volodya's other big concern? On Jan 30, 2011, at 10:06 AM, Vladimir Prus wrote:
Suppose you have library X with 200 new changes. For next release, it is necessary to include one of those changes. How do you do that? With current release process, it's a single command (obviously, run by a single person).
And regardless, SVN has all the same issues w.r.t. picking individual changes, doesn't it?
No. When you merge a single revision from branch A to branch B, SVN records accurate information about this merge. When you do 'git cherry-pick', git does not record any information whatsoever and is fundementally unable to record it.
On Mon, Jan 31, 2011 at 2:13 AM, Gordon Woodhull <gordon@woodhull.com> wrote:
On Jan 30, 2011, at 12:24 PM, Dean Michael Berris wrote:
Why would it be more work? I don't see why you think it would be more work. If the server choices are github, gitorious, sourceforge git repos, or your own publicly accessible server, who does it have to be more work for? And to that point, if pulling from multiple sources is just one command, why would that even be more work?
Aha. Â I thought there must be some misunderstanding here. Â Distributed repositories shouldn't mean that everyone has to keep track of dozens of servers.
Dean, or other Git advocates, can you also answer Volodya's other big concern?
Let me try...
On Jan 30, 2011, at 10:06 AM, Vladimir Prus wrote:
Suppose you have library X with 200 new changes. For next release, it is necessary to include one of those changes. How do you do that? With current release process, it's a single command (obviously, run by a single person).
Simple: if these 200 changes are in a STABLE branch, then all those are pulled by the release managers. If only one of those 200 changes are to be released by the library developer, then he shouldn't have moved 200 changes into the STABLE branch -- and the release managers can just choose to ignore the 200 changes and just pick the one that "fixes" the issue for the release and be done with it. I don't see *why* this is a big problem. Am I missing something?
And regardless, SVN has all the same issues w.r.t. picking individual changes, doesn't it?
No. When you merge a single revision from branch A to branch B, SVN records accurate information about this merge. When you do 'git cherry-pick', git does not record any information whatsoever and is fundementally unable to record it.
In git, individual changes have two important things that come with it: the patch/changes, and the parent from which these changes apply. If you want to cherry-pick a commit from one branch to another, 'git cherry-pick' is *exactly* the command you use -- and you can even edit the message that gets committed when you do this cherry picking. Now, let's say you finally merge the changes from the whole branch from which you cherry-picked a commit into the branch that has this cherry-picked change already applied, git is actually smart enough to see that the change is already there and would not have to apply it again -- and since the original parent from the cherry-picked commit is recording in the commit for the cherry-pick, that's all transparent for you. Heck in git, even if you have the same change/patch applied in 100 different branches and try to merge them all into a single branch, it's smart enough to see that "oh, these patches look alike so I don't really need to apply them over each other and just pick one". This is what dela-merging and delta-fingerprints do that is *not* there in Subversion. http://www.kernel.org/pub/software/scm/git/docs/git-cherry-pick.html http://progit.org/book/ch5-3.html -- Dean Michael Berris about.me/deanberris
Dean Michael Berris wrote:
Now, let's say you finally merge the changes from the whole branch from which you cherry-picked a commit into the branch that has this cherry-picked change already applied, git is actually smart enough to see that the change is already there and would not have to apply it again
Sorry, this is false. If you have branch F with commits C1, C2 and C3, and you first cherry-pick C2 into M, and then merge F into M, then git will try to merge entire C1+C2+C3 delta. Therefore, you have modified, in M, any lines that C2 touches -- for examples because you've run into conflict when doing cherry-pick and changed something -- you will get a conflict, again. Please try the attached script -- which results in conflict with git 1.7.1, and used to result in conflict in every other version. Am I missing something? - Volodya -- Vladimir Prus Mentor Graphics +7 (812) 677-68-40 #!/bin/bash rm -rf /tmp/merge.git mkdir /tmp/merge.git cd /tmp/merge.git git init for i in {1..100}; do echo A$i >> file; done git add file git commit -m "Initial version" git checkout -b feature perl -pi -e 's|A1|A1_|' file git commit -a -m "C1" perl -pi -e 's|A50|A50_|' file git commit -a -m "C2" perl -pi -e 's|A90|A90_|' file git commit -a -m "C2" git checkout master git cherry-pick feature^ perl -pi -e 's|A50_|A50__|' file git commit -a -m "Do something after cherry-pick" git merge feature
On Mon, Jan 31, 2011 at 3:07 AM, Vladimir Prus <vladimir@codesourcery.com> wrote:
Dean Michael Berris wrote:
Now, let's say you finally merge the changes from the whole branch from which you cherry-picked a commit into the branch that has this cherry-picked change already applied, git is actually smart enough to see that the change is already there and would not have to apply it again
Sorry, this is false. If you have branch F with commits C1, C2 and C3, and you first cherry-pick C2 into M, and then merge F into M, then git will try to merge entire C1+C2+C3 delta. Therefore, you have modified, in M, any lines that C2 touches -- for examples because you've run into conflict when doing cherry-pick and changed something -- you will get a conflict, again. Please try the attached script -- which results in conflict with git 1.7.1, and used to result in conflict in every other version.
Am I missing something?
You might be... I had to modify it a bit, because 'feature^' is really not correct, but basically here is the new script (based on yours); note that I see no merge conflicts whatsoever and git doesn't complain. Are you sure this is supposed to fail because it doesn't fail from my end -- I'm using git 1.7.1 on Ubuntu Linux 10.10 64-bit... -- Dean Michael Berris about.me/deanberris
Dean Michael Berris wrote:
On Mon, Jan 31, 2011 at 3:07 AM, Vladimir Prus <vladimir@codesourcery.com> wrote:
Dean Michael Berris wrote:
Now, let's say you finally merge the changes from the whole branch from which you cherry-picked a commit into the branch that has this cherry-picked change already applied, git is actually smart enough to see that the change is already there and would not have to apply it again
Sorry, this is false. If you have branch F with commits C1, C2 and C3, and you first cherry-pick C2 into M, and then merge F into M, then git will try to merge entire C1+C2+C3 delta. Therefore, you have modified, in M, any lines that C2 touches -- for examples because you've run into conflict when doing cherry-pick and changed something -- you will get a conflict, again. Please try the attached script -- which results in conflict with git 1.7.1, and used to result in conflict in every other version.
Am I missing something?
You might be... I had to modify it a bit, because 'feature^' is really not correct,
Uhm, why? feature is a refname that resolves to C3 (top commit on the 'feature' branch), feature^ is therefore C2. And 'git cherry-pick feature^' appears to work fine here.
but basically here is the new script (based on yours); note that I see no merge conflicts whatsoever and git doesn't complain. Are you sure this is supposed to fail because it doesn't fail from my end -- I'm using git 1.7.1 on Ubuntu Linux 10.10 64-bit...
Well, your new script cherry-picks C3, not C2, and since other bits are designed to generate conflict on C2, it's no wonder you get no conflict. - Volodya -- Vladimir Prus Mentor Graphics +7 (812) 677-68-40
On Mon, Jan 31, 2011 at 4:16 AM, Vladimir Prus <vladimir@codesourcery.com> wrote:
Dean Michael Berris wrote:
On Mon, Jan 31, 2011 at 3:07 AM, Vladimir Prus <vladimir@codesourcery.com> wrote:
Am I missing something?
You might be... I had to modify it a bit, because 'feature^' is really not correct,
Uhm, why? feature is a refname that resolves to C3 (top commit on the 'feature' branch), feature^ is therefore C2. And 'git cherry-pick feature^' appears to work fine here.
Right, I had to run your script with chmod +x directly so that it actually works locally. Silly me trying to do 'sh script.sh'.
but basically here is the new script (based on yours); note that I see no merge conflicts whatsoever and git doesn't complain. Are you sure this is supposed to fail because it doesn't fail from my end -- I'm using git 1.7.1 on Ubuntu Linux 10.10 64-bit...
Well, your new script cherry-picks C3, not C2, and since other bits are designed to generate conflict on C2, it's no wonder you get no conflict.
Right. So I ran it and saw you had a conflict. Now looking at this article (after a quick googling) : http://davitenio.wordpress.com/2008/09/27/git-merge-after-git-cherry-pick-av... -- it explains one way of doing it. Basically you rebase first before you merge (which works for the case you point out). Attached is the script that is modified to address the specific issue you raise. HTH PS. Of course this only makes sense on repositories you actually control -- i.e. locally. If you're tracking a remote branch, this could screw up your local branch -- in those cases it's best that you actually have a branch from the remote tracking branch and rebase that before you actually merge back into the "main" branch from which the remote tracking branch is forked off. Your branch and the remote branch can diverge (which actually happens quite a bit with git) but with proper re-alignment procedures -- i.e. pulling or making sure your changes are upstream -- these things can largely be avoided. -- Dean Michael Berris about.me/deanberris
Dean Michael Berris wrote:
On Mon, Jan 31, 2011 at 4:16 AM, Vladimir Prus <vladimir@codesourcery.com> wrote:
Dean Michael Berris wrote:
On Mon, Jan 31, 2011 at 3:07 AM, Vladimir Prus <vladimir@codesourcery.com> wrote:
Am I missing something?
You might be... I had to modify it a bit, because 'feature^' is really not correct,
Uhm, why? feature is a refname that resolves to C3 (top commit on the 'feature' branch), feature^ is therefore C2. And 'git cherry-pick feature^' appears to work fine here.
Right, I had to run your script with chmod +x directly so that it actually works locally. Silly me trying to do 'sh script.sh'.
but basically here is the new script (based on yours); note that I see no merge conflicts whatsoever and git doesn't complain. Are you sure this is supposed to fail because it doesn't fail from my end -- I'm using git 1.7.1 on Ubuntu Linux 10.10 64-bit...
Well, your new script cherry-picks C3, not C2, and since other bits are designed to generate conflict on C2, it's no wonder you get no conflict.
Right. So I ran it and saw you had a conflict. Now looking at this article (after a quick googling) : http://davitenio.wordpress.com/2008/09/27/git-merge-after-git-cherry-pick-av... -- it explains one way of doing it. Basically you rebase first before you merge (which works for the case you point out). ... PS. Of course this only makes sense on repositories you actually control -- i.e. locally.
Exactly, so this does not work for the case of cherry-pick from long-lived public development branch of library X to release branch. Now, this use of git rebase appears to contradict my earlier statements that git cannot record cherry-pick merges, so I went to investigate, and, unfortunately, discovered a bug in my script. Instead of simulating the 'I did cherry-pick, found conflicts, and edited them' use case it simulates 'I did cherry-pick, and then edited the same lines in another commit' use case. So, I've modified the script to do the right thing (see below), and rebase trick no longer works. In fact, rebase itself causes a conflict. Looks like git cherry-pick is still deficient. - Volodya -- Vladimir Prus Mentor Graphics +7 (812) 677-68-40 #!/bin/bash set -e rm -rf /tmp/merge.git mkdir /tmp/merge.git cd /tmp/merge.git git init for i in {1..100}; do echo A$i >> file; done git add file git commit -m "Initial version" git checkout -b feature perl -pi -e 's|A1|A1_|' file git commit -a -m "C1" perl -pi -e 's|A50|A50_|' file git commit -a -m "C2" perl -pi -e 's|A90|A90_|' file git commit -a -m "C3" git checkout master # Cherry-pick C2. To simulate conflict resolution, don't # commit it yet. git cherry-pick -n feature^ # Edit C2, as if we had a conflict and resolved it perl -pi -e 's|A50_|A50__|' file # Commit the result. git commit -a -C feature^ # Do irrelevant change. perl -pi -e 's|A65|A65_|' file git commit -a -m "Do something after cherry-pick" # Uncomment to try to use 'rebase' trick # git checkout feature # git rebase master # Try to merge feature into master git checkout master git merge -s resolve feature
Vladimir Prus wrote:
Right. So I ran it and saw you had a conflict. Now looking at this article (after a quick googling) : http://davitenio.wordpress.com/2008/09/27/git-merge-after-git-cherry-pick-av... -- it explains one way of doing it. Basically you rebase first before you merge (which works for the case you point out). ... PS. Of course this only makes sense on repositories you actually control -- i.e. locally.
Exactly, so this does not work for the case of cherry-pick from long-lived public development branch of library X to release branch.
Now, this use of git rebase appears to contradict my earlier statements that git cannot record cherry-pick merges, so I went to investigate, and, unfortunately, discovered a bug in my script. Instead of simulating the 'I did cherry-pick, found conflicts, and edited them' use case it simulates 'I did cherry-pick, and then edited the same lines in another commit' use case. So, I've modified the script to do the right thing (see below), and rebase trick no longer works. In fact, rebase itself causes a conflict.
Looks like git cherry-pick is still deficient.
So, we spoke with Dean on IRC. His suggestion was to use git merge -s recursive -X ours for merging, but that would not work either -- it will prefer local versions for all conflics -- not something you want during regular development. No better ideas exist. So, to summarize, if I have a trunk branch and a release branch, Subversion allows me to pick a change on trunk branch, merge it to release branch, resolve any conflicts that arise, and forget about it, knowing that when I get to doing wholesale merge of the trunk branch, Subversion will not try to re-apply that change again. No similar mechanism for git has been demonstrated. If I do cherry-pick and get a conflict, then even if I resolve that conflict and is happy, there's no way to instruct git not to merge that revision again. - Volodya -- Vladimir Prus Mentor Graphics +7 (812) 677-68-40
On 30 Jan 2011, at 21:05, Vladimir Prus wrote:
Exactly, so this does not work for the case of cherry-pick from long-lived public development branch of library X to release branch.
Now, this use of git rebase appears to contradict my earlier statements that git cannot record cherry-pick merges, so I went to investigate, and, unfortunately, discovered a bug in my script. Instead of simulating the 'I did cherry-pick, found conflicts, and edited them' use case it simulates 'I did cherry-pick, and then edited the same lines in another commit' use case. So, I've modified the script to do the right thing (see below), and rebase trick no longer works. In fact, rebase itself causes a conflict.
Looks like git cherry-pick is still deficient.
I agree this is a problem with git. I think the problem is having a "trunk" which is merged piece-meal into a release branch. This is an unusual method of development, I'm not sure of any other large projects which have the a long-lived trunk and release, with merges from trunk to release. git does not support this well. Of course, boost should not change it's development model just to support a different tool. But, if the development model did change, git might accommodate that better. Chris
At Sun, 30 Jan 2011 23:14:53 +0000, Christopher Jefferson wrote:
On 30 Jan 2011, at 21:05, Vladimir Prus wrote:
Exactly, so this does not work for the case of cherry-pick from long-lived public development branch of library X to release branch.
Now, this use of git rebase appears to contradict my earlier statements that git cannot record cherry-pick merges, so I went to investigate, and, unfortunately, discovered a bug in my script. Instead of simulating the 'I did cherry-pick, found conflicts, and edited them' use case it simulates 'I did cherry-pick, and then edited the same lines in another commit' use case. So, I've modified the script to do the right thing (see below), and rebase trick no longer works. In fact, rebase itself causes a conflict.
Looks like git cherry-pick is still deficient.
I agree this is a problem with git. I think the problem is having a "trunk" which is merged piece-meal into a release branch. This is an unusual method of development, I'm not sure of any other large projects which have the a long-lived trunk and release, with merges from trunk to release. git does not support this well.
Actually it does; if all you ever do with Git is merge stuff, then you're golden. What Git doesn't support well is something else entirely... I just got the low-down on #git and it was definitely an eye-opener. Basically, there's a rule (maybe even an "ethic") that you're supposed to follow with Git, and if you don't, you'll have this kind of trouble eventually. IIUC, the ethic is that you don't keep the same "logical change" around with two different commit histories (and sharing a logical change with other people is considered "keeping it around," because you've lost control of its lifetime. Thus: * Rewriting history should only be done on private commits, *and then you should throw away your reference to the old history so you don't get yourself in trouble* * You never rebase or cherry-pick commits from a _public_ feature branch onto another public branch * You never squash public commits onto another public branch. Because each commit is a unique object, and its history is part of its identity, doing any of these things is considered to be almost like lying about history. The people on #git that I talked to were pretty hard-line about this, and said that if you buy into the ethic you almost never need to break it. Instead of cherry picking a fix off the publically-visible 1.43 release branch for incorporation into the current trunk, what you *should* do (according to this ethic) is merge the 1.43 branch into trunk. Now, to make this work in practice you need to accept a slightly flexible meaning of "public" and "private." That is, for a number of purposes you may need to show people commits that they should expect to get rewritten, and you need to label these as "not for public consumption." For example, if you want someone to pull from your repository, and there's any chance they might look at your changes and say "I like all of them except that one in the middle," the branch you publish should be marked "volatile", so that people know to expect those same logical changes might reappear in a different commit history. You can actually name the branch "volatile/myfeature" if you want! If the upstream maintainer rejects your changes, you can rewrite them and publish them as volatile again, until they are finally merged. So, to reiterate, what Git doesn't support well is _maintaining_ the same logical change in two different commit histories. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On Sun, Jan 30, 2011 at 11:14:53PM +0000, Christopher Jefferson wrote:
I think the problem is having a "trunk" which is merged piece-meal into a release branch. This is an unusual method of development, I'm not sure of any other large projects which have the a long-lived trunk and release, with merges from trunk to release. git does not support this well.
It's not just git, in my experience, but humans also have trouble with this model. One of the main problems being that bugs are fixed in the trunk then languish, forgotten, while Boost is released with the same bugs, sometimes for more than one release. -Steve
AMDG On 1/30/2011 7:30 PM, Steve M. Robbins wrote:
On Sun, Jan 30, 2011 at 11:14:53PM +0000, Christopher Jefferson wrote:
I think the problem is having a "trunk" which is merged piece-meal into a release branch. This is an unusual method of development, I'm not sure of any other large projects which have the a long-lived trunk and release, with merges from trunk to release. git does not support this well.
It's not just git, in my experience, but humans also have trouble with this model. One of the main problems being that bugs are fixed in the trunk then languish, forgotten, while Boost is released with the same bugs, sometimes for more than one release.
I don't deny that the current system has problems, but it does exist for a good reason. Does anyone remember how long it took to release 1.34? In Christ, Steven Watanabe
On Mon, Jan 31, 2011 at 11:55 AM, Steven Watanabe <watanabesj@gmail.com> wrote:
AMDG
On 1/30/2011 7:30 PM, Steve M. Robbins wrote:
On Sun, Jan 30, 2011 at 11:14:53PM +0000, Christopher Jefferson wrote:
I think the problem is having a "trunk" which is merged piece-meal into a release branch. This is an unusual method of development, I'm not sure of any other large projects which have the a long-lived trunk and release, with merges from trunk to release. git does not support this well.
It's not just git, in my experience, but humans also have trouble with this model. Â One of the main problems being that bugs are fixed in the trunk then languish, forgotten, while Boost is released with the same bugs, sometimes for more than one release.
I don't deny that the current system has problems, but it does exist for a good reason. Â Does anyone remember how long it took to release 1.34?
I do remember that. Back then I don't think git even existed yet though or wasn't getting much usage outside of the Linux development community -- FWIW It's only been around since 2005. IMO, if back then (or especially now) libraries had their own repos to begin with and published "stable" versions and had people stabilizing each library in the stable version, then pulling together a release would largely be a "controlled" process instead of having to wait for everyone to agree that it's ready to release. Really a good example of how this works on a large scale is looking at how Linux distributions are developed. In a Linux distribution every included library, application, and kernel have their own repository -- these are called the upstream repositories -- and the distribution developers maintain a fork of these libraries/applications/kernel. Changes local to the "downstream" repositories (things like packaging, special changes local to the distribution, etc.) are sometimes submitted upstream to the independent project repositories but that means the distribution can be released with the patches local to that distribution. For example RedHat has its own repo's of glibc, gcc, the Linux kernel, etc. which have their own changes that may or may not be merged upstream (but usually the quality of these patches are so good that they make it to the projects up stream, or the maintainers are actually employed by RedHat that it's largely not too much of an issue). Distributions like Ubuntu also do the same thing. It's short of a minor miracle how Linux distributions actually get released with a regular cadence and with high enough quality using this system. What enables it is not just the process and policies but the tools used to support the process and policies better. I hate to sound like a used car salesman but really you should try try it for yourself. ;) -- Dean Michael Berris about.me/deanberris
On Sun, Jan 30, 2011 at 07:55:10PM -0800, Steven Watanabe wrote:
AMDG
On 1/30/2011 7:30 PM, Steve M. Robbins wrote:
On Sun, Jan 30, 2011 at 11:14:53PM +0000, Christopher Jefferson wrote:
I think the problem is having a "trunk" which is merged piece-meal into a release branch. This is an unusual method of development, I'm not sure of any other large projects which have the a long-lived trunk and release, with merges from trunk to release. git does not support this well.
It's not just git, in my experience, but humans also have trouble with this model. One of the main problems being that bugs are fixed in the trunk then languish, forgotten, while Boost is released with the same bugs, sometimes for more than one release.
I don't deny that the current system has problems, but it does exist for a good reason.
I'm sure you're right. Moreover, upon reflection, I believe my issue can be fixed in ways other than changing the merging model. For example, one could have an extra state in the issue tracker: open -> fixed-in-trunk -> fixed-in-release. Ideally, the latter transition could be made automatically when the change set containing the fix is merged. Cheers, -Steve
On Sun, Jan 30, 2011 at 6:14 PM, Christopher Jefferson <chris@bubblescope.net> wrote:
... I think the problem is having a "trunk" which is merged piece-meal into a release branch. This is an unusual method of development, I'm not sure of any other large projects which have the a long-lived trunk and release, with merges from trunk to release. git does not support this well.
Of course, boost should not change it's development model just to support a different tool. But, if the development model did change, git might accommodate that better.
We use the "trunk + release" model because of historical limitations. Hopefully we can retire the "trunk" part of the model soon, yet still be able to widely test libraries before any merging to the "release" part of the model. For example, see https://github.com/ryppl/ryppl/wiki/Test-and-Test-Reporting-Wish-List --Beman
At Sat, 29 Jan 2011 10:55:21 +0300, Vladimir Prus wrote:
Dave Abrahams wrote:
However, my vision for Boost is a bit different:
* each library has its own Git repo
* each library maintainer tests and produces versioned releases of that library on his/her own timetable
* the latest release of a library is tagged "STABLE"
* When assembling a new Boost release, the release manager marks the "STABLE" revision from each library and begins boost-wide integration/interoperability testing on those revisions
I am rather sceptical about this approach. The goodness of what we have now is that there's a single trunk that holds code that is:
- supposedly not worse in test results in all other code, - is the newest one satisfying the above constraint
First of all, I don't think there's general agreement that trunk should hold code "supposedly not worse in test results." That's one reason our trunk test results are not all that useful. Some people like to use trunk for "TDD," which apparently means checking in tests that are known to be broken and fixing them incrementally. Secondly, I don't see having "a single trunk" with these properties as bringing with it any particular "goodness." If you're looking for a fix you still need to know which code to look at. So just go to that library's GitHub repo.
Therefore, if I wish to make sure my code works with new release of Boost, of if I wish to check if some bug is fixed, that is the place to go.
But in reality, it doesn't tell you anything about the next release of Boost, because things don't always get moved from trunk to release. Trunk
Further, this place is jointly updated by every boost developer.
That's a problem right there, not an advantage.
What you propose breaks this useful thing, because:
- Only release managers "pull" into the unified thing
Not really. The individual library developers determined which versions will be *automatically* pulled into the unified thing by tagging their individual releases as STABLE.
- Release managers only pull "releases",
Which can be much more frequent than what we do at Boost.
which, together with delays added by release managers, means that a fix to a component will be added to the unified thing after considerable delay
I don't see where the delay comes from.
- In practice, release managers will only pull a week or two before - the release,
No, a collection of the latest STABLE versions will be continuously and automatically tested.
therefore the unified thing we have now will no longer exists, and two weeks before the release we'll get a frankenstein creature that might technically pass all tests (though I doubt that), but won't be tested by any users in real projects.
Welcome to the real world. That's how every OS distribution and every other collection of semi-independent modules works, and works well. No sane user of a real project is going to slow down his development by regularly testing against an unSTABLE Boost trunk, and that's increasingly true as Boost grows.
So, to reiterate, no matter what tools are proposed, I strongly thing that there should always be UNIFIED in-development tree of ENTIRE BOOST to which all library maintainers can add changes without coordination with release managers.
I would want to opt out of adding changes to such a tree because it seems to me like a giant waste of everybody's time. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
Dave Abrahams wrote:
At Sat, 29 Jan 2011 10:55:21 +0300, Vladimir Prus wrote:
Dave Abrahams wrote:
However, my vision for Boost is a bit different:
* each library has its own Git repo
* each library maintainer tests and produces versioned releases of that library on his/her own timetable
* the latest release of a library is tagged "STABLE"
* When assembling a new Boost release, the release manager marks the "STABLE" revision from each library and begins boost-wide integration/interoperability testing on those revisions
I am rather sceptical about this approach. The goodness of what we have now is that there's a single trunk that holds code that is:
- supposedly not worse in test results in all other code, - is the newest one satisfying the above constraint
First of all, I don't think there's general agreement that trunk should hold code "supposedly not worse in test results." That's one reason our trunk test results are not all that useful. Some people like to use trunk for "TDD," which apparently means checking in tests that are known to be broken and fixing them incrementally.
That seems like violating basic principles of software development. Maybe, stricter published guidelines, together with automatic integration testing and nagging and reverting, is what we need here.
Secondly, I don't see having "a single trunk" with these properties as bringing with it any particular "goodness." If you're looking for a fix you still need to know which code to look at. So just go to that library's GitHub repo.
What if you don't know which library is that? Say, if you use multiindex, serialization autolink kicks in. As a developer who could not care less about all this "Boost C++ Libraries" thing, you probably want to just get in-development version to see if either library got fixed, before looking at the code for real.
Therefore, if I wish to make sure my code works with new release of Boost, of if I wish to check if some bug is fixed, that is the place to go.
But in reality, it doesn't tell you anything about the next release of Boost, because things don't always get moved from trunk to release.
That's the consequence of our non-standard release process. All other software projects create release branches off trunk, and Boost is the only one I know who has long-lived release branch. I though it was a conscious decision, but all the recent discussions makes me doubt that, so maybe we should briefly discuss relative merits. 1. The 'make release branch off trunk' process is good for all purposes, really, except it requires that at some point in time, trunk is in good shape. Historically, for lack of tools and release management resources, we often could not get trunk in good shape quick. 2. Ours 'merge to release branch' process has the advantage that even if somebody breaks library X in trunk completely, he might just forget to merge it to release. So, we get improved stability, at the cost of slow progress. It's often the case that somebody forgets to merge changes to release branch, especially for patches and fixes applied outside of one's official libraries. So, in comparison, to achieve the same rate of changes and quality, (1) requires discipline with commits and checking test results, while (2) requires the same, and additional dancing and coordination with merges. So, (2) is strictly more complex and error prone, and should only be done if specific maintainers want separate branches for their components.
Further, this place is jointly updated by every boost developer.
That's a problem right there, not an advantage.
Ok, we disagree here.
What you propose breaks this useful thing, because:
- Only release managers "pull" into the unified thing
Not really. The individual library developers determined which versions will be *automatically* pulled into the unified thing by tagging their individual releases as STABLE.
There are two problems here: 1. Just like library developers forget to merge to release, they will forget to tag those 'releases' 2. Because 'git cherry-pick' is fundamentally unable to record any mergeinfo, this means that any time you want to merge a single specific fix to release branch, you would have problems.
- Release managers only pull "releases",
Which can be much more frequent than what we do at Boost.
Oh. I think everybody thinks that our current rate of releases is almost-too-often already.
which, together with delays added by release managers, means that a fix to a component will be added to the unified thing after considerable delay
I don't see where the delay comes from.
Either release managers look at the thing they merge (which adds considerable delay), or they don't, in which case direct push to release branch is more efficient.
therefore the unified thing we have now will no longer exists, and two weeks before the release we'll get a frankenstein creature that might technically pass all tests (though I doubt that), but won't be tested by any users in real projects.
Welcome to the real world. That's how every OS distribution and every other collection of semi-independent modules works, and works well. No sane user of a real project is going to slow down his development by regularly testing against an unSTABLE Boost trunk, and that's increasingly true as Boost grows.
Strangely, I know of other examples. Say, KDE, which is zillion times bigger than Boost, has relatively small number of separate git repositories, whereas most of the core (which is half-zillion times bigger than Boost), lives in two modules, which contain wildly different functionality, are updated by multiple people, and then built and used and tested by users all the time. So, this proves that Boost has a lot of grows ahead before having all code in place place will becomes, even in theory, a problem. - Volodya -- Vladimir Prus Mentor Graphics +7 (812) 677-68-40
At Sun, 30 Jan 2011 16:43:17 +0300, Vladimir Prus wrote:
Dave Abrahams wrote:
First of all, I don't think there's general agreement that trunk should hold code "supposedly not worse in test results." That's one reason our trunk test results are not all that useful. Some people like to use trunk for "TDD," which apparently means checking in tests that are known to be broken and fixing them incrementally.
That seems like violating basic principles of software development.
It depends whose "basic principles" you subscribe to, apparently.
Maybe, stricter published guidelines, together with automatic integration testing and nagging and reverting, is what we need here.
Secondly, I don't see having "a single trunk" with these properties as bringing with it any particular "goodness." If you're looking for a fix you still need to know which code to look at. So just go to that library's GitHub repo.
What if you don't know which library is that?
A big monolithic repo has the same problem.
Therefore, if I wish to make sure my code works with new release of Boost, of if I wish to check if some bug is fixed, that is the place to go.
But in reality, it doesn't tell you anything about the next release of Boost, because things don't always get moved from trunk to release.
That's the consequence of our non-standard release process. All other software projects create release branches off trunk, and Boost is the only one I know who has long-lived release branch. I though it was a conscious decision, but all the recent discussions makes me doubt that, so maybe we should briefly discuss relative merits.
1. The 'make release branch off trunk' process is good for all purposes, really, except it requires that at some point in time, trunk is in good shape. Historically, for lack of tools and release management resources, we often could not get trunk in good shape quick.
And that requires far too much coordination to be practical on a project of this size.
2. Ours 'merge to release branch' process has the advantage that even if somebody breaks library X in trunk completely, he might just forget to merge it to release. So, we get improved stability, at the cost of slow progress. It's often the case that somebody forgets to merge changes to release branch, especially for patches and fixes applied outside of one's official libraries.
So, in comparison, to achieve the same rate of changes and quality, (1) requires discipline with commits and checking test results, while (2) requires the same, and additional dancing and coordination with merges. So, (2) is strictly more complex and error prone, and should only be done if specific maintainers want separate branches for their components.
Anyone can make a separate branch at any time no matter which procedure is used.
What you propose breaks this useful thing, because:
- Only release managers "pull" into the unified thing
Not really. The individual library developers determined which versions will be *automatically* pulled into the unified thing by tagging their individual releases as STABLE.
There are two problems here:
1. Just like library developers forget to merge to release, they will forget to tag those 'releases'
That's totally up to the library developer. Library developers generally care Boost contains an up-to-date version of their libraries, so they will be motivated to remember.
2. Because 'git cherry-pick' is fundamentally unable to record any mergeinfo, this means that any time you want to merge a single specific fix to release branch, you would have problems.
What release branch? Who said anything about cherry-pick? What problems? And regardless, SVN has all the same issues w.r.t. picking individual changes, doesn't it?
- Release managers only pull "releases",
Which can be much more frequent than what we do at Boost.
Oh. I think everybody thinks that our current rate of releases is almost-too-often already.
Now it seems like you're just looking for a snappy refutation without really thinking. Surely you realize that the release rate for individual libraries does not have to affect the release rate for Boost?
which, together with delays added by release managers, means that a fix to a component will be added to the unified thing after considerable delay
I don't see where the delay comes from.
Either release managers look at the thing they merge (which adds considerable delay), or they don't, in which case direct push to release branch is more efficient.
They don't look, unless there are test failures or other problems. And they don't merge. At most we're talking about updating submodule references.
therefore the unified thing we have now will no longer exists, and two weeks before the release we'll get a frankenstein creature that might technically pass all tests (though I doubt that), but won't be tested by any users in real projects.
Welcome to the real world. That's how every OS distribution and every other collection of semi-independent modules works, and works well. No sane user of a real project is going to slow down his development by regularly testing against an unSTABLE Boost trunk, and that's increasingly true as Boost grows.
Strangely, I know of other examples. Say, KDE, which is zillion times bigger than Boost, has relatively small number of separate git repositories,
They're trying to fix that, IIUC.
whereas most of the core (which is half-zillion times bigger than Boost), lives in two modules, which contain wildly different functionality, are updated by multiple people, and then built and used and tested by users all the time. So, this proves that Boost has a lot of grows ahead before having all code in place place will becomes, even in theory, a problem.
The problem isn't where the code is located, it's: a. how much coordination is required to get things done b. the mental drag of having to deal with all the parts of Boost that simply aren't relevant to you. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
Dave Abrahams wrote:
At Sun, 30 Jan 2011 16:43:17 +0300, Vladimir Prus wrote:
Dave Abrahams wrote:
First of all, I don't think there's general agreement that trunk should hold code "supposedly not worse in test results." That's one reason our trunk test results are not all that useful. Some people like to use trunk for "TDD," which apparently means checking in tests that are known to be broken and fixing them incrementally.
That seems like violating basic principles of software development.
It depends whose "basic principles" you subscribe to, apparently.
Of course -- it could be there are projects were you can commit broken code to trunk any time you like, and everybody is fine with that -- and it's just me who don't have enough experience to have met such projects. (FAOD, "trunk" is not specific to any version control system, this is just code that everybody is working on together to get shipped).
Maybe, stricter published guidelines, together with automatic integration testing and nagging and reverting, is what we need here.
Secondly, I don't see having "a single trunk" with these properties as bringing with it any particular "goodness." If you're looking for a fix you still need to know which code to look at. So just go to that library's GitHub repo.
What if you don't know which library is that?
A big monolithic repo has the same problem.
Not really -- you need a single checkout operation to find whether the problem is fixed or not with 99% probability. And only then, to investigate further.
Therefore, if I wish to make sure my code works with new release of Boost, of if I wish to check if some bug is fixed, that is the place to go.
But in reality, it doesn't tell you anything about the next release of Boost, because things don't always get moved from trunk to release.
That's the consequence of our non-standard release process. All other software projects create release branches off trunk, and Boost is the only one I know who has long-lived release branch. I though it was a conscious decision, but all the recent discussions makes me doubt that, so maybe we should briefly discuss relative merits.
1. The 'make release branch off trunk' process is good for all purposes, really, except it requires that at some point in time, trunk is in good shape. Historically, for lack of tools and release management resources, we often could not get trunk in good shape quick.
And that requires far too much coordination to be practical on a project of this size.
I don't think this statement has been proven.
2. Ours 'merge to release branch' process has the advantage that even if somebody breaks library X in trunk completely, he might just forget to merge it to release. So, we get improved stability, at the cost of slow progress. It's often the case that somebody forgets to merge changes to release branch, especially for patches and fixes applied outside of one's official libraries.
So, in comparison, to achieve the same rate of changes and quality, (1) requires discipline with commits and checking test results, while (2) requires the same, and additional dancing and coordination with merges. So, (2) is strictly more complex and error prone, and should only be done if specific maintainers want separate branches for their components.
Anyone can make a separate branch at any time no matter which procedure is used.
And? (2) still requires more effort.
What you propose breaks this useful thing, because:
- Only release managers "pull" into the unified thing
Not really. The individual library developers determined which versions will be automatically pulled into the unified thing by tagging their individual releases as STABLE.
There are two problems here:
1. Just like library developers forget to merge to release, they will forget to tag those 'releases'
That's totally up to the library developer. Library developers generally care Boost contains an up-to-date version of their libraries, so they will be motivated to remember.
"will"? In practice, right now, developers do forget. And again, there are often commits made by those who are not official maintaners of library X (because X might not even have maintainers). The chances of such changes of falling through are even higher.
2. Because 'git cherry-pick' is fundamentally unable to record any mergeinfo, this means that any time you want to merge a single specific fix to release branch, you would have problems.
What release branch? Who said anything about cherry-pick? What problems?
Suppose you have library X with 200 new changes. For next release, it is necessary to include one of those changes. How do you do that? With current release process, it's a single command (obviously, run by a single person).
And regardless, SVN has all the same issues w.r.t. picking individual changes, doesn't it?
No. When you merge a single revision from branch A to branch B, SVN records accurate information about this merge. When you do 'git cherry-pick', git does not record any information whatsoever and is fundementally unable to record it.
- Release managers only pull "releases",
Which can be much more frequent than what we do at Boost.
Oh. I think everybody thinks that our current rate of releases is almost-too-often already.
Now it seems like you're just looking for a snappy refutation without really thinking.
Of course, that's precisely what I am doing all the time.
Surely you realize that the release rate for individual libraries does not have to affect the release rate for Boost?
First, in "release managers only pull releases", above, we are surely talking about Boost release managers? Second, could you name, exactly, all libraries whose developers expressed a need for individual releases? I know of one.
which, together with delays added by release managers, means that a fix to a component will be added to the unified thing after considerable delay
I don't see where the delay comes from.
Either release managers look at the thing they merge (which adds considerable delay), or they don't, in which case direct push to release branch is more efficient.
They don't look, unless there are test failures or other problems. And they don't merge. At most we're talking about updating submodule references.
Which still has to be done manually?
therefore the unified thing we have now will no longer exists, and two weeks before the release we'll get a frankenstein creature that might technically pass all tests (though I doubt that), but won't be tested by any users in real projects.
Welcome to the real world. That's how every OS distribution and every other collection of semi-independent modules works, and works well. No sane user of a real project is going to slow down his development by regularly testing against an unSTABLE Boost trunk, and that's increasingly true as Boost grows.
Strangely, I know of other examples. Say, KDE, which is zillion times bigger than Boost, has relatively small number of separate git repositories,
They're trying to fix that, IIUC.
Oh? It does not seem like any split of kdelibs is immediately forthcoming, or even discussed with such activity as we're having here. Maybe, that suggest that we're actually solving wrong problem?
whereas most of the core (which is half-zillion times bigger than Boost), lives in two modules, which contain wildly different functionality, are updated by multiple people, and then built and used and tested by users all the time. So, this proves that Boost has a lot of grows ahead before having all code in place place will becomes, even in theory, a problem.
The problem isn't where the code is located, it's:
a. how much coordination is required to get things done
Yes. And given that right now, coordination basically boils down to library authors merging their things before deadline (or not merging), and given that merge is a single command, it's not clear to me how you are improving on that.
b. the mental drag of having to deal with all the parts of Boost that simply aren't relevant to you.
What do you mean by mental drag? Having extra directories in your checkout? - Volodya Vladimir Prus Mentor Graphics +7 (812) 677-68-40
AMDG On 1/30/2011 7:06 AM, Vladimir Prus wrote:
Dave Abrahams wrote:
At Sun, 30 Jan 2011 16:43:17 +0300, Vladimir Prus wrote:
2. Because 'git cherry-pick' is fundamentally unable to record any mergeinfo, this means that any time you want to merge a single specific fix to release branch, you would have problems.
What release branch? Who said anything about cherry-pick? What problems?
Suppose you have library X with 200 new changes. For next release, it is necessary to include one of those changes. How do you do that? With current release process, it's a single command (obviously, run by a single person).
And regardless, SVN has all the same issues w.r.t. picking individual changes, doesn't it?
No. When you merge a single revision from branch A to branch B, SVN records accurate information about this merge. When you do 'git cherry-pick', git does not record any information whatsoever and is fundementally unable to record it.
I cherry-pick changesets to merge to the release branch on a pretty regular basis and it works fine with svn. If git really doesn't support this well, you can count me as strongly against switching. In Christ, Steven Watanabe
At Sun, 30 Jan 2011 11:34:52 -0800, Steven Watanabe wrote:
I cherry-pick changesets to merge to the release branch on a pretty regular basis and it works fine with svn. If git really doesn't support this well, you can count me as strongly against switching.
Git supports it "pretty well" because its merge algorithms are pretty smart (I've never had a problem in practice), but it can't do as well as SVN does for the particular use case Volodya has mentioned. For everyone's reference, here's where he ended up: Commits in HEAD 4faabacc * master Do something after cherry-pick 0468cdd0 * C2 f5b6b987 | * feature C3 fb0de3f5 | * C2 7c8f9509 | * C1 |/ 04e628f9 * Initial version The last commit on master changed lines that were also changed by C2. Because 0468cdd0 and fb0de3f5 (the two C2's) are separate commits, Git has no way to recognize that fb0de3f5 has been incorporated into 4faabacc, so it will try to make those changes again when "feature" is merged into "master." -- Dave Abrahams BoostPro Computing http://www.boostpro.com
At Sun, 30 Jan 2011 18:06:28 +0300, Vladimir Prus wrote:
Dave Abrahams wrote:
At Sun, 30 Jan 2011 16:43:17 +0300, Vladimir Prus wrote:
Maybe, stricter published guidelines, together with automatic integration testing and nagging and reverting, is what we need here.
Secondly, I don't see having "a single trunk" with these properties as bringing with it any particular "goodness." If you're looking for a fix you still need to know which code to look at. So just go to that library's GitHub repo.
What if you don't know which library is that?
A big monolithic repo has the same problem.
Not really -- you need a single checkout operation to find whether the problem is fixed or not with 99% probability. And only then, to investigate further.
You mean, via a "try it and see" approach? Making it a single user operation to update to the latest released version of any or all libraries is under my proposal so trivial as to be unworthy of much discussion IMO.
But in reality, it doesn't tell you anything about the next release of Boost, because things don't always get moved from trunk to release.
That's the consequence of our non-standard release process. All other software projects create release branches off trunk, and Boost is the only one I know who has long-lived release branch. I though it was a conscious decision, but all the recent discussions makes me doubt that, so maybe we should briefly discuss relative merits.
1. The 'make release branch off trunk' process is good for all purposes, really, except it requires that at some point in time, trunk is in good shape. Historically, for lack of tools and release management resources, we often could not get trunk in good shape quick.
And that requires far too much coordination to be practical on a project of this size.
I don't think this statement has been proven.
Nothing in this discussion is going to be "proven."
2. Ours 'merge to release branch' process has the advantage that even if somebody breaks library X in trunk completely, he might just forget to merge it to release. So, we get improved stability, at the cost of slow progress. It's often the case that somebody forgets to merge changes to release branch, especially for patches and fixes applied outside of one's official libraries.
So, in comparison, to achieve the same rate of changes and quality, (1) requires discipline with commits and checking test results, while (2) requires the same, and additional dancing and coordination with merges. So, (2) is strictly more complex and error prone, and should only be done if specific maintainers want separate branches for their components.
Anyone can make a separate branch at any time no matter which procedure is used.
And? (2) still requires more effort.
No argument. I am just saying that the one condition under which you say 2 "should only be done" isn't even really valid.
What you propose breaks this useful thing, because:
- Only release managers "pull" into the unified thing
Not really. The individual library developers determined which versions will be automatically pulled into the unified thing by tagging their individual releases as STABLE.
There are two problems here:
1. Just like library developers forget to merge to release, they will forget to tag those 'releases'
That's totally up to the library developer. Library developers generally care Boost contains an up-to-date version of their libraries, so they will be motivated to remember.
"will"? In practice, right now, developers do forget.
This line of argument eventually devolves into "library maintainers will forget to check their changes in at all, so they should work on a network share that we snapshot for release." I think marking a change as ready for release should be a fairly conscious decision, and one that's separate from saying "let's test this code out." As long as that's the case, someone can forget. IMO the trade-off is well worth it.
And again, there are often commits made by those who are not official maintaners of library X (because X might not even have maintainers). The chances of such changes of falling through are even higher.
I don't see how. Right now someone has to explicitly decide which changes are moved to the release branch, and something equivalent will happen
2. Because 'git cherry-pick' is fundamentally unable to record any mergeinfo, this means that any time you want to merge a single specific fix to release branch, you would have problems.
What release branch? Who said anything about cherry-pick? What problems?
Suppose you have library X with 200 new changes. For next release, it is necessary to include one of those changes.
You cherry-pick that commit onto the branch in X's repo that currently points at X's last release.
How do you do that? With current release process, it's a single command (obviously, run by a single person).
And regardless, SVN has all the same issues w.r.t. picking individual changes, doesn't it?
No. When you merge a single revision from branch A to branch B, SVN records accurate information about this merge. When you do 'git cherry-pick', git does not record any information whatsoever and is fundementally unable to record it.
Ah. I was just complaining to a friend about the fact that Git drops that information. Mostly I don't miss it for any practical purpose, but conceptually I'd like it to be there.
- Release managers only pull "releases",
Which can be much more frequent than what we do at Boost.
Oh. I think everybody thinks that our current rate of releases is almost-too-often already.
Surely you realize that the release rate for individual libraries does not have to affect the release rate for Boost?
First, in "release managers only pull releases", above, we are surely talking about Boost release managers?
Yes. We are talking about them "pulling" the releases of individual libraries, not of Boost.
Second, could you name, exactly, all libraries whose developers expressed a need for individual releases?
No, of course I can't name all of them exactly. Can you? I doubt it.
I know of one.
I know I have expressed that need, and I've heard others express it.
which, together with delays added by release managers, means that a fix to a component will be added to the unified thing after considerable delay
I don't see where the delay comes from.
Either release managers look at the thing they merge (which adds considerable delay), or they don't, in which case direct push to release branch is more efficient.
They don't look, unless there are test failures or other problems. And they don't merge. At most we're talking about updating submodule references.
Which still has to be done manually?
No. From above: "The individual library developers determined which versions will be automatically pulled into the unified thing" by which I mean there's a "button" release managers can push to assemble a collection of the latest release versions of all the libraries in the collection.
Strangely, I know of other examples. Say, KDE, which is zillion times bigger than Boost, has relatively small number of separate git repositories,
They're trying to fix that, IIUC.
Oh? It does not seem like any split of kdelibs is immediately forthcoming, or even discussed with such activity as we're having here.
IIUC there is strong interest within at least part of the KDE community in following a Ryppl-like modularized approach. They have been looking into various kinds of package management infrastructure for this job. I'll be happy to try to find out more if you want.
Maybe, that suggest that we're actually solving wrong problem?
Exactly the opposite. KDE's moving in that direction IIUC; FreeBSD is moving in that direction. I'm confident we could quickly find 10 other projects that are trying to modularize and zero modularized projects that are trying to switch to a monolithic organization. When have you ever heard of that happening?
The problem isn't where the code is located, it's:
a. how much coordination is required to get things done
Yes. And given that right now, coordination basically boils down to library authors merging their things before deadline (or not merging), and given that merge is a single command,
svn switch, svn merge, svn switch == a single command? But anyway, that's irrelevant because...
it's not clear to me how you are improving on that.
...it's not about the number of commands, it's about the sense of speed and freedom with which library developers can operate.
b. the mental drag of having to deal with all the parts of Boost that simply aren't relevant to you.
What do you mean by mental drag? Having extra directories in your checkout?
Yes, among other things. For me personally, Boost's monolithic structure takes much of the fun out of working on it. Totally subjective and un-provable, I know. so-sue-me-ly y'rs, -- Dave Abrahams BoostPro Computing http://www.boostpro.com
AMDG On 1/30/2011 2:16 PM, Dave Abrahams wrote:
At Sun, 30 Jan 2011 18:06:28 +0300, Vladimir Prus wrote:
Dave Abrahams wrote:
b. the mental drag of having to deal with all the parts of Boost that simply aren't relevant to you.
What do you mean by mental drag? Having extra directories in your checkout?
Yes, among other things. For me personally, Boost's monolithic structure takes much of the fun out of working on it. Totally subjective and un-provable, I know.
I don't get it. I just don't get it. What is so hard about finding the directories for the library that you're working on and ignoring everything else? In Christ, Steven Watanabe
At Sun, 30 Jan 2011 16:11:51 -0800, Steven Watanabe wrote:
On 1/30/2011 2:16 PM, Dave Abrahams wrote:
For me personally, Boost's monolithic structure takes much of the fun out of working on it. Totally subjective and un-provable, I know.
I don't get it. I just don't get it. What is so hard about finding the directories for the library that you're working on and ignoring everything else?
I didn't say it was hard. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On 1/30/2011 8:33 PM, Dave Abrahams wrote:
At Sun, 30 Jan 2011 16:11:51 -0800, Steven Watanabe wrote:
On 1/30/2011 2:16 PM, Dave Abrahams wrote:
For me personally, Boost's monolithic structure takes much of the fun out of working on it. Totally subjective and un-provable, I know.
I don't get it. I just don't get it. What is so hard about finding the directories for the library that you're working on and ignoring everything else?
I didn't say it was hard.
There is another issue with Boost's structure which has little to do with library developers. Boost, after all, is for end-users also. Many end-users, whether rightly or wrongly, want to distribute with their product only those libraries ( header files, shared libraries, miscellaneous files ) that they need and Boost working on an all or nothing basis inhibits that, although it provides greater stability that way. While BCP might work fairly well, I believe it is hardly foolproof, whereas some means for being able to easily get just those Boost libraries, with their dependencies, which an end-user needs should be doable in a future vision of individually maintained/distributed Boost libraries. On this basis my own concern, whether a DVCS is used or not, is the dependencies between libraries. With different versions of different libraries existing, within there own distributions, it is absolutely necessary that the dependencies between different versions of libraries be resolved correctly. While this is not overwhelmingly difficult, it needs to be done in a way that is transparent as possible and provides the least problem for end-users, else no one is going to want to deal with the headaches involved on a practical basis. This means some way of incorporating version information in header files, some way of incorporating version information in shared libraries, some way of checking both transparently, and all of this must work in whatever environments a future individually distributed Boost targets. Just thought I would mention this minor fact <g>.
At Sun, 30 Jan 2011 22:06:07 -0500, Edward Diener wrote:
On this basis my own concern, whether a DVCS is used or not, is the dependencies between libraries.
...
This means some way of incorporating version information in header files, some way of incorporating version information in shared libraries, some way of checking both transparently, and all of this must work in whatever environments a future individually distributed Boost targets.
Those are all "nice-to-haves" that are not practiced by the many other projects that include such dependency relationships. Care to design that system? -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On 1/31/2011 12:38 AM, Dave Abrahams wrote:
At Sun, 30 Jan 2011 22:06:07 -0500, Edward Diener wrote:
On this basis my own concern, whether a DVCS is used or not, is the dependencies between libraries.
...
This means some way of incorporating version information in header files, some way of incorporating version information in shared libraries, some way of checking both transparently, and all of this must work in whatever environments a future individually distributed Boost targets.
Those are all "nice-to-haves" that are not practiced by the many other projects that include such dependency relationships. Care to design that system?
If this is not practiced by many other projects, perhaps that is because there are few other projects potentially seeking to deliver their libraries separately. But of course I think you are not entirely correct as the various problems of DLL dependencies on Windows and the tools on Linux to resolve dependencies when upgrading some application on Linux show. I would only seek to design such a system if there would be felt a need to do so by others also. In the face of "nobody else finds a need to do this so why should we" why would I expend my energy designing such a system or discussing it with others ? I will only offer that if Boost does in the future decide to deliver individual libraries rather than a single monolothic distribution as it does now, that end-users finding that library X version N.N does not work with library Y version N.N, of which it has a dependency relationship, will not be happy campers. If Boost developers think that is fine in such a system and that it is up to the end-users to figure out what works with what each time some upgraded library comes out, that is fine with me although I disagree that such a state of affairs would be good for Boost.
At Mon, 31 Jan 2011 09:54:23 -0500, Edward Diener wrote:
On 1/31/2011 12:38 AM, Dave Abrahams wrote:
At Sun, 30 Jan 2011 22:06:07 -0500, Edward Diener wrote:
On this basis my own concern, whether a DVCS is used or not, is the dependencies between libraries.
...
This means some way of incorporating version information in header files, some way of incorporating version information in shared libraries, some way of checking both transparently, and all of this must work in whatever environments a future individually distributed Boost targets.
Those are all "nice-to-haves" that are not practiced by the many other projects that include such dependency relationships. Care to design that system?
If this is not practiced by many other projects, perhaps that is because there are few other projects potentially seeking to deliver their libraries separately. But of course I think you are not entirely correct as the various problems of DLL dependencies on Windows and the tools on Linux to resolve dependencies when upgrading some application on Linux show.
I'm not correct about what? I never denied that problems arise.
I would only seek to design such a system if there would be felt a need to do so by others also. In the face of "nobody else finds a need to do this so why should we" why would I expend my energy designing such a system or discussing it with others ?
You complete misinterpret me. I totally understand _why_ we should design such a system, and, as I said, I think it would be really nice to have. I just don't think we need "some way of incorporating version information in header files, some way of incorporating version information in shared libraries, some way of checking both transparently, and all of this ...work[ing] in whatever environments a future individually distributed Boost targets" in order to move forward with modularization. These are independent problems that can be solved independently. In fact, people already have these kinds of issues with a non-modularized Boost, so it would be good to solve them today.
I will only offer that if Boost does in the future decide to deliver individual libraries rather than a single monolothic distribution as it does now, that end-users finding that library X version N.N does not work with library Y version N.N, of which it has a dependency relationship, will not be happy campers.
Encoding version dependency metadata and using it to make sure a coherent set of interoperable libraries are used together is a basic, fundamental requirement of Ryppl. However, I never envisioned it involving any of the things you listed above. Again, though, that would be really nice to have, and I'd like to see such a system. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On 1/31/2011 10:22 AM, Dave Abrahams wrote:
At Mon, 31 Jan 2011 09:54:23 -0500, Edward Diener wrote:
On 1/31/2011 12:38 AM, Dave Abrahams wrote:
At Sun, 30 Jan 2011 22:06:07 -0500, Edward Diener wrote:
On this basis my own concern, whether a DVCS is used or not, is the dependencies between libraries.
...
This means some way of incorporating version information in header files, some way of incorporating version information in shared libraries, some way of checking both transparently, and all of this must work in whatever environments a future individually distributed Boost targets.
Those are all "nice-to-haves" that are not practiced by the many other projects that include such dependency relationships. Care to design that system?
If this is not practiced by many other projects, perhaps that is because there are few other projects potentially seeking to deliver their libraries separately. But of course I think you are not entirely correct as the various problems of DLL dependencies on Windows and the tools on Linux to resolve dependencies when upgrading some application on Linux show.
I'm not correct about what? I never denied that problems arise.
Not correct that some form of dependency checking "are not practiced by the many other projects that include such dependency relationships."
I would only seek to design such a system if there would be felt a need to do so by others also. In the face of "nobody else finds a need to do this so why should we" why would I expend my energy designing such a system or discussing it with others ?
You complete misinterpret me. I totally understand _why_ we should design such a system, and, as I said, I think it would be really nice to have. I just don't think we need "some way of incorporating version information in header files, some way of incorporating version information in shared libraries, some way of checking both transparently, and all of this ...work[ing] in whatever environments a future individually distributed Boost targets" in order to move forward with modularization. These are independent problems that can be solved independently. In fact, people already have these kinds of issues with a non-modularized Boost, so it would be good to solve them today.
Do you think that some form of programatically checking dependencies between modules should exist in a modularized Boost ? If not do you see a manual form of checking dependencies to be the only way, or do you feel that no form of checking dependencies is necessary ?
I will only offer that if Boost does in the future decide to deliver individual libraries rather than a single monolothic distribution as it does now, that end-users finding that library X version N.N does not work with library Y version N.N, of which it has a dependency relationship, will not be happy campers.
Encoding version dependency metadata and using it to make sure a coherent set of interoperable libraries are used together is a basic, fundamental requirement of Ryppl.
I would be interested in how Ryppl plans to do this. Are there presently docs about this for Ryppl ?
However, I never envisioned it involving any of the things you listed above. Again, though, that would be really nice to have, and I'd like to see such a system.
I admit my background in Windows programming and my more limited use of Linux shows me that a programmatic method of dependency checking would be very valuable. While some form of header file dependency checking has never been popular, this has been largely due, I believe, to the fact that nearly all separate libraries are distributed in the form of shared libraries with version information. In C++ template programming this is not the case, so my suggestion about a form of header file dependency checking was meant to cover this area. But I am certainly willing to hear your, or anybody else's, better ideas. I just feel, as an end-user, that going from the current monolothic Boost to a modularized Boost will produce serious headaches if libraries are not in sync when an end-user attempts to use them for his product. This is especially true of potentially individual Boost libraries which are released at their own pace with new functionality being periodically added and fixes being made. I am sure you know all this, but sometimes developers do not focus on end-user problems because they are so keen on the programming task they do within their own development.
At Mon, 31 Jan 2011 13:20:28 -0500, Edward Diener wrote:
On 1/31/2011 10:22 AM, Dave Abrahams wrote:
At Mon, 31 Jan 2011 09:54:23 -0500, Edward Diener wrote:
On 1/31/2011 12:38 AM, Dave Abrahams wrote:
At Sun, 30 Jan 2011 22:06:07 -0500, Edward Diener wrote:
On this basis my own concern, whether a DVCS is used or not, is the dependencies between libraries.
...
This means some way of incorporating version information in header files, some way of incorporating version information in shared libraries, some way of checking both transparently, and all of this must work in whatever environments a future individually distributed Boost targets.
Those are all "nice-to-haves" that are not practiced by the many other projects that include such dependency relationships. Care to design that system?
If this is not practiced by many other projects, perhaps that is because there are few other projects potentially seeking to deliver their libraries separately. But of course I think you are not entirely correct as the various problems of DLL dependencies on Windows and the tools on Linux to resolve dependencies when upgrading some application on Linux show.
I'm not correct about what? I never denied that problems arise.
Not correct that some form of dependency checking "are not practiced by the many other projects that include such dependency relationships."
I didn't say that either. I asserted that the specific list of forms of dependency checking you mentioned are not practiced in general. I should have added "in my experience." Of course I could be wrong, but I haven't seen it.
These are independent problems that can be solved independently. In fact, people already have these kinds of issues with a non-modularized Boost, so it would be good to solve them today.
Do you think that some form of programatically checking dependencies between modules should exist in a modularized Boost ?
Yes
I will only offer that if Boost does in the future decide to deliver individual libraries rather than a single monolothic distribution as it does now, that end-users finding that library X version N.N does not work with library Y version N.N, of which it has a dependency relationship, will not be happy campers.
Encoding version dependency metadata and using it to make sure a coherent set of interoperable libraries are used together is a basic, fundamental requirement of Ryppl.
I would be interested in how Ryppl plans to do this. Are there presently docs about this for Ryppl ?
Current plans are to leverage libsatsolver for this purpose. No docs available from Ryppl, but feel free to search the ryppl-dev list on Google groups.
While some form of header file dependency checking has never been popular, this has been largely due, I believe, to the fact that nearly all separate libraries are distributed in the form of shared libraries with version information. In C++ template programming this is not the case, so my suggestion about a form of header file dependency checking was meant to cover this area.
It's a good idea, really! I just don't see it as an absolute requirement, which is what you seemed to be saying.
But I am certainly willing to hear your, or anybody else's, better ideas. I just feel, as an end-user, that going from the current monolothic Boost to a modularized Boost will produce serious headaches if libraries are not in sync when an end-user attempts to use them for his product.
Agreed. The ryppl tool can manage that when you ask it to fetch/install modules and their dependencies, and it must. Some kind of compile-time check in header files, like you suggest, would be a bonus. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
Am Montag, den 31.01.2011, 13:20 -0500 schrieb Edward Diener:
I just feel, as an end-user, that going from the current monolothic Boost to a modularized Boost will produce serious headaches if libraries are not in sync when an end-user attempts to use them for his product. This is especially true of potentially individual Boost libraries which are released at their own pace with new functionality being periodically added and fixes being made.
This is very true. However, changes in the development process do not necessarily imply changes for the end user: we can use individual projects for development, aggregate them to a release, and deliver a monolithic package. cheers, Daniel
At Mon, 31 Jan 2011 21:26:51 +0100, Daniel Pfeifer wrote:
Am Montag, den 31.01.2011, 13:20 -0500 schrieb Edward Diener:
I just feel, as an end-user, that going from the current monolothic Boost to a modularized Boost will produce serious headaches if libraries are not in sync when an end-user attempts to use them for his product. This is especially true of potentially individual Boost libraries which are released at their own pace with new functionality being periodically added and fixes being made.
This is very true. However, changes in the development process do not necessarily imply changes for the end user: we can use individual projects for development, aggregate them to a release, and deliver a monolithic package.
It has always been my intention that Boost continue to do that. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On 1/31/2011 3:26 PM, Daniel Pfeifer wrote:
Am Montag, den 31.01.2011, 13:20 -0500 schrieb Edward Diener:
I just feel, as an end-user, that going from the current monolothic Boost to a modularized Boost will produce serious headaches if libraries are not in sync when an end-user attempts to use them for his product. This is especially true of potentially individual Boost libraries which are released at their own pace with new functionality being periodically added and fixes being made.
This is very true. However, changes in the development process do not necessarily imply changes for the end user: we can use individual projects for development, aggregate them to a release, and deliver a monolithic package.
Agreed. But I think it is the goal of a modularized Boost, unless I am wrong, to be able to deliver individual libraries also as new releases of those libraries occur. Individual libraries are easier to move around locally on a developer's machine and lose the connection to their dependencies which a monolitihic Boost, in its own Boost tree structure, does not encourage.
Edward Diener wrote:
On 1/31/2011 3:26 PM, Daniel Pfeifer wrote:
This is very true. However, changes in the development process do not necessarily imply changes for the end user: we can use individual projects for development, aggregate them to a release, and deliver a monolithic package.
Agreed. But I think it is the goal of a modularized Boost, unless I am wrong, to be able to deliver individual libraries also as new releases of those libraries occur. Individual libraries are easier to move around locally on a developer's machine and lose the connection to their dependencies which a monolitihic Boost, in its own Boost tree structure, does not encourage.
True - but a long term goal - not necessarily an immediate one. If we can take incremental steps in the right direction, we should arrive there by default. Robert Ramey
Dave Abrahams wrote:
2. Ours 'merge to release branch' process has the advantage that even if somebody breaks library X in trunk completely, he might just forget to merge it to release. So, we get improved stability, at the cost of slow progress. It's often the case that somebody forgets to merge changes to release branch, especially for patches and fixes applied outside of one's official libraries.
So, in comparison, to achieve the same rate of changes and quality, (1) requires discipline with commits and checking test results, while (2) requires the same, and additional dancing and coordination with merges. So, (2) is strictly more complex and error prone, and should only be done if specific maintainers want separate branches for their components.
Anyone can make a separate branch at any time no matter which procedure is used.
And? (2) still requires more effort.
No argument. I am just saying that the one condition under which you say 2 "should only be done" isn't even really valid.
I disagree. Everybody can make a branch, but that's always additional overhead, and it's extra overhead when you have to coordinate merging (or other form of inclusion) of that branch into release with release managers. So, you are proposing that using a branch and coordinating with release managers be required from every Boost developer, which is just creating more work.
What you propose breaks this useful thing, because:
- Only release managers "pull" into the unified thing
Not really. The individual library developers determined which versions will be automatically pulled into the unified thing by tagging their individual releases as STABLE.
There are two problems here:
1. Just like library developers forget to merge to release, they will forget to tag those 'releases'
That's totally up to the library developer. Library developers generally care Boost contains an up-to-date version of their libraries, so they will be motivated to remember.
"will"? In practice, right now, developers do forget.
This line of argument eventually devolves into "library maintainers will forget to check their changes in at all, so they should work on a network share that we snapshot for release." I think marking a change as ready for release should be a fairly conscious decision, and one that's separate from saying "let's test this code out." As long as that's the case, someone can forget. IMO the trade-off is well worth it.
I think that running "xxx commit" should be a conscious decision. It's not at all clear to me that after that, additional conscious decisions should be required.
And again, there are often commits made by those who are not official maintaners of library X (because X might not even have maintainers). The chances of such changes of falling through are even higher.
I don't see how. Right now someone has to explicitly decide which changes are moved to the release branch, and something equivalent will happen
Right now, any developer can do this himself, without depending on somebody else to pick up the changes.
2. Because 'git cherry-pick' is fundamentally unable to record any mergeinfo, this means that any time you want to merge a single specific fix to release branch, you would have problems.
What release branch? Who said anything about cherry-pick? What problems?
Suppose you have library X with 200 new changes. For next release, it is necessary to include one of those changes.
You cherry-pick that commit onto the branch in X's repo that currently points at X's last release.
And as we've determined, we have chances of running into merge conflicts later.
- Release managers only pull "releases",
Which can be much more frequent than what we do at Boost.
Oh. I think everybody thinks that our current rate of releases is almost-too-often already.
Surely you realize that the release rate for individual libraries does not have to affect the release rate for Boost?
First, in "release managers only pull releases", above, we are surely talking about Boost release managers?
Yes. We are talking about them "pulling" the releases of individual libraries, not of Boost.
Second, could you name, exactly, all libraries whose developers expressed a need for individual releases?
No, of course I can't name all of them exactly. Can you? I doubt it.
It's you who is on the mission to prove this modularization effort is gonna help anybody.
which, together with delays added by release managers, means that a fix to a component will be added to the unified thing after considerable delay
I don't see where the delay comes from.
Either release managers look at the thing they merge (which adds considerable delay), or they don't, in which case direct push to release branch is more efficient.
They don't look, unless there are test failures or other problems. And they don't merge. At most we're talking about updating submodule references.
Which still has to be done manually?
No. From above: "The individual library developers determined which versions will be automatically pulled into the unified thing" by which I mean there's a "button" release managers can push to assemble a collection of the latest release versions of all the libraries in the collection.
When is this button pushed? If it's pushed right before release, it means the release state was never tested together. If you want to have it pushed regularly, what are guarantees it will be pushed? And if you want it to be pushed automatically, why don't just have developers push into release branch directly?
Strangely, I know of other examples. Say, KDE, which is zillion times bigger than Boost, has relatively small number of separate git repositories,
They're trying to fix that, IIUC.
Oh? It does not seem like any split of kdelibs is immediately forthcoming, or even discussed with such activity as we're having here.
IIUC there is strong interest within at least part of the KDE community in following a Ryppl-like modularized approach. They have been looking into various kinds of package management infrastructure for this job. I'll be happy to try to find out more if you want.
Mailing list pointers will help.
Maybe, that suggest that we're actually solving wrong problem?
Exactly the opposite. KDE's moving in that direction IIUC; FreeBSD is moving in that direction. I'm confident we could quickly find 10 other projects that are trying to modularize and zero modularized projects that are trying to switch to a monolithic organization. When have you ever heard of that happening?
Nice question. Let me avoid answering it and ask whether you are aware of many modularizations whereby a component the size of boost::any is split into a separate git repository?
The problem isn't where the code is located, it's:
a. how much coordination is required to get things done
Yes. And given that right now, coordination basically boils down to library authors merging their things before deadline (or not merging), and given that merge is a single command,
svn switch, svn merge, svn switch == a single command?
I somehow manage with just svn merge ;-)
But anyway, that's irrelevant because...
it's not clear to me how you are improving on that.
...it's not about the number of commands, it's about the sense of speed and freedom with which library developers can operate.
Okay. Though for me, nothing beats the speed and freedom of having my changed put to release branch directly. - Volodya -- Vladimir Prus Mentor Graphics +7 (812) 677-68-40
At Mon, 31 Jan 2011 12:27:24 +0300, Vladimir Prus wrote:
Dave Abrahams wrote:
2. Ours 'merge to release branch' process has the advantage that even if somebody breaks library X in trunk completely, he might just forget to merge it to release. So, we get improved stability, at the cost of slow progress. It's often the case that somebody forgets to merge changes to release branch, especially for patches and fixes applied outside of one's official libraries.
So, in comparison, to achieve the same rate of changes and quality, (1) requires discipline with commits and checking test results, while (2) requires the same, and additional dancing and coordination with merges. So, (2) is strictly more complex and error prone, and should only be done if specific maintainers want separate branches for their components.
Anyone can make a separate branch at any time no matter which procedure is used.
And? (2) still requires more effort.
No argument. I am just saying that the one condition under which you say 2 "should only be done" isn't even really valid.
I disagree. Everybody can make a branch, but that's always additional overhead, and it's extra overhead when you have to coordinate merging (or other form of inclusion)
Again, the form of inclusion is a single command that says "update my idea of Boost to reference the latest release of every submodule."
of that branch into release with release managers. So, you are proposing that using a branch and coordinating with release managers be required from every Boost developer, which is just creating more work.
Meh. What coordination?
"will"? In practice, right now, developers do forget.
This line of argument eventually devolves into "library maintainers will forget to check their changes in at all, so they should work on a network share that we snapshot for release." I think marking a change as ready for release should be a fairly conscious decision, and one that's separate from saying "let's test this code out." As long as that's the case, someone can forget. IMO the trade-off is well worth it.
I think that running "xxx commit" should be a conscious decision. It's not at all clear to me that after that, additional conscious decisions should be required.
Understood. Some people don't like to do their work on a branch. It does cost one extra step when you're ready to declare the work production-ready and for public consumption. If you really want to do that, you can commit all your work on the STABLE branch of your repository and it will get sucked into the next Boost release automatically (unless it causes problems).
And again, there are often commits made by those who are not official maintaners of library X (because X might not even have maintainers). The chances of such changes of falling through are even higher.
I don't see how. Right now someone has to explicitly decide which changes are moved to the release branch, and something equivalent will happen
Right now, any developer can do this himself, without depending on somebody else to pick up the changes.
IIUC you are really pointing to the fact that every developer can modify every file in Boost, and thus can make changes to any library that will automatically go out in the next release unless someone reverts them. We could maintain that system of course. I don't think it would be a very good idea, but we could.
2. Because 'git cherry-pick' is fundamentally unable to record any mergeinfo, this means that any time you want to merge a single specific fix to release branch, you would have problems.
What release branch? Who said anything about cherry-pick? What problems?
Suppose you have library X with 200 new changes. For next release, it is necessary to include one of those changes.
You cherry-pick that commit onto the branch in X's repo that currently points at X's last release.
And as we've determined, we have chances of running into merge conflicts later.
Yes, it can happen, and this is one scenario where SVN will outperform Git. However, it doesn't happen often, and apparently if you're using Git "properly" you don't really get into such a situation. If I apply the "logic of the Git ethic" here, I come up with this: * Your 200 new changes are not yet publicly available (in a non-volatile branch), so if they're visible at all they are in a "look-but-don't-touch" condition * You cherry-pick the one change onto that release branch * You *immediately* rewrite the branch that contained the change so that it: a) no longer contains that change, and b) is rebased on the release branch Operating strictly via this ethic is a new concept to me. I have violated it blatantly in the past and seldom run into merge conflicts, so you can probably usually get away with not following it. However, I think the more collaborators you have, the more necessary it becomes.
Second, could you name, exactly, all libraries whose developers expressed a need for individual releases?
No, of course I can't name all of them exactly. Can you? I doubt it.
It's you who is on the mission to prove this modularization effort is gonna help anybody.
No, I know I can't prove any of this... and frankly I don't think I need to. Most people understand that decoupling, where practical, is a good idea and that a monolithic Boost has lots of downsides.
> which, together with delays added by release managers, means that > a fix to a component will be added to the unified thing after > considerable delay
I don't see where the delay comes from.
Either release managers look at the thing they merge (which adds considerable delay), or they don't, in which case direct push to release branch is more efficient.
They don't look, unless there are test failures or other problems. And they don't merge. At most we're talking about updating submodule references.
Which still has to be done manually?
No. From above: "The individual library developers determined which versions will be automatically pulled into the unified thing" by which I mean there's a "button" release managers can push to assemble a collection of the latest release versions of all the libraries in the collection.
When is this button pushed? If it's pushed right before release, it means the release state was never tested together.
It's pushed some suitable time before the next scheduled release, and then there's a period where that release state is tested, examined, and prepared to go out.
If you want to have it pushed regularly, what are guarantees it will be pushed?
C'mon, man! The release managers have to do *something* every three months. Pushing a button doesn't seem too burdensome.
And if you want it to be pushed automatically, why don't just have developers push into release branch directly?
They do! They push into the release branch of their library's repository. That's the only release branch there is. Keeping the modules well-separated *guarantees* that at integration time there's no chance of merge conflicts. They are simply checked out into separate directory trees.
IIUC there is strong interest within at least part of the KDE community in following a Ryppl-like modularized approach. They have been looking into various kinds of package management infrastructure for this job. I'll be happy to try to find out more if you want.
Mailing list pointers will help.
working on it.
Maybe, that suggest that we're actually solving wrong problem?
Exactly the opposite. KDE's moving in that direction IIUC; FreeBSD is moving in that direction. I'm confident we could quickly find 10 other projects that are trying to modularize and zero modularized projects that are trying to switch to a monolithic organization. When have you ever heard of that happening?
Nice question. Let me avoid answering it
No, thanks. Please answer. A one-word answer is easy to type :-)
and ask whether you are aware of many modularizations whereby a component the size of boost::any is split into a separate git repository?
Yes, it happens very commonly with emacs packages.
The problem isn't where the code is located, it's:
a. how much coordination is required to get things done
Yes. And given that right now, coordination basically boils down to library authors merging their things before deadline (or not merging), and given that merge is a single command,
svn switch, svn merge, svn switch == a single command?
I somehow manage with just svn merge ;-)
You at least have to commit. If you do most of your work on the trunk, you must keep the release branch checked out somewhere all the time, so you don't need to switch. I eventually started doing that too.
But anyway, that's irrelevant because...
it's not clear to me how you are improving on that.
...it's not about the number of commands, it's about the sense of speed and freedom with which library developers can operate.
Okay. Though for me, nothing beats the speed and freedom of having my changed put to release branch directly.
Fair enough. I am the first to admit this is highly subjective. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
AMDG On 1/31/2011 7:53 AM, Dave Abrahams wrote:
At Mon, 31 Jan 2011 12:27:24 +0300, Vladimir Prus wrote:
Second, could you name, exactly, all libraries whose developers expressed a need for individual releases?
No, of course I can't name all of them exactly. Can you? I doubt it.
It's you who is on the mission to prove this modularization effort is gonna help anybody.
No, I know I can't prove any of this... and frankly I don't think I need to.
Yes you do. Simply throwing out catchwords gets you nowhere in my book. From what I've seen, this accounts for the vast majority of the arguments in favor of "modularization." Any /real/ benefits have gotten lost in the noise. It's quite possible that there would be advantages. I just don't know what they are.
Most people understand that decoupling, where practical, is a good idea and that a monolithic Boost has lots of downsides.
Cynical translation: Most people see the word "decoupling" and think good and the word "monolithic" and think "bad" without necessarily having any understanding of what it actually means in context. In Christ, Steven Watanabe
At Mon, 31 Jan 2011 09:34:56 -0800, Steven Watanabe wrote:
No, I know I can't prove any of this... and frankly I don't think I need to.
Yes you do. Simply throwing out catchwords gets you nowhere in my book.
Here's what I meant: I think most people understand what I'm talking about. There will be a few people who just disagree, or don't get it, or will object for some other reason, and that's OK. I don't need to convince everybody. That said, my assessment could be way off the mark. I've been having this discussion in good faith, trying to make substantive arguments, being careful not to blow smoke, and trying to be fair to all points of view. If that looks to you like "simply throwing out catchphrases," I certainly don't see any point in expending effort trying to convince you.
From what I've seen, this accounts for the vast majority of the arguments in favor of "modularization." Any /real/ benefits have gotten lost in the noise. It's quite possible that there would be advantages. I just don't know what they are.
They've been discussed many times in the past and I'm just out of time to repeat myself at the moment, but we can try to have that conversation again soon. But how will we keep the "noise" (on all sides) down?
Most people understand that decoupling, where practical, is a good idea and that a monolithic Boost has lots of downsides.
Cynical translation: Most people see the word "decoupling" and think good and the word "monolithic" and think "bad" without necessarily having any understanding of what it actually means in context.
I usually try hard to avoid translating the things other people say cynically. It's just not a very useful (or generous) way to listen, especially when the participants ostensibly have the same aims. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
participants (12)
-
Beman Dawes -
Christopher Jefferson -
Daniel Pfeifer -
Dave Abrahams -
Dean Michael Berris -
Edward Diener -
Gordon Woodhull -
Robert Ramey -
Sean Chittenden -
Steve M. Robbins -
Steven Watanabe -
Vladimir Prus