At Sun, 30 Jan 2011 18:06:28 +0300, Vladimir Prus wrote:
Dave Abrahams wrote:
At Sun, 30 Jan 2011 16:43:17 +0300, Vladimir Prus wrote:
Maybe, stricter published guidelines, together with automatic integration testing and nagging and reverting, is what we need here.
Secondly, I don't see having "a single trunk" with these properties as bringing with it any particular "goodness." If you're looking for a fix you still need to know which code to look at. So just go to that library's GitHub repo.
What if you don't know which library is that?
A big monolithic repo has the same problem.
Not really -- you need a single checkout operation to find whether the problem is fixed or not with 99% probability. And only then, to investigate further.
You mean, via a "try it and see" approach? Making it a single user operation to update to the latest released version of any or all libraries is under my proposal so trivial as to be unworthy of much discussion IMO.
But in reality, it doesn't tell you anything about the next release of Boost, because things don't always get moved from trunk to release.
That's the consequence of our non-standard release process. All other software projects create release branches off trunk, and Boost is the only one I know who has long-lived release branch. I though it was a conscious decision, but all the recent discussions makes me doubt that, so maybe we should briefly discuss relative merits.
1. The 'make release branch off trunk' process is good for all purposes, really, except it requires that at some point in time, trunk is in good shape. Historically, for lack of tools and release management resources, we often could not get trunk in good shape quick.
And that requires far too much coordination to be practical on a project of this size.
I don't think this statement has been proven.
Nothing in this discussion is going to be "proven."
2. Ours 'merge to release branch' process has the advantage that even if somebody breaks library X in trunk completely, he might just forget to merge it to release. So, we get improved stability, at the cost of slow progress. It's often the case that somebody forgets to merge changes to release branch, especially for patches and fixes applied outside of one's official libraries.
So, in comparison, to achieve the same rate of changes and quality, (1) requires discipline with commits and checking test results, while (2) requires the same, and additional dancing and coordination with merges. So, (2) is strictly more complex and error prone, and should only be done if specific maintainers want separate branches for their components.
Anyone can make a separate branch at any time no matter which procedure is used.
And? (2) still requires more effort.
No argument. I am just saying that the one condition under which you say 2 "should only be done" isn't even really valid.
What you propose breaks this useful thing, because:
- Only release managers "pull" into the unified thing
Not really. The individual library developers determined which versions will be automatically pulled into the unified thing by tagging their individual releases as STABLE.
There are two problems here:
1. Just like library developers forget to merge to release, they will forget to tag those 'releases'
That's totally up to the library developer. Library developers generally care Boost contains an up-to-date version of their libraries, so they will be motivated to remember.
"will"? In practice, right now, developers do forget.
This line of argument eventually devolves into "library maintainers will forget to check their changes in at all, so they should work on a network share that we snapshot for release." I think marking a change as ready for release should be a fairly conscious decision, and one that's separate from saying "let's test this code out." As long as that's the case, someone can forget. IMO the trade-off is well worth it.
And again, there are often commits made by those who are not official maintaners of library X (because X might not even have maintainers). The chances of such changes of falling through are even higher.
I don't see how. Right now someone has to explicitly decide which changes are moved to the release branch, and something equivalent will happen
2. Because 'git cherry-pick' is fundamentally unable to record any mergeinfo, this means that any time you want to merge a single specific fix to release branch, you would have problems.
What release branch? Who said anything about cherry-pick? What problems?
Suppose you have library X with 200 new changes. For next release, it is necessary to include one of those changes.
You cherry-pick that commit onto the branch in X's repo that currently points at X's last release.
How do you do that? With current release process, it's a single command (obviously, run by a single person).
And regardless, SVN has all the same issues w.r.t. picking individual changes, doesn't it?
No. When you merge a single revision from branch A to branch B, SVN records accurate information about this merge. When you do 'git cherry-pick', git does not record any information whatsoever and is fundementally unable to record it.
Ah. I was just complaining to a friend about the fact that Git drops that information. Mostly I don't miss it for any practical purpose, but conceptually I'd like it to be there.
- Release managers only pull "releases",
Which can be much more frequent than what we do at Boost.
Oh. I think everybody thinks that our current rate of releases is almost-too-often already.
Surely you realize that the release rate for individual libraries does not have to affect the release rate for Boost?
First, in "release managers only pull releases", above, we are surely talking about Boost release managers?
Yes. We are talking about them "pulling" the releases of individual libraries, not of Boost.
Second, could you name, exactly, all libraries whose developers expressed a need for individual releases?
No, of course I can't name all of them exactly. Can you? I doubt it.
I know of one.
I know I have expressed that need, and I've heard others express it.
which, together with delays added by release managers, means that a fix to a component will be added to the unified thing after considerable delay
I don't see where the delay comes from.
Either release managers look at the thing they merge (which adds considerable delay), or they don't, in which case direct push to release branch is more efficient.
They don't look, unless there are test failures or other problems. And they don't merge. At most we're talking about updating submodule references.
Which still has to be done manually?
No. From above: "The individual library developers determined which versions will be automatically pulled into the unified thing" by which I mean there's a "button" release managers can push to assemble a collection of the latest release versions of all the libraries in the collection.
Strangely, I know of other examples. Say, KDE, which is zillion times bigger than Boost, has relatively small number of separate git repositories,
They're trying to fix that, IIUC.
Oh? It does not seem like any split of kdelibs is immediately forthcoming, or even discussed with such activity as we're having here.
IIUC there is strong interest within at least part of the KDE community in following a Ryppl-like modularized approach. They have been looking into various kinds of package management infrastructure for this job. I'll be happy to try to find out more if you want.
Maybe, that suggest that we're actually solving wrong problem?
Exactly the opposite. KDE's moving in that direction IIUC; FreeBSD is moving in that direction. I'm confident we could quickly find 10 other projects that are trying to modularize and zero modularized projects that are trying to switch to a monolithic organization. When have you ever heard of that happening?
The problem isn't where the code is located, it's:
a. how much coordination is required to get things done
Yes. And given that right now, coordination basically boils down to library authors merging their things before deadline (or not merging), and given that merge is a single command,
svn switch, svn merge, svn switch == a single command? But anyway, that's irrelevant because...
it's not clear to me how you are improving on that.
...it's not about the number of commands, it's about the sense of speed and freedom with which library developers can operate.
b. the mental drag of having to deal with all the parts of Boost that simply aren't relevant to you.
What do you mean by mental drag? Having extra directories in your checkout?
Yes, among other things. For me personally, Boost's monolithic structure takes much of the fun out of working on it. Totally subjective and un-provable, I know. so-sue-me-ly y'rs, -- Dave Abrahams BoostPro Computing http://www.boostpro.com