Dave Abrahams wrote:

IIUC we already agreed long ago to use gitflow:

It's not entirely clear to me what using gitflow would mean in the context of a Boost superproject with per-library submodules. Who is "we" who would be using gitflow, the Boost release managers or the library maintainers? When Boost release 1.64.0 is started, who is going to create the gitflow release branch, and from what?

On 21 Oct 2013 at 12:28, Beman Dawes wrote:

1) What git/TortoiseGit/other commands would a Boost library maintainer have to execute if, after following https://svn.boost.org/trac/boost/wiki/TryModBoost, they wish to fix a simple bug in their "develop" branch?

TortoiseGit hides "gitness" from the user by often executing more than one git command than is obvious. Hardcore git users get upset by this, but I personally don't mind except when command line git doesn't do what I expect because I've become too used to TortoiseGit (e.g. changing a file while editing a commit message commits the changes on TortoiseGit, but does not on command line).

For that matter, does a Boost library maintainer even start by https://svn.boost.org/trac/boost/wiki/TryModBoost? Is there a more preferable approach?

That page basically clones and checks out a kitchen sink Boost with all possible git submodules. That rather makes modularisation pointless. What we really need is for each Boost module to specify its dependency Boost modules, and that way only the git submodules needed are cloned. Therefore if I want to use Boost.AFIO in my project, AFIO will specify it needs Thread and ASIO and those get git submodule update --init --recursive.

2) What git/TortoiseGit/other commands would a Boost library maintainer have to execute if, after (1) they wanted to merge it into their library's "master' branch?

I think the only sane way is to have the master Boost master repo track which SHA in a submodule's master branch is the sanctified known true stable incarnation of the library. A library can commit what they like to whatever branch - and we can't stop them. But we can gatekeep which SHA in that repo is considered valid by master Boost master. We do indeed need a process for automating that, unless you want to give write access to the master Boost master repo to all Boost authors (I assume you don't!). I think the hard part there is who admin's that process. Niall -- Currently unemployed and looking for work. Work Portfolio: http://careers.stackoverflow.com/nialldouglas/

On 21 Oct 2013 at 21:18, Beman Dawes wrote:

Some developers will prefer GitFlow. Others will use the command line. And still others with use other tools. We have to support all of them with at least minimal documentation.

I assume you mean the tools rather than the processes here. In which case fine, let each library maintainer do as they wish.

...

What we really need is for each Boost module to specify its dependency Boost modules, and that way only the git submodules needed are cloned. Therefore if I want to use Boost.AFIO in my project, AFIO will specify it needs Thread and ASIO and those get git submodule update --init --recursive.

Maybe someday, but not now.

It's much easier than you think you know. If you insist on a mandatory script in the root of each submodule e.g. fetch-boost-dependencies.sh fetch-boost-dependencies.cmd (which simply CMD's the former BTW) All that does is call git update --init --recursive for those libraries it depends on. This can work because it is safe to repeatedly call that command on an already initialised submodule. It surely is then extremely easy to have Boost.Build check for the presence of that script and execute it if present. Voila, automatic dependency fetching.

...

I think the only sane way is to have the master Boost master repo track which SHA in a submodule's master branch is the sanctified known true stable incarnation of the library. A library can commit what they like to whatever branch - and we can't stop them. But we can gatekeep which SHA in that repo is considered valid by master Boost master.

Un, didn't you just describe how a git sub-module works?

In implementation, yes. I was trying to describe the work flow as that's what I thought was being asked. (I did so because I have noticed occasional confusion about git here, so I thought it would set the context and help avoid confusion)

...

We do indeed need a process for automating that, unless you want to give write access to the master Boost master repo to all Boost authors (I assume you don't!).

Wouldn't the sub-module developer just send a pull request to the super-module?

They could, but it's awkward because they have to: 1. Pull changes into their fork of super-module from base. 2. Commit submodule ref update to their fork. 2a. Check that newly merged super-module and module compiles and passes all tests. 3. Issue pull request to base. I worry you see about 2a: will people tend to skip that part? One part of a solution could be requiring Travis CI report good before a pull request will be accepted (Travis becomes an option with modularised Boost, few individual libraries will take more than 30 minutes to compile). I worry that the super-project admin is very likely to accept all pull requests sent without deeply checking what they mean - after all, they're just a submodule ref update with no explanatory context. I agree we already accept this in SVN, but people commit to SVN knowing they are being watched. I also worry that every library maintainer having to keep a complete set of upstream forks for issuing pull requests might scare people a bit. It's also a bit involved. Lastly, I worry that bisecting all this to find cause of breakages is probably hard. Niall -- Currently unemployed and looking for work. Work Portfolio: http://careers.stackoverflow.com/nialldouglas/

on Mon Oct 21 2013, Beman Dawes wrote:

On Mon, Oct 21, 2013 at 10:15 AM, Peter Dimov wrote:

...

Dave Abrahams wrote:

...

IIUC we already agreed long ago to use gitflow:

Yes, but how do Boost developers and release managers "use gitflow" ?

See my reply to Peter.

...

It's not entirely clear to me what using gitflow would mean in the context of a Boost superproject with per-library submodules. Who is "we" who would be using gitflow, the Boost release managers or the library maintainers? When Boost release 1.64.0 is started, who is going to create the gitflow release branch, and from what?

Yes, those are exactly the sort of questions I had in mind. Sorry for not being more explicit. Let me try again:

1) What git/TortoiseGit/other commands would a Boost library maintainer have to execute if, after following https://svn.boost.org/trac/boost/wiki/TryModBoost, they wish to fix a simple bug in their "develop" branch?

I don't understand the source of mystery here. This is bog-standard Git usage: cd my/submodule/directory git checkout develop # make edits git commit git push origin develop [I'm not going to translate to TortoiseGit/other]

For that matter, does a Boost library maintainer even start by https://svn.boost.org/trac/boost/wiki/TryModBoost? Is there a more preferable approach?

Not today.

2) What git/TortoiseGit/other commands would a Boost library maintainer have to execute if, after (1) they wanted to merge it into their library's "master' branch?

In gitflow, merging into master means creating a release. See http://nvie.com/posts/a-successful-git-branching-model/ starting with "Creating a release branch" and ending at "Hotfix branches" for the raw git commands. But those are complex. Just install gitflow and git flow release start v git flow release finish v As you can see, that's a lot simpler. I strongly recommend using the gitflow tool because it will save lots of trouble and prevent mistakes. [I just found http://danielkummer.github.io/git-flow-cheatsheet/ which is a really good guide.] -- Dave Abrahams

Umm, shouldn't you create a new branch here for the change rather than changing develop directly:

git checkout -b issue_8_fix develop #make edits git commit -a git checkout develop git merge issue_8_fix git commit -a

I think this is only worth it for when there will be at least two commits in that branch. Creating a branch for one commit sounds a bit pointless to me. Also, git merge without --no-ff is also a bit pointless in that case, because you lose the "branch" information. Philippe

On Tue, Oct 22, 2013 at 11:06 PM, Niall Douglas wrote:

On 22 Oct 2013 at 19:45, Dave Abrahams wrote:

...

I don't understand the source of mystery here. This is bog-standard Git usage:

cd my/submodule/directory git checkout develop # make edits

Umm, shouldn't you create a new branch here for the change rather than changing develop directly:

git checkout -b issue_8_fix develop #make edits git commit -a git checkout develop git merge issue_8_fix git commit -a

My (very tentative) plan for the docs is to recommend always branching for new features, even if trivial, and also for other change if they are anything beyond trivial. But not for a totally simple change like fixing a single typo level bug. I'm not sure I want to sidetrack in what is basically beginners documentation to take about how practices often differ among projects, particularly along the spectrum from small single developer projects all the way up to very large multi-developer mega projects. But maybe I'll say something brief.

...

git commit git push origin develop [snip] git flow release start v git flow release finish v

As you can see, that's a lot simpler. I strongly recommend using the gitflow tool because it will save lots of trouble and prevent mistakes.

I agree. Though I must admit I do it by hand personally, mainly because TortoiseGit doesn't assist gitflow to my knowledge.

The TortoiseGit folks have been very responsive to the few requests for improvements I've been involved with. So if we find gitflow tools would be helpful, consider submitting a request to them.

For those interested: point is with gitflow that git branches are incredibly cheap. You branch for every self contained change, no matter how small. Think a branch per fully compiling source tree, and you're about right granularity. It's normal to generate hundreds of new branches per month - don't worry, there is a git command to auto purge all merged branches.

Yes, I'll probably add that as an aside.

Thanks, --Beman

On Thu, Oct 24, 2013 at 9:31 AM, Daniel James wrote:

It might be worth looking at sourcetree, as it has support for git-flow (I haven't tried it myself):

http://www.sourcetreeapp.com/

It was a bit surprising at first for me because it just have one big button for gitflow, but it's interactive so it made me understand better how gitflow works: each time you want to do something related to branching, first hit the big gitflow button and it will present you the options available at the time (except the first time post-clone when you need to set it up, just use the default config). So when you begin a feature, next time you will hit the button and you are in a feature branch, it will propose only to close the feature.

On 24 Oct 2013 03:14, "Dave Abrahams" wrote:

on Tue Oct 22 2013, "Niall Douglas" wrote:

...

On 22 Oct 2013 at 19:45, Dave Abrahams wrote:

...

I don't understand the source of mystery here. This is bog-standard Git usage:

cd my/submodule/directory git checkout develop # make edits

Umm, shouldn't you create a new branch here for the change rather than changing develop directly:

git checkout -b issue_8_fix develop #make edits git commit -a git checkout develop git merge issue_8_fix git commit -a

No, working directly on the develop branch is an allowed option; the lack of a feature branch affects nobody but the developer. If you won't have multiple features in flight at once, there's no reason you have to create feature branches.

It can get a bit awkward if you want to start a new feature while you're still using develop to work on a different feature.

What about accumulation of small features, small fixes and improvements which do not necessary qualify as hotfix release managed with the gitflow hotfix branches? I've been using gitflow to maintain SOCI for months now and it's proved that master/develop is not always sufficient to cover all cases, especially with various GitHub pull requests flowing from users/contributors. I personally found a need for Subversion's equivalent of long running /branches/X.Y, hence I'm trying to incorporate support branches. Here is my post to gitflow-users with the problem/case explained, not sure if it applies to Boost directly, but I'm curious how Boost will manage such cases: https://groups.google.com/d/msg/gitflow-users/uvy0b4OrgnY/B1E-hb9lS6wJ By the way, the original nvine/gitflow seems not actively maintained. It's worth to check AVH Edition of gitflow, it fully implements the support branches. Another interesting clone of the gitflow tools/workflow is HubFlow http://datasift.github.io/gitflow/ Mateusz Loskot (Sent from mobile, apology for top-posting a wheznd broken quotes)

On 21 Oct 2013 at 17:15, Peter Dimov wrote:

Dave Abrahams wrote:

...

IIUC we already agreed long ago to use gitflow:

It's not entirely clear to me what using gitflow would mean in the context of a Boost superproject with per-library submodules. Who is "we" who would be using gitflow, the Boost release managers or the library maintainers? When Boost release 1.64.0 is started, who is going to create the gitflow release branch, and from what?

I should firstly stress that I think people are confusing git flow as a process and git flow as the tool which you install into git. The latter simply is a script which implements the former of course, but I'll be frank in saying I think it is too toy for useful use in modularised Boost. An elaboration of that tool could be just fine, and I think that's what Beman is asking: for a set of canned git command sequences, preferably with TortoiseGit equivalents, for various common operations. The former, git flow as a process, I think for a single repo is totally uncontroversial. It's how I and anyone else I know would naturally use Git once you're used to it. For modularised Boost, git flow as a process needs extending to cope with how we've done modules, because how we've done modules I think is fairly unique. I think that topic - how do we extend git flow the process to cope with Boost modularisation - is where we should be focusing our attention, not on git flow the tool, git flow the process for a single repo, whether we agreed on git flow, whether git flow is good or bad etc. Once agreed, then we know how to extend git flow the tool usefully. And possibly TortoiseGit too. Niall -- Currently unemployed and looking for work. Work Portfolio: http://careers.stackoverflow.com/nialldouglas/

on Mon Oct 21 2013, "Peter Dimov" wrote:

Dave Abrahams wrote:

...

IIUC we already agreed long ago to use gitflow:

It's not entirely clear to me what using gitflow would mean in the context of a Boost superproject with per-library submodules. Who is "we" who would be using gitflow, the Boost release managers or the library maintainers? When Boost release 1.64.0 is started, who is going to create the gitflow release branch, and from what?

The release managers (or a continuous integration script) assembles a release candidate that references, by default, the latest commits to the "master" branch of all submodules. Test failures may result in rolling back to earlier commits on that branch in a given submodule. In other words, library authors release their code on whatever schedule they like, and Boost release managers assemble a combination of library releases into a Boost release. -- Dave Abrahams

on Wed Oct 23 2013, Julian Gonggrijp wrote:

Dave Abrahams wrote:

...

The release managers (or a continuous integration script) assembles a release candidate that references, by default, the latest commits to the "master" branch of all submodules. Test failures may result in rolling back to earlier commits on that branch in a given submodule.

In other words, library authors release their code on whatever schedule they like, and Boost release managers assemble a combination of library releases into a Boost release.

I'm a proponent of the use of Git and GitFlow in boost (in fact I *might* have been the first to suggest GitFlow [1]) as well as of modularization, but I can't help seeing a potential problem with this approach.

What do you do if module releases conflict? For example, if the latest release of module A is only compatible with older releases of module B, or if one module depends on the latest version of B while another depends on the previous? When such effects propagate you might have to be very conservative with the global update even though most modules have updated several times in the meanwhile.

One or more modules don't advance their version in the official Boost distribution until one or both of the maintainers solve the problem.

In my opinion it would be better to make hotfixes in the offending modules (using standard gitflow hotfix branches) so that every Boost release can include the latest versions of all modules.

I disagree.

Of course, that requires more coordination between the release managers and the library maintainers.

Exactly. One of the main points of modularizing is to minimize the coordination burdens associated with our processes. Especially when you have an organization of volunteers, creating a situation where one person's non-responsiveness can stymie overall progress is a bad idea. -- Dave Abrahams

On 24 October 2013 08:50, Julian Gonggrijp wrote:

Dave Abrahams wrote:

...

Exactly. One of the main points of modularizing is to minimize the coordination burdens associated with our processes. Especially when you have an organization of volunteers, creating a situation where one person's non-responsiveness can stymie overall progress is a bad idea.

Point taken. Still, you haven't taken away my worry about "conflict propagation": what if a couple of conflicting modules block the newest versions of 20 (or more!) other modules?

Is that somehow not possible? Am I missing something?

If that happens in subversion, we have the same problem. Perhaps a larger one since merging to release is such a mess.

Daniel James wrote:

On 24 October 2013 08:50, Julian Gonggrijp wrote:

...

Dave Abrahams wrote:

...

Exactly. One of the main points of modularizing is to minimize the coordination burdens associated with our processes. Especially when you have an organization of volunteers, creating a situation where one person's non-responsiveness can stymie overall progress is a bad idea.

Point taken. Still, you haven't taken away my worry about "conflict propagation": what if a couple of conflicting modules block the newest versions of 20 (or more!) other modules?

Is that somehow not possible? Am I missing something?

If that happens in subversion, we have the same problem. Perhaps a larger one since merging to release is such a mess.

I didn't think of that. Thanks for clearing up! -Julian

On Wed, Oct 23, 2013 at 10:26 PM, Dave Abrahams wrote:

on Wed Oct 23 2013, Julian Gonggrijp wrote: ...

...

In my opinion it would be better to make hotfixes in the offending modules (using standard gitflow hotfix branches) so that every Boost release can include the latest versions of all modules.

I disagree.

...

Of course, that requires more coordination between the release managers and the library maintainers.

Exactly. One of the main points of modularizing is to minimize the coordination burdens associated with our processes. Especially when you have an organization of volunteers, creating a situation where one person's non-responsiveness can stymie overall progress is a bad idea.

Dave's point is critical. As Boost continues to add libraries, there will always be some non-responsive maintainers. We have to be able to cope with that. Furthermore, library maintainers will occasionally become permanently unavailable. That's why we are (at least initially) requiring a library's public repo be hosted at GitHub, thus allowing us to appoint a new maintainer to take over the library and eliminate the chance of the public repo disappearing unexpectedly. --Beman