On 04.12.2013 10:14, Andrey Semashev wrote:

On Wed, Dec 4, 2013 at 10:00 AM, Vladimir Prus wrote:

...

I am not sure how maintenance of subproject references in the super-project will work.

- For 'master' branch, I assume that as developers update 'master' branch in their libraries, they would submit pull requests, so that the super-project points at new commits. That shall work, except that pull requests like this are hard to examine. E.g:

https://github.com/boostorg/boost/pull/4

If I did not specify direct links to commits, I don't know how anybody would decide whether new revision is OK or not.

- For 'develop' branch, I certainly don't think that for every change to 'develop' branch of any library, there will be a pull request on superproject? Doing so would create a lot of administrative overhead, and not doing that means that checking out 'develop' branch of the superproject does not necessary pull heads of 'develop' branches of each library.

Am I missing something?

As far as I understand, you don't have to create pull requests for master and develop branches. The actual revision is pulled from submodules automatically as you check out the monolithic boost (the boostorg superproject). The pull requests might be needed during the release cycle, if you want specific last minute changes pulled by the release managers so that they get into release.

Well, the superproject has references to specific revisions of each component. Or am I mistaken on this part? - Volodya

On Tue, Dec 3, 2013 at 11:35 PM, Vladimir Prus wrote:

On 04.12.2013 10:28, Vladimir Prus wrote:

As far as I understand, you don't have to create pull requests for

...

...

master and develop branches. The actual revision is pulled from submodules automatically as you check out the monolithic boost (the boostorg superproject). The pull requests might be needed during the release cycle, if you want specific last minute changes pulled by the release managers so that they get into release.

Well, the superproject has references to specific revisions of each component. Or am I mistaken on this part?

Looks like I'm not. I did this:

ghost@solid:~/Sources/boost-modular$ git checkout master M libs/program_options Switched to branch 'master' ghost@solid:~/Sources/boost-modular$ git submodule update --init Submodule path 'libs/program_options': checked out ' fb4f36f3eecbef1e3b4ace5b263e01d56e42d5c7'

This revision being checked out is what the current superproject repo specifies, which is 2 commits behind master of boostorg/program_options library.

So, yes, without pull request, or manual updates, or some scripting, checking out the superproject will not get either 'master' or 'develop' state of all libraries.

The following should get you what your asking for, if I understand you correctly: git clone --recursive -b develop https://github.com/boostorg/boost git submodule foreach git checkout develop Now I'm not sure that's what you need. You will get the latest changes from each of the repositories, but what are the entrance criteria for adding something to develop or master? Does it have to compile? Do all the changes have unit-tests that pass? I would assume the criteria for master would be more strict that develop, e.g. all changes on develop for a repository have to build with the latest changes on all the other repositories while changes on master have to also pass all their unit-tests.

- Volodya

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/ mailman/listinfo.cgi/boost

Given that git develop == svn trunk and git master == svn branches/release, I expect the criteria to be the same as with svn. The essential point is that both develop and master should be checked out at the latest revision by default, so that the actual code is tested and released. The updates to the latest revision should happen transparently, without requiring library maintainers to issue pull requests or anything.

Amen. I don't think I understood half of this thread, what I did understand was deeply "will to live" removing :-( Lets keep it simple folks, John.

On 12/04/2013 08:39 AM, Vladimir Prus wrote:

On 04.12.2013 11:27, Cox, Michael wrote:

...

On Tue, Dec 3, 2013 at 11:35 PM, Vladimir Prus wrote:

...

On 04.12.2013 10:28, Vladimir Prus wrote:

As far as I understand, you don't have to create pull requests for

...

...

master and develop branches. The actual revision is pulled from submodules automatically as you check out the monolithic boost (the boostorg superproject). The pull requests might be needed during the release cycle, if you want specific last minute changes pulled by the release managers so that they get into release.

Well, the superproject has references to specific revisions of each component. Or am I mistaken on this part?

Looks like I'm not. I did this:

ghost@solid:~/Sources/boost-modular$ git checkout master M libs/program_options Switched to branch 'master' ghost@solid:~/Sources/boost-modular$ git submodule update --init Submodule path 'libs/program_options': checked out ' fb4f36f3eecbef1e3b4ace5b263e01d56e42d5c7'

This revision being checked out is what the current superproject repo specifies, which is 2 commits behind master of boostorg/program_options library.

So, yes, without pull request, or manual updates, or some scripting, checking out the superproject will not get either 'master' or 'develop' state of all libraries.

The following should get you what your asking for, if I understand you correctly:

git clone --recursive -b develop https://github.com/boostorg/boost

Let's assume that I've already did this, and others checked in more changes, in which case...

...

git submodule foreach git checkout develop

... this command won't be sufficient, as you'd something like:

git submodule foreach git fetch

?

I am probably confused as what you are trying to do, but I would simply do. git checkout master git submodule update ... everything is master (svn branches/release) git checkout develop git submodule update ... everything is develop (svn trunk) if your working trees or index is not clean, git may be cautious about letting you do this to save you from loosing data. -- Bjørn

...

I am probably confused as what you are trying to do,but I would simply do.

git checkout master git submodule update ... everything is master (svn branches/release)

Uhm, no. Everything is detached head corresponding to submodule references in superproject.

Which means, again, that whatever I put to 'master' of my library is not ending up as part of regression tests, or next release, unless I also do additional dance, which is inconvenient for me, and extra work for those with write access to 'boost' superproject.

the testers should probably ignore the superproject and check out the master or develop branches on *all* submodules. otherwise i don't see how we can have consistent regression tests ... tim

On Dec 4, 2013 12:40 AM, "Vladimir Prus" wrote:

On 04.12.2013 11:27, Cox, Michael wrote:

...

On Tue, Dec 3, 2013 at 11:35 PM, Vladimir Prus wrote:

...

On 04.12.2013 10:28, Vladimir Prus wrote:

As far as I understand, you don't have to create pull requests for

...

...

master and develop branches. The actual revision is pulled from submodules automatically as you check out the monolithic boost (the boostorg superproject). The pull requests might be needed during the release cycle, if you want specific last minute changes pulled by the release managers so that they get into release.

Well, the superproject has references to specific revisions of each component. Or am I mistaken on this part?

Looks like I'm not. I did this:

ghost@solid:~/Sources/boost-modular$ git checkout master M libs/program_options Switched to branch 'master' ghost@solid:~/Sources/boost-modular$ git submodule update

--init

...

...

Submodule path 'libs/program_options': checked out ' fb4f36f3eecbef1e3b4ace5b263e01d56e42d5c7'

This revision being checked out is what the current superproject repo specifies, which is 2 commits behind master of boostorg/program_options library.

So, yes, without pull request, or manual updates, or some scripting, checking out the superproject will not get either 'master' or 'develop' state of all libraries.

The following should get you what your asking for, if I understand you correctly:

git clone --recursive -b develop https://github.com/boostorg/boost

Let's assume that I've already did this, and others checked in more changes, in which case...

...

git submodule foreach git checkout develop

... this command won't be sufficient, as you'd something like:

git submodule foreach git fetch

Yes, I was assuming you were starting from scratch, hence the "git clone ...". But this really isn't sufficient either, since fetch doesn't affect the working directories. So my new revised recipes are: To create a brand new local repository: git clone --recursive -b develop https://github.com/boostorg/boost git submodule foreach --recursive git checkout develop For a "stale" repository initially checked out as above, to freshen it up to the latest in boostorg: git pull # Updates the super-project. git submodule foreach --recursive git pull BTW, there's nothing stopping developers from checking out multiple local repos. From my brief exposure to Mecurial, that seemed to be the preferred method early on (I think they've improved Hg considerably since I looked at it last). Even with git's extensive branch handling capabilities, for a given work flow of work you can substitute a repository for a branch and vice versa, e.g. one could create a local repository, boost-develop.git where the develop branch was named master that tracked the upstream repository boost.git's branch develop. That would be the more traditional non-gitflow usage of git. git clone --recursive -b develop https://github.com/boostorg/boost-devel git checkout -b master -t origin/develop # Makes the local master track the remotes develop branch I'm not recommending this, just showing how a more traditional git branch usage could be setup. Michael

- Volodya

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On 04.12.2013 13:13, Daniel James wrote:

On 4 December 2013 07:27, Cox, Michael wrote:

...

The following should get you what your asking for, if I understand you correctly:

git clone --recursive -b develop https://github.com/boostorg/boost

git submodule foreach git checkout develop

Most developers won't do this, they'll be using the master branch of the super project, and only checking out develop for the modules they're concerned with.

And then any incompatibilities are discovered a week before release, I suppose? - Volodya

On 04.12.2013 13:44, Daniel James wrote:

On 4 December 2013 09:40, Vladimir Prus wrote:

...

On 04.12.2013 13:13, Daniel James wrote:

...

On 4 December 2013 07:27, Cox, Michael wrote:

...

The following should get you what your asking for, if I understand you correctly:

git clone --recursive -b develop https://github.com/boostorg/boost

git submodule foreach git checkout develop

Most developers won't do this, they'll be using the master branch of the super project, and only checking out develop for the modules they're concerned with.

And then any incompatibilities are discovered a week before release, I suppose?

Well, you should only be making bug fixes a week before release, and changes for an upcoming release shouldn't be made in the develop branch. The idea is to use the git flow process to manage changes.

The point is I change module X and you change module Y and these changes collide, then having us test against previously-released version of Boost, or some random-state-of-submodule-references, as opposed to master branches of everything, will necessary mean the collision will be detected later in release cycle. - Volodya

On 04.12.2013 15:13, Daniel James wrote:

On 4 December 2013 10:34, Vladimir Prus wrote:

...

On 04.12.2013 13:44, Daniel James wrote:

...

On 4 December 2013 09:40, Vladimir Prus wrote:

...

On 04.12.2013 13:13, Daniel James wrote:

...

On 4 December 2013 07:27, Cox, Michael wrote:

...

The following should get you what your asking for, if I understand you correctly:

git clone --recursive -b develop https://github.com/boostorg/boost

git submodule foreach git checkout develop

Most developers won't do this, they'll be using the master branch of the super project, and only checking out develop for the modules they're concerned with.

And then any incompatibilities are discovered a week before release, I suppose?

Well, you should only be making bug fixes a week before release, and changes for an upcoming release shouldn't be made in the develop branch. The idea is to use the git flow process to manage changes.

The point is I change module X and you change module Y and these changes collide, then having us test against previously-released version of Boost, or some random-state-of-submodule-references, as opposed to master branches of everything, will necessary mean the collision will be detected later in release cycle.

You'll be testing against the master branches of anything.

We're back to square one. Checking out master branch of the superproject will only get you master branches of everything if: - as soon as anybody commits anything to master branch of his library, he creates a pull request for super project - this pull request is merged quickly Do you expect both conditions will hold? - Volodya

On 04.12.2013 15:27, Daniel James wrote:

On 4 December 2013 11:16, Vladimir Prus wrote:

...

We're back to square one.

You made a lot of assumptions and had a discussion where no one knew what they were talking about. So you have to go back to square one.

...

Checking out master branch of the superproject will only get you master branches of everything if:

- as soon as anybody commits anything to master branch of his library, he creates a pull request for super project

- this pull request is merged quickly

Do you expect both conditions will hold?

Why do you think we're going to be using pull requests for the master repository?

Because no other mechanism has been documented anywhere that I can see, and because at this point, if some other mechanism is to be used, it would have been designed and documented already, prior to switching to git? - Volodya

In the git world, it's more common for develop to occur on "master" and for set releases to go to a "release" branch (with tags). On Wed, Dec 4, 2013 at 6:33 AM, Vladimir Prus wrote:

On 04.12.2013 15:27, Daniel James wrote:

...

On 4 December 2013 11:16, Vladimir Prus wrote:

...

We're back to square one.

You made a lot of assumptions and had a discussion where no one knew what they were talking about. So you have to go back to square one.

Checking out master branch of the superproject

...

will only get you master branches of everything if:

- as soon as anybody commits anything to master branch of his library, he creates a pull request for super project

- this pull request is merged quickly

Do you expect both conditions will hold?

Why do you think we're going to be using pull requests for the master repository?

Because no other mechanism has been documented anywhere that I can see, and because at this point, if some other mechanism is to be used, it would have been designed and documented already, prior to switching to git?

- Volodya

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/ mailman/listinfo.cgi/boost

On 4 December 2013 11:36, Jeremy Ong wrote:

In the git world, it's more common for develop to occur on "master" and for set releases to go to a "release" branch (with tags).

In the git world, "master" is nothing else than just a default branch. In the git world, you *never* do development on "master", but a topic branch. In the git world, "master" is not equivalent to SVN trunk, but again, it's just a default branch. How you treat your default branch (latest vanilla upstream or latest stable or any other way), depends on a workflow of your choice. In the git world, numerous (or hardly finite number) workflows are possible, all are equally valid. Boost uses gitflow. Best regards, -- Mateusz Łoskot, http://mateusz.loskot.net

On Wed, Dec 4, 2013 at 1:13 PM, Daniel James wrote:

On 4 December 2013 07:27, Cox, Michael wrote:

...

The following should get you what your asking for, if I understand you correctly:

git clone --recursive -b develop https://github.com/boostorg/boost

git submodule foreach git checkout develop

Most developers won't do this, they'll be using the master branch of the super project, and only checking out develop for the modules they're concerned with.

Although our testing system won't be doing that at first, and probably not for some time. Which is a problem.

As long as the testing is done against develop or master branch of *all* modules I don't see why individual developers should be doing that.

On Wed, Dec 4, 2013 at 3:14 PM, Daniel James wrote:

On 4 December 2013 09:53, Andrey Semashev wrote:

...

On Wed, Dec 4, 2013 at 1:13 PM, Daniel James wrote:

...

On 4 December 2013 07:27, Cox, Michael wrote:

...

The following should get you what your asking for, if I understand you correctly:

git clone --recursive -b develop https://github.com/boostorg/boost

git submodule foreach git checkout develop

Most developers won't do this, they'll be using the master branch of the super project, and only checking out develop for the modules they're concerned with.

Although our testing system won't be doing that at first, and probably not for some time. Which is a problem.

As long as the testing is done against develop or master branch of *all* modules I don't see why individual developers should be doing that.

Maybe you should try looking at older threads where this was discussed.

I remember the discussions but still stand by my words. There's no point in testing locally something that will possibly break in the official test system.

On 04.12.2013 11:32, Andrey Semashev wrote:

I just checked out the super-project:

git clone --recursive http://github.com/boostorg/boost.git modular-boost

It actually said:

Submodule path 'libs/program_options': checked out 'fb4f36f3eecbef1e3b4ace5b263e01d56e42d5c7'

But after these commands I saw the newer revisions:

cd modular-boost/libs/program_options git checkout master Switched to branch 'master'

git log -1 commit 9d7c9875265ec7fec03aabf18e3d53c3e5e1a46c Author: Vladimir Prus Date: Wed Dec 4 09:17:17 2013 +0400

Remove tabs.

I'm not sure what the "Submodule" line means, but it seems the repository is cloned with the newer revisions.

That's because git does complete repository clone. But then, it checks out the revision specified for each submodule (see 'git ls-tree master:libs' the the exact specification). I don't think it can automatically checkout symbolic reference (such as 'master'), so either we need to document things like 'git submodule foreach' use, or use some scripting to update submodule references periodically. - Volodya

Andrey Semashev writes:

On Wed, Dec 4, 2013 at 12:06 PM, Vladimir Prus wrote:

...

On 04.12.2013 11:32, Andrey Semashev wrote:

...

I just checked out the super-project:

git clone --recursive http://github.com/boostorg/boost.git modular-boost

It actually said:

Submodule path 'libs/program_options': checked out 'fb4f36f3eecbef1e3b4ace5b263e01d56e42d5c7'

But after these commands I saw the newer revisions:

cd modular-boost/libs/program_options git checkout master Switched to branch 'master'

git log -1 commit 9d7c9875265ec7fec03aabf18e3d53c3e5e1a46c Author: Vladimir Prus Date: Wed Dec 4 09:17:17 2013 +0400

Remove tabs.

I'm not sure what the "Submodule" line means, but it seems the repository is cloned with the newer revisions.

man git-submodule says that the submodule involves something called "gitlink", which refers to the particular revision. I don't know what this is but is it possible to specify HEAD instead of the specific revisions in gitlinks?

It's not possible. Git submodules were purposely designed to only let you link to a specific revision. This is A Good Thing and different from how svn:externals work. The reason for fixed revision links is that a particular commit in the super project is then a snapshot of the entire source tree, submodules and everything. If you could link to a submodule HEAD it would never be possible to get a stable snapshot of superproject commit blah023blah034blah093blah because the source of the submodule may have changed since blah023blah034blah093blah was committed. In short, a dependency nightmare. Git submodules make it very explicit what was intended to link to what. If you want to change the linkage you have to do it _explicitly_ by committing an updated submodule link. That way noone is left confused about what the commit author really intended. Yes, this confused me horribly when I started using git too. Now I can't imagine why I ever wanted it otherwise. Alex -- Swish - Easy SFTP for Windows Explorer (http://www.swish-sftp.org)

Philippe Vaucher writes:

...

Git submodules make it very explicit what was intended to link to what. If you want to change the linkage you have to do it _explicitly_ by committing an updated submodule link. That way noone is left confused about what the commit author really intended.

Yes, this confused me horribly when I started using git too. Now I can't imagine why I ever wanted it otherwise.

I think it's worth mentionning that git can track the tip of a branch now: http://stackoverflow.com/questions/9189575/git-submodule-tracking-latest/918...

If I'm understanding it correctly, that doesn't change things to automatically update the submodule to the tip; it just makes it easier to guide the manual updating along a particualr branch? Alex -- Swish - Easy SFTP for Windows Explorer (http://www.swish-sftp.org)

On 4 December 2013 06:35, Vladimir Prus wrote:

So, yes, without pull request, or manual updates, or some scripting, checking out the superproject will not get either 'master' or 'develop' state of all libraries.

Yes. I think it should eventually be scripted, but at the moment it's manual.

On Wed, Dec 4, 2013 at 8:06 AM, Philippe Vaucher

wrote:

As said on irc I believe there are 3 main solutions to this problem:

1. every library maintainer also has write access to the super repo and updates the super repo submodule references themselves. 2. someone is responsible for updating the super repo (think linus for the linux kernel), and makes a bit of review before updating the submodule ref that changes 3. automatic submodule update for the master/develop branches whenever something was pushed to master or develop in the submodule

Philippe, please do not top post. See the Boost discussion guidelines http://www.boost.org/community/policy.html#quoting --Beman

On 5/12/2013 10:13, Quoth Beman Dawes:

The eventual goal is an automated (i.e. continuous integration) approach that detects changes to each library's public repo master branch, updates a local working copy of the super-project (on a test slave) to the new master head, then if and only if acceptance conditions are met, commits and pushes the super-project. The maintainer(s) are automatically notified of acceptance or rejection, with reasons.

The conditions for acceptance would include passing tests (or markups), inspection, etc.

Given that (as I understand it) the super-project master was produced from the SVN release branch and develop from the SVN trunk, I assume that means that: 1. the future automated process you describe above would be updating only "develop" on the super-project (but from "master" on the sub-projects). 2. the super-project "master" will be updated manually by the release managers, either periodically or just when it gets close to release time, from some combination of prior sub-project "releases" (aka master commits) that appears to be stable according to testing from "develop". Is that correct?