Git Modularization Review no vote heads-up

older
[Atomic] Rationale for preventing...

Beman Dawes

9 May 2013 9 May '13

11:43 a.m.

If the steering committee's Git Modularization Review vote were held today, I've vote no since I think that we aren't yet ready. Since my concerns are apparently easy to fix technically, I'm mentioning them here to give Dave and Daniel a chance to address them before the actual Git Modularization Review starts. Concerns: 1) The mapping of svn branch names to modular git branch names needs to be revised: Svn "trunk" needs to become modular git "develop", not "master". Modular git needs to have a branch "master" that represents the latest stable release. Whether the content is identical to the last boost release or to branches/release at point of conversion needs to be decided, as does what the history, if any, of this branch looks like. 2) The procedures described in https://svn.boost.org/trac/boost/wiki/TryModBoost need to be updated, the dependency on CMake needs to be removed, and the procedures need to work. These are blocking issues because they prevent development and testing of modular boost testing procedures, developers procedures, release procedures, installation procedures, and documentation. Until they are resolved, the entire modular boost conversion rests on the shoulders of Dave and Daniel. Once they are resolved, others can pitch in and help since from that point forward we will be in a pure git environment and detailed knowledge of the conversion process from svn is not required. --Beman

Show replies by date

Bjørn Roald

9 May 9 May

1:46 p.m.

On Thu, 2013-05-09 at 07:43 -0400, Beman Dawes wrote:

...

2) The procedures described in https://svn.boost.org/trac/boost/wiki/TryModBoost need to be updated, the dependency on CMake needs to be removed, and the procedures need to work.

I tried to check this out, I think Steven Watanabe did some test with a modification to b2 to automate this. So my intention was to look for it and possibly try it or ping Steven. However trying the steps posted on the wiki seems to fail on the git submodule update --init step. There is no .gitmodules file in the top directory modular-boost master branch HEAD, so I guess there is no way git submodule know what and where to fetch from. It silently completes as there is nothing to do. There must be something that is disconnected here. -- Bjørn Roald

Beman Dawes

2:40 p.m.

On Thu, May 9, 2013 at 9:46 AM, Bjørn Roald wrote:

...

On Thu, 2013-05-09 at 07:43 -0400, Beman Dawes wrote:

...
2) The procedures described in https://svn.boost.org/trac/boost/wiki/TryModBoost need to be updated, the dependency on CMake needs to be removed, and the procedures need to work.

I tried to check this out, I think Steven Watanabe did some test with a modification to b2 to automate this. So my intention was to look for it and possibly try it or ping Steven.

However trying the steps posted on the wiki seems to fail on the

git submodule update --init

step.

There is no .gitmodules file in the top directory modular-boost master branch HEAD, so I guess there is no way git submodule know what and where to fetch from. It silently completes as there is nothing to do. There must be something that is disconnected here.

Yes, and that's part of why I'm concerned. The modularization of the individual libraries is now working nicely with full history for all branches, but the modular-boost master hasn't been handling submodules for quite a while now. --Beman

Nasos Iliopoulos

6:29 p.m.

Hello, It seems to me also that there should be some more iterations or some more relevant documentation. I would feel more comfortable using submodules, but without them my best attempt up to now is (ublas as an example here): git clone https://github.com/boostorg/boost modular-boost cd modular-boost {manually relpace /v2 with /buildv2 in boost-build.jam and bootstrap.sh} cd tools git clone https://github.com/boostorg/build cd .. ./bootstrap.sh cd libs/numeric git clone https://github.com/boostorg/ublas cd ../../ ./b2 include=libs/numeric/ublas/include libs/numeric/ublas/bench1 (to compile bench1 for example) This relies on the assumption that the rest of boost is in an accessible location in the system. -Nasos On 05/09/2013 10:40 AM, Beman Dawes wrote:

...

On Thu, May 9, 2013 at 9:46 AM, Bjørn Roald wrote:

...
On Thu, 2013-05-09 at 07:43 -0400, Beman Dawes wrote:

...
2) The procedures described in https://svn.boost.org/trac/boost/wiki/TryModBoost need to be updated, the dependency on CMake needs to be removed, and the procedures need to work. I tried to check this out, I think Steven Watanabe did some test with a modification to b2 to automate this. So my intention was to look for it and possibly try it or ping Steven.

However trying the steps posted on the wiki seems to fail on the

git submodule update --init

step.

There is no .gitmodules file in the top directory modular-boost master branch HEAD, so I guess there is no way git submodule know what and where to fetch from. It silently completes as there is nothing to do. There must be something that is disconnected here. Yes, and that's part of why I'm concerned. The modularization of the individual libraries is now working nicely with full history for all branches, but the modular-boost master hasn't been handling submodules for quite a while now.

--Beman

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Dave Abrahams

10 May 10 May

1:52 p.m.

I personally find the subject line of this posting needlessly alarming. on Thu May 09 2013, Beman Dawes wrote:

...

If the steering committee's Git Modularization Review vote

What vote? We asked for a review period; maybe I shouldn't have used that word. The intention was to give people a chance to make corrections to the way things were being modularized so we could move on to the next step. https://github.com/ryppl/Boost2Git/wiki#vetting-period

...

were held today, I've vote no since I think that we aren't yet ready. Since my concerns are apparently easy to fix technically, I'm mentioning them here to give Dave and Daniel a chance to address them before the actual Git Modularization Review starts.

Concerns:

1) The mapping of svn branch names to modular git branch names needs to be revised: Svn "trunk" needs to become modular git "develop", not "master". Modular git needs to have a branch "master" that represents the latest stable release. Whether the content is identical to the last boost release or to branches/release at point of conversion needs to be decided, as does what the history, if any, of this branch looks like.

Decided by whom? If it's up to us, we'll do whatever is expedient to avoid the threat of a "no vote"

...

2) The procedures described in https://svn.boost.org/trac/boost/wiki/TryModBoost need to be updated, the dependency on CMake needs to be removed, and the procedures need to work.

Where did these prerequisites come from? You wrote the web page; are you going to update it? https://svn.boost.org/trac/boost/wiki/TryModBoost?action=history

...

These are blocking issues because they prevent development and testing of modular boost testing procedures, developers procedures, release procedures, installation procedures, and documentation. Until they are resolved, the entire modular boost conversion rests on the shoulders of Dave and Daniel. Once they are resolved, others can pitch in and help since from that point forward we will be in a pure git environment and detailed knowledge of the conversion process from svn is not required.

I understand that these are blocking issues for the switchover. We're not asking people to approve an immediate switchover; we're just asking for people to make fixes and requests regarding how things are sorted into modules. -- Dave Abrahams

Beman Dawes

12 May 12 May

7:52 p.m.

On Fri, May 10, 2013 at 9:52 AM, Dave Abrahams wrote:

...

I personally find the subject line of this posting needlessly alarming.

...

From my viewpoint, it seems at least somewhat alarming that we are going into a review soon with the critical "develop" and "master" branches still not having the right names, and the https://github.com/boostorg/boost super-project for modular boost not usable for evaluation (or at least "git submodule update --init" apparently no longer working).

...

on Thu May 09 2013, Beman Dawes wrote:

...
If the steering committee's Git Modularization Review vote

What vote? We asked for a review period; maybe I shouldn't have used that word.

The reviews usually end with a Yes, No, or perhaps Conditionally Yes vote from the review submitter.

...

The intention was to give people a chance to make corrections to the way things were being modularized so we could move on to the next step. https://github.com/ryppl/Boost2Git/wiki#vetting-period

...
were held today, I've vote no since I think that we aren't yet ready. Since my concerns are apparently easy to fix technically, I'm mentioning them here to give Dave and Daniel a chance to address them before the actual Git Modularization Review starts.

Concerns:

1) The mapping of svn branch names to modular git branch names needs to be revised: Svn "trunk" needs to become modular git "develop", not "master". Modular git needs to have a branch "master" that represents the latest stable release. Whether the content is identical to the last boost release or to branches/release at point of conversion needs to be decided, as does what the history, if any, of this branch looks like.

Decided by whom? If it's up to us, we'll do whatever is expedient to avoid the threat of a "no vote"

Let's break that down into two questions. First, "Whether the content [of Master] is identical to the last boost release or to branches/release at point of conversion" Suggestion: We freeze the entire svn repo when the 1.54 release candidate is build. That becomes the point in time of conversion to git. IOW, the last boost release == branches/release at point of conversion to git, so the question becomes moot. Can we reach a consensus on that, and if necessary get it ratified by the Steering Committee? Second, "What does the history, if any, of master look like?" I've heard three possibilities: 1) History based on the difference between release tags. 2) History of the current svn branches/release. 3) No history at all. (The tags are still available, so if anyone cares they diff the differences.) (1) or (2) are my preference, but (3) is also acceptable. I wouldn't want the choice to delay the schedule. My feeling is that you should make the call, after consulting with Daniel. You have a broad overview of the needs of the community and also understand the history mechanism much better than most of us.

...

...
2) The procedures described in https://svn.boost.org/trac/boost/wiki/TryModBoost need to be updated, the dependency on CMake needs to be removed, and the procedures need to work.

Where did these prerequisites come from?

By looking at the prerequisites for each of the command line steps in https://svn.boost.org/trac/boost/wiki/TryModBoost.

...

You wrote the web page; are you going to update it? https://svn.boost.org/trac/boost/wiki/TryModBoost?action=history

I'm being blocked by the "git submodule update --init" step not working, and not knowing what the replacement is for the "cmake -P forward_headers.cmake" step.

...

...
These are blocking issues because they prevent development and testing of modular boost testing procedures, developers procedures, release procedures, installation procedures, and documentation. Until they are resolved, the entire modular boost conversion rests on the shoulders of Dave and Daniel. Once they are resolved, others can pitch in and help since from that point forward we will be in a pure git environment and detailed knowledge of the conversion process from svn is not required.

I understand that these are blocking issues for the switchover. We're not asking people to approve an immediate switchover; we're just asking for people to make fixes and requests regarding how things are sorted into modules.

It isn't just that they are blocking issues for the switchover. They are also blocking issues for making progress on documentation and developing test and release procedures. That said, the separation of libraries into separate modules with full history is looking very much improved! --Beman

Dave Abrahams

13 May 13 May

5 a.m.

on Sun May 12 2013, Beman Dawes wrote:

...

On Fri, May 10, 2013 at 9:52 AM, Dave Abrahams wrote:

...
I personally find the subject line of this posting needlessly alarming.

From my viewpoint, it seems at least somewhat alarming that we are going into a review soon with the critical "develop" and "master" branches still not having the right names,

It's clear that you are somewhat alarmed, yes. The "review" that we're asking for is there in part so that the community can decide what "the right names" are. As Daniel pointed out quite nicely I think, there is no single obvious right answer. That said, if you think the right answer is obvious, we can make the necessary change at a moment's notice. Just tell us what you want. Anything we do is easily reversed.

...

and the https://github.com/boostorg/boost super-project for modular boost not usable for evaluation (or at least "git submodule update --init" apparently no longer working).

That's because when people asked for modularized history, what we were doing previously—just creating a snapshot of the submodules where they are, going forward—was no longer the correct answer. I could easily provide you with a 10-line bash script that checks the head of all the submodules out at their proper locations, but getting them to be correctly represented though the entire history of Boost is requiring a lot of coding (I think I'm very close, though).

...

...
on Thu May 09 2013, Beman Dawes wrote:

...
If the steering committee's Git Modularization Review vote

What vote? We asked for a review period; maybe I shouldn't have used that word.

The reviews usually end with a Yes, No, or perhaps Conditionally Yes vote from the review submitter.

Yes, "review" was the wrong word; I see that now.

...

...
The intention was to give people a chance to make corrections to the way things were being modularized so we could move on to the next step. https://github.com/ryppl/Boost2Git/wiki#vetting-period

...
were held today, I've vote no since I think that we aren't yet ready. Since my concerns are apparently easy to fix technically, I'm mentioning them here to give Dave and Daniel a chance to address them before the actual Git Modularization Review starts.

Concerns:

1) The mapping of svn branch names to modular git branch names needs to be revised: Svn "trunk" needs to become modular git "develop", not "master". Modular git needs to have a branch "master" that represents the latest stable release. Whether the content is identical to the last boost release or to branches/release at point of conversion needs to be decided, as does what the history, if any, of this branch looks like.

Decided by whom? If it's up to us, we'll do whatever is expedient to avoid the threat of a "no vote"

Let's break that down into two questions.

First, "Whether the content [of Master] is identical to the last boost release or to branches/release at point of conversion"

Suggestion: We freeze the entire svn repo when the 1.54 release candidate is build. That becomes the point in time of conversion to git. IOW, the last boost release == branches/release at point of conversion to git, so the question becomes moot. Can we reach a consensus on that, and if necessary get it ratified by the Steering Committee?

I don't understand how this relates to what we're asking people to evaluate. We want to know if they're happy with where the files are going in the Git repos and what branches and/or tags they're being directed to. If people aren't happy, we want to make adjustments. A perfect example of the kind of information we need is in http://boost.2283326.n4.nabble.com/Git-Modularization-Ready-for-Review-tp464... where Vicente Botet wrote, | Could I move the following from repository thread to core? | | "boost/detail/atomic_redef_macros.hpp" : | "include/boost/detail/atomic_redef_macros.hpp"; | "boost/detail/atomic_undef_macros.hpp" : | "include/boost/detail/atomic_undef_macros.hpp"; I added these | files. They are used now by Boost.Thread, but it should be used by | SmartPtr (See https://svn.boost.org/trac/boost/ticket/6842 and | https://svn.boost.org/trac/boost/ticket/6843).

...

Second, "What does the history, if any, of master look like?"

Wait; don't we care about the history of any of the other branches? Daniel and I have spent months getting that right because the community insisted on it.

...

I've heard three possibilities:

1) History based on the difference between release tags. 2) History of the current svn branches/release. 3) No history at all. (The tags are still available, so if anyone cares they diff the differences.)

(1) or (2) are my preference,

2 is trivially accomplished. 1 would take more work.

...

but (3) is also acceptable. I wouldn't want the choice to delay the schedule. My feeling is that you should make the call, after consulting with Daniel. You have a broad overview of the needs of the community and also understand the history mechanism much better than most of us.

Well, I don't personally think history matters that much, and I think having a history that matches git-flow's usage matters even less, since after all we haven't actually been using git-flow, so it would be artificial. It would be nice to do a "perfect" job, but I'm not sure it's worth the effort.

...

...
...
2) The procedures described in https://svn.boost.org/trac/boost/wiki/TryModBoost need to be updated, the dependency on CMake needs to be removed, and the procedures need to work.

Where did these prerequisites come from?

By looking at the prerequisites for each of the command line steps in https://svn.boost.org/trac/boost/wiki/TryModBoost.

...
You wrote the web page; are you going to update it? https://svn.boost.org/trac/boost/wiki/TryModBoost?action=history

I'm being blocked by the "git submodule update --init" step not working, and not knowing what the replacement is for the "cmake -P forward_headers.cmake" step.

These things are both important for making the transition, but they're not important to the particular review we were asking for.

...

...
...
These are blocking issues because they prevent development and testing of modular boost testing procedures, developers procedures, release procedures, installation procedures, and documentation. Until they are resolved, the entire modular boost conversion rests on the shoulders of Dave and Daniel. Once they are resolved, others can pitch in and help since from that point forward we will be in a pure git environment and detailed knowledge of the conversion process from svn is not required.

I understand that these are blocking issues for the switchover. We're not asking people to approve an immediate switchover; we're just asking for people to make fixes and requests regarding how things are sorted into modules.

It isn't just that they are blocking issues for the switchover. They are also blocking issues for making progress on documentation and developing test and release procedures.

Sure. I'm working hard on it.

...

That said, the separation of libraries into separate modules with full history is looking very much improved!

Well, I know it isn't glamorous and it doesn't satisfy the need to say "look it builds," but *that* is what we're asking people to evaluate now. -- Dave Abrahams

Vladimir Prus

18 May 18 May

8:14 a.m.

On 13.05.2013 09:00, Dave Abrahams wrote:

...

...
That said, the separation of libraries into separate modules with full history is looking very much improved!

Well, I know it isn't glamorous and it doesn't satisfy the need to say "look it builds," but *that* is what we're asking people to evaluate now.

So what are the plans for "look it builds"? At the very least, I would prefer that requirement to use cmake to put together symlinks be removed. What's the plan for that? Jürgen has posted a patch that teaches Boost.Build to do that, and I can give it a try, but before that I think cloning of the master repository should work (others already reported some issue with chrono) and at least Boost.Build should have its final layout. Also, what would be the process for tweaking any infrastructure code? Commit to SVN and wait until git mirror is updated? Or create a special branch/repo in git and do changes there, rebasing against new versions? Thanks, Volodya

Dave Abrahams

20 May 20 May

6:20 a.m.

on Sat May 18 2013, Vladimir Prus wrote:

...

On 13.05.2013 09:00, Dave Abrahams wrote:

...
...
That said, the separation of libraries into separate modules with full history is looking very much improved!

Well, I know it isn't glamorous and it doesn't satisfy the need to say "look it builds," but *that* is what we're asking people to evaluate now.

So what are the plans for "look it builds"? At the very least, I would prefer that requirement to use cmake to put together symlinks be removed. What's the plan for that?

Steven Watanabe is working on that. It looks like Jürgen Hunold and Eric Cornelius may have picked up where he left off? Jürgen has asked Eric to post about this on the boost-build list, so I presume you've seen some of it.

...

Jürgen has posted a patch that teaches Boost.Build to do that, and I can give it a try, but before that I think cloning of the master repository should work

It does work.

...

(others already reported some issue with chrono)

I don't know what's being referred to here.

...

and at least Boost.Build should have its final layout.

I think maybe I disagree. If you want to rearrange Boost.Build in SVN before the transition, then be my guest. Modularization will capture those changes and you'll have a corresponding layout in Git. However, I don't think it's reasonable to build a fake *history* in Git that doesn't reflect the way things went in SVN. That would make those running the Git transition responsible for any adjustments required for your preferred layout. Make sense?

...

Also, what would be the process for tweaking any infrastructure code? Commit to SVN and wait until git mirror is updated?

That would be my preference. The mirror updates fairly quickly. If rules or modularization code doesn't change, it's very fast (about 3.5 minutes). Otherwise, it takes about an hour, but we're not pushing ruleset or modularization code changes very often.

...

Or create a special branch/repo in git and do changes there, rebasing against new versions?

The process essentially clobbers and reconstructs the git repos continuously, so if you want to work in Git using the modularized repos, expect to have to rebase against new branch heads, and don't expect to be able to push to the github repos. -- Dave Abrahams

Dave Abrahams

6:36 a.m.

on Sun May 19 2013, Dave Abrahams wrote:

...

...
Jürgen has posted a patch that teaches Boost.Build to do that, and I can give it a try, but before that I think cloning of the master repository should work

It does work.

...
(others already reported some issue with chrono)

I don't know what's being referred to here.

OK, I've seen those posts, but I think the problem is fixed now. -- Dave Abrahams

Daniel Pfeifer

8:48 a.m.

2013/5/20 Dave Abrahams

...

on Sat May 18 2013, Vladimir Prus wrote:

...
and at least Boost.Build should have its final layout. I think maybe I disagree.

We definetly need to make some changes to the Boost.Build layout. Currently, we direct both "tools/build" and "tools/jam" to the "build" repository, then we want that repository to be a submodule in boost at the location "build". Hence, the directory "tools/build/v2" will appear as "tools/build/build/v2" which is not the same layout as in SVN. We could either use "tools/build" or "tools/build/v2" the root of repository "build". But we need to put the submodule at that exact location. Everything that is not matched (eg. jam/, index.html), can remain in the meta-repository "boost". Which one do you prefer? cheers, Daniel

Jürgen Hunold

6:41 p.m.

HI Daniel. On Monday, 20. May 2013 10:48:20 Daniel Pfeifer wrote:

...

2013/5/20 Dave Abrahams

...
on Sat May 18 2013, Vladimir Prus wrote:

...
and at least Boost.Build should have its final layout.

I think maybe I disagree.

We definetly need to make some changes to the Boost.Build layout.

The question is, when to do them.

...

Currently, we direct both "tools/build" and "tools/jam" to the "build" repository, then we want that repository to be a submodule in boost at the location "build". Hence, the directory "tools/build/v2" will appear as "tools/build/build/v2" which is not the same layout as in SVN.

Yes, this is clearly not what we want. But where does "tools/jam" come from? It is not in the current svn layout anymore. Seems to be some v1 relict. Can you investigate this?

...

We could either use "tools/build" or "tools/build/v2" the root of repository "build". But we need to put the submodule at that exact location. Everything that is not matched (eg. jam/, index.html), can remain in the meta-repository "boost".

We wanted "tools/build" to contain the current "tools/build/v2" of svn. I wonder if we should do the final move after the transition to git. Or just move the files _now_ and adjust the toplevel scripts before the transition.

...

Which one do you prefer?

Nevertheless +1 for "tools/build" from my side if this works smoothly. The main culprit is that I don't even have a "tools/build" submodule at the moment. Yours, Jürgen -- * Dipl.-Math. Jürgen Hunold ! * voice: ++49 4257 300 ! Fährstraße 1 * fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen * jhunold@gmx.eu ! Germany

Bjørn Roald

9:21 p.m.

On Mon, 2013-05-20 at 20:41 +0200, Jürgen Hunold wrote:

...

HI Daniel.

On Monday, 20. May 2013 10:48:20 Daniel Pfeifer wrote:

...
2013/5/20 Dave Abrahams

...
on Sat May 18 2013, Vladimir Prus wrote:

...
and at least Boost.Build should have its final layout.

I think maybe I disagree.

We definetly need to make some changes to the Boost.Build layout.

The question is, when to do them.

do them before or after conversion to git, not as part of it.

...

...
Currently, we direct both "tools/build" and "tools/jam" to the "build" repository, then we want that repository to be a submodule in boost at the location "build". Hence, the directory "tools/build/v2" will appear as "tools/build/build/v2" which is not the same layout as in SVN.

Yes, this is clearly not what we want. But where does "tools/jam" come from? It is not in the current svn layout anymore. Seems to be some v1 relict. Can you investigate this?

...
We could either use "tools/build" or "tools/build/v2" the root of repository "build". But we need to put the submodule at that exact location. Everything that is not matched (eg. jam/, index.html), can remain in the meta-repository "boost".

We wanted "tools/build" to contain the current "tools/build/v2" of svn. I wonder if we should do the final move after the transition to git. Or just move the files _now_ and adjust the toplevel scripts before the transition.

do you have a plan for how the top level scripts can work in both the svn and the git environments prior to transition if you change tools/build/v2 to tools/build?

...

...
Which one do you prefer?

Nevertheless +1 for "tools/build" from my side if this works smoothly.

I propose to keep tools/build/v2 until all work is transitioned to git. Then moving content of tools/build/v2 to tools/build is simple within build submodule repository and the corresponding boost meta repository can have modified top level scripts.

...

The main culprit is that I don't even have a "tools/build" submodule at the moment.

yes, why is that submodule not included in .gitmodules? It seems to be listed as submodule of boost in ryppl/Boost2Git/repositories.txt and there is a build repository in http://github.com/boostorg/build I guess, for now you may add that one manually to a branch in your clone: git checkout -b feature/fix-broken-build git submodule add http://github.com/boostorg/build tools/build but it suffers from the restructure of tools/build into tools/build/build :-( # you could edit the submodule for your own needs - whatever they # are, e.g: cd tools/build git checkout -b feature/fix-broken-build git mv build/* . git commit -m "only tools/build, not tools/build/build" # then add the changed build submodule to index and commit # the boost meta repository cd ../.. git add tools/build git commit -m "only tools/build, not tools/build/build" # then later, when you get updates upstream you could rebase your # changes on top of upstream commits from Boost2Git conversion # with something like git fetch git rebase origin/master -- Bjørn

Vladimir Prus

21 May 21 May

2:52 p.m.

On 21.05.2013 01:21, Bjørn Roald wrote:

...

do you have a plan for how the top level scripts can work in both the svn and the git environments prior to transition if you change tools/build/v2 to tools/build?

Even with my limited bash-fu, that is perfectly doable.

...

...
...
...
...
Which one do you prefer?

Nevertheless +1 for "tools/build" from my side if this works smoothly.

I propose to keep tools/build/v2 until all work is transitioned to git. Then moving content of tools/build/v2 to tools/build is simple within build submodule repository and the corresponding boost meta repository can have modified top level scripts.

To be honest, I don't trust git move (or lack thereof) and not really looking forward to spending an evening with company of 'git filter-branch'. If an infrastructure for svn to git conversion is already developed, why not create the layout that is right? Thanks, Volodya

Dave Abrahams

22 May 22 May

5:30 a.m.

on Tue May 21 2013, Vladimir Prus wrote:

...

On 21.05.2013 01:21, Bjørn Roald wrote:

...
do you have a plan for how the top level scripts can work in both the svn and the git environments prior to transition if you change tools/build/v2 to tools/build?

Even with my limited bash-fu, that is perfectly doable.

...
...
...
...
...
Which one do you prefer?

Nevertheless +1 for "tools/build" from my side if this works smoothly.

I propose to keep tools/build/v2 until all work is transitioned to git. Then moving content of tools/build/v2 to tools/build is simple within build submodule repository and the corresponding boost meta repository can have modified top level scripts.

To be honest, I don't trust git move (or lack thereof)

Well, there's nothing really we can do about that lack of trust. It's a simple matter to move files in Git.

...

and not really looking forward to spending an evening with company of 'git filter-branch'.

Why do you want to revise history?

...

If an infrastructure for svn to git conversion is already developed, why not create the layout that is right?

How do you define "right?" -- Dave Abrahams

Jürgen Hunold

7:56 a.m.

Hi Dave, On Tuesday, 21. May 2013 22:30:10 Dave Abrahams wrote:

...

on Tue May 21 2013, Vladimir Prus wrote:

...
On 21.05.2013 01:21, Bjørn Roald wrote: To be honest, I don't trust git move (or lack thereof)

Well, there's nothing really we can do about that lack of trust. It's a simple matter to move files in Git.

It this is the only rationale, then let us please abort the idea of history rewriting and start making things right after the conversion. git "move" just works fine, even with git svn ;-) And please remember that svn still has no atomic "mv", but just "rm/add" combination which is quite error prone, especially with svn merge. Been there, still doing that.

...

...
and not really looking forward to spending an evening with company of 'git filter-branch'.

Why do you want to revise history?

...
If an infrastructure for svn to git conversion is already developed, why not create the layout that is right?

How do you define "right?"

In this case reducing directory depth and removing artefacts from v1 -> v2 transition, I presume. But this can easily be done after the conversion. I think we already lost too much time with failed rewrite attempts, so let us get a working repository first. Then Volodya can test git mv v2/* . and report results. Afterwards, we can always try rewriting in a separate clone. Yours, Jürgen -- * Dipl.-Math. Jürgen Hunold ! * voice: ++49 4257 300 ! Fährstraße 1 * fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen * jhunold@gmx.eu ! Germany

Daniel James

5:38 p.m.

On 22 May 2013 08:56, Jürgen Hunold wrote:

...

And please remember that svn still has no atomic "mv", but just "rm/add" combination which is quite error prone,

That's what git does. Subversion's move command copies the file and then removes the original. Using subversion as if it is git will certainly create a mess.

Dave Abrahams

23 May 23 May

5:13 a.m.

on Wed May 22 2013, Daniel James wrote:

...

On 22 May 2013 08:56, Jürgen Hunold wrote:

...
And please remember that svn still has no atomic "mv", but just "rm/add" combination which is quite error prone,

That's what git does. Subversion's move command copies the file and then removes the original.

Daniel is correct. -- Dave Abrahams

Dave Abrahams

5:28 a.m.

on Wed May 22 2013, Jürgen Hunold wrote:

...

But this can easily be done after the conversion.

I think we already lost too much time with failed rewrite attempts, so let us get a working repository first. Then Volodya can test git mv v2/* . and report results. Afterwards, we can always try rewriting in a separate clone.

Rewriting _published_ history is most strongly discouraged by the Git people, for several good reasons. *If* there is to be any rearrangement, it should happen before the switchover, so it doesn't bork people who are doing work based on the history originally published. However, I am loath to do any rearrangement that doesn't (reasonably) faithfully reflect how things were set up in the past. Otherwise, someone will check out an old state of the super-module and find that things have the wrong path relationships. Of course, path relationships will not match SVN anyway (because we don't have a modular layout in SVN), but people on this list made it quite clear that modularizing history was important to them, so I presume they want the Git history to reflect reality with maximal fidelity. That said, if the consensus is that things should be rearranged in the build repository, we can do that. We just need clear and explicit instructions that cover what to do with *all* the paths that have been used in Boost.Build (including branches and tags) throughout time. As you can see from https://github.com/ryppl/Boost2Git/blob/master/repositories.txt#L2261 and following lines, just preserving the existing structure was nontrivial. -- Dave Abrahams

Dave Abrahams

6:15 a.m.

on Wed May 22 2013, Dave Abrahams wrote:

...

on Wed May 22 2013, Jürgen Hunold wrote:

...
But this can easily be done after the conversion.

I think we already lost too much time with failed rewrite attempts, so let us get a working repository first. Then Volodya can test git mv v2/* . and report results. Afterwards, we can always try rewriting in a separate clone.

Rewriting _published_ history is most strongly discouraged by the Git people, for several good reasons. *If* there is to be any rearrangement, it should happen before the switchover, so it doesn't bork people who are doing work based on the history originally published.

However, I am loath to do any rearrangement that doesn't (reasonably) faithfully reflect how things were set up in the past. Otherwise, someone will check out an old state of the super-module and find that things have the wrong path relationships. Of course, path relationships will not match SVN anyway (because we don't have a modular layout in SVN), but people on this list made it quite clear that modularizing history was important to them, so I presume they want the Git history to reflect reality with maximal fidelity.

That said, if the consensus is that things should be rearranged in the build repository, we can do that. We just need clear and explicit instructions that cover what to do with *all* the paths that have been used in Boost.Build (including branches and tags) throughout time. As you can see from https://github.com/ryppl/Boost2Git/blob/master/repositories.txt#L2261 and following lines, just preserving the existing structure was nontrivial.

Caveat: what I mean by "preserving existing structure" is that branches that include just a slice of the tools/build + tools/jam hierarchy have all the files located in the same places where they appear under trunk/. That's the appropriate way to map things for Git. -- Dave Abrahams

Daniel James

8:12 a.m.

On 23 May 2013 06:28, Dave Abrahams wrote:

...

Rewriting _published_ history is most strongly discouraged by the Git people, for several good reasons. *If* there is to be any rearrangement, it should happen before the switchover, so it doesn't bork people who are doing work based on the history originally published.

Maybe that could be avoided by having two repositories: the historical repo, which would retain perfect history, and the working repo, which would have the desired layout. The working repo could be created after the conversion, and have enough history to be useful for general development. The meta project could switch its reference over once the new repository has been set up. Would also prevent any extra delay to deal with this. Since the odeint developers want to use their git repository (https://github.com/headmyshoulder/odeint-v2) rather than the one created by the conversion, that distinction might be required there anyway.

"Jürgen Hunold"

8:30 a.m.

Hi Daniel,

...

Gesendet: Donnerstag, 23. Mai 2013 um 10:12 Uhr Von: "Daniel James" An: boost@lists.boost.org Betreff: Re: [boost] Git Modularization Review no vote heads-up

On 23 May 2013 06:28, Dave Abrahams wrote:

...
Rewriting _published_ history is most strongly discouraged by the Git people, for several good reasons. *If* there is to be any rearrangement, it should happen before the switchover, so it doesn't bork people who are doing work based on the history originally published.

Maybe that could be avoided by having two repositories: the historical repo, which would retain perfect history, and the working repo, which would have the desired layout. The working repo could be created after the conversion, and have enough history to be useful for general development. The meta project could switch its reference over once the new repository has been set up. Would also prevent any extra delay to deal with this.

Since the odeint developers want to use their git repository (https://github.com/headmyshoulder/odeint-v2) rather than the one created by the conversion, that distinction might be required there anyway.

Well, odeint is a rather new library. And for those it makes sense to just add the existing repository as a submodule on acceptance. AFAIK the odeint maintainers are just waiting for this to be possible. Yours, Jürgen -- * Dipl.-Math. Jürgen Hunold ! juergen.hunold@gmx.eu * voice: 0049 4257 300 ! Fährstraße 1 * fax : 0049 4257 300 ! 31609 Balge/Sebbenhausen * mobil: 0049 178 186 1566 ! Germany

Dave Abrahams

1:47 p.m.

on Thu May 23 2013, Daniel James wrote:

...

On 23 May 2013 06:28, Dave Abrahams wrote:

...
Rewriting _published_ history is most strongly discouraged by the Git people, for several good reasons. *If* there is to be any rearrangement, it should happen before the switchover, so it doesn't bork people who are doing work based on the history originally published.

Maybe that could be avoided by having two repositories: the historical repo, which would retain perfect history, and the working repo, which would have the desired layout. The working repo could be created after the conversion, and have enough history to be useful for general development. The meta project could switch its reference over once the new repository has been set up. Would also prevent any extra delay to deal with this.

If that were going to be acceptable to the community, presumably everyone would have been happy with the original plan to graft an accurate SVN history when people want to refer to it, no? Please don't tell me that the last several months of work were wasted!

...

Since the odeint developers want to use their git repository (https://github.com/headmyshoulder/odeint-v2) rather than the one created by the conversion, that distinction might be required there anyway.

It would be simpler to just replace the one generated by the conversion process with the one they're using,, IMO. -- Dave Abrahams

Daniel James

24 May 24 May

6:49 a.m.

On 23 May 2013 14:47, Dave Abrahams wrote:

...

on Thu May 23 2013, Daniel James wrote:

...

...
Maybe that could be avoided by having two repositories: the historical repo, which would retain perfect history, and the working repo, which would have the desired layout. The working repo could be created after the conversion, and have enough history to be useful for general development. The meta project could switch its reference over once the new repository has been set up. Would also prevent any extra delay to deal with this.

If that were going to be acceptable to the community, presumably everyone would have been happy with the original plan to graft an accurate SVN history when people want to refer to it, no? Please don't tell me that the last several months of work were wasted!

What's acceptable to one person is quite different to what's acceptable to another. It should ultimately be up to the maintainers of individual modules. Also, you would still have an accurate history in the historical module(s). Which was the point, it was an attempt at a compromise between two conflicting desires (to have an accurate history, and to have a repository with a different directory layout).

...

...
Since the odeint developers want to use their git repository (https://github.com/headmyshoulder/odeint-v2) rather than the one created by the conversion, that distinction might be required there anyway.

It would be simpler to just replace the one generated by the conversion process with the one they're using,, IMO.

I thought that would mess up the history of the meta-module, as it will refer to changesets generated by the svn to git conversion.

Dave Abrahams

30 May 30 May

10:27 p.m.

on Thu May 23 2013, Daniel James wrote:

...

On 23 May 2013 14:47, Dave Abrahams wrote:

...
on Thu May 23 2013, Daniel James wrote:

...
...
Maybe that could be avoided by having two repositories: the historical repo, which would retain perfect history, and the working repo, which would have the desired layout. The working repo could be created after the conversion, and have enough history to be useful for general development. The meta project could switch its reference over once the new repository has been set up. Would also prevent any extra delay to deal with this.

If that were going to be acceptable to the community, presumably everyone would have been happy with the original plan to graft an accurate SVN history when people want to refer to it, no? Please don't tell me that the last several months of work were wasted!

What's acceptable to one person is quite different to what's acceptable to another.

No kidding!

...

It should ultimately be up to the maintainers of individual modules.

If anyone expects the Boost master repository to contain accurate submodule references through history, maintainers must not be allowed to delete old branches or rewrite old history.

...

Also, you would still have an accurate history in the historical module(s). Which was the point, it was an attempt at a compromise between two conflicting desires (to have an accurate history, and to have a repository with a different directory layout).

As I've said before, Git *cannot* represent exactly what SVN represents. If you want an accurate history, use SVN to get it. The best we can do with Git is to approximate actual history.

...

...
...
Since the odeint developers want to use their git repository (https://github.com/headmyshoulder/odeint-v2) rather than the one created by the conversion, that distinction might be required there anyway.

It would be simpler to just replace the one generated by the conversion process with the one they're using, IMO.

I thought that would mess up the history of the meta-module, as it will refer to changesets generated by the svn to git conversion.

It would, you're right. -- Dave Abrahams

Daniel James

31 May 31 May

3:27 p.m.

On 30 May 2013 23:27, Dave Abrahams wrote:

...

on Thu May 23 2013, Daniel James wrote:

...
On 23 May 2013 14:47, Dave Abrahams wrote:

...
If that were going to be acceptable to the community, presumably everyone would have been happy with the original plan to graft an accurate SVN history when people want to refer to it, no? Please don't tell me that the last several months of work were wasted!

What's acceptable to one person is quite different to what's acceptable to another.

No kidding!

So saying that it would be acceptable for one module, is not the same as saying that it would be acceptable to the community.

...

...
It should ultimately be up to the maintainers of individual modules.

If anyone expects the Boost master repository to contain accurate submodule references through history, maintainers must not be allowed to delete old branches or rewrite old history.

But the old branches would remain untouched in the historical repo. Before the switch to the new layout, the master module would use that for the submodule, aftewards it would use the working repo. Rather than storing the historical record in a separate repo, it could be stored in separate branches, if that's preferred. And I'm not suggesting that every repository has such a split. Btw. I think people will want to delete branches that aren't relevant to their modules. It might be a good idea to tag them (say, 'svn-branch-name') so that the actual branches can be safely deleted.

...

...
Also, you would still have an accurate history in the historical module(s). Which was the point, it was an attempt at a compromise between two conflicting desires (to have an accurate history, and to have a repository with a different directory layout).

As I've said before, Git *cannot* represent exactly what SVN represents. If you want an accurate history, use SVN to get it. The best we can do with Git is to approximate actual history.

When I said 'accurate', I didn't mean exact. Just good enough that an historical checkout would look equivalent to the corresponding checkout from subversion.

4181

Age (days ago)

4203

Last active (days ago)

List overview

Download

25 comments

9 participants

participants (9)

"Jürgen Hunold"
Beman Dawes
Bjørn Roald
Daniel James
Daniel Pfeifer
Dave Abrahams
Jürgen Hunold
Nasos Iliopoulos
Vladimir Prus