[git] annotated SVN synchronization for boost.test (example only)
Below is an annotated sequence of (bash) command-line git operations that I used in a local copy of Boost.Test to understand what was happening in it, with no intention of proposing this be what the maintainers actually do with the module. Boost.Test is particularly interesting because master has been essentially unchanged for four years, while there are significant modifications pending in develop. A few of the changes on master make the SVN synchronization step tricky. People who want to hone their git-fu might find it workwhile to follow these steps in a local workspace to see how they work. I make no claim that they're the only way the same goal could be reached. I also have not used this for further development, so there the resulting structure may be flawed or contrary to strict git-flow expectations. WARNING: DO NOT paste the entire sequence into a shell window: at a couple of places you must manually do what the comment says (e.g., edit the contents of files to resolve conflicts) in order to reach a successful state. # Revert to a clean state where local branches match the origin, and the # working branches from any previous run of the script have been # removed. git fetch origin git checkout master git reset --hard origin/master git checkout develop git reset --hard origin/develop git branch -D synch/master synch/develop basis # git-merge-base identifies the commit to be used when merging things. # We're looking for something better, so no need to go beyond that one. MERGE_BASE=$(git merge-base develop master) # 635b1e38dd643af264c2ea8f0ba08dfb58337e37 echo ${MERGE_BASE} # Examine the history along master and develop, finding the most recent # SVN revision merged from develop into master. This only works if all # previous SVN revisions were also merged; if only select ones were # merged, it's probably best to skip them and keep looking. For # example, master on Boost.Test merged 81913, 63640, 62426, and a couple # others, but the last version that incorporated most prior fixes was # 57993 merged in ${MST_MP_SHA1}. git log ${MERGE_BASE}..master MST_MP_SHA1=2e4f64197956ccb94b24be4ed32abf25ee4b9dd1 # Now find the develop commit that most recently preceded that one in # SVN history. Basically, this is reviewing the log of differences # for the last commit with a SVN revision less than the one selected in # the previous step. BEWARE: If the revision you picked above doesn't # correspond to a develop branch change (e.g., if you picked 81913 which # didn't go into develop until 82756) you will create pain for yourself. git log ${MERGE_BASE}..develop # Here it's commit 57864 in ${DEV_MP_SHA1} DEV_MP_SHA1=fd8d9b3e4e4ab9656b0d880e8c519eabd134638a # Now take a look at what's different between those versions. That's # what'll get lost in a "-s ours" merge if it subsequently appeared in # develop. git diff ${DEV_MP_SHA1}..${MST_MP_SHA1} # There's some whitespace differences, some casts and error messages, # file permission changes, and two small changes from r81913 that we'll # talk about later. # Here's where my technique differs: You could do an "-s ours" merge # from the selected develop commit into master's HEAD as recommended at # https://svn.boost.org/trac/boost/wiki/StartModMaint#Firstpost-svnconversionm... # but if you do you might lose cherry-pick commits on master that are # also on develop. Instead, we'll create a new merge point on master at # the synchronization point, and merge from develop. git checkout -b basis ${MST_MP_SHA1} git merge -s ours -m 'SVN merge basis preserving master' ${DEV_MP_SHA1} # Verify that the differences are the ones we accepted earlier: git diff basis..${DEV_MP_SHA1} # Now, let's do some cleanup, and get master and develop branches that # actually fork at the merge point, rather than simply assume the merge # point is identical. This indirectly helps manage the differences that # we threw away with "-s ours". git checkout -b synch/master master git rebase basis # This should show no differences, because we just inserted a commit # that records parentage but changes no content. git diff master synch/master # Do the same at develop. Here we're going to have to resolve any # conflicts between the selected merge point git checkout -b synch/develop develop git rebase basis # 66 patches in we hit one of the differences: git stat # (MANUAL STEP) There's whitespace at the end of build/Jamfile.v2. Edit # the file to remove the diff markers, stage the changes, and tell # rebase to continue. git add -u git rebase --continue # After 204 patches there's another. git stat # (MANUAL STEP) Here you'll see everything is clean except # include/boost/test/impl/framework.ipp. Search for <<<<<< and edit to # keep the one with the upper-case TUT_SUITE. git add -u git rebase --continue # And we're done. Now let's see what changed: git diff develop synch/develop # Some of the differences between master and develop at the SVN merge # point have gone away; these are the ones I suspect but have not # confirmed would have been lost had we merged to master's HEAD instead # of to the base version. A few changes remain that were in master but # not in develop. # You have a choice at this point. If the repository wasn't already # public, I'd reset master (develop) to synch/master (synch/develop) and # move on. But since they were public, merge them; keeps history, # and makes future actions cleaner. git checkout master git merge -m 'Merge in rebased master' synch/master git checkout develop git merge -m 'Merge in rebased develop' synch/develop # Finally, what would happen if we merged develop into master now? git checkout -b test master git merge develop # Very nearly a clean merge. See what's not staged: git stat -u | grep '^.[^ ]' # ## test # UU include/boost/test/utils/runtime/cla/parameter.hpp # UU include/boost/test/utils/runtime/env/variable.hpp # # Again, open the files and look for the diff markers: one uses m!=0 and # the other !!m. Which is right? Use git-blame: git blame origin/master include/boost/test/utils/runtime/cla/parameter.hpp # The master change came in 954d3ab6 which was SVN commit r81913 by # Vicente J. Botet Escriba. Show the change: git log -p 954d3ab6^! git blame origin/develop include/boost/test/utils/runtime/cla/parameter.hpp # The develop change came in 4f7fc06c which was SVN commit r82756 by # Gennadiy Rozental. Show the change: git log -p 4f7fc06c^! # Note that if, back when selecting the SVN merge point, we'd gone with # r81913 things would be completely bolluxed at this point since the # corresponding change didn't come from the develop branch. # (MANUAL STEP) It's Rozental's project, so pick the develop solution # (!!m). Then stage the changes and commit, editing the message to # indicate how you resolved the conflict. git add -u git commit
Peter A. Bigot <pab <at> pabigot.com> writes:
Below is an annotated sequence of (bash) command-line git operations that I used in a local copy of Boost.Test to understand what was happening in it, with no intention of proposing this be what the maintainers actually do with the module.
Thank you. I need to get my hands dirty and finally switch to git locally. At some point I'll need this (I hope ;)) Gennadiy
On 01/13/2014 06:28 AM, Gennadiy Rozental wrote:
Peter A. Bigot <pab <at> pabigot.com> writes:
Below is an annotated sequence of (bash) command-line git operations that I used in a local copy of Boost.Test to understand what was happening in it, with no intention of proposing this be what the maintainers actually do with the module. Thank you. I need to get my hands dirty and finally switch to git locally. At some point I'll need this (I hope ;))
In the interests of reducing pain: I recommend against using those steps. I wrote that description while I was still getting a handle on how Boost development works and how git would be used to support it. At the time, I assumed develop branches were for module-specific enhancements and remained decoupled from other libraries until they had reached a releasable state and were merged to master. I now understand that daily Boost integration is done with develop branches, so the git update steps that involve a rebase are inappropriate. Peter
On Mon, Jan 13, 2014 at 12:36 PM, Peter A. Bigot <pab@pabigot.com> wrote:
On 01/13/2014 06:28 AM, Gennadiy Rozental wrote:
Peter A. Bigot <pab <at> pabigot.com> writes:
Below is an annotated sequence of (bash) command-line git operations
that I used in a local copy of Boost.Test to understand what was happening in it, with no intention of proposing this be what the maintainers actually do with the module.
Thank you. I need to get my hands dirty and finally switch to git locally. At some point I'll need this (I hope ;))
In the interests of reducing pain: I recommend against using those steps.
The problem, of course, is that your procedure is overwhelmingly complex for a new git user. That's why the docs we've been working on try to stick to very basic git. Since Gennadiy hasn't even begun to use git, he might be better off starting with https://svn.boost.org/trac/boost/wiki/ModularBoost --Beman
On 01/14/2014 11:00 AM, Beman Dawes wrote:
On Mon, Jan 13, 2014 at 12:36 PM, Peter A. Bigot <pab@pabigot.com> wrote:
On 01/13/2014 06:28 AM, Gennadiy Rozental wrote:
Peter A. Bigot <pab <at> pabigot.com> writes:
Below is an annotated sequence of (bash) command-line git operations
that I used in a local copy of Boost.Test to understand what was happening in it, with no intention of proposing this be what the maintainers actually do with the module.
Thank you. I need to get my hands dirty and finally switch to git locally. At some point I'll need this (I hope ;))
In the interests of reducing pain: I recommend against using those steps.
The problem, of course, is that your procedure is overwhelmingly complex for a new git user. That's why the docs we've been working on try to stick to very basic git.
True. The problem with the docs is that they don't explain what the commands are doing and why (or why not) use them. Rote application works until something goes wrong, then people get confused about what happened. (Recall that the original email was roughly contemporaneous with, and motivated by, issues a maintainer had following https://svn.boost.org/trac/boost/wiki/PostCvtMergePoint in multi_index.) The sole value, if any, in what I wrote was that it does try to make clear the goals of each step, and to introduce the underlying git concepts and process trade-offs that influence the decision of how to react to the situations that are presented when you follow them. It's an opportunity to engage in a learning exercise.
Since Gennadiy hasn't even begun to use git, he might be better off starting with https://svn.boost.org/trac/boost/wiki/ModularBoost
Agreed. Peter
-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Peter A. Bigot Sent: Tuesday, January 14, 2014 5:24 PM To: boost@lists.boost.org Subject: Re: [boost] [git] annotated SVN synchronization for boost.test (example only)
On 01/14/2014 11:00 AM, Beman Dawes wrote:
On Mon, Jan 13, 2014 at 12:36 PM, Peter A. Bigot <pab@pabigot.com> wrote:
On 01/13/2014 06:28 AM, Gennadiy Rozental wrote:
Peter A. Bigot <pab <at> pabigot.com> writes:
Below is an annotated sequence of (bash) command-line git operations
that I used in a local copy of Boost.Test to understand what was happening in it, with no intention of proposing this be what the maintainers actually do with the module.
Thank you. I need to get my hands dirty and finally switch to git locally. At some point I'll need this (I hope ;))
In the interests of reducing pain: I recommend against using those steps.
The problem, of course, is that your procedure is overwhelmingly complex for a new git user. That's why the docs we've been working on try to stick to very basic git.
True. The problem with the docs is that they don't explain what the commands are doing and why (or why not) use them.
Rote application works until something goes wrong, then people get confused about what happened.
Definitely - this is a serious flaw with very, very many sets of instructions. * They don't say why the next step is being taken. * They don't say how one can tell it has worked OK. * Or how to check that it has worked OK. * And what to do if it hasn't. For example, even the bootstrap process description fails this test. (I'm still unclear what b2 headers does! - and I'm not certain I fully understand what bootstrap does either). I know this makes the instructions longer but IMO it's well worth it. Paul --- Paul A. Bristow, Prizet Farmhouse, Kendal LA8 8AB UK +44 1539 561830 07714330204 pbristow@hetp.u-net.com
On Jan 14, 2014, at 12:51 PM, "Paul A. Bristow" <pbristow@hetp.u-net.com> wrote:
From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Peter A. Bigot
On 01/14/2014 11:00 AM, Beman Dawes wrote:
The problem, of course, is that your procedure is overwhelmingly complex for a new git user. That's why the docs we've been working on try to stick to very basic git.
True. The problem with the docs is that they don't explain what the commands are doing and why (or why not) use them.
Rote application works until something goes wrong, then people get confused about what happened.
Definitely - this is a serious flaw with very, very many sets of instructions.
* They don't say why the next step is being taken.
* They don't say how one can tell it has worked OK.
* Or how to check that it has worked OK.
* And what to do if it hasn't.
[snip]
I know this makes the instructions longer but IMO it's well worth it.
Explanations and rationales are always a good idea, but after one is familiar with the process, but enough time elapses to need the instructions again, one wants just the instructions. I suggest links to pop-ups, or to separate pages, that provide the details while not cluttering the sequence of commands. ___ Rob (Sent from my portable computation engine)
-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Rob Stewart Sent: Wednesday, January 15, 2014 1:52 AM To: boost@lists.boost.org Subject: Re: [boost] [git] annotated SVN synchronization for boost.test (example only)
On Jan 14, 2014, at 12:51 PM, "Paul A. Bristow" <pbristow@hetp.u-net.com> wrote:
From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Peter A. Bigot
On 01/14/2014 11:00 AM, Beman Dawes wrote:
The problem, of course, is that your procedure is overwhelmingly complex for a new git user. That's why the docs we've been working on try to stick to very basic git.
True. The problem with the docs is that they don't explain what the commands are doing and why (or why not) use them.
Rote application works until something goes wrong, then people get confused about what happened.
Definitely - this is a serious flaw with very, very many sets of instructions.
* They don't say why the next step is being taken.
* They don't say how one can tell it has worked OK.
* Or how to check that it has worked OK.
* And what to do if it hasn't.
[snip]
I know this makes the instructions longer but IMO it's well worth it.
Explanations and rationales are always a good idea, but after one is familiar with the process, but enough time elapses to need the instructions again, one wants just the instructions. I suggest
links to
pop-ups, or to separate pages, that provide the details while not cluttering the sequence of commands.
After you've done it once, won't you use just remember the commands anyway ;-) Paul PS Yes - of course - but it's much more work, and I'm not sure how with a wiki 'language' - or Quickbook. So while we are all newbies at modular-boost, I think we need all the help we can get.
participants (5)
-
Beman Dawes -
Gennadiy Rozental -
Paul A. Bristow -
Peter A. Bigot -
Rob Stewart