Hi all, First post here, so please bear with me. A lot has been said in the last year on the subject of a possible migration from svn to git. However, after reading pretty much every message on the issue on this list (sic!), I have the impression that some of the most important aspects did not get the consideration they require. I would therefore like to add my outside perspective. SUMMARY: 1. an svn --> git migration does cause some major, but one-time disruption. However, svn right now causes minor, but continuous disruption on a daily basis - which often goes unnoticed. 2. git's distributed concept is nothing less than one generation ahead of centralized VCSs like svn. The benefits usually do not become apparent until git is actually used in practice. 3. at the moment, boost is significantly missing out on fresh enthusiasm that new contributors could bring into the project; svn and the policies it mandates are partly too blame for that. 4. svn trunk (i.e. boost library) maintainers are too limited by the svn concept in their decision which changes they merge at what point in time and may be pressured into rushing suboptimal decisions; this can make maintenance work painful and frustrating. git greatly alleviates that problem. 5. IMO, it's not 'if', it is 'when'. The longer a migration is delayed, the longer boost development will be slowed down without even noticing. DETAILS: I am aware that some of my points technically are mentioned on the boost git wiki page at https://svn.boost.org/trac/boost/wiki/Git/WhyGit, but they still seem somewhat neglected. This is probably not surprising as many participants in the discussion 1. presumably are established boost contributors and thus share a 'from the inside' view of the subject and 2. are only marginally affected by svn's conceptual problems This is what happened here in the last couple of days: I needed a tool for a job in my current project and I couldn't find one, so I thought about what components I need to build my own. A little web research - Boost.Iostreams looks perfect for the purpose. You have all been through what comes next, be it with boost or with other open source software: Download the latest release, install the libraries, start reading the docs, run some sample code, etc. While reading, I spot a typo in an HTML doc, then another in an include file comment, a third one in another file... the fourth one at the latest makes me think - this is open source, I should sync the latest sources, fix all those typos as I come across them while reading and contribute them back to the project. A quick read on submitting bugs on boost.org, svn co yadi yadi, fix the typos locally. Of course I don't have write privileges to the repo, so the best I can do for now is create a patch and attach that to a bug report. Within minutes, someone takes care of the bug, comments and takes ownership. Excellent! Smithers, money fight! I go back to reading, find more typos. My patch has not been committed to the trunk yet, so my local changes are still sitting there on my local hard drive. If I do another svn diff > file_2.patch, the patch will also contain the changes I have already attached to the bug report. What are my options now ? Revert my changes, base the new ones on the head revision and send another patch ? Create a new patch file and manually remove the diffs of the old changes I already sent ? How will the upstream maintainer know which changes are based on what version ? Also, I notice a few other things that in my opinion could be done better to facilitate adoption of the iostreams library. I would be willing to do the work and I would in the perfect position as I am walking in the shoes of an adopter right now - but these are separate from the typo fixes and they are larger issues; considering I can't check my changes in, would I really want to have them sit on my local harddrive, waiting possibly for months for someone upstream to review and hopefully merge them ? A couple of hours later: I get an email from boost bugs/svn; the upstream maintainer has committed my changes. However, I actually receive not one, but two emails, because the upstream guy chose to split my patch into changes on documentation and changes on source code (which presumably required extra work on his part). I do an svn up and now have to deal with conflicts between my local changes and the new commits: I make a note of my second set of changes and overwrite my local files with the new repo versions, then bring the second set back in. Uff! That's a lot of work just to fix some typos! And these are just cosmetic issues in discrete chunks that do not require any testing; contributing changes to source code would take even more effort and more caution - build and test on several platforms, peer review, etc. Also, the upstream maintainer jumped on the bug report right away and integrated my changes within hours. This is the ideal situation (thanks again Daniel, great job! :-) - usually, things don't happen so quickly - especially around major release time. HOW THIS IS DONE IN GIT: 1. google 'github iostreams' --> click first search result: https://github.com/boost-lib/iostreams (this can be even reduced to 'gh: iostreams' by integrating github into chrome search engines, but that's another story...) 2. on githup website, click 'fork', wait a few seconds --> this auto-creates my personal online github iostreams repo 3. on my local machine, open a shell, enter 'git clone ', copy & paste the repo .git url shown on the website in there and run e.g. 'git clone git@github.com:<username>/iostreams.git' --> this creates a local fork of my personal online iostreams repo + the mailing list probably changes the line above to 'hidden' + there is github/desktop integration and a plethora of gui clients 4. i change local files and commit to the local repo. once a set of changes is complete, i push them online into my personal online repo; once my work has reached some maturity, I send a pull request to the upstream maintainer of the official iostreams repo, aka the keeper of the trunk. disclaimer: I have omitted one-time steps to set up local git, github.com account, ssh keys, lastpass integration, yadi yadi. these steps are abundantly documented online, done in minutes and required only once. THE GIT ADVANTAGE: I don't have to wait for the first set of changes to be merged into the trunk before I resync, resolve conflicts and base the second set on the new head revision; I simply create a new branch for a new change set and send a new pull request once my work is complete. The upstream maintainer has no peer pressure to merge any changes in: He/she is sitting on the head revision and is looking at a number of pull requests for change sets in various personal forks like mine; these are more like 'change offers' the maintainer is free to merge in if he/she so chooses (cherry-picking) - or postpone if this is not the time because for example a major release is due and they seem too risky. I as a contributor do not really care so much: Of course I am stoked if my work actually makes it into the project eventually, but the point in time when this happens technically does not matter to me - pending changes will not get in my way if the merge takes a little longer. This is the major difference between svn and git - and I can not stress this advantage enough: I am decoupled from, but still connected to the official trunk. I can make arbitrary changes and group them in any way I see fit. I offer them to upstream and move on to new development. As a side effect, this also provides an overview impression of quality and continuity of the work of a potential new contributor. Of course it is technically correct that svn also supports the concepts of forks and branches, but in terms of tooling and mentality, they are considered much more heavy-weight than in git and might get created a couple of times a year - while in git, they might come and go a couple of times a day. Finally, the 'series of tubes' metaphor IMO does not really hit it. Subversion to me seems more like a freight train where the waggons move at different speeds, so they constantly bump into the car behind and in front of them; movement of the entire train is jerky. Git on the other hand is a bunch of space ships linked by hyperelastic tractor beams: Even if one travels out as far as the delta quadrant, the upstream connection will always make sure the collective does benefit from any new development it brings back from there. [snipped several paragraphs about cultural and other aspects; maybe some other time.] # ssc
On Wed, Feb 8, 2012 at 7:30 AM, Steven Samuel Cole <steven.samuel.cole@gmail.com> wrote:
...
I've never used git, but I have used bzr. Having worked with a dvcs, the disadvantages of svn become quite clear. Svn *might* be fine for personal (one dev) projects, but for bigger projects it's just not the right option. I can't think of any real downsides. -- Olaf
Olaf van der Spek wrote:
On Wed, Feb 8, 2012 at 7:30 AM, Steven Samuel Cole <steven.samuel.cole@gmail.com> wrote:
...
I've never used git, but I have used bzr. Having worked with a dvcs, the disadvantages of svn become quite clear. Svn *might* be fine for personal (one dev) projects, but for bigger projects it's just not the right option.
I can't think of any real downsides.
I found out that John Wiegley has been working on a faithful conversion of the boost svn tree to git, see [1]. Is that effort still ongoing? Also, there has been a very interesting discussion on branching model and release process, which seems to have started with [2]. Two subthreads particularly interested me, i.e. the one starting in [3] and another one starting just before [4] (especially the dialogue between Vladimir Prus and Dave Abrahams). This might have been mentioned before, but gitflow seems to be ideal. See [5] for an explanation of the branching model and [6] for an optional tool which automates workflows that adopt the model. -Julian 1. http://lists.boost.org/Archives/boost/2011/01/176378.php 2. http://lists.boost.org/Archives/boost/2011/01/176002.php 3. http://lists.boost.org/Archives/boost/2011/01/176245.php 4. http://lists.boost.org/Archives/boost/2011/01/176226.php 5. http://nvie.com/posts/a-successful-git-branching-model/ 6. https://github.com/nvie/gitflow
On 8 February 2012 14:37, Julian Gonggrijp <j.gonggrijp@gmail.com> wrote:
This might have been mentioned before, but gitflow seems to be ideal. See [5] for an explanation of the branching model and [6] for an optional tool which automates workflows that adopt the model.
The problem that we face with something like gitflow is testing. We only have the infrastructure for testing two branches so we can't adequately test feature branches. We really do need to test on a wide variety of platforms as early as possible. But we wouldn't want to merge half done features into the develop branch just to get testing (this is one of the biggest problems we have at the moment, changes are often in trunk for some time before they're merged into release, if at all). A possible solution would be to use the 'develop' branch as the equivalent of our current release branch (I'm not sure how to branch for release, we might have to do without and just restrict additions to develop as we currently do, at least at first) and add another 'testing' branch. This would branch from develop, and feature branches would be merged into it for testing. Once testing has shown the feature branch seems to be release worthy, it would be merged into develop. The testing branch would never be merged into anything (its history would be very messy). We would probably create a new testing branch regularly (once per release cycle?) so that neglected features would be dropped. Or maybe, rather than being a traditional branch, the test repo could be created by a script which merges in active feature branches. It'd have to deal with conflicts though, maybe just keeping track of commits that are known to be good and sticking with them when a conflict is found. But something like that could be developed later - especially after having some experience of how the process works. This testing mechanism would be fairly separate from the main flow structure so it could be provided by an additional tool to gitflow. Hopefully wouldn't be too hard to develop. Does that make any sense? I know it isn't ideal, but it seems plausible given our current infrastructure. And we could work on improving it as we go.
Daniel James wrote:
On 8 February 2012 14:37, Julian Gonggrijp <j.gonggrijp@gmail.com> wrote:
This might have been mentioned before, but gitflow seems to be ideal. See [5] for an explanation of the branching model and [6] for an optional tool which automates workflows that adopt the model.
The problem that we face with something like gitflow is testing. We only have the infrastructure for testing two branches so we can't adequately test feature branches.
Stop right here. You seem to be writing this on the assumption that with git, we'd still have two separate source trees, one for the development branch and one for the release/master branch. One of the exciting things of git is exactly that you can abandon that situation. There is only one file tree, and the branches act like "parallel worlds" in which that same file tree received different update histories. So a feature branch is just another instance of the same tree, with the same available testing infrastructure.
We really do need to test on a wide variety of platforms as early as possible. But we wouldn't want to merge half done features into the develop branch just to get testing (this is one of the biggest problems we have at the moment, changes are often in trunk for some time before they're merged into release, if at all).
So this is actually an advantage of git: you don't need to merge a feature branch first in order to test it.
A possible solution would be to use the 'develop' branch as the equivalent of our current release branch (I'm not sure how to branch for release, we might have to do without and just restrict additions to develop as we currently do, at least at first) and add another 'testing' branch. This would branch from develop, and feature branches would be merged into it for testing. [...]
Or maybe, rather than being a traditional branch, the test repo could be created by a script [...]
This testing mechanism would be fairly separate from the main flow structure so it could be provided by an additional tool to gitflow. Hopefully wouldn't be too hard to develop. Does that make any sense? I know it isn't ideal, but it seems plausible given our current infrastructure. And we could work on improving it as we go.
No, no, no. No. Something that seems plausible given the current svn- based approach would actually make things more complicated and less straight-forward. Here's how I think the standard gitflow branching model would fit snugly around the current Boost workflow: - The develop branch corresponds to the current trunk. This is where library authors can continually make (small) changes to their projects. There is one important difference: when authors decide to add big new features, to completely overhaul their library or to add a whole new library (basically anything that might take more than a day), they fork a new feature branch for that purpose. - As indicated, the feature branches act like sandboxes where big changes and additions can be prepared before they're merged back into the develop branch. Or they can just be abandoned if an experiment turns out to fail. Authors can run full regression tests within the feature branches, since every branch contains the same file tree with the same tool configuration. - The master branch corresponds to the current release branch, again with an important difference. In the current situation, trunk is merged directly into release, with the (manual) exception of libraries that are in too turbulent state, and after that the release branch is updated until everything is really ready for release. In the new situation, a new (temporary) release branch is forked from develop first (which automatically excludes turbulent changes because those reside in separate feature branches) and then updated until all tests pass (ideally), at which point it is finally merged into master (as well as back into develop). So the master branch only includes the actual releases, which of course are also tagged. - As indicated, release branches act like quarantines in which the code from develop is made ready for release/merge into master. Again, because it's the same file tree the same tools are available. Because release branches are only merged into master when the code is deemed ready for release, the merge into master can be a trigger for running scripts that may be used in the release process. Similarly, commits to release branches can be set as a trigger for automatic regression testing. - Hotfix branches can be forked from master whenever a release turns out to contains bugs afterall, like the update from 1.46 to 1.46.1. Again they contain the same file tree and the same tools, and regression testing can also be automated in this case. - BoostPro might want to use support branches for older releases. Again, a support branch contains the full tree with all tools, and so forth. For as far as I can tell, the gitflow model allows Boost to do everything it does now, in many cases more elegantly. -Julian
On 9 February 2012 13:46, Julian Gonggrijp <j.gonggrijp@gmail.com> wrote:
Daniel James wrote:
On 8 February 2012 14:37, Julian Gonggrijp <j.gonggrijp@gmail.com> wrote:
This might have been mentioned before, but gitflow seems to be ideal. See [5] for an explanation of the branching model and [6] for an optional tool which automates workflows that adopt the model.
The problem that we face with something like gitflow is testing. We only have the infrastructure for testing two branches so we can't adequately test feature branches.
Stop right here. You seem to be writing this on the assumption that with git, we'd still have two separate source trees, one for the development branch and one for the release/master branch. One of the exciting things of git is exactly that you can abandon that situation.
I was writing about the limits of our regression testing system, not the limits of our version control system. We need to frequently run our tests on compilers and platforms that we don't have access to, so we rely on volunteers running the test suite regularly. They currently only do that on two branches.
Daniel James wrote:
On 9 February 2012 13:46, Julian Gonggrijp <j.gonggrijp@gmail.com> wrote:
Daniel James wrote:
On 8 February 2012 14:37, Julian Gonggrijp <j.gonggrijp@gmail.com> wrote:
This might have been mentioned before, but gitflow seems to be ideal. See [5] for an explanation of the branching model and [6] for an optional tool which automates workflows that adopt the model.
The problem that we face with something like gitflow is testing. We only have the infrastructure for testing two branches so we can't adequately test feature branches.
Stop right here. You seem to be writing this on the assumption that with git, we'd still have two separate source trees, one for the development branch and one for the release/master branch. One of the exciting things of git is exactly that you can abandon that situation.
I was writing about the limits of our regression testing system, not the limits of our version control system. We need to frequently run our tests on compilers and platforms that we don't have access to, so we rely on volunteers running the test suite regularly. They currently only do that on two branches.
So the perceived problem is that it would take too much time for the volunteers to check out all of the numerous feature branches that might exist at any given moment and test all of them, in addition to the develop branch and the current release branch. The practical consequence is that features under active development cannot be tested on all platforms until they're merged back into the develop branch. Here's why I think this isn't a real issue. Generally, when you make a change to your project, you do it in two stages. First, you implement the changes; next, you keep tweaking and polishing your work until it does the right thing on all relevant platforms. The first stage is typically something you'll do in a feature branch. During the work, you run unit tests on your local platform to establish that your approach should work in principle. If you reach the end of this stage, you'd generally expect that your changes will work on other platforms as well, with some corrections. So that's the right moment to merge back into develop and continue your work over there. The tweaking and polishing stage is then facilitated by the fact that the develop branch is regularly tested by volunteers. You can still use temporary feature branches of shorter duration if some compiler needs more attention; just merge them back before the next regular test session is run. Such a workflow is completely normal with git. Concluding, the volunteers still only need to test two branches, i.e. develop and the current temporary release branch (if there is one). The only difference that feature branches make to testing, is that changes for which testing on all platforms doesn't really make sense yet are isolated. HTH, -Julian
On 02/11/2012 01:42 PM, Julian Gonggrijp wrote:
Daniel James wrote:
On 9 February 2012 13:46, Julian Gonggrijp<j.gonggrijp@gmail.com> wrote:
Daniel James wrote:
On 8 February 2012 14:37, Julian Gonggrijp<j.gonggrijp@gmail.com> wrote:
This might have been mentioned before, but gitflow seems to be ideal. See [5] for an explanation of the branching model and [6] for an optional tool which automates workflows that adopt the model.
The problem that we face with something like gitflow is testing. We only have the infrastructure for testing two branches so we can't adequately test feature branches.
Stop right here. You seem to be writing this on the assumption that with git, we'd still have two separate source trees, one for the development branch and one for the release/master branch. One of the exciting things of git is exactly that you can abandon that situation.
I was writing about the limits of our regression testing system, not the limits of our version control system. We need to frequently run our tests on compilers and platforms that we don't have access to, so we rely on volunteers running the test suite regularly. They currently only do that on two branches.
So the perceived problem is that it would take too much time for the volunteers to check out all of the numerous feature branches that might exist at any given moment and test all of them, in addition to the develop branch and the current release branch. The practical consequence is that features under active development cannot be tested on all platforms until they're merged back into the develop branch.
Here's why I think this isn't a real issue.
Generally, when you make a change to your project, you do it in two stages. First, you implement the changes; next, you keep tweaking and polishing your work until it does the right thing on all relevant platforms.
The first stage is typically something you'll do in a feature branch. During the work, you run unit tests on your local platform to establish that your approach should work in principle.
If you reach the end of this stage, you'd generally expect that your changes will work on other platforms as well, with some corrections. So that's the right moment to merge back into develop and continue your work over there.
Right, that all makes sense. However, the whole branching stuff seems like a lot of overhead (even if the tool is able to handle it very good). Mind you, we, the developers don't always have all the toolchains available on our developing platforms. What would make this whole branching very cool is if we could somehow tie it with testing such that we are able to tell the testrunners to use our feature branch to run the tests (of course this must be automated in a way). I think in this scenario, a modularized boost will come in very handy ... Think of the VHDL model, you have unit tests, but the module under test might be very different, though the unit tests stay the same. This is something i would really like to see.
The tweaking and polishing stage is then facilitated by the fact that the develop branch is regularly tested by volunteers. You can still use temporary feature branches of shorter duration if some compiler needs more attention; just merge them back before the next regular test session is run. Such a workflow is completely normal with git.
Concluding, the volunteers still only need to test two branches, i.e. develop and the current temporary release branch (if there is one). The only difference that feature branches make to testing, is that changes for which testing on all platforms doesn't really make sense yet are isolated.
HTH, -Julian
_______________________________________________ Unsubscribe& other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Thomas Heller wrote:
On 02/11/2012 01:42 PM, Julian Gonggrijp wrote:
So the perceived problem is that it would take too much time for the volunteers to check out all of the numerous feature branches that might exist at any given moment and test all of them, in addition to the develop branch and the current release branch. The practical consequence is that features under active development cannot be tested on all platforms until they're merged back into the develop branch.
Here's why I think this isn't a real issue.
[...]
Right, that all makes sense. However, the whole branching stuff seems like a lot of overhead (even if the tool is able to handle it very good).
Please take my word that the branching stuff doesn't add any overhead. Forks and merges are just special cases of commits: Commit - create a new commit with one previous commit as parent. Fork - create a new commit which will have a sibling from the same parent. Merge - create a new commit with two parents. That's all there is to branching in git. History is a directed acyclic graph of commits with parent relationships. Under the hood, a branch is nothing more than a bookmark to a leaf commit.
Mind you, we, the developers don't always have all the toolchains available on our developing platforms. What would make this whole branching very cool is if we could somehow tie it with testing such that we are able to tell the testrunners to use our feature branch to run the tests (of course this must be automated in a way).
I agree something like that could be nice, but I think the gitflow model is already very cool without it. :-)
I think in this scenario, a modularized boost will come in very handy ... Think of the VHDL model,
Do you mean this? http://en.wikipedia.org/wiki/VHDL
you have unit tests, but the module under test might be very different, though the unit tests stay the same. This is something i would really like to see.
Could you clarify? I don't see why we would want to test a different module with the same unit tests. -Julian
On 02/11/2012 03:48 PM, Julian Gonggrijp wrote:
Thomas Heller wrote:
On 02/11/2012 01:42 PM, Julian Gonggrijp wrote:
So the perceived problem is that it would take too much time for the volunteers to check out all of the numerous feature branches that might exist at any given moment and test all of them, in addition to the develop branch and the current release branch. The practical consequence is that features under active development cannot be tested on all platforms until they're merged back into the develop branch.
Here's why I think this isn't a real issue.
[...]
Right, that all makes sense. However, the whole branching stuff seems like a lot of overhead (even if the tool is able to handle it very good).
Please take my word that the branching stuff doesn't add any overhead. Forks and merges are just special cases of commits:
Commit - create a new commit with one previous commit as parent. Fork - create a new commit which will have a sibling from the same parent. Merge - create a new commit with two parents.
You gotta be kidding me, right? Even with the most awesome tool in the world, it is trivial to see that this adds overhead! While by simply committing stuff, i just have to issue one command and be able to observer that everything still is alright. You already listed forking and merging as two different steps! Come on ...
That's all there is to branching in git. History is a directed acyclic graph of commits with parent relationships. Under the hood, a branch is nothing more than a bookmark to a leaf commit.
Yeah, this all sounds very nifty ... in theory. This all means overhead in terms of commands to issue and quality assurance that everything went right, including running tests after the merge (did it really merge everything correctly), resolving possible conflicts etc. While I admit that all this branching sounds very tempting, please don't try to sell me as a fool that all this comes without a cost. This is just not true.
Mind you, we, the developers don't always have all the toolchains available on our developing platforms. What would make this whole branching very cool is if we could somehow tie it with testing such that we are able to tell the testrunners to use our feature branch to run the tests (of course this must be automated in a way).
I agree something like that could be nice, but I think the gitflow model is already very cool without it. :-)
I think in this scenario, a modularized boost will come in very handy ... Think of the VHDL model,
Do you mean this? http://en.wikipedia.org/wiki/VHDL
you have unit tests, but the module under test might be very different, though the unit tests stay the same. This is something i would really like to see.
Could you clarify? I don't see why we would want to test a different module with the same unit tests.
Yes, I mean the hardware description language. Let me explain: In VHDL you usually start prototyping your new design by writing some behavioral description at a very high level, almost like you do with a "normal" programming language. After you are done, you start to write your unit tests for that. You do that by writing a very generic unit test and refer to your new design as "device under test". After you successfully passed all the tests, you start by getting down one level in the abstraction, lets say RTL level (Register transfer level). After that you plug that new device into your unit tests to ensure correct behavior. Now, imagine that different branches are seen as different devices, so to say. Instead of letting the test runners run the unit tests defined in the main branch, you let'em run the unit tests in your new feature branch, completely transparent for them. This means that you don't need to disrupt regular service by committing a half done feature to the main branch, but get the full advantage of the different test runners.
-Julian
_______________________________________________ Unsubscribe& other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Thomas Heller wrote:
On 02/11/2012 03:48 PM, Julian Gonggrijp wrote:
Please take my word that the branching stuff doesn't add any overhead. Forks and merges are just special cases of commits:
Commit - create a new commit with one previous commit as parent. Fork - create a new commit which will have a sibling from the same parent. Merge - create a new commit with two parents.
You gotta be kidding me, right? Even with the most awesome tool in the world, it is trivial to see that this adds overhead! While by simply committing stuff, i just have to issue one command and be able to observer that everything still is alright. You already listed forking and merging as two different steps! Come on ...
Alright, sorry. I admit I was a bit too enthousiast there.
That's all there is to branching in git. History is a directed acyclic graph of commits with parent relationships. Under the hood, a branch is nothing more than a bookmark to a leaf commit.
Yeah, this all sounds very nifty ... in theory. This all means overhead in terms of commands to issue and quality assurance that everything went right, including running tests after the merge (did it really merge everything correctly), resolving possible conflicts etc.
Yes, resolving conflicts after a merge is probably the main term of added overhead when branching. I think that gitflow allows for a working style that reduces merge conflicts, though, and ironically that's mostly because of the use of additional branches.
While I admit that all this branching sounds very tempting, please don't try to sell me as a fool that all this comes without a cost. This is just not true.
Agreed, the proposed branching model does have a cost.
Mind you, we, the developers don't always have all the toolchains available on our developing platforms. What would make this whole branching very cool is if we could somehow tie it with testing such that we are able to tell the testrunners to use our feature branch to run the tests (of course this must be automated in a way). I think in this scenario, a modularized boost will come in very handy ... Think of the VHDL model,
Do you mean this? http://en.wikipedia.org/wiki/VHDL
you have unit tests, but the module under test might be very different, though the unit tests stay the same. This is something i would really like to see.
Could you clarify? I don't see why we would want to test a different module with the same unit tests.
Yes, I mean the hardware description language. Let me explain: In VHDL you usually start prototyping your new design by writing some behavioral description at a very high level, almost like you do with a "normal" programming language. After you are done, you start to write your unit tests for that. You do that by writing a very generic unit test and refer to your new design as "device under test". After you successfully passed all the tests, you start by getting down one level in the abstraction, lets say RTL level (Register transfer level). After that you plug that new device into your unit tests to ensure correct behavior. Now, imagine that different branches are seen as different devices, so to say. Instead of letting the test runners run the unit tests defined in the main branch, you let'em run the unit tests in your new feature branch, completely transparent for them. This means that you don't need to disrupt regular service by committing a half done feature to the main branch, but get the full advantage of the different test runners.
I need more time to absorb this, but it does sound very cool indeed. -Julian
Thomas Heller wrote:
On 02/11/2012 03:48 PM, Julian Gonggrijp wrote:
Thomas Heller wrote:
... Think of the VHDL model, Do you mean this? http://en.wikipedia.org/wiki/VHDL Yes, I mean the hardware description language. ... successfully passed all the tests, you start by getting down one level in the abstraction, lets say RTL level (Register transfer level).
huh ?!? hardware description language ?? VHDL ?? RTL ?? can we please stay in software development here and use terms and metaphors that everyone can relate to ?
On 02/12/2012 04:56 AM, Steven Samuel Cole wrote:
Thomas Heller wrote:
On 02/11/2012 03:48 PM, Julian Gonggrijp wrote:
Thomas Heller wrote:
... Think of the VHDL model, Do you mean this? http://en.wikipedia.org/wiki/VHDL Yes, I mean the hardware description language. ... successfully passed all the tests, you start by getting down one level in the abstraction, lets say RTL level (Register transfer level).
huh ?!? hardware description language ?? VHDL ?? RTL ??
can we please stay in software development here and use terms and metaphors that everyone can relate to ?
I am terribly sorry that this blows your mind. This wasn't a metaphor. This was an example how testing is done in other languages. Writing VHDL code is the same C++ code. It needs testing. All I did was trying to find a real advantage for all this branching stuff. If narrow mindedness and the failure of thinking out of the box is the git way, then let me quote you: "--> yeah, thanks but no thanks."
Thomas Heller wrote:
On 02/11/2012 03:48 PM, Julian Gonggrijp wrote:
Thomas Heller wrote:
What would make this whole branching very cool is if we could somehow tie it with testing such that we are able to tell the testrunners to use our feature branch to run the tests (of course this must be automated in a way). I think in this scenario, a modularized boost will come in very handy ... Think of the VHDL model,
Do you mean this? http://en.wikipedia.org/wiki/VHDL
you have unit tests, but the module under test might be very different, though the unit tests stay the same. This is something i would really like to see.
Could you clarify? I don't see why we would want to test a different module with the same unit tests.
Yes, I mean the hardware description language. Let me explain: In VHDL you usually start prototyping your new design by writing some behavioral description at a very high level, almost like you do with a "normal" programming language. After you are done, you start to write your unit tests for that. You do that by writing a very generic unit test and refer to your new design as "device under test". After you successfully passed all the tests, you start by getting down one level in the abstraction, lets say RTL level (Register transfer level). After that you plug that new device into your unit tests to ensure correct behavior. Now, imagine that different branches are seen as different devices, so to say. Instead of letting the test runners run the unit tests defined in the main branch, you let'em run the unit tests in your new feature branch, completely transparent for them. This means that you don't need to disrupt regular service by committing a half done feature to the main branch, but get the full advantage of the different test runners.
Thomas, I've given this some thought and I can't help but think that the VHDL comparison doesn't really work. That's mainly because Boost libraries or hypothetical Boost modules are not like subdevices that cooperate on a common task, while that does seem to be the case with the VHDL example. The VHDL business is causing us to miss your point, which is a waste since your idea does seem to be very interesting. My guess is that, given a branching model like gitflow, you envision something along the following lines: There is a "testing image" which the testing volunteers run their periodic tests on. By default, the image mirrors the latest commit on the develop branch. When working on a feature branch, library authors can reversibly and transparently substitute the version of their library from the feature branch for the default version in the image. All of this is facilitated by some kind of automated tool which has yet to be found. The result is that most libraries in the image are from develop, but some are from feature branches, without the testing volunteers needing to worry about it. They can run a single testing cycle on the heterogeneous image. For library authors, this brings the advantage that they can already have their changes tested on all platforms before they feel confident that it should be possible to make them work. (Or while they emply a long-term feature branch specifically in order to support a platform, per Daniel James.) Please let us know to what extend this resembles what you had in mind. I suggest that you omit VHDL-related examples for the time being. (After reading Daniel's early post again -- i.e. the second link in the PS -- I realize Daniel already proposed something very similar. I might have messed up by attributing ideas to the wrong persons; my apologies.) Daniel, I would like to invite you specifically to take part in this discussion/brainstorm session. -Julian PS: some context for those who need it, in chronological order. http://nvie.com/posts/a-successful-git-branching-model/ Introduction of the gitflow branching model by the inventor. http://lists.boost.org/Archives/boost/2012/02/190215.php Post by Daniel James, in which he raised his concerns about testing in gitflow. The post contained an idea similar to the one presented here. http://lists.boost.org/Archives/boost/2012/02/190269.php Further clarification of Daniel's concern. http://dl.dropbox.com/u/3512486/boost_branching_model.pdf An illustration that compares the current Boost workflow to one in which gitflow is applied. It doesn't cover testing, however. I introduced it in the following post: http://lists.boost.org/Archives/boost/2012/02/190312.php http://lists.boost.org/Archives/boost/2012/02/190327.php Thomas' concern about testing in a gitflow-like branching model. http://lists.boost.org/Archives/boost/2012/02/190352.php My reply to Daniel, arguing that gitflow wouldn't hurt the existing test cycle. http://lists.boost.org/Archives/boost/2012/02/190355.php My reply to Thomas, arguing that the svn-based workflow that he describes is like a stripped down version of the workflow he would get with gitflow. Note that Thomas' interesting idea discussed in the present post was not taken into account. The phrase "as observed by Julien" refers to this post: http://lists.boost.org/Archives/boost/2012/02/190324.php Daniel sent a reply to my reply to Thomas, which made me aware of Daniel's earlier similar proposal. At the time of writing, this post cannot be retrieved from the archives yet.
On 02/12/2012 12:34 PM, Julian Gonggrijp wrote:
Thomas Heller wrote:
On 02/11/2012 03:48 PM, Julian Gonggrijp wrote:
Thomas Heller wrote:
What would make this whole branching very cool is if we could somehow tie it with testing such that we are able to tell the testrunners to use our feature branch to run the tests (of course this must be automated in a way). I think in this scenario, a modularized boost will come in very handy ... Think of the VHDL model,
Do you mean this? http://en.wikipedia.org/wiki/VHDL
you have unit tests, but the module under test might be very different, though the unit tests stay the same. This is something i would really like to see.
Could you clarify? I don't see why we would want to test a different module with the same unit tests.
Yes, I mean the hardware description language. Let me explain: In VHDL you usually start prototyping your new design by writing some behavioral description at a very high level, almost like you do with a "normal" programming language. After you are done, you start to write your unit tests for that. You do that by writing a very generic unit test and refer to your new design as "device under test". After you successfully passed all the tests, you start by getting down one level in the abstraction, lets say RTL level (Register transfer level). After that you plug that new device into your unit tests to ensure correct behavior. Now, imagine that different branches are seen as different devices, so to say. Instead of letting the test runners run the unit tests defined in the main branch, you let'em run the unit tests in your new feature branch, completely transparent for them. This means that you don't need to disrupt regular service by committing a half done feature to the main branch, but get the full advantage of the different test runners.
Thomas, I've given this some thought and I can't help but think that the VHDL comparison doesn't really work. That's mainly because Boost libraries or hypothetical Boost modules are not like subdevices that cooperate on a common task, while that does seem to be the case with the VHDL example.
The VHDL business is causing us to miss your point, which is a waste since your idea does seem to be very interesting. My guess is that, given a branching model like gitflow, you envision something along the following lines:
Yes you are right, the VHDL way of testing different components was just an inspiration to what could be created. Of course, a new feature or a refactoring is not the same as different architectural descriptions of a piece of hardware ... Anyway ...
There is a "testing image" which the testing volunteers run their periodic tests on. By default, the image mirrors the latest commit on the develop branch. When working on a feature branch, library authors can reversibly and transparently substitute the version of their library from the feature branch for the default version in the image. All of this is facilitated by some kind of automated tool which has yet to be found.
The result is that most libraries in the image are from develop, but some are from feature branches, without the testing volunteers needing to worry about it. They can run a single testing cycle on the heterogeneous image. For library authors, this brings the advantage that they can already have their changes tested on all platforms before they feel confident that it should be possible to make them work. (Or while they emply a long-term feature branch specifically in order to support a platform, per Daniel James.)
Just for reference, and for better understanding of the current test model i suggest to get familiar with it: http://www.boost.org/development/running_regression_tests.html
Please let us know to what extend this resembles what you had in mind. I suggest that you omit VHDL-related examples for the time being.
*nod* looks like you absorbed it and summarized my ideas, thanks ;)
(After reading Daniel's early post again -- i.e. the second link in the PS -- I realize Daniel already proposed something very similar. I might have messed up by attributing ideas to the wrong persons; my apologies.)
Daniel, I would like to invite you specifically to take part in this discussion/brainstorm session.
-Julian
PS: some context for those who need it, in chronological order.
http://nvie.com/posts/a-successful-git-branching-model/ Introduction of the gitflow branching model by the inventor.
http://lists.boost.org/Archives/boost/2012/02/190215.php Post by Daniel James, in which he raised his concerns about testing in gitflow. The post contained an idea similar to the one presented here.
http://lists.boost.org/Archives/boost/2012/02/190269.php Further clarification of Daniel's concern.
http://dl.dropbox.com/u/3512486/boost_branching_model.pdf An illustration that compares the current Boost workflow to one in which gitflow is applied. It doesn't cover testing, however. I introduced it in the following post: http://lists.boost.org/Archives/boost/2012/02/190312.php
http://lists.boost.org/Archives/boost/2012/02/190327.php Thomas' concern about testing in a gitflow-like branching model.
http://lists.boost.org/Archives/boost/2012/02/190352.php My reply to Daniel, arguing that gitflow wouldn't hurt the existing test cycle.
http://lists.boost.org/Archives/boost/2012/02/190355.php My reply to Thomas, arguing that the svn-based workflow that he describes is like a stripped down version of the workflow he would get with gitflow. Note that Thomas' interesting idea discussed in the present post was not taken into account. The phrase "as observed by Julien" refers to this post: http://lists.boost.org/Archives/boost/2012/02/190324.php
Daniel sent a reply to my reply to Thomas, which made me aware of Daniel's earlier similar proposal. At the time of writing, this post cannot be retrieved from the archives yet.
_______________________________________________ Unsubscribe& other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Thomas Heller wrote:
On 02/12/2012 12:34 PM, Julian Gonggrijp wrote:
[...] My guess is that, given a branching model like gitflow, you envision something along the following lines:
There is a "testing image" which the testing volunteers run their periodic tests on. By default, the image mirrors the latest commit on the develop branch. When working on a feature branch, library authors can reversibly and transparently substitute the version of their library from the feature branch for the default version in the image. All of this is facilitated by some kind of automated tool which has yet to be found.
The result is that most libraries in the image are from develop, but some are from feature branches, without the testing volunteers needing to worry about it. They can run a single testing cycle on the heterogeneous image. For library authors, this brings the advantage that they can already have their changes tested on all platforms before they feel confident that it should be possible to make them work. (Or while they emply a long-term feature branch specifically in order to support a platform, per Daniel James.)
*nod* looks like you absorbed it and summarized my ideas, thanks ;)
That's good to know. I think I figured out a way to do this. It's a more elegant variant of the outline provided by Daniel in [1], quoted here. Daniel James wrote:
Or maybe, rather than being a traditional branch, the test repo could be created by a script which merges in active feature branches. It'd have to deal with conflicts though, maybe just keeping track of commits that are known to be good and sticking with them when a conflict is found. But something like that could be developed later - especially after having some experience of how the process works.
This testing mechanism would be fairly separate from the main flow structure so it could be provided by an additional tool to gitflow. Hopefully wouldn't be too hard to develop. Does that make any sense? I know it isn't ideal, but it seems plausible given our current infrastructure. And we could work on improving it as we go.
I think it would be both wise and feasible to require (1) that authors edit a given library either on the develop branch or on any number of feature branches, but not on both in parallel, and (2) that a public feature branch always affects only one library. Apart from avoiding confusion, this offers a strong guarantee that a feature branch can always be merged back into develop without conflicts. Given such a convention, I think the test imaging tool is almost trivial to implement. The core implementation takes only two additional small files in the /status directory: a list of feature branches that authors want to be substituted for the default in the develop branch, and a small script that will be run by the volunteers. Assuming that the list is a newline-separated text file like this (just an example, take it with some grains of salt), substitute.txt (must be edited on the develop branch only):
newlibs/application math/octonion/hardwareaccel data/heap
the core of the script would be something like this, testcomposite.sh:
# must be run from $BOOST_ROOT/status git checkout develop git pull awk 'begin { print "git merge --no-commit --no-rerere-autoupdate" } ; { printf "remote/%s" , $0 }' substitute.txt >mergecommand.sh sh mergecommand.sh rm mergecommand.sh bjam # an additional command to report the results? bjam --clean git checkout develop
Note that the last line undoes the custom merge, so the test image is only a temporary state of the volunteer's working directory. The script I outlined and the existing testing tools will need some adjustments to make the summary correctly display which commits have been tested. I believe this shouldn't be an obstacle. It should be easy to port to most platforms, as it's a plain command list and the only external tools used are portable themselves, i.e. git, awk and bjam. I suspect it would be possible to realize this with a svn-based implementation of the branching model as well. That would however have some drawbacks with regard to the merge operation, in terms of robustness and speed: - substitute.txt and the awk command would be more complex, since svn merge requires authors to not only list the name of the branch but also the path to the library and the revision number; - it would require multiple calls to svn merge, because svn can merge only two branches at a time; - it would involve more server interaction because the volunteer has no local copy of the history. Putting the VCS considerations aside, it seems like we have an exciting opportunity here. I'm looking forward to all feedback. -Julian 1. http://lists.boost.org/Archives/boost/2012/02/190215.php
on Wed Feb 08 2012, Daniel James <dnljms-AT-gmail.com> wrote:
On 8 February 2012 14:37, Julian Gonggrijp <j.gonggrijp@gmail.com> wrote:
This might have been mentioned before, but gitflow seems to be ideal. See [5] for an explanation of the branching model and [6] for an optional tool which automates workflows that adopt the model.
The problem that we face with something like gitflow is testing. We only have the infrastructure for testing two branches so we can't adequately test feature branches. We really do need to test on a wide variety of platforms as early as possible.
The way we plan to handle this with Ryppl is that you check in a testing specification with your project. The testing specification is just a text file, something like this JSON: { <configname>: { dependencies: { /* dependency versions */ <libname>: <git tree-ish>, ... }, cmake: { /* properties passed to cmake */ <key>: <value>, ... }, platforms: [ <platform-string>, ... ] }, ... } Where you can request multiple configurations to be tested for commits on each branch. To get your in-development work tested, just publish a feature branch containing a test specification file to your public repository. The test results are then added to the commit using git notes.
But we wouldn't want to merge half done features into the develop branch just to get testing (this is one of the biggest problems we have at the moment, changes are often in trunk for some time before they're merged into release, if at all).
A possible solution would be to use the 'develop' branch as the equivalent of our current release branch
I take for granted that we're going to have a modularized boost with a separate repository for each library (<https://github.com/boost-lib>) and each library will have its own develop branch. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
Dave Abrahams wrote:
on Wed Feb 08 2012, Daniel James <dnljms-AT-gmail.com> wrote:
The problem that we face with something like gitflow is testing. We only have the infrastructure for testing two branches so we can't adequately test feature branches. We really do need to test on a wide variety of platforms as early as possible.
The way we plan to handle this with Ryppl is that you check in a testing specification with your project. The testing specification is just a text file, something like this JSON:
{ <configname>: { dependencies: { /* dependency versions */ <libname>: <git tree-ish>, ... }, cmake: { /* properties passed to cmake */ <key>: <value>, ... }, platforms: [ <platform-string>, ... ] }, ... }
Where you can request multiple configurations to be tested for commits on each branch. To get your in-development work tested, just publish a feature branch containing a test specification file to your public repository.
The test results are then added to the commit using git notes.
How convenient will that be for the testers? How does this approach compare to the one discussed in http://lists.boost.org/Archives/boost/2012/02/190418.php ?
I take for granted that we're going to have a modularized boost with a separate repository for each library (<https://github.com/boost-lib>) and each library will have its own develop branch.
This sounds like a situation that calls for git-subtree, but maybe I understood git-subtree wrongly. -Julian
on Tue Feb 28 2012, Julian Gonggrijp <j.gonggrijp-AT-gmail.com> wrote:
Dave Abrahams wrote:
on Wed Feb 08 2012, Daniel James <dnljms-AT-gmail.com> wrote:
The problem that we face with something like gitflow is testing. We only have the infrastructure for testing two branches so we can't adequately test feature branches. We really do need to test on a wide variety of platforms as early as possible.
The way we plan to handle this with Ryppl is that you check in a testing specification with your project. The testing specification is just a text file, something like this JSON:
{ <configname>: { dependencies: { /* dependency versions */ <libname>: <git tree-ish>, ... }, cmake: { /* properties passed to cmake */ <key>: <value>, ... }, platforms: [ <platform-string>, ... ] }, ... }
Where you can request multiple configurations to be tested for commits on each branch. To get your in-development work tested, just publish a feature branch containing a test specification file to your public repository.
The test results are then added to the commit using git notes.
How convenient will that be for the testers?
If by testers you mean the people that manage machines used as testing resources, it will not require any manual intervention on their part. We may eventually need a way for them to specify which tests they are willing to run, though. For example, I may want my test slave to "opt out" out of all tests of feature branches.
How does this approach compare to the one discussed in http://lists.boost.org/Archives/boost/2012/02/190418.php ?
The message above seems to be (mostly) describing a mechanism for automatically merging feature branches. As far as I can tell, the approaches are apples and oranges, and thus are basically compatible. We do something like that at Boostpro for handling upstream merges to Clang. Once Boost is properly modularized, I'm not sure how important such a mechanism will be for most libraries, which are maintained by individuals. That said, *if* you have to manage lots of long-lived feature branches, a tool like topgit can be used very effectively along with git-flow. There's one important thing about the git-flow model that I'm not sure you've accounted for, though: on feature branches, completely broken commits are de-rigeur.
I take for granted that we're going to have a modularized boost with a separate repository for each library (<https://github.com/boost-lib>) and each library will have its own develop branch.
This sounds like a situation that calls for git-subtree, but maybe I understood git-subtree wrongly.
git-subtree, IIUC, is a mechanism for merging/extracting individual repository histories into/from separate subdirectories of a larger repository. My main question is, why does Boost want to do that? I understand why we want separate repositories, but I don't yet understand why anyone wants a monolithic whole. On the other hand, I *can* see how it would be very useful as a basis for the one-time history-preserving modularization step. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
Dave Abrahams wrote:
on Tue Feb 28 2012, Julian Gonggrijp <j.gonggrijp-AT-gmail.com> wrote:
Dave Abrahams wrote:
on Wed Feb 08 2012, Daniel James <dnljms-AT-gmail.com> wrote:
The problem that we face with something like gitflow is testing. [...]
The way we plan to handle this with Ryppl is that you check in a testing specification with your project. The testing specification is just a text file, something like this JSON:
[...]
Where you can request multiple configurations to be tested for commits on each branch. To get your in-development work tested, just publish a feature branch containing a test specification file to your public repository.
The test results are then added to the commit using git notes.
How does this approach compare to the one discussed in http://lists.boost.org/Archives/boost/2012/02/190418.php ?
The message above seems to be (mostly) describing a mechanism for automatically merging feature branches. As far as I can tell, the approaches are apples and oranges, and thus are basically compatible. We do something like that at Boostpro for handling upstream merges to Clang.
I'm not sure whether this is correct. The script I outlined does involve a git merge, but it's only a temporary one in the working directory that will be reset afterwards. The purpose of the script is not to merge (in fact the intent is to leave all branches unchanged) but to provide a transparent image to the testing volunteer, which includes exactly one branch of each Boost library. So my impression is that the planned Ryppl testing model and the testing model concept from the thread in the Boost archives do have similar goals, i.e. to automate testing of heterogeneous selections of branches from the various libraries. After some more thought I suspect the planned approach from Ryppl is superior, though. :-)
There's one important thing about the git-flow model that I'm not sure you've accounted for, though: on feature branches, completely broken commits are de-rigeur.
I was aware of that; the idea would be that a library developer pushes the feature branch to be tested to the public repository when the last commit on that branch is expected to be stable, then waits until it's tested before pushing again. After all, the developer isn't interested in testing completely broken commits. I think this is the same thing that the planned Ryppl approach aims to accomodate for by listing the exact commits to be tested.
I take for granted that we're going to have a modularized boost with a separate repository for each library (<https://github.com/boost-lib>) and each library will have its own develop branch.
This sounds like a situation that calls for git-subtree, but maybe I understood git-subtree wrongly.
git-subtree, IIUC, is a mechanism for merging/extracting individual repository histories into/from separate subdirectories of a larger repository. My main question is, why does Boost want to do that? I understand why we want separate repositories, but I don't yet understand why anyone wants a monolithic whole.
/No Boost supertree at all/, that was an eye opener for me. It seems so natural to have a supertree, but I now realise it's not that obvious.
On the other hand, I *can* see how it would be very useful as a basis for the one-time history-preserving modularization step.
I was going to respond to your previous email and ask why there is still a Boost "super-project" at https://github.com/boost-lib , but you answered that question over here. Thanks. -Julian
Does the https://github.com/boost-lib on github mean we can send pull request to boost? Thanks, Paul
on Wed Feb 29 2012, paul Fultz <pfultz2-AT-yahoo.com> wrote:
Does the https://github.com/boost-lib on github mean we can send pull request to boost?
You can send, but for the time being, nobody will respond :-) Those repos are being maintained automatically (see https://svn.boost.org/trac/boost/wiki/CMakeModularizationStatus) -- Dave Abrahams BoostPro Computing http://www.boostpro.com
on Wed Feb 29 2012, Julian Gonggrijp <j.gonggrijp-AT-gmail.com> wrote:
Dave Abrahams wrote:
on Tue Feb 28 2012, Julian Gonggrijp <j.gonggrijp-AT-gmail.com> wrote:
Dave Abrahams wrote:
on Wed Feb 08 2012, Daniel James <dnljms-AT-gmail.com> wrote:
The problem that we face with something like gitflow is testing. [...]
The way we plan to handle this with Ryppl is that you check in a testing specification with your project. The testing specification is just a text file, something like this JSON:
[...]
Where you can request multiple configurations to be tested for commits on each branch. To get your in-development work tested, just publish a feature branch containing a test specification file to your public repository.
The test results are then added to the commit using git notes.
How does this approach compare to the one discussed in http://lists.boost.org/Archives/boost/2012/02/190418.php ?
The message above seems to be (mostly) describing a mechanism for automatically merging feature branches. As far as I can tell, the approaches are apples and oranges, and thus are basically compatible. We do something like that at Boostpro for handling upstream merges to Clang.
I'm not sure whether this is correct. The script I outlined does involve a git merge, but it's only a temporary one in the working directory that will be reset afterwards. The purpose of the script is not to merge (in fact the intent is to leave all branches unchanged) but to provide a transparent image to the testing volunteer, which includes exactly one branch of each Boost library.
Sorry, I don't understand what you mean by a "transparent image."
So my impression is that the planned Ryppl testing model and the testing model concept from the thread in the Boost archives do have similar goals, i.e. to automate testing of heterogeneous selections of branches from the various libraries. After some more thought I suspect the planned approach from Ryppl is superior, though. :-)
Why do you think that? (you give up pretty easily!)
There's one important thing about the git-flow model that I'm not sure you've accounted for, though: on feature branches, completely broken commits are de-rigeur.
I was aware of that; the idea would be that a library developer pushes the feature branch to be tested to the public repository when the last commit on that branch is expected to be stable, then waits until it's tested before pushing again. After all, the developer isn't interested in testing completely broken commits. I think this is the same thing that the planned Ryppl approach aims to accomodate for by listing the exact commits to be tested.
Well, you don't need to list "exact commits". A tree-ish could be a branch name, so every change on that branch gets tested. I suppose that if you're checking in known-broken work it might make sense to have a special notation in the commit message that would prevent that commit from being tested when it appears on a feature branch. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
(Citation levels: Dave - Julian - Dave - Julian)
[...] How does this approach compare to the one discussed in http://lists.boost.org/Archives/boost/2012/02/190418.php ?
The message above seems to be (mostly) describing a mechanism for automatically merging feature branches. As far as I can tell, the approaches are apples and oranges, and thus are basically compatible. We do something like that at Boostpro for handling upstream merges to Clang.
I'm not sure whether this is correct. The script I outlined does involve a git merge, but it's only a temporary one in the working directory that will be reset afterwards. The purpose of the script is not to merge (in fact the intent is to leave all branches unchanged) but to provide a transparent image to the testing volunteer, which includes exactly one branch of each Boost library.
Sorry, I don't understand what you mean by a "transparent image."
Oh. I think it was explained at the top of the message I pointed to, though. Basically the "testing image" is just a working copy of the (presumed monolithic) Boost tree, but not from one branch; most subdirectories come from the (super-project) develop branch, while others may come from any feature branch that the corresponding developers happened to "subscribe for testing". For a more complete explanation I think you could best just (re-)read the discussion that led up to that post, starting here: http://lists.boost.org/Archives/boost/2012/02/190269.php you could from there follow the bottom "Reply:" after every message, which will bring you to the end in 8 posts.
So my impression is that the planned Ryppl testing model and the testing model concept from the thread in the Boost archives do have similar goals, i.e. to automate testing of heterogeneous selections of branches from the various libraries. After some more thought I suspect the planned approach from Ryppl is superior, though. :-)
Why do you think that? (you give up pretty easily!)
Basically, from what you said the planned Ryppl approach seems more flexible: - developers can subscribe multiple branches to be tested instead of only one (which implicitly defaults to the develop branch); - there is a possibility to explicitly state which commit should be tested; - test slave owners can flexibly opt out of certain branches while still testing the other branches of the same library. I didn't mean to "give up", though. My proposed realisation of Thomas' idea is much simpler and seems more concrete. It's almost guaranteed to work given a "conservative" gitflow reincarnation of the current monolithic Boost tree. It might also work with a modularized Boost, in which case I'd consider it some kind of "primitive predecessor" of the planned Rypple testing model. But I didn't think very hard about the latter possibility. -Julian
On 28 February 2012 20:03, Dave Abrahams <dave@boostpro.com> wrote:
The way we plan to handle this with Ryppl is that you check in a testing specification with your project. The testing specification is just a text file, something like this JSON:
Ryppl comes with a few problems. If the aim is to use git, then it's an expensive precondition.
on Wed Feb 29 2012, Daniel James <dnljms-AT-gmail.com> wrote:
On 28 February 2012 20:03, Dave Abrahams <dave@boostpro.com> wrote:
The way we plan to handle this with Ryppl is that you check in a testing specification with your project. The testing specification is just a text file, something like this JSON:
Ryppl comes with a few problems.
Specifically?
If the aim is to use git, then it's an expensive precondition.
What is? If you mean Ryppl... Ryppl is not a precondition for the transition to Git. In some sense it's the other way around: a modularized Boost is a precondition for Boost's transition to Ryppl. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On 1 March 2012 00:44, Dave Abrahams <dave@boostpro.com> wrote:
on Wed Feb 29 2012, Daniel James <dnljms-AT-gmail.com> wrote:
On 28 February 2012 20:03, Dave Abrahams <dave@boostpro.com> wrote:
The way we plan to handle this with Ryppl is that you check in a testing specification with your project. The testing specification is just a text file, something like this JSON:
Ryppl comes with a few problems.
Specifically?
You should be telling us that. It's vapourware, it's conceptually unproven, it's coupled to a controversial build system, it's a large disruptive change.
If the aim is to use git, then it's an expensive precondition.
What is?
If you mean Ryppl... Ryppl is not a precondition for the transition to Git. In some sense it's the other way around: a modularized Boost is a precondition for Boost's transition to Ryppl.
OK then, a modularised boost is an expensive precondition for moving to git. Perhaps we should be discussing how to use an alternative version control system while changing as little as possible. The other stuff can come later.
on Thu Mar 01 2012, Daniel James <dnljms-AT-gmail.com> wrote:
On 1 March 2012 00:44, Dave Abrahams <dave@boostpro.com> wrote:
on Wed Feb 29 2012, Daniel James <dnljms-AT-gmail.com> wrote:
On 28 February 2012 20:03, Dave Abrahams <dave@boostpro.com> wrote:
The way we plan to handle this with Ryppl is that you check in a testing specification with your project. The testing specification is just a text file, something like this JSON:
Ryppl comes with a few problems.
Specifically?
You should be telling us that. It's vapourware, it's conceptually unproven, it's coupled to a controversial build system, it's a large disruptive change.
Yes, all of these things are true. I'm not sure why I should be telling you that, though. I only asked because I wanted to know to what you were referring.
If the aim is to use git, then it's an expensive precondition.
What is?
If you mean Ryppl... Ryppl is not a precondition for the transition to Git. In some sense it's the other way around: a modularized Boost is a precondition for Boost's transition to Ryppl.
OK then, a modularised boost is an expensive precondition for moving to git.
It isn't a precondition for moving to Git. We can move to Git and then do the modularization step; it's not a problem. However, it does mean two transitions.
Perhaps we should be discussing how to use an alternative version control system while changing as little as possible. The other stuff can come later.
Sure, by all means. I think you misunderstood my posting for some sort of advocacy. We've done some thinking about these problems and I thought the conversation could benefit from that thinking; that's all. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On 1 March 2012 13:06, Dave Abrahams <dave@boostpro.com> wrote:
on Thu Mar 01 2012, Daniel James <dnljms-AT-gmail.com> wrote:
On 1 March 2012 00:44, Dave Abrahams <dave@boostpro.com> wrote:
on Wed Feb 29 2012, Daniel James <dnljms-AT-gmail.com> wrote:
Ryppl comes with a few problems.
Specifically?
You should be telling us that. It's vapourware, it's conceptually unproven, it's coupled to a controversial build system, it's a large disruptive change.
Yes, all of these things are true. I'm not sure why I should be telling you that, though. I only asked because I wanted to know to what you were referring.
Sorry, having re-read that, it was too aggressive, I take it back. I thought these things were well established, which is why I didn't mention them in the first place.
OK then, a modularised boost is an expensive precondition for moving to git.
It isn't a precondition for moving to Git. We can move to Git and then do the modularization step; it's not a problem. However, it does mean two transitions.
Which can be a good thing. Breaking a process down into smaller stages can make it easier. It seems to me that we've discussed git several times, and it's always part of a grand scheme. If git is considered desirable enough, then it might be best to just switch to it, keeping everything else the same.
on Thu Mar 01 2012, Daniel James <dnljms-AT-gmail.com> wrote:
On 1 March 2012 13:06, Dave Abrahams <dave@boostpro.com> wrote:
on Thu Mar 01 2012, Daniel James <dnljms-AT-gmail.com> wrote:
OK then, a modularised boost is an expensive precondition for moving to git.
It isn't a precondition for moving to Git. We can move to Git and then do the modularization step; it's not a problem. However, it does mean two transitions.
Which can be a good thing. Breaking a process down into smaller stages can make it easier. It seems to me that we've discussed git several times, and it's always part of a grand scheme. If git is considered desirable enough, then it might be best to just switch to it, keeping everything else the same.
And the process of making that possible is already underway: https://github.com/ryppl/boost-svn Actually, I think the switch is mostly just awaiting a decision from Beman and any necessary associated discussion here on this list. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On 03/01/2012 08:07 AM, Dave Abrahams wrote:
And the process of making that possible is already underway: https://github.com/ryppl/boost-svn Actually, I think the switch is mostly just awaiting a decision from Beman and any necessary associated discussion here on this list. Sounds like the discussion should start and a milestone identified for making it happen, i.e. release 1.50 with current procedures and 1.51 will move to git? My, admittedly little, experience with github and git in general is that I'm anxiously awaiting for boost to make this change.
Dave Abrahams wrote:
on Thu Mar 01 2012, Daniel James <dnljms-AT-gmail.com> wrote:
On 1 March 2012 13:06, Dave Abrahams <dave@boostpro.com> wrote:
on Thu Mar 01 2012, Daniel James <dnljms-AT-gmail.com> wrote:
OK then, a modularised boost is an expensive precondition for moving to git.
It isn't a precondition for moving to Git. We can move to Git and then do the modularization step; it's not a problem. However, it does mean two transitions.
Which can be a good thing. Breaking a process down into smaller stages can make it easier. It seems to me that we've discussed git several times, and it's always part of a grand scheme. If git is considered desirable enough, then it might be best to just switch to it, keeping everything else the same.
And the process of making that possible is already underway: https://github.com/ryppl/boost-svn
Thanks for pointing to the ryppl account on github. It raised some questions for me: 1. There are two git versions of the Boost tree in there, boost-svn and boost-history, which to my uninformed eye seem to be doing the same thing. boost-svn is a "live mirror" of the Boost svn repository while boost-history is a "faithful conversion" of the Boost svn repository. Apart from the fact that boost-svn has more branches than boost-history and that boost-history seems to be updated less often, I see no difference. Why do these two git repositories exist in parallel? 2. Again to my uninformed eye, all those branches of boost-svn look a bit disorganised. Wouldn't it need some cleaning and reorganising either shortly before or shortly after the switch? Or is that perhaps exactly the purpose of boost-history?
Actually, I think the switch is mostly just awaiting a decision from Beman and any necessary associated discussion here on this list.
on Fri Mar 02 2012, Julian Gonggrijp <j.gonggrijp-AT-gmail.com> wrote:
Dave Abrahams wrote:
And the process of making that possible is already underway: https://github.com/ryppl/boost-svn
Thanks for pointing to the ryppl account on github. It raised some questions for me:
1. There are two git versions of the Boost tree in there, boost-svn and boost-history, which to my uninformed eye seem to be doing the same thing. boost-svn is a "live mirror" of the Boost svn repository while boost-history is a "faithful conversion" of the Boost svn repository. Apart from the fact that boost-svn has more branches than boost-history and that boost-history seems to be updated less often, I see no difference. Why do these two git repositories exist in parallel?
2. Again to my uninformed eye, all those branches of boost-svn look a bit disorganised. Wouldn't it need some cleaning and reorganising either shortly before or shortly after the switch? Or is that perhaps exactly the purpose of boost-history?
I'm going to leave the definitive answer to John Wiegley, who is doing the conversion, but I think it's like this: boost-svn is being generated by the "traditional" git-svn tool. The results of git-svn are, however, not adequate for correctly preserving Boost's history boost-history is being generated by https://github.com/jwiegley/subconvert, a tool John wrote to do a "faithful" conversion. If you want to know the precise differences between the two, John will have to explain them to you... but if you look at the former repository you can find lots of spurious-looking branches, e.g. "trunk@38326," "trunk@38328," etc. So I wouldn't be concerned by the smaller number of branches in the latter mirror. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
on Fri Mar 02 2012, Julian Gonggrijp <j.gonggrijp-AT-gmail.com> wrote:
Dave Abrahams wrote:
And the process of making that possible is already underway: https://github.com/ryppl/boost-svn
Thanks for pointing to the ryppl account on github. It raised some questions for me:
1. There are two git versions of the Boost tree in there, boost-svn and boost-history, which to my uninformed eye seem to be doing the same thing. boost-svn is a "live mirror" of the Boost svn repository while boost-history is a "faithful conversion" of the Boost svn repository. Apart from the fact that boost-svn has more branches than boost-history and that boost-history seems to be updated less often, I see no difference. Why do these two git repositories exist in parallel?
2. Again to my uninformed eye, all those branches of boost-svn look a bit disorganised. Wouldn't it need some cleaning and reorganising either shortly before or shortly after the switch? Or is that perhaps exactly the purpose of boost-history?
I'm going to leave the definitive answer to John Wiegley, who is doing the conversion, but I think it's like this: boost-svn is being generated by the "traditional" git-svn tool. The results of git-svn are, however, not adequate for correctly preserving Boost's history boost-history is being generated by https://github.com/jwiegley/subconvert, a tool John wrote to do a "faithful" conversion. If you want to know the precise differences between the two, John will have to explain them to you... but if you look at the former repository you can find lots of spurious-looking branches, e.g. "trunk@38326," "trunk@38328," etc. So I wouldn't be concerned by the smaller number of branches in the latter mirror. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
Dave Abrahams <dave@boostpro.com> writes:
If you want to know the precise differences between the two, John will have to explain them to you... but if you look at the former repository you can find lots of spurious-looking branches, e.g. "trunk@38326," "trunk@38328," etc. So I wouldn't be concerned by the smaller number of branches in the latter mirror.
One difference between git-svn and subconvert is that git-svn doesn't preserve ancient branches which had the same name as current branches. For example, you create a branch foo, then later delete it, then create a new branch foo. git-svn will throw away any history related to the first branch foo. There are a few other cases where git-svn tosses history to make its job simpler. Case in point: git-svn version of Boost history: 660,068 Git objects subconvert version of Boost history: 1,204,586 Git objects Clearly a lot of something is getting lost on the cutting floor with git-svn. subconvert's goal is preserve every Subversion revision that every existed somewhere behind a branch or a tag in the resulting Git repository. -- John Wiegley BoostPro Computing, Inc. http://www.boostpro.com
on Fri Mar 02 2012, John Wiegley <jwiegley-AT-gmail.com> wrote:
Dave Abrahams <dave@boostpro.com> writes:
If you want to know the precise differences between the two, John will have to explain them to you... but if you look at the former repository you can find lots of spurious-looking branches, e.g. "trunk@38326," "trunk@38328," etc. So I wouldn't be concerned by the smaller number of branches in the latter mirror.
One difference between git-svn and subconvert is that git-svn doesn't preserve ancient branches which had the same name as current branches.
For example, you create a branch foo, then later delete it, then create a new branch foo. git-svn will throw away any history related to the first branch foo.
There are a few other cases where git-svn tosses history to make its job simpler. Case in point:
git-svn version of Boost history: 660,068 Git objects subconvert version of Boost history: 1,204,586 Git objects
Clearly a lot of something is getting lost on the cutting floor with git-svn. subconvert's goal is preserve every Subversion revision that every existed somewhere behind a branch or a tag in the resulting Git repository.
Thanks, John --- It'd be awesome if you could update the github repos' comment areas with that information. Also, I'd be happy to throw out the git-svn version and stick with the subconvert version from here on out, provided you could get the subconvert repo to update itself more frequently. What do you think? -- Dave Abrahams BoostPro Computing http://www.boostpro.com
Dave Abrahams <dave@boostpro.com> writes:
> Dave Abrahams <dave@boostpro.com> writes:
Thanks, John --- It'd be awesome if you could update the github repos' comment areas with that information.
Will do!
Also, I'd be happy to throw out the git-svn version and stick with the subconvert version from here on out, provided you could get the subconvert repo to update itself more frequently.
What do you think?
I've set it up to update nightly on my computer, but I've noticed some tags not getting created in the converted repo, so let me track that down first. Once that is done, I'll let you know and we can go from there. Exciting stuff! -- John Wiegley BoostPro Computing, Inc. http://www.boostpro.com
Daniel James wrote:
On 1 March 2012 13:06, Dave Abrahams <dave@boostpro.com> wrote:
on Thu Mar 01 2012, Daniel James <dnljms-AT-gmail.com> wrote:
OK then, a modularised boost is an expensive precondition for moving to git.
It isn't a precondition for moving to Git. We can move to Git and then do the modularization step; it's not a problem. However, it does mean two transitions.
Which can be a good thing. Breaking a process down into smaller stages can make it easier. It seems to me that we've discussed git several times, and it's always part of a grand scheme.
If by "part of a grand scheme" you mean CMake, modularization or the Ryppl model in its entirety, I agree; but if you mean gitflow branching or the crude test image model that emerged from our discussion with Thomas Heller, I don't. I think both of the latter two ideas would involve only small, gradual changes which are best implemented (shortly) after the conversion to git is made. These are just a matter of "adapting to new opportunities". In fact, it seems that https://github.com/ryppl/boost-svn almost conforms to the gitflow branching model already.
If git is considered desirable enough, then it might be best to just switch to it, keeping everything else the same.
On 2 March 2012 11:18, Julian Gonggrijp <j.gonggrijp@gmail.com> wrote:
Daniel James wrote:
on Thu Mar 01 2012, Daniel James <dnljms-AT-gmail.com> wrote:
Which can be a good thing. Breaking a process down into smaller stages can make it easier. It seems to me that we've discussed git several times, and it's always part of a grand scheme.
If by "part of a grand scheme" you mean CMake, modularization or the Ryppl model in its entirety, I agree; but if you mean gitflow branching or the crude test image model that emerged from our discussion with Thomas Heller, I don't. I think both of the latter two ideas would involve only small, gradual changes which are best implemented (shortly) after the conversion to git is made. These are just a matter of "adapting to new opportunities".
I was referring more to previous discussions (including things that I've said). I have no idea how difficult it would be to switch to something like gitflow. I think I understand how gitflow is supposed to work in theory, but have no experience of how it works in practise.
on Wed Feb 08 2012, Julian Gonggrijp <j.gonggrijp-AT-gmail.com> wrote:
Olaf van der Spek wrote:
On Wed, Feb 8, 2012 at 7:30 AM, Steven Samuel Cole <steven.samuel.cole@gmail.com> wrote:
...
I've never used git, but I have used bzr. Having worked with a dvcs, the disadvantages of svn become quite clear. Svn *might* be fine for personal (one dev) projects, but for bigger projects it's just not the right option.
I can't think of any real downsides.
I found out that John Wiegley has been working on a faithful conversion of the boost svn tree to git, see [1]. Is that effort still ongoing?
Yes
Also, there has been a very interesting discussion on branching model and release process, which seems to have started with [2]. Two subthreads particularly interested me, i.e. the one starting in [3] and another one starting just before [4] (especially the dialogue between Vladimir Prus and Dave Abrahams). This might have been mentioned before, but gitflow seems to be ideal.
+1 I hope we'll be using git-flow. https://github.com/nvie/gitflow -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On Wed, Feb 8, 2012 at 1:30 AM, Steven Samuel Cole <steven.samuel.cole@gmail.com> wrote:
... extensive description of Git and its advantages to Boost.
I've become a firm believer in the advantages of Git, both for my personal projects and for larger projects. I believe Boost should move to Git sooner rather than later. I've proposed a BoostCon/C++Now tutorial and workshop on moving Boost to Git. The Steering Committee will be meeting, so perhaps we can get a go ahead for Git! Thanks to Steven for his informative and timely post! --Beman PS: FWIW, the C++ committee is now using Git (via GitHub) for the working paper and other documents.
On Wed, Feb 8, 2012 at 8:09 PM, Beman Dawes <bdawes@acm.org> wrote:
I've become a firm believer in the advantages of Git, both for my personal projects and for larger projects. I believe Boost should move to Git sooner rather than later.
This topic reminded me of the ryppl project [1]. As I remember, it uses git for VCS and there were ideas to move Boost under ryppl one day. Does anyone know the status of this project? [1] www.ryppl.org
On Fri, Feb 10, 2012 at 10:32, Andrey Semashev <andrey.semashev@gmail.com>wrote:
On Wed, Feb 8, 2012 at 8:09 PM, Beman Dawes <bdawes@acm.org> wrote:
I've become a firm believer in the advantages of Git, both for my personal projects and for larger projects. I believe Boost should move to Git sooner rather than later.
This topic reminded me of the ryppl project [1]. As I remember, it uses git for VCS and there were ideas to move Boost under ryppl one day. Does anyone know the status of this project?
[1] www.ryppl.org
Hi, Very recent discussions on the Ryppl mailing list are about refocus on what problem(s) Ryppl should fix, Apparently parts of the solution imply currently widely used tools like CMake and git, but also 0install that could fix the cross-platform distribution problem. Joël Lamotte
On 02/08/2012 07:30 AM, Steven Samuel Cole wrote:
Hi all,
First post here, so please bear with me.
A lot has been said in the last year on the subject of a possible migration from svn to git. However, after reading pretty much every message on the issue on this list (sic!), I have the impression that some of the most important aspects did not get the consideration they require. I would therefore like to add my outside perspective.
Thanks for your comments.
SUMMARY:
1. an svn --> git migration does cause some major, but one-time disruption. However, svn right now causes minor, but continuous disruption on a daily basis - which often goes unnoticed.
2. git's distributed concept is nothing less than one generation ahead of centralized VCSs like svn. The benefits usually do not become apparent until git is actually used in practice.
3. at the moment, boost is significantly missing out on fresh enthusiasm that new contributors could bring into the project; svn and the policies it mandates are partly too blame for that.
4. svn trunk (i.e. boost library) maintainers are too limited by the svn concept in their decision which changes they merge at what point in time and may be pressured into rushing suboptimal decisions; this can make maintenance work painful and frustrating. git greatly alleviates that problem.
5. IMO, it's not 'if', it is 'when'. The longer a migration is delayed, the longer boost development will be slowed down without even noticing.
<snip>
This is what happened here in the last couple of days:
Now let's play the same game with git.
I needed a tool for a job in my current project and I couldn't find one, so I thought about what components I need to build my own. A little web research - Boost.Iostreams looks perfect for the purpose.
You have all been through what comes next, be it with boost or with other open source software: Download the latest release, install the libraries, start reading the docs, run some sample code, etc.
While reading, I spot a typo in an HTML doc, then another in an include file comment, a third one in another file... the fourth one at the latest makes me think - this is open source, I should sync the latest sources, fix all those typos as I come across them while reading and contribute them back to the project.
Nothing really changes here.
A quick read on submitting bugs on boost.org, svn co yadi yadi, fix the typos locally. Of course I don't have write privileges to the repo, so the best I can do for now is create a patch and attach that to a bug report. Within minutes, someone takes care of the bug, comments and takes ownership. Excellent! Smithers, money fight!
Great! Now with something like github you create your own fork, push to that and eventually create a pull request and wait until it gets applied ... as you described. The process didn't change. You just named the steps differently.
I go back to reading, find more typos. My patch has not been committed to the trunk yet, so my local changes are still sitting there on my local hard drive. If I do another svn diff > file_2.patch, the patch will also contain the changes I have already attached to the bug report.
What are my options now ? Revert my changes, base the new ones on the head revision and send another patch ? Create a new patch file and manually remove the diffs of the old changes I already sent ? How will the upstream maintainer know which changes are based on what version ?
Simple. Update the bug you already filed with the patch containing all fixes. What would be the options with git? Update the pull request? Again, same thing, different name.
Also, I notice a few other things that in my opinion could be done better to facilitate adoption of the iostreams library. I would be willing to do the work and I would in the perfect position as I am walking in the shoes of an adopter right now - but these are separate from the typo fixes and they are larger issues; considering I can't check my changes in, would I really want to have them sit on my local harddrive, waiting possibly for months for someone upstream to review and hopefully merge them ?
So with git you pushed them to your personal fork. Over time, your fork and the upstream version eventually diverge. You have to maintain your changes. Same thing when the changes lurk on your local svn working.
A couple of hours later: I get an email from boost bugs/svn; the upstream maintainer has committed my changes. However, I actually receive not one, but two emails, because the upstream guy chose to split my patch into changes on documentation and changes on source code (which presumably required extra work on his part).
Frankly, I don't see how this would change with git.
I do an svn up and now have to deal with conflicts between my local changes and the new commits: I make a note of my second set of changes and overwrite my local files with the new repo versions, then bring the second set back in.
Same again. How is this significantly different to what you'd do with git?
Uff! That's a lot of work just to fix some typos! And these are just cosmetic issues in discrete chunks that do not require any testing; contributing changes to source code would take even more effort and more caution - build and test on several platforms, peer review, etc.
Right, all that wouldn't change with having git as the tool of choice.
Also, the upstream maintainer jumped on the bug report right away and integrated my changes within hours. This is the ideal situation (thanks again Daniel, great job! :-) - usually, things don't happen so quickly - especially around major release time.
Right, and that won't change with git. Or do I miss an essential point? <snip>
THE GIT ADVANTAGE:
I don't have to wait for the first set of changes to be merged into the trunk before I resync, resolve conflicts and base the second set on the new head revision; I simply create a new branch for a new change set and send a new pull request once my work is complete.
Errm ... you can still do that with svn. just continue working. Make a new patch, update the patch in the bug report you filed, yes trac can do that. There really is no need to wait for anything.
The upstream maintainer has no peer pressure to merge any changes in: He/she is sitting on the head revision and is looking at a number of pull requests for change sets in various personal forks like mine; these are more like 'change offers' the maintainer is free to merge in if he/she so chooses (cherry-picking) - or postpone if this is not the time because for example a major release is due and they seem too risky.
So, how is it really different from the current trac system?
I as a contributor do not really care so much: Of course I am stoked if my work actually makes it into the project eventually, but the point in time when this happens technically does not matter to me - pending changes will not get in my way if the merge takes a little longer.
This is the major difference between svn and git - and I can not stress this advantage enough: I am decoupled from, but still connected to the official trunk. I can make arbitrary changes and group them in any way I see fit. I offer them to upstream and move on to new development.
I think i miss something fundamental here. While there are shortcomings in the current process, they won't be solved by changing tools. Just my 0.02€. Sorry to be the party pooper ... Regards, Thomas P.S: I find it way harder to properly maintain a couple of git branches and forks than doing my stuff with svn. The git userinterface is just a pain, i regularly get confused and have no idea what to do. P.P.S: The only real advantage I see in DVCSs is the better local working copy managing, as in you can create local branches, commiting locally etc. but while having dismissed git in favor of svn for a while now, I didn't really miss these particular features.
On 02/08/2012 08:57 AM, Thomas Heller wrote:
P.S: I find it way harder to properly maintain a couple of git branches and forks than doing my stuff with svn. The git userinterface is just a pain, i regularly get confused and have no idea what to do.
When I first started using git, from being very comfortable with svn, it was very frustrating for me. The things that got me past the frustration and onto reasonable work flows were gitk, git gui, and git stash. gitk is a GUI that shows the branches, commits, etc git gui shows working copy/branch things, commits. Amending last commit and cherry picking are a great features built into gui. git stash is a stack of patches I started using git via git-svn which complicates things because of the necessity to commit back to svn properly. git fetch git svn rebase git stash are essential commands to know about to work with git-svn The upshot is, the initial learning curve, especially if coming from a cvs/svn background, is nonzero. One of my favorite quotes is that "learning to use c++/Unix/(now Git)" is hard, but using c++/Unix/git is easy." My opinion, the time investment in all of these tools will repay you 100 fold.
Also, I notice a few other things that in my opinion could be done better to facilitate adoption of the iostreams library. I would be willing to do the work and I would in the perfect position as I am walking in the shoes of an adopter right now - but these are separate from the typo fixes and they are larger issues; considering I can't check my changes in, would I really want to have them sit on my local harddrive, waiting possibly for months for someone upstream to review and hopefully merge them ?
So with git you pushed them to your personal fork. Over time, your fork and the upstream version eventually diverge. You have to maintain your changes. Same thing when the changes lurk on your local svn working.
well, your svn working copy is just a bunch of files, while git's personal branches are part of the repository ... you can merge, rebase, stash, etc ... with svn your working copy gets easily out of sync ... the nice thing of git is that it does not really harm if your branch diverges from upstream, as its merge facilities are much more sophisticated than everything that subversion has to offer ... frankly, since i moved my code into boost's svn, i find it way more fragile to maintain than before tim
On 02/08/2012 06:29 PM, Tim Blechmann wrote:
Also, I notice a few other things that in my opinion could be done better to facilitate adoption of the iostreams library. I would be willing to do the work and I would in the perfect position as I am walking in the shoes of an adopter right now - but these are separate from the typo fixes and they are larger issues; considering I can't check my changes in, would I really want to have them sit on my local harddrive, waiting possibly for months for someone upstream to review and hopefully merge them ?
So with git you pushed them to your personal fork. Over time, your fork and the upstream version eventually diverge. You have to maintain your changes. Same thing when the changes lurk on your local svn working.
well, your svn working copy is just a bunch of files, while git's personal branches are part of the repository ... you can merge, rebase, stash, etc ...
Right, you have those. Now what's left is to know when to use what, etc.
with svn your working copy gets easily out of sync ...
svn update is all you need. ever. There are conflicts from time to time. Man up resolve them.
the nice thing of git is that it does not really harm if your branch diverges from upstream, as its merge facilities are much more sophisticated than everything that subversion has to offer ...
I've heard that a couple of times now. My personal experience with git however is quite the contrary. I regularly mess up my local repository.
frankly, since i moved my code into boost's svn, i find it way more fragile to maintain than before
Yes, I used to advertise git too. I had the exact same experience. That goes away. After a while, you won't look back to git, and enjoy "svn up" and "svn commit" and start to question the usefulness of decentralized version control systems for a centralized boost altogether.
tim
_______________________________________________ Unsubscribe& other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
with svn your working copy gets easily out of sync ...
svn update is all you need. ever. There are conflicts from time to time. Man up resolve them.
true, but in my experience resolving conflicts with svn is much more fragile than with git ...
the nice thing of git is that it does not really harm if your branch diverges from upstream, as its merge facilities are much more sophisticated than everything that subversion has to offer ...
I've heard that a couple of times now. My personal experience with git however is quite the contrary. I regularly mess up my local repository.
well, you need to have an understanding of what you are doing ... but at least all you changes are stored inside the local repository. if you mess with your local changes, you can revert to them, if you mess with a working copy, you mess with files that you cannot got back to ...
frankly, since i moved my code into boost's svn, i find it way more fragile to maintain than before
Yes, I used to advertise git too. I had the exact same experience. That goes away. After a while, you won't look back to git, and enjoy "svn up" and "svn commit" and
svn was a relief for me after using cvs ... i've had the same feeling when switching to git a few years later. using svn again (for boost), it really feels like a bad joke ...
start to question the usefulness of decentralized version control systems for a centralized boost altogether.
just because git is decentralized does not mean you cannot use it for centralized projects. tim
Tim Blechmann wrote:
just because git is decentralized does not mean you cannot use it for centralized projects.
+1 Considering the challenge of changing the test infrastructure from the current parallel "trunk" and "release" repositories to anything else, it could make sense to move to git, but stays with the current organization and "discourage" public branches (or more precisely "pull requests" from public branches). Regards, Thomas
On 02/09/2012 12:08 PM, Tim Blechmann wrote:
with svn your working copy gets easily out of sync ...
svn update is all you need. ever. There are conflicts from time to time. Man up resolve them.
true, but in my experience resolving conflicts with svn is much more fragile than with git ...
the nice thing of git is that it does not really harm if your branch diverges from upstream, as its merge facilities are much more sophisticated than everything that subversion has to offer ...
I've heard that a couple of times now. My personal experience with git however is quite the contrary. I regularly mess up my local repository.
well, you need to have an understanding of what you are doing ... but at least all you changes are stored inside the local repository. if you mess with your local changes, you can revert to them, if you mess with a working copy, you mess with files that you cannot got back to ...
Right, and there are plethora of ways to mess up local copies with git. While it might be possible to get back in a sane state it is extremely hard. As said, especially with the features everything advertises here, it gets really hard to maintain something like a mainline git repository.
frankly, since i moved my code into boost's svn, i find it way more fragile to maintain than before
Yes, I used to advertise git too. I had the exact same experience. That goes away. After a while, you won't look back to git, and enjoy "svn up" and "svn commit" and
svn was a relief for me after using cvs ... i've had the same feeling when switching to git a few years later. using svn again (for boost), it really feels like a bad joke ...
start to question the usefulness of decentralized version control systems for a centralized boost altogether.
just because git is decentralized does not mean you cannot use it for centralized projects.
All I am asking is: what's the point? If it is used as we use svn currently, why change at all? So far, I've only heard stuff like: "X is extremely easy to do with git (If it is done right, and you know what you are doing, otherwise you are screwed)" (paraphrased). As others hinted, git should probably seen as a "meta VCS", the workflow matters. I definitely can see that something like the "linux model" makes sense for certain libraries. Git was obvously designed for it. However, whenever I tried to use all those nice and funky features (ease of branching and merging etc.). I always lost a lot of time because something didn't work as i expected. After asking people who are supposed to know the stuff, they told i did it wrong and i should use another tool in the git toolset. So I did that the next time and ended up right at the beginning. Sorry, I really tried to use git. It is all nice if you are alone and keep developing for yourself and push feature. However, when there are more people involved, with all kinds of branches etc. I always lost track and messed something up. Be it that something didn't compile for others cause merging failed, be it that history was completely borked or that I myself had the hassle to rewrite history so it all worked out again. In the end my experience was that all the nice features you are describing are not worth the hassle and do more harm than they are worth. Of course, chances are high that i just did it wrong. So, what i would like to see from you guys is a definite guide to use git. A description of workflow for both maintainer, contributor and whatever other role you can think of, and a playground to test that out. Otherwise, I don't see a lot of value in such discussions.
tim
Hi, On Thu, Feb 9, 2012 at 12:53, Thomas Heller <thom.heller@googlemail.com>wrote:
Sorry, I really tried to use git.
If my understanding is correct, it's the git implementation and interface that made you loose time. Did you have the opportunity to try another decentralized source control tool since? (I ask because I had problems with git that I did were made simpler with another DSVC so if it's an interface problem, it could be potentially be fixed by choosing another tool. However I'm not sure if it's the only problem you had with DSVC.) Joël Lamotte
On 02/09/2012 01:22 PM, Klaim - Joël Lamotte wrote:
Hi,
On Thu, Feb 9, 2012 at 12:53, Thomas Heller<thom.heller@googlemail.com>wrote:
Sorry, I really tried to use git.
If my understanding is correct, it's the git implementation and interface that made you loose time. Did you have the opportunity to try another decentralized source control tool since?
No, I didn't only thing i used so far is git and svn.
(I ask because I had problems with git that I did were made simpler with another DSVC so if it's an interface problem, it could be potentially be fixed by choosing another tool. However I'm not sure if it's the only problem you had with DSVC.)
Joël Lamotte
well, you need to have an understanding of what you are doing ... but at least all you changes are stored inside the local repository. if you mess with your local changes, you can revert to them, if you mess with a working copy, you mess with files that you cannot got back to ...
Right, and there are plethora of ways to mess up local copies with git. While it might be possible to get back in a sane state it is extremely hard.
what is hard about running `git reset'?
Sorry, I really tried to use git. It is all nice if you are alone and keep developing for yourself and push feature. However, when there are more people involved, with all kinds of branches etc. I always lost track and messed something up.
git has a lerning curve and if you do not know what you are doing, it can be pretty frustrating ... instead of getting your hands dirty by simply trying stuff, reading about its concepts before will save you some headache ...
So, what i would like to see from you guys is a definite guide to use git. A description of workflow for both maintainer, contributor and whatever other role you can think of, and a playground to test that out. Otherwise, I don't see a lot of value in such discussions.
much has beenwritten about possible work flows. the progit book for example has a nice section about different possible workflows [1] tim [1] http://progit.org/book/ch5-1.html
Right, and there are plethora of ways to mess up local copies with git. While it might be possible to get back in a sane state it is extremely hard. As said, especially with the features everything advertises here, it gets really hard to maintain something like a mainline git repository.
Maybe it has already been said, but I disagree. It's extremely simple to get back to a sane state if you fuck up your repo with git: simply type "git reflog", find the line showing the commit where everything was still correct, then "git reset --hard SHA1_OF_CORRECT_COMMIT" and you're back to where you started. This feature removes the pressure of _having_ to do your merges correctly the first time like you have to in svn, which makes the use of topic branches and friends a pleasure and not a pain. My 0.02$ Philippe
On 7 Mar 2012, at 16:10, Philippe Vaucher wrote:
Right, and there are plethora of ways to mess up local copies with git. While it might be possible to get back in a sane state it is extremely hard. As said, especially with the features everything advertises here, it gets really hard to maintain something like a mainline git repository.
Maybe it has already been said, but I disagree. It's extremely simple to get back to a sane state if you fuck up your repo with git: simply type "git reflog", find the line showing the commit where everything was still correct, then "git reset --hard SHA1_OF_CORRECT_COMMIT" and you're back to where you started.
This feature removes the pressure of _having_ to do your merges correctly the first time like you have to in svn, which makes the use of topic branches and friends a pleasure and not a pain.
That is an extremely useful git hint. I am sure I will not be alone in having never seen it. Even looking briefly at the documentation for reflog doesn't make it easy to understand that is what I wanted. This is the biggest problem with git -- there are a hundred tiny things to learn which are hard to find when you do not know what you are looking for. Chris
Christopher Jefferson wrote:
On 7 Mar 2012, at 16:10, Philippe Vaucher wrote: This is the biggest problem with git -- there are a hundred tiny things to learn which are hard to find when you do not know what you are looking for.
This is a big problem with all software these days - and boost is no exception. Robert Ramey
On 7 March 2012 19:21, Robert Ramey <ramey@rrsd.com> wrote:
Christopher Jefferson wrote:
On 7 Mar 2012, at 16:10, Philippe Vaucher wrote: This is the biggest problem with git -- there are a hundred tiny things to learn which are hard to find when you do not know what you are looking for.
This is a big problem with all software these days
More so with git than its rivals.
Hi, Am 7. März 2012 20:48 schrieb Daniel James <dnljms@gmail.com>:
On 7 March 2012 19:21, Robert Ramey <ramey@rrsd.com> wrote:
Christopher Jefferson wrote:
On 7 Mar 2012, at 16:10, Philippe Vaucher wrote: This is the biggest problem with git -- there are a hundred tiny things to learn which are hard to find when you do not know what you are looking for.
This is a big problem with all software these days
More so with git than its rivals.
One key advantage of git over its rivals is that as soon as you get to pick git as a version system you lose all the git trolls that tell you to use git. No other version system can do that! But seriously I prefer a magic tool box that does everything you think of and as soon as you find something uncomfortable a short time looking around yields a comfortable solution. regards, Andreas
Andreas Pokorny wrote:
Am 7. März 2012 20:48 schrieb Daniel James <dnljms@gmail.com>:
On 7 March 2012 19:21, Robert Ramey <ramey@rrsd.com> wrote:
Christopher Jefferson wrote:
On 7 Mar 2012, at 16:10, Philippe Vaucher wrote: This is the biggest problem with git -- there are a hundred tiny things to learn which are hard to find when you do not know what you are looking for.
This is a big problem with all software these days
More so with git than its rivals.
One key advantage of git over its rivals is that as soon as you get to pick git as a version system you lose all the git trolls that tell you to use git. No other version system can do that!
+1
But seriously I prefer a magic tool box that does everything you think of and as soon as you find something uncomfortable a short time looking around yields a comfortable solution.
This, and in all honesty I think git is at least as well-documented as svn. Both have an online help function; in addition, git has extensive and well-organised man pages. With a good tutorial like Git Immersion [1] you should be good to go in little time, all the other tiny things will come with time. If the thought of not knowing all those tiny things in advance worries you, reading Pro Git [2] should be somewhat reassuring. The git-flow tool that is available to automate the gitflow branching model is less well documented, but fortunately there's not much to know about it. There is a more-or-less complete overview of the subcommands and their options on their GitHub wiki [3], as well as a very terse online help command. -Julian 1. http://gitimmersion.com/ 2. http://progit.org/ 3. https://github.com/nvie/gitflow/wiki/Command-Line-Arguments
I haven't seen anyone mention what seems to me the most obvious advantage of "git" for a large collaborative project like Boost: It is easy for a small group to clone their own version of the repository somewhere accessible (like git-hub of course, but any participant's development machine with the right Internet connection will do, as would something like dropbox) and work there until they are ready to expose it to wider audience. Similarly, a project started in git -- not necessarily with Boost in mind -- can potentially be merged into the central repository, after being edited to meet standards, complete with its history (less likely, of course, without some attention paid to Boost). This would be particularly useful when work is "speculative". People who want to work on something that they think might not work out are not discouraged from proper source control or integration because of fear of embarrasment because it turns out to have been a "stupid idea" -- either regarding the whole attempt, or regarding some dead end investigations. My experience is that innovation is frequently helped by some privacy in the early stages. Topher
I haven't seen anyone mention what seems to me the most obvious advantage of "git" for a large collaborative project like Boost:
For me the most obvious advantage compared to the current solution is that it's decentralized. This implies much more than what people unexperienced with a decentralized revision system imagine... it allows people to contribute much more easily and also it allows booster to review modifications more easily too imho. Philippe
I've had only a limited opportunity to use git, but I recognized something about it pretty quickly -- something which I think is behind some of these strong differences of opinion. Basically, git takes a different approach to project source control -- it "thinks" about the problems in a different way. To use it properly you have to think the same way. When you do, the way to respond to a particular situation is obvious precisely when it seems arbitrary to someone thinking in more traditional terms, and documentation is clear and helpful where those who haven't learned the new perspective find it confusing and worthless. What makes the problem worse is that the same word may actually mean something subtlty but importantly different, while a new term is frequently interpretted mistakenly as being "pretty much the same thing" as a more familiar term. Whether this new way of thinking is better, as the supporters claim, is yet to be seen. Clearly, it is a barrier that needs to be taken into account in any changeover, but which, I think, can be lowered with a set of well documented procedures and scripts designed for the particular repository design. For those who want to pass through this kind of looking glass, the most important advise is 1) Don't assume that you know what is meant by any term or phrase, whether familiar or not; 2) Concentrate on descriptions of "here's a problem and here is how you-could/we-did solve it"; 3) Resist the temptation to assume people are doing something stupid -- always presume that people actually know what they are talking about (sometimes they don't, of course, but even then they are likely to be less stupid than it might appear and you'll usually learn more making the same mistake than trying to avoid it). I haven't, by the way, been able to cross the conceptual threshold for git myself yet, but this is a pattern I have seen frequently in software innovations -- the "symptoms" are clear, and -- worthwhile or not -- I'm confident that there really is stuff going on that I don't understand yet. The most relevant example to this group was back when almost nobody had heard of object oriented programming and some of us tried to promote it and explain just what it was ("Your making a big deal out of nothing -- 'objects' are just data structures, 'methods', are just fancy procedures, and its much more flexible to use cut-and-paste than 'inheritance'."). And, of course, this is pretty much THE forum for "generic programming". Some other major examples I've seen is the "Real Programmers use assembly language" battle and the "structured control flow" battle (a battle so thoroughly won, that most younger prgrammers don't seem to know the term or, even when they do, have any real concept of the alternative; yes, I remember when "if ... then ... otherwise..." -- not yet "else" -- was unusual in programming languages, or was considered just some occasionally handy "syntactic sugar"). Topher
On 7 March 2012 19:21, Robert Ramey<ramey@rrsd.com> wrote:
Christopher Jefferson wrote:
On 7 Mar 2012, at 16:10, Philippe Vaucher wrote: This is the biggest problem with git -- there are a hundred tiny things to learn which are hard to find when you do not know what you are looking for.
This is a big problem with all software these days
More so with git than its rivals.
_______________________________________________ Unsubscribe& other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
on Wed Mar 07 2012, Daniel James <dnljms-AT-gmail.com> wrote:
On 7 March 2012 19:21, Robert Ramey <ramey@rrsd.com> wrote:
Christopher Jefferson wrote:
On 7 Mar 2012, at 16:10, Philippe Vaucher wrote: This is the biggest problem with git -- there are a hundred tiny things to learn which are hard to find when you do not know what you are looking for.
This is a big problem with all software these days
More so with git than its rivals.
That is true. And some of Git's commands are simply arranged asymmetrically such that they're hard to remember. For example, I find the lack of consistency between the options to "git branch" and "git tag" to be maddening. All that said, I still think, on balance, that Git is a winning technology and the right choice for our future. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
on Wed Mar 14 2012, Dave Abrahams <dave-AT-boostpro.com> wrote:
on Wed Mar 07 2012, Daniel James <dnljms-AT-gmail.com> wrote:
On 7 March 2012 19:21, Robert Ramey <ramey@rrsd.com> wrote:
Christopher Jefferson wrote:
On 7 Mar 2012, at 16:10, Philippe Vaucher wrote: This is the biggest problem with git -- there are a hundred tiny things to learn which are hard to find when you do not know what you are looking for.
This is a big problem with all software these days
More so with git than its rivals.
That is true. And some of Git's commands are simply arranged asymmetrically such that they're hard to remember. For example, I find the lack of consistency between the options to "git branch" and "git tag" to be maddening.
Which inconsistency I now can't find. :-P -- Dave Abrahams BoostPro Computing http://www.boostpro.com
Thomas Heller wrote:
svn update is all you need. ever. ... you won't look back to git, and enjoy "svn up" and "svn commit"
these statements give me the impression you have actually not yet walked in the shoes of a branch maintainer / integrator; i don't see how you would do that without at least 'svn merge'. as such, i'm not sure if you have experienced the problems of merging / integrating a development line first hand and can appreciate the conceptual advantages git has to offer in that area. Thomas Heller wrote:
There are conflicts from time to time.
are you talking about conflicts you get after an svn up ?? those are certainly not the reason why DCVSs exist - but git helps with that, too ;-) Thomas Heller wrote:
Man up resolve them.
i don't know the details of how things are run in boost, but in my experience, the job of an integrator is the one with the worst responsibility to appreciation ratio in a software project: if they screw up, everyone will notice due to the bad release; if they deliver 100%, they rarely ever get the praise they deserve. i've seen companies where developers who have fallen from favor with management were forced onto the integration chair until they either resign or go mad. telling them to man up without having been down that road in a high profile commercial or large, popular open source project is... not something they will like to hear. of course you can not know this, but the mere memory of this situation still makes me twitch. i do hope that doesn't show in my tone, my apologies if it does. Thomas Heller wrote:
The git userinterface is just a pain, i regularly get confused and have no idea what to do. ... I regularly mess up my local repository.
I find myself doing a lot of 'git status' in a version-controlled folder which gives me info like host:folder user$ git status # On branch some_feature_branch # Changes not staged for commit: # (use "git add <file>..." to update what will be committed) # (use "git checkout -- <file>..." to discard changes in working directory) # # modified: main.cpp # or host:folder user$ git status # On branch some_feature_branch # Changes to be committed: # (use "git reset HEAD <file>..." to unstage) # # modified: main.cpp # one single command and the tool tells you what you could do next. how much simpler and more user-friendly could it possibly be ? Thomas Heller wrote:
Right, and there are plethora of ways to mess up local copies with git. While it might be possible to get back in a sane state it is extremely hard.
what it 'extremely hard' about 'git reset --hard' ? (no pun intended). host:folder user$ git status # On branch some_feature_branch nothing to commit (working directory clean) btw: i've come across http://githowto.com today, maybe that helps to get past the initial learning curve. # ssc
On 02/09/2012 03:16 PM, Steven Samuel Cole wrote:
Thomas Heller wrote:
svn update is all you need. ever. ... you won't look back to git, and enjoy "svn up" and "svn commit"
these statements give me the impression you have actually not yet walked in the shoes of a branch maintainer / integrator; i don't see how you would do that without at least 'svn merge'. as such, i'm not sure if you have experienced the problems of merging / integrating a development line first hand and can appreciate the conceptual advantages git has to offer in that area.
Thomas Heller wrote:
There are conflicts from time to time.
are you talking about conflicts you get after an svn up ?? those are certainly not the reason why DCVSs exist - but git helps with that, too ;-)
Thomas Heller wrote:
Man up resolve them.
i don't know the details of how things are run in boost, but in my experience, the job of an integrator is the one with the worst responsibility to appreciation ratio in a software project: if they screw up, everyone will notice due to the bad release; if they deliver 100%, they rarely ever get the praise they deserve.
i've seen companies where developers who have fallen from favor with management were forced onto the integration chair until they either resign or go mad. telling them to man up without having been down that road in a high profile commercial or large, popular open source project is... not something they will like to hear. of course you can not know this, but the mere memory of this situation still makes me twitch. i do hope that doesn't show in my tone, my apologies if it does.
In boost, library maintainers merge into the release branch before every release. I did that for the last several boost releases for Boost.Phoenix. I did not have any problems.
Thomas Heller wrote:
The git userinterface is just a pain, i regularly get confused and have no idea what to do. ... I regularly mess up my local repository.
I find myself doing a lot of 'git status' in a version-controlled folder which gives me info like
host:folder user$ git status # On branch some_feature_branch # Changes not staged for commit: # (use "git add <file>..." to update what will be committed) # (use "git checkout -- <file>..." to discard changes in working directory) # # modified: main.cpp #
or
host:folder user$ git status # On branch some_feature_branch # Changes to be committed: # (use "git reset HEAD <file>..." to unstage) # # modified: main.cpp #
one single command and the tool tells you what you could do next. how much simpler and more user-friendly could it possibly be ?
Thomas Heller wrote:
Right, and there are plethora of ways to mess up local copies with git. While it might be possible to get back in a sane state it is extremely hard.
what it 'extremely hard' about 'git reset --hard' ? (no pun intended).
And lose all the work? I am not sure what exactly you were trying to tell me.
host:folder user$ git status # On branch some_feature_branch nothing to commit (working directory clean)
btw: i've come across http://githowto.com today, maybe that helps to get past the initial learning curve.
All I was trying to say is: why should i bother learning a new tool, when it doesn't solve anything. All I was saying is that you described the exact same process in your initial mail that is possible with SVN. It is not a tool problem. It might be true that git or any other DCVS enable different workflows. But as you described it, it is exactly the same. The problems of bugs not being solved, or patches not being applied does not magically go away. The fact the regression test setup is suboptimal does not suddenly go up in smoke. My personal feeling is that this will even be worse with git: Which repository do i choose to fork from? Mainline? The one of the maintainer? A special branch with some new features? What if it is not really up to date (Yes, last time i checked you still have to manually update your source)? How will new features be tested? These are all things that noone considered. All you hear is that git is so great, and you can branch and merge back however you want. I did not have that experience. I experienced serious trouble with git. So I am asking you again: What exactly do you propose?
All I was trying to say is: why should i bother learning a new tool, when it doesn't solve anything.
The popularity of git by itself suggests to me that it's worth paying attention to. I've seen many projects switch from svn to git. And none switching from git to svn.
All I was saying is that you described the exact same process in your initial mail that is possible with SVN. It is not a tool problem. It might be true that git or any other DCVS enable different workflows.
The key difference with git is that there is less of a client/server distinction. Every client (where development happens) is the same as a server, in terms of having all the revision history. Changes flow from one repository to another, one branch in one place, to some other branch somewhere else, generally without manual merging, diff or patch steps. Occasionally there are conflicts that need manual resolution. But generally, rebasing is painless.
Which repository do i choose to fork from? Mainline? The one of the maintainer? A special branch with some new features?
Precisely the one that is most relevant. In a nutshell - the one you're most likely to want to upstream your changes to. If the maintainer of boost.foo wants to accept patches into a repository entirely under their control - then periodically upstream those to an official boost branch - so be it.
What if it is not really up to date (Yes, last time i checked you still have to manually update your source)?
Then rebase against something else, the common history will be well utilised in the process.
All you hear is that git is so great, and you can branch and merge back however you want.
The practical problem I face is maintaining a patch set for an old version of boost. Without the "local full revision history" I need to do a lot of manual rebasing to bring that patch set forward to a modern version of boost. Indeed, without doing this work there isn't much prospect of getting those finally accepted back into the mainline boost. I would fully expect this to be far less of an issue using git, if only because I've had good experience managing a few divergent branches of codebases with git, versus what seems extremely limited features of svn. One of the surprising aspects of git is the "malleability of change history" quite unlike cvs, svn or perforce. The timestamps need not necessarily be monotonically increasing from one revision to the next. I guess in principle I could refactor a change in order to merge some subset of it to some other branch/repo. Try doing that with svn. I mean really... - Nigel
on Fri Feb 10 2012, Nigel Stewart <nigels.com-AT-gmail.com> wrote:
All I was trying to say is: why should i bother learning a new tool, when it doesn't solve anything.
The popularity of git by itself suggests to me that it's worth paying attention to. I've seen many projects switch from svn to git. And none switching from git to svn.
+1 This may seem like it's just a popularity contest without technical meaning, but it is actually important. It means more people will know how to use Git effectively, more tools will be developed around Git, more improvements will go into Git, etc., etc. Git is a winning technology. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On 2/8/2012 9:29 AM, Tim Blechmann wrote:
Also, I notice a few other things that in my opinion could be done better to facilitate adoption of the iostreams library. I would be willing to do the work and I would in the perfect position as I am walking in the shoes of an adopter right now - but these are separate from the typo fixes and they are larger issues; considering I can't check my changes in, would I really want to have them sit on my local harddrive, waiting possibly for months for someone upstream to review and hopefully merge them ?
So with git you pushed them to your personal fork. Over time, your fork and the upstream version eventually diverge. You have to maintain your changes. Same thing when the changes lurk on your local svn working.
well, your svn working copy is just a bunch of files, while git's personal branches are part of the repository ... you can merge, rebase, stash, etc ... with svn your working copy gets easily out of sync ... the nice thing of git is that it does not really harm if your branch diverges from upstream, as its merge facilities are much more sophisticated than everything that subversion has to offer ...
frankly, since i moved my code into boost's svn, i find it way more fragile to maintain than before
tim
Hi Tim - I am interested in your views here. I have had good success with git and monolithic projects. How would you go about working on your library and then pushing to the central repo. Would you envision that the central repo is nothing more than a shell pointing to submodules that are the "real" library repos? Or would you just rebase all of boost before committing back modifications to your library? Or some other process completely? I'm sure people have thought about this problem and the solutions. You might be one of them. Unfortunately I mostly hear how great it would be to move to git but few have provided the usage for a project like boost : a loosely coupled and independently developed collection of libraries. Thanks! michael -- ---------------------------------- Michael Caisse Object Modeling Designs www.objectmodelingdesigns.com
On Wed, Feb 8, 2012 at 7:35 PM, Michael Caisse <boost@objectmodelingdesigns.com> wrote:
I'm sure people have thought about this problem and the solutions. You might be one of them. Unfortunately I mostly hear how great it would be to move to git but few have provided the usage for a project like boost : a loosely coupled and independently developed collection of libraries.
Maybe that's because most people see value in git that's unrelated to (supporting) decoupling. -- Olaf
On 2/8/2012 11:40 AM, Olaf van der Spek wrote:
On Wed, Feb 8, 2012 at 7:35 PM, Michael Caisse <boost@objectmodelingdesigns.com> wrote:
I'm sure people have thought about this problem and the solutions. You might be one of them. Unfortunately I mostly hear how great it would be to move to git but few have provided the usage for a project like boost : a loosely coupled and independently developed collection of libraries.
Maybe that's because most people see value in git that's unrelated to (supporting) decoupling.
I can appreciate that. I find lots of value in git for many of my projects; however, that isn't the subject at hand. I'm truly interested in how to structure git (work flow, repositories, what-not) to work for a project like Boost. Maybe there is a link to this already. michael -- ---------------------------------- Michael Caisse Object Modeling Designs www.objectmodelingdesigns.com
I am interested in your views here. I have had good success with git and monolithic projects. How would you go about working on your library and then pushing to the central repo. Would you envision that the central repo is nothing more than a shell pointing to submodules that are the "real" library repos?
the semantics of git submodules is a bit weird: if you check out a new revision of the top-level repository, the submodules won't be updated automatically, but you need to run `git submodules update' manually ... i am using submodules for tracking some external dependencies and in my experience 70% of the compile failures of my users are related to submodules getting out of sync. for a project like boost, i'd therefore suggest to avoid submodules. the advantage however is that the full git repository of boost is few hundred mb ...
Or would you just rebase all of boost before committing back modifications to your library? Or some other process completely?
if changes are trivial, i usually rebase to the current master before pushing new changes to the public repository. the reason is mainly to avoid unnecessary merge commits in the history. if changes get too complex, merging is usually better ... also more friendly for bisecting bugs.
I'm sure people have thought about this problem and the solutions. You might be one of them. Unfortunately I mostly hear how great it would be to move to git but few have provided the usage for a project like boost
: a loosely coupled and independently developed collection of libraries.
well, just because git provides some features, we do not necessarily have to use them ... it can be used in a similar manner to the current svn repository, only managing two branches cheers, tim
On Thu, Feb 9, 2012 at 12:07 PM, Tim Blechmann <tim@klingt.org> wrote:
the semantics of git submodules is a bit weird: if you check out a new revision of the top-level repository, the submodules won't be updated automatically, but you need to run `git submodules update' manually ... i am using submodules for tracking some external dependencies and in my experience 70% of the compile failures of my users are related to submodules getting out of sync.
for a project like boost, i'd therefore suggest to avoid submodules. the advantage however is that the full git repository of boost is few hundred mb ...
Submodules are probably misunderstood and misused most of the time. I think it was not meant to be used to have separate repositories for sub-part of the projects and a everything evolving together in sync. I see the use of submodules more like integrators would do, retrieving specific revisions of various sub projects, and testing these versions alltogether. Whenever parts integrates well, the submodule versions that work fine together are frozen in a commit. This is not very usable if you try to develop directly in the "main" repository that has the submodules. For Boost, I would say it could be used and even a powerful tool for integrating libraries together, testing them and managing release lifecycle, but could hardly be used for day to day library development. -- Beren Minor
Beren Minor <beren.minor+boost@gmail.com> writes:
On Thu, Feb 9, 2012 at 12:07 PM, Tim Blechmann <tim@klingt.org> wrote:
for a project like boost, i'd therefore suggest to avoid submodules. the advantage however is that the full git repository of boost is few hundred mb ...
Submodules are probably misunderstood and misused most of the time. I think it was not meant to be used to have separate repositories for sub-part of the projects and a everything evolving together in sync. I see the use of submodules more like integrators would do, retrieving specific revisions of various sub projects, and testing these versions alltogether. Whenever parts integrates well, the submodule versions that work fine together are frozen in a commit. This is not very usable if you try to develop directly in the "main" repository that has the submodules.
I think that's right.
For Boost, I would say it could be used and even a powerful tool for integrating libraries together, testing them and managing release lifecycle, but could hardly be used for day to day library development.
I would recommend using git-subtree. It's the submodule-like tool more appropriate for what boost is: a collection of project units that can be built and tested independently but also make sense as a cohesive whole. git-subtree is getting integrated into the official git repository now and should be available in the next month or two. Of you you can always get it here: https://github.com/apenwarr/git-subtree -Dave
On 2/9/2012 2:24 PM, David A. Greene wrote:
I would recommend using git-subtree. It's the submodule-like tool more appropriate for what boost is: a collection of project units that can be built and tested independently but also make sense as a cohesive whole.
git-subtree is getting integrated into the official git repository now and should be available in the next month or two. Of you you can always get it here:
https://github.com/apenwarr/git-subtree
-Dave
_______________________________________________ Unsubscribe& other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Thank you Dave - git-subtree looks like something I've been missing in git. I'll take a closer look. Thanks for the link. michael -- ---------------------------------- Michael Caisse Object Modeling Designs www.objectmodelingdesigns.com
Michael Caisse wrote:
On 2/9/2012 2:24 PM, David A. Greene wrote:
I would recommend using git-subtree. [...]
Thank you Dave -
git-subtree looks like something I've been missing in git. I'll take a closer look. Thanks for the link.
I agree that this seems like a very interesting approach to modularity. -Julian
on Thu Feb 09 2012, greened-AT-obbligato.org wrote:
Beren Minor <beren.minor+boost@gmail.com> writes:
On Thu, Feb 9, 2012 at 12:07 PM, Tim Blechmann <tim@klingt.org> wrote:
for a project like boost, i'd therefore suggest to avoid submodules. the advantage however is that the full git repository of boost is few hundred mb ...
Submodules are probably misunderstood and misused most of the time. I think it was not meant to be used to have separate repositories for sub-part of the projects and a everything evolving together in sync. I see the use of submodules more like integrators would do, retrieving specific revisions of various sub projects, and testing these versions alltogether. Whenever parts integrates well, the submodule versions that work fine together are frozen in a commit. This is not very usable if you try to develop directly in the "main" repository that has the submodules.
I think that's right.
For Boost, I would say it could be used and even a powerful tool for integrating libraries together, testing them and managing release lifecycle, but could hardly be used for day to day library development.
I would recommend using git-subtree. It's the submodule-like tool more appropriate for what boost is: a collection of project units that can be built and tested independently but also make sense as a cohesive whole.
git-subtree is getting integrated into the official git repository now and should be available in the next month or two. Of you you can always get it here:
I don't think git-subtree is the right tool for us. Suppose libraries A and B both depend on libraries C and D. It would IMO be a disaster if the repositories for A and B each included the source to C and D. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
Dave Abrahams wrote:
on Thu Feb 09 2012, greened-AT-obbligato.org wrote:
[...]
I don't think git-subtree is the right tool for us. Suppose libraries A and B both depend on libraries C and D. It would IMO be a disaster if the repositories for A and B each included the source to C and D.
I might have misunderstood, but my impression was that git-subtree was not intended to be used like that in the first place. If I understood correctly, A, B, C and D would all be subtrees of the Boost supertree, and developers would be able to tranparently treat any subtree either as a separate git repository or as an ordinary directory within the Boost repository at any given time. So both A and B would only include itself, and C and D wouldn't be included by anything other than the Boost supertree. For as far as I can see, it's really a more transparent and more flexible alternative to submodules. -Julian
on Tue Feb 28 2012, Julian Gonggrijp <j.gonggrijp-AT-gmail.com> wrote:
Dave Abrahams wrote:
on Thu Feb 09 2012, greened-AT-obbligato.org wrote:
[...]
I don't think git-subtree is the right tool for us. Suppose libraries A and B both depend on libraries C and D. It would IMO be a disaster if the repositories for A and B each included the source to C and D.
I might have misunderstood, but my impression was that git-subtree was not intended to be used like that in the first place. If I understood correctly, A, B, C and D would all be subtrees of the Boost supertree, and developers would be able to tranparently treat any subtree either as a separate git repository or as an ordinary directory within the Boost repository at any given time. So both A and B would only include itself, and C and D wouldn't be included by anything other than the Boost supertree.
I don't see any advantage at all in having a "Boost supertree." If I need a cluster of five libraries to get a job done, I don't see a reason to mess around with anything more than those five. And with the dependencies properly encoded, I won't even have to ask for all of those explicitly. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
on Thu Feb 09 2012, Beren Minor <beren.minor+boost-AT-gmail.com> wrote:
On Thu, Feb 9, 2012 at 12:07 PM, Tim Blechmann <tim@klingt.org> wrote:
the semantics of git submodules is a bit weird: if you check out a new revision of the top-level repository, the submodules won't be updated automatically, but you need to run `git submodules update' manually ... i am using submodules for tracking some external dependencies and in my experience 70% of the compile failures of my users are related to submodules getting out of sync.
for a project like boost, i'd therefore suggest to avoid submodules. the advantage however is that the full git repository of boost is few hundred mb ...
Submodules are probably misunderstood and misused most of the time. I think it was not meant to be used to have separate repositories for sub-part of the projects and a everything evolving together in sync. I see the use of submodules more like integrators would do, retrieving specific revisions of various sub projects, and testing these versions alltogether. Whenever parts integrates well, the submodule versions that work fine together are frozen in a commit. This is not very usable if you try to develop directly in the "main" repository that has the submodules.
For Boost, I would say it could be used and even a powerful tool for integrating libraries together, testing them and managing release lifecycle, but could hardly be used for day to day library development.
+1 -- Dave Abrahams BoostPro Computing http://www.boostpro.com
on Thu Feb 09 2012, Tim Blechmann <tim-AT-klingt.org> wrote:
I am interested in your views here. I have had good success with git and monolithic projects. How would you go about working on your library and then pushing to the central repo. Would you envision that the central repo is nothing more than a shell pointing to submodules that are the "real" library repos?
the semantics of git submodules is a bit weird: if you check out a new revision of the top-level repository, the submodules won't be updated automatically, but you need to run `git submodules update' manually ... i am using submodules for tracking some external dependencies and in my experience 70% of the compile failures of my users are related to submodules getting out of sync.
for a project like boost, i'd therefore suggest to avoid submodules. the advantage however is that the full git repository of boost is few hundred mb ...
We don't need to use submodules. /Maybe/ the people who are packaging integrated boost releases (i.e. the release team) might want to use them as a way of recording an integration state, but most Boost developers and users should never need to touch them. Instead, we want to use an installation system like 0install to gather a library's dependencies and make them available to the build system. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
Hi all, I share the views of Steven. I have worked with svn for a couple of years, but for the last 4 years I have been using git every day at work. One of GITs strengths is how easy it is to keep branches in sync. Collaboration is in my opinion a lot easier compared with SVN. Being able to create commits in a local repository, then push these commits upstream rather than pushing a patch makes your local GIT checkout able to know where it is when these fixes have been included upstream and you want to pull the changes. Generally, very little manual merging necessary. As Steven also said, the upstream maintainer can cherry-pick just the changes he wants to include. As long as a change in the local git repo is contained in one or more commits, the upstream maintainer can cherry-pick this/these commits into the upstream repo. Even after this, syncing back to the local repo is no problem. The only downside (I can see) with GIT is that it has a bit of a learning curve. -Øivind
participants (24)
-
Andreas Pokorny -
Andrey Semashev -
Beman Dawes -
Beren Minor -
Brian Schrom -
Christopher Jefferson -
Daniel James -
Dave Abrahams -
greened＠obbligato.org -
John Wiegley -
Julian Gonggrijp -
Klaim - Joël Lamotte -
Michael Caisse -
Nigel Stewart -
Olaf van der Spek -
paul Fultz -
Philippe Vaucher -
Robert Ramey -
Steven Samuel Cole -
Thomas Heller -
Thomas Klimpel -
Tim Blechmann -
Topher Cooper -
Øivind Loe