[Git] Documentation update
The "Git and Modular Boost" documentation has been updated: * The GitHub mothership with be github.com/boostorg * Full history for the current trunk and branches/release will be available. * There are now active milestone and issue trackers for the conversion. See https://svn.boost.org/trac/boost/wiki/WikiStart#GitandModularBoost --Beman
On Sun, Feb 10, 2013 at 11:08 AM, Beman Dawes <bdawes@acm.org> wrote:
The "Git and Modular Boost" documentation has been updated:
* The GitHub mothership with be github.com/boostorg
Is that currently being updated like the bitbucket test repos? I ask because the testing will rely on dulwich git which does not talk at all with bitbucket (it seems the bitbucket git server works differently.. not that I blame it given the disfunction that is the git server protocol). Hence I need something useable for development of the testing scripts.
* There are now active milestone and issue trackers for the conversion.
Will the people implementing those milestones have access to update/change them? And, shouldn't they be in the boostorg project instead of ryppl? After all, the testing milestone I'm working on has nothing to do with ryppl. Sorry if I'm horribly confused about where things have been and where they are now. -- -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo
on Sun Feb 10 2013, Rene Rivera <grafikrobot-AT-gmail.com> wrote:
On Sun, Feb 10, 2013 at 11:08 AM, Beman Dawes <bdawes@acm.org> wrote:
The "Git and Modular Boost" documentation has been updated:
* The GitHub mothership with be github.com/boostorg
Is that currently being updated like the bitbucket test repos?
Exactly the same way.
I ask because the testing will rely on dulwich git which does not talk at all with bitbucket (it seems the bitbucket git server works differently.. not that I blame it given the disfunction that is the git server protocol).
Oh, that's rather lame. Is there a reason you're not going to use libgit2 instead of dulwich?
Hence I need something useable for development of the testing scripts.
* There are now active milestone and issue trackers for the conversion.
Will the people implementing those milestones have access to update/change them? And, shouldn't they be in the boostorg project instead of ryppl?
Well, they'd need a repository if we're going to use a GH tracker. What repo would you use?
After all, the testing milestone I'm working on has nothing to do with ryppl.
Ryppl is where we're hosting all tools associated with the transition, so we've got a tracker there for managing the work (https://github.com/ryppl/boost-svn2git/issues?state=open). Beman has some creative ideas about how that tracker should be used that I'll let him explain himself, since I don't really understand them. -- Dave Abrahams
On Mon, Feb 11, 2013 at 11:51 AM, Dave Abrahams <dave@boostpro.com> wrote:
on Sun Feb 10 2013, Rene Rivera <grafikrobot-AT-gmail.com> wrote:
On Sun, Feb 10, 2013 at 11:08 AM, Beman Dawes <bdawes@acm.org> wrote:
I ask because the testing will rely on dulwich git which does not talk at all with bitbucket (it seems the bitbucket git server works differently.. not that I blame it given the disfunction that is the git server protocol).
Oh, that's rather lame. Is there a reason you're not going to use libgit2 instead of dulwich?
Yes.. The python bindings for libgit2 are terribly/unusably incomplete. -- -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo
on Mon Feb 11 2013, Rene Rivera <grafikrobot-AT-gmail.com> wrote:
On Mon, Feb 11, 2013 at 11:51 AM, Dave Abrahams <dave@boostpro.com> wrote:
on Sun Feb 10 2013, Rene Rivera <grafikrobot-AT-gmail.com> wrote:
On Sun, Feb 10, 2013 at 11:08 AM, Beman Dawes <bdawes@acm.org> wrote:
I ask because the testing will rely on dulwich git which does not talk at all with bitbucket (it seems the bitbucket git server works differently.. not that I blame it given the disfunction that is the git server protocol).
Oh, that's rather lame. Is there a reason you're not going to use libgit2 instead of dulwich?
Yes.. The python bindings for libgit2 are terribly/unusably incomplete.
Surely they're complete enough to allow you to clone and pull, no? What more do you need? -- Dave Abrahams
NOTE: Part of this discussion was within the release team. But it was suggested that it should also be generally public. Hence I'm raising some of the same issues from the private discussion. On Mon, Feb 11, 2013 at 12:58 PM, Dave Abrahams <dave@boostpro.com> wrote:
on Mon Feb 11 2013, Rene Rivera <grafikrobot-AT-gmail.com> wrote:
On Mon, Feb 11, 2013 at 11:51 AM, Dave Abrahams <dave@boostpro.com> wrote:
on Sun Feb 10 2013, Rene Rivera <grafikrobot-AT-gmail.com> wrote:
On Sun, Feb 10, 2013 at 11:08 AM, Beman Dawes <bdawes@acm.org> wrote:
I ask because the testing will rely on dulwich git which does not talk at all with bitbucket (it seems the bitbucket git server works differently.. not that I blame it given the disfunction that is the git server protocol).
Oh, that's rather lame. Is there a reason you're not going to use libgit2 instead of dulwich?
Yes.. The python bindings for libgit2 are terribly/unusably incomplete.
Surely they're complete enough to allow you to clone and pull, no? What more do you need?
Why would one assume that? First there is no binding to "git_clone" < https://github.com/search?q=git_clone+repo%3Alibgit2%2Fpygit2&type=Repositories&ref=advsearch&l=>. So it's a nonstarter at step zero. Second there are no bindings for submodules/subrepos. Which might be an additional nonstarter if the git_clone didn't recurse clone/pull subrepos. I was faced with either implementing some of the additional clone/pull/checkout/update code with dulwich or likely implement even more for libgit2. Hence I'm sticking with dulwich. Since at least that gives the advantage of being pure Python. My initial problems where mostly because I was trying to do things that where not actually supported by the git client/server design.. Of course I didn't know that because the git design is essentially non-existent and anything that exists is ad-hoc and minimal.. AFAICT. My latest problems don't have much to do with dulwich at this point.. More of not being able to do anything interesting (i.e. efficient) because of the limitation of the git server and system. Yes, sad to say I've had to learn way more than I really wanted about git. And it has only confirmed my educated suspicions about it. But I'll get something working eventually. Just will not be soon. Sadly-and-somewhat-frustratedly.. Rene. -- -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo
On Mon, Feb 11, 2013 at 11:18 PM, Rene Rivera <grafikrobot@gmail.com> wrote:
NOTE: Part of this discussion was within the release team. But it was suggested that it should also be generally public. Hence I'm raising some of the same issues from the private discussion.
On Mon, Feb 11, 2013 at 12:58 PM, Dave Abrahams <dave@boostpro.com> wrote:
on Mon Feb 11 2013, Rene Rivera <grafikrobot-AT-gmail.com> wrote:
On Mon, Feb 11, 2013 at 11:51 AM, Dave Abrahams <dave@boostpro.com> wrote:
on Sun Feb 10 2013, Rene Rivera <grafikrobot-AT-gmail.com> wrote:
On Sun, Feb 10, 2013 at 11:08 AM, Beman Dawes <bdawes@acm.org> wrote:
I ask because the testing will rely on dulwich git which does not talk at all with bitbucket (it seems the bitbucket git server works differently.. not that I blame it given the disfunction that is the git server protocol).
Oh, that's rather lame. Is there a reason you're not going to use libgit2 instead of dulwich?
Yes.. The python bindings for libgit2 are terribly/unusably incomplete.
Surely they're complete enough to allow you to clone and pull, no? What more do you need?
Why would one assume that? First there is no binding to "git_clone" < https://github.com/search?q=git_clone+repo%3Alibgit2%2Fpygit2&type=Repositories&ref=advsearch&l=>. So it's a nonstarter at step zero. Second there are no bindings for submodules/subrepos. Which might be an additional nonstarter if the git_clone didn't recurse clone/pull subrepos.
I was faced with either implementing some of the additional clone/pull/checkout/update code with dulwich or likely implement even more for libgit2. Hence I'm sticking with dulwich. Since at least that gives the advantage of being pure Python.
I must be missing something. Why not just work through the python subprocess interface? For example, here is how Marshall issues an "svn export" in the snapshot.py script: import subprocess ... def svnExport(url, eol, revisionStr, dest): command_arr = [ "svn", "export", "--non-interactive", "--native-eol", eol, "-r", revisionStr, url, dest ] subprocess.check_output ( command_arr ) ... IIRC, he does it this way because of an svn problem that was somewhat similar to what you are facing with git - it was hard to find a python svn binding that could be relied on in a script that had to run on both Windows and the Mac. As Dave points out, the git interaction is really simple, so isn't the subprocess approach sufficient? --Beman
On 12 February 2013 14:25, Beman Dawes <bdawes@acm.org> wrote:
On Mon, Feb 11, 2013 at 11:18 PM, Rene Rivera <grafikrobot@gmail.com> wrote:
NOTE: Part of this discussion was within the release team. But it was suggested that it should also be generally public. Hence I'm raising some of the same issues from the private discussion.
On Mon, Feb 11, 2013 at 12:58 PM, Dave Abrahams <dave@boostpro.com> wrote:
on Mon Feb 11 2013, Rene Rivera <grafikrobot-AT-gmail.com> wrote:
On Mon, Feb 11, 2013 at 11:51 AM, Dave Abrahams <dave@boostpro.com> wrote:
on Sun Feb 10 2013, Rene Rivera <grafikrobot-AT-gmail.com> wrote:
On Sun, Feb 10, 2013 at 11:08 AM, Beman Dawes <bdawes@acm.org> wrote:
I ask because the testing will rely on dulwich git which does not talk at all with bitbucket (it seems the bitbucket git server works differently.. not that I blame it given the disfunction that is the git server protocol).
Oh, that's rather lame. Is there a reason you're not going to use libgit2 instead of dulwich?
Yes.. The python bindings for libgit2 are terribly/unusably incomplete.
Surely they're complete enough to allow you to clone and pull, no? What more do you need?
Why would one assume that? First there is no binding to "git_clone" < https://github.com/search?q=git_clone+repo%3Alibgit2%2Fpygit2&type=Repositories&ref=advsearch&l=>. So it's a nonstarter at step zero. Second there are no bindings for submodules/subrepos. Which might be an additional nonstarter if the git_clone didn't recurse clone/pull subrepos.
I was faced with either implementing some of the additional clone/pull/checkout/update code with dulwich or likely implement even more for libgit2. Hence I'm sticking with dulwich. Since at least that gives the advantage of being pure Python.
I must be missing something. Why not just work through the python subprocess interface? For example, here is how Marshall issues an "svn export" in the snapshot.py script:
import subprocess ...
def svnExport(url, eol, revisionStr, dest): command_arr = [ "svn", "export", "--non-interactive", "--native-eol", eol, "-r", revisionStr, url, dest ] subprocess.check_output ( command_arr ) ...
Such approach has proved to work very well for myself too: https://github.com/mloskot/pygit-svn-mirror/blob/master/lib/gitsvnmirror.py Best regards, -- Mateusz Loskot, http://mateusz.loskot.net
On Tue, Feb 12, 2013 at 8:25 AM, Beman Dawes <bdawes@acm.org> wrote:
I was faced with either implementing some of the additional clone/pull/checkout/update code with dulwich or likely implement even more for libgit2. Hence I'm sticking with dulwich. Since at least that gives
On Mon, Feb 11, 2013 at 11:18 PM, Rene Rivera <grafikrobot@gmail.com> wrote: the
advantage of being pure Python.
I must be missing something. Why not just work through the python subprocess interface? As Dave points out, the git interaction is really simple, so isn't the subprocess approach sufficient?
It's not sufficient.. As I can't rewrite the subrepo references to use the HTTPS protocol instead of the GIT protocol (which is how they are currently encoded) with the git CLI. And I really want to avoid having testers install git, if possible. As it's not an easy install as say svn and hg. -- -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo
On Tue, Feb 12, 2013 at 10:37 AM, Rene Rivera <grafikrobot@gmail.com> wrote:
On Tue, Feb 12, 2013 at 8:25 AM, Beman Dawes <bdawes@acm.org> wrote: ...
I must be missing something. Why not just work through the python subprocess interface? As Dave points out, the git interaction is really simple, so isn't the subprocess approach sufficient?
It's not sufficient.. As I can't rewrite the subrepo references to use the HTTPS protocol instead of the GIT protocol (which is how they are currently encoded) with the git CLI. And I really want to avoid having testers install git, if possible. As it's not an easy install as say svn and hg.
Huh! With GitHub alone having 3 million users, Git can't be too hard to install:-) Requiring testers to have a command line git client installed seems totally reasonable, IMO. --Beman
On 12 February 2013 15:53, Beman Dawes <bdawes@acm.org> wrote:
On Tue, Feb 12, 2013 at 10:37 AM, Rene Rivera <grafikrobot@gmail.com> wrote:
On Tue, Feb 12, 2013 at 8:25 AM, Beman Dawes <bdawes@acm.org> wrote: ...
I must be missing something. Why not just work through the python subprocess interface? As Dave points out, the git interaction is really simple, so isn't the subprocess approach sufficient?
It's not sufficient.. As I can't rewrite the subrepo references to use the HTTPS protocol instead of the GIT protocol (which is how they are currently encoded) with the git CLI. And I really want to avoid having testers install git, if possible. As it's not an easy install as say svn and hg.
Huh! With GitHub alone having 3 million users, Git can't be too hard to install:-)
Requiring testers to have a command line git client installed seems totally reasonable, IMO.
FYI, GitHub for Windows package delivers the command line client too and it all works well 1) without painful certificates juggling with pageant 2) is friendly to those who are allergic to Cygwin or msysGit bash. Best regards, -- Mateusz Loskot, http://mateusz.loskot.net
on Tue Feb 12 2013, Mateusz Loskot <mateusz-AT-loskot.net> wrote:
On 12 February 2013 15:53, Beman Dawes <bdawes@acm.org> wrote:
On Tue, Feb 12, 2013 at 10:37 AM, Rene Rivera <grafikrobot@gmail.com> wrote:
On Tue, Feb 12, 2013 at 8:25 AM, Beman Dawes <bdawes@acm.org> wrote: ...
I must be missing something. Why not just work through the python subprocess interface?
As Dave points out, the git interaction is really simple, so isn't the subprocess approach sufficient?
It's not sufficient.. As I can't rewrite the subrepo references to use the HTTPS protocol instead of the GIT protocol (which is how they are currently encoded) with the git CLI. And I really want to avoid having testers install git, if possible. As it's not an easy install as say svn and hg.
Huh! With GitHub alone having 3 million users, Git can't be too hard to install:-)
Requiring testers to have a command line git client installed seems totally reasonable, IMO.
FYI, GitHub for Windows package delivers the command line client too and it all works well 1) without painful certificates juggling with pageant 2) is friendly to those who are allergic to Cygwin or msysGit bash.
Nice! -- Dave Abrahams
on Tue Feb 12 2013, Beman Dawes <bdawes-AT-acm.org> wrote:
On Tue, Feb 12, 2013 at 10:37 AM, Rene Rivera <grafikrobot@gmail.com> wrote:
On Tue, Feb 12, 2013 at 8:25 AM, Beman Dawes <bdawes@acm.org> wrote: ...
I must be missing something. Why not just work through the python subprocess interface? As Dave points out, the git interaction is really simple, so isn't the subprocess approach sufficient?
It's not sufficient.. As I can't rewrite the subrepo references to use the HTTPS protocol instead of the GIT protocol (which is how they are currently encoded)
Sure you can; Just git clone non-recursively and then edit the .gitmodules file.
with the git CLI. And I really want to avoid having testers install git, if possible. As it's not an easy install as say svn and hg.
Huh! With GitHub alone having 3 million users, Git can't be too hard to install:-)
Requiring testers to have a command line git client installed seems totally reasonable, IMO.
Is git for windows dead-simple to install yet? -- Dave Abrahams
on Tue Feb 12 2013, "Jeffrey Lee Hellrung, Jr." <jeffrey.hellrung-AT-gmail.com> wrote:
On Tue, Feb 12, 2013 at 8:07 AM, Dave Abrahams <dave@boostpro.com> wrote: [...]
Is git for windows dead-simple to install yet?
I've always thought so.
IMO it wasn't always dead-simple. Too many choices of what to install, confusing documentation about what you're actually getting with each choice, etc., were all problems. I feel safe recommending http://windows.github.com, though. I guess I should test it myself before being so confident... -- Dave Abrahams
On Tue, Feb 12, 2013 at 4:39 PM, Dave Abrahams <dave@boostpro.com> wrote:
on Tue Feb 12 2013, "Jeffrey Lee Hellrung, Jr." < jeffrey.hellrung-AT-gmail.com> wrote:
On Tue, Feb 12, 2013 at 8:07 AM, Dave Abrahams <dave@boostpro.com> wrote: [...]
Is git for windows dead-simple to install yet?
I've always thought so.
IMO it wasn't always dead-simple. Too many choices of what to install, confusing documentation about what you're actually getting with each choice, etc., were all problems. I feel safe recommending http://windows.github.com, though. I guess I should test it myself before being so confident...
I guess there was the choice of SSH client, IIRC, but I don't remember anything else remotely trippy :/ I've always just used the git bash (okay, more recently git gui), I don't know if that simplifies things. - Jeff
on Tue Feb 12 2013, "Jeffrey Lee Hellrung, Jr." <jeffrey.hellrung-AT-gmail.com> wrote:
On Tue, Feb 12, 2013 at 4:39 PM, Dave Abrahams <dave@boostpro.com> wrote:
on Tue Feb 12 2013, "Jeffrey Lee Hellrung, Jr." < jeffrey.hellrung-AT-gmail.com> wrote:
On Tue, Feb 12, 2013 at 8:07 AM, Dave Abrahams <dave@boostpro.com> wrote: [...]
Is git for windows dead-simple to install yet?
I've always thought so.
IMO it wasn't always dead-simple. Too many choices of what to install, confusing documentation about what you're actually getting with each choice, etc., were all problems. I feel safe recommending http://windows.github.com, though. I guess I should test it myself before being so confident...
I guess there was the choice of SSH client, IIRC, but I don't remember anything else remotely trippy :/ I've always just used the git bash (okay, more recently git gui), I don't know if that simplifies things.
http://stackoverflow.com/questions/3672272/msysgit-vs-git-for-windows is an example -- Dave Abrahams
On Wed, Feb 13, 2013 at 1:39 AM, Dave Abrahams <dave@boostpro.com> wrote:
on Tue Feb 12 2013, "Jeffrey Lee Hellrung, Jr." < jeffrey.hellrung-AT-gmail.com> wrote:
On Tue, Feb 12, 2013 at 8:07 AM, Dave Abrahams <dave@boostpro.com> wrote: [...]
Is git for windows dead-simple to install yet?
I've always thought so.
IMO it wasn't always dead-simple. Too many choices of what to install, confusing documentation about what you're actually getting with each choice, etc., were all problems. I feel safe recommending http://windows.github.com, though. I guess I should test it myself before being so confident...
Here is my experience on this. I am a user of git who retrieve libraries and sometime patch them and provide PR on GitHub and Bitbucket, but I don't fully use git as I find Mercurial far more clear on the command line interface (and TortoiseHg helps a lot too but I suspect a future Windows release of SourceTree will change things radically). First I must say that git itself on windows is less obscure than it was few years ago. Still, errors Currently, I have 3 tools that deal with git: - TortoiseGit (+MSysGit) : it install everything but git so you have to install MSysGit first. Once done the GUI is pretty much inspired fromg - GitGUI -
Sorry I posted before finishing writing, here is the complete email. Here is my experience on this. I am a user of git who retrieve libraries and sometime patch them and provide PR on GitHub and Bitbucket, but I don't fully use git as I find Mercurial far more clear on the command line interface (and TortoiseHg helps a lot too but I suspect a future Windows release of SourceTree will change things radically). First I must say that git itself on windows is less obscure than it was few years ago. Still, errors are very obscure compared to mercurial on windows, but this is another discussion. Currently, I have 3 tools that help using git: - TortoiseGit (+MSysGit) : it install everything but git so you have to install MSysGit first (which is not very intuitive at first because you always ask "why not install it for me?"). Once done the GUI is pretty much inspired from TortoiseSVN. It's ugly (never correctly sized windows and fields) but works correctly so far. - GitGUI: easy to install but incredibly intrusive in interfaces and also have a very obscure UI (I find) I don't recommand it. - GitHub (the app): clean and simple. It's excellent working with GitHub but it becomes quickly too simple when you want to do some advanced manipulations. The display is really nice but I don't like not being able to sort easily the changes I want to commit. In all these tools you can invoke the command line with git available inside a repository directory so you can always easily work with command line. My thinking is that if you want to work with GitHub.com only, then GitHub is your best choice BUT you should learn the git command line syntax fully in case you need to do advanced stuffs. However if you want to work with other support systems, use TortoiseGit. Notice that TortoiseGit have nothing like TortoiseHg's Workbench which is too bad, but as I said I hope a coming Windows version of SourceTree will eliminate the GUI problem for SVN, GIT and HG repositories. So yeah, git on windows is getting easier, it's just a bit slow to get easy to use. Joel Lamotte
On Tue, Feb 12, 2013 at 10:07 AM, Dave Abrahams <dave@boostpro.com> wrote:
on Tue Feb 12 2013, Beman Dawes <bdawes-AT-acm.org> wrote:
On Tue, Feb 12, 2013 at 8:25 AM, Beman Dawes <bdawes@acm.org> wrote: ...
I must be missing something. Why not just work through the python subprocess interface? As Dave points out, the git interaction is really simple, so isn't the subprocess approach sufficient?
It's not sufficient.. As I can't rewrite the subrepo references to use
On Tue, Feb 12, 2013 at 10:37 AM, Rene Rivera <grafikrobot@gmail.com> wrote: the
HTTPS protocol instead of the GIT protocol (which is how they are currently encoded)
Sure you can; Just git clone non-recursively and then edit the .gitmodules file.
But then I have even more code that is managing a file I don't really want to manage. As opposed to now where I have about 150 lines of easy to read portable python code that does all the subrepo stuff while limiting the clone/pull/checkout to only the tested branch. And hence at least saving some disk space. PS.. The line count is likely to decrease as I clean up code now that it's working. -- -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo
on Wed Feb 13 2013, Rene Rivera <grafikrobot-AT-gmail.com> wrote:
On Tue, Feb 12, 2013 at 10:07 AM, Dave Abrahams <dave@boostpro.com> wrote:
on Tue Feb 12 2013, Beman Dawes <bdawes-AT-acm.org> wrote:
On Tue, Feb 12, 2013 at 8:25 AM, Beman Dawes <bdawes@acm.org> wrote: ...
I must be missing something. Why not just work through the python subprocess interface? As Dave points out, the git interaction is really simple, so isn't the subprocess approach sufficient?
It's not sufficient.. As I can't rewrite the subrepo references to use
On Tue, Feb 12, 2013 at 10:37 AM, Rene Rivera <grafikrobot@gmail.com> wrote: the
HTTPS protocol instead of the GIT protocol (which is how they are currently encoded)
We can fix that on our end of course.
Sure you can; Just git clone non-recursively and then edit the .gitmodules file.
But then I have even more code that is managing a file I don't really want to manage.
As opposed to now where I have about 150 lines of easy to read portable python code that does all the subrepo stuff while limiting the clone/pull/checkout to only the tested branch. And hence at least saving some disk space.
If you're trying to save disk, you could try using a shallow clone(?) -- Dave Abrahams
On Wed, Feb 13, 2013 at 11:18 AM, Dave Abrahams <dave@boostpro.com> wrote:
on Wed Feb 13 2013, Rene Rivera <grafikrobot-AT-gmail.com> wrote:
On Tue, Feb 12, 2013 at 10:07 AM, Dave Abrahams <dave@boostpro.com> wrote:
on Tue Feb 12 2013, Beman Dawes <bdawes-AT-acm.org> wrote:
On Tue, Feb 12, 2013 at 10:37 AM, Rene Rivera <grafikrobot@gmail.com> wrote:
On Tue, Feb 12, 2013 at 8:25 AM, Beman Dawes <bdawes@acm.org> wrote: ...
I must be missing something. Why not just work through the python subprocess interface? As Dave points out, the git interaction is really simple, so isn't
the
subprocess approach sufficient?
It's not sufficient.. As I can't rewrite the subrepo references to use the HTTPS protocol instead of the GIT protocol (which is how they are currently encoded)
We can fix that on our end of course.
Sure.. But this is one of those cases where the git design is less than ideal (not the svn does any better in this regard). In an ideal world the subrepo references would have a list of alternatives for protocols and client code could pick the correct one to match the protocol the client prefers. Which is a long way of saying it's going to be less hassles in the long run if I adjust the references to suit the client needs manually since I know the possible alternatives as we are only dealing with github.
Sure you can; Just git clone non-recursively and then edit the
.gitmodules file.
But then I have even more code that is managing a file I don't really want to manage.
As opposed to now where I have about 150 lines of easy to read portable python code that does all the subrepo stuff while limiting the clone/pull/checkout to only the tested branch. And hence at least saving some disk space.
If you're trying to save disk, you could try using a shallow clone(?)
That would be the first thing I tried almost two months ago with the git CLI. And it turns out to not be currently possible as fetching subrepos recursively ignores the shallow clone for the subrepos. I subsequently spent from that time up to a short time ago trying to do a shallow clone with dulwich without "intellectual" success (i.e. I don't understand enough of the protocol and API to do it). And gave up on that for now. So it's now not space efficient.. But at least it works. I'll see about finding out a way to do the shallow clone in the future. NOTE.. I'm referring to calendar time there.. Actual effort time is rather small. -- -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo
On 14 February 2013 05:30, Rene Rivera <grafikrobot@gmail.com> wrote:
Sure.. But this is one of those cases where the git design is less than ideal (not the svn does any better in this regard). In an ideal world the subrepo references would have a list of alternatives for protocols and client code could pick the correct one to match the protocol the client prefers.
This information shouldn't really be stored in the repo, but externally (since it can change with both time and location). I'm not sure if it's what you're looking for, and it certainly isn't ideal, but git does have a configuration option to map urls to different locations. Details are at: http://www.kernel.org/pub/software/scm/git/docs/git-clone.html#URLS You need to scroll down a tad, look for 'insteadOf'. As with all git configuration it's a bit of a pain, as it can't be done automatically. But you maybe could write a script to add the necessary configuration to a local clone, or add it to the user's .gitconfig file. I actually haven't tried it yet, but I'm hoping I can use it to speed up a script I have which fetches from the same repo to multiple local clones.
on Thu Feb 14 2013, Rene Rivera <grafikrobot-AT-gmail.com> wrote:
On Wed, Feb 13, 2013 at 11:18 AM, Dave Abrahams <dave@boostpro.com> wrote:
If you're trying to save disk, you could try using a shallow clone(?)
That would be the first thing I tried almost two months ago with the git CLI. And it turns out to not be currently possible as fetching subrepos recursively ignores the shallow clone for the subrepos.
You don't have to fetch recursively, though. You seem to want to do something unorthodox with Git *and* at the same time want it to take only a few simple lines of code. I don't think that's necessarily reasonable.
I subsequently spent from that time up to a short time ago trying to do a shallow clone with dulwich without "intellectual" success (i.e. I don't understand enough of the protocol and API to do it). And gave up on that for now. So it's now not space efficient.. But at least it works.
That's really what matters.
I'll see about finding out a way to do the shallow clone in the future.
I think you have your priorities straight :-) -- Dave Abrahams
on Tue Feb 12 2013, Beman Dawes <bdawes-AT-acm.org> wrote:
As Dave points out, the git interaction is really simple, so isn't the subprocess approach sufficient?
I think Rene is trying to save testers from having to install Git itself. We could of course ask them to get 0install, and do everything automatically from there. That's where I hope to be eventually, but I presume we don't want to take that leap along with the Git leap(?) It's certainly easy enough to set up... -- Dave Abrahams
On Tue, Feb 12, 2013 at 10:05 AM, Dave Abrahams <dave@boostpro.com> wrote:
on Tue Feb 12 2013, Beman Dawes <bdawes-AT-acm.org> wrote:
As Dave points out, the git interaction is really simple, so isn't the subprocess approach sufficient?
I think Rene is trying to save testers from having to install Git itself.
Yes. And for the simple reason that I want to reduce the number of requirements on testers. Regardless.. I have enough of what we need with using dulwich already working (assuming my final testing with the full boost superproject works when the project is fixed). Now it's time to work on the b2 support for the includes. And the changes to accomodate the modular layout. -- -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo
On Wed, Feb 13, 2013 at 12:01 AM, Rene Rivera <grafikrobot@gmail.com> wrote:
On Tue, Feb 12, 2013 at 10:05 AM, Dave Abrahams <dave@boostpro.com> wrote:
on Tue Feb 12 2013, Beman Dawes <bdawes-AT-acm.org> wrote:
As Dave points out, the git interaction is really simple, so isn't the subprocess approach sufficient?
I think Rene is trying to save testers from having to install Git itself.
Yes. And for the simple reason that I want to reduce the number of requirements on testers.
Regardless.. I have enough of what we need with using dulwich already working (assuming my final testing with the full boost superproject works when the project is fixed).
Excellent! If there are initial rough edges, that's to be expected and they can be smoothed out over time if they are bothering people.
Now it's time to work on the b2 support for the includes.
Right. We need that support soon.
And the changes to accomodate the modular layout.
What particular changes are you concerned about. I would like to open issues and/or milestones for any issues we aren't already tracking. --Beman
on Mon Feb 11 2013, Rene Rivera <grafikrobot-AT-gmail.com> wrote:
NOTE: Part of this discussion was within the release team. But it was suggested that it should also be generally public. Hence I'm raising some of the same issues from the private discussion.
On Mon, Feb 11, 2013 at 12:58 PM, Dave Abrahams <dave@boostpro.com> wrote:
on Mon Feb 11 2013, Rene Rivera <grafikrobot-AT-gmail.com> wrote:
On Mon, Feb 11, 2013 at 11:51 AM, Dave Abrahams <dave@boostpro.com> wrote:
on Sun Feb 10 2013, Rene Rivera <grafikrobot-AT-gmail.com> wrote:
On Sun, Feb 10, 2013 at 11:08 AM, Beman Dawes <bdawes@acm.org> wrote:
I ask because the testing will rely on dulwich git which does not talk at all with bitbucket (it seems the bitbucket git server works differently.. not that I blame it given the disfunction that is the git server protocol).
Oh, that's rather lame. Is there a reason you're not going to use libgit2 instead of dulwich?
Yes.. The python bindings for libgit2 are terribly/unusably incomplete.
Surely they're complete enough to allow you to clone and pull, no? What more do you need?
Why would one assume that?
Because anything less would kind of suck.
First there is no binding to "git_clone" < https://github.com/search?q=git_clone+repo%3Alibgit2%2Fpygit2&type=Repositories&ref=advsearch&l=>. So it's a nonstarter at step zero.
That kind of sucks.
Second there are no bindings for submodules/subrepos. Which might be an additional nonstarter if the git_clone didn't recurse clone/pull subrepos.
I was faced with either implementing some of the additional clone/pull/checkout/update code with dulwich or likely implement even more for libgit2.
libgit2 covers it (http://stackoverflow.com/a/8553826/125349); it's just not in the pygit2 bindings, I guess.
Hence I'm sticking with dulwich. Since at least that gives the advantage of being pure Python.
My initial problems where mostly because I was trying to do things that where not actually supported by the git client/server design.. Of course I didn't know that because the git design is essentially non-existent and anything that exists is ad-hoc and minimal.. AFAICT.
Did you ask the "git people" about how to do these things?
My latest problems don't have much to do with dulwich at this point.. More of not being able to do anything interesting (i.e. efficient) because of the limitation of the git server and system.
That's surprising to me since I've always found git to be quite speedy.
Yes, sad to say I've had to learn way more than I really wanted about git.
This also surprises me. What, exactly, are you trying to do that is so exotic?
And it has only confirmed my educated suspicions about it. But I'll get something working eventually. Just will not be soon.
Sadly-and-somewhat-frustratedly.. Rene.
Sorry this has been frustrating/saddening. -- Dave Abrahams
Hi, On Mon, Feb 11, 2013 at 1:47 PM, Rene Rivera <grafikrobot@gmail.com> wrote:
Is that currently being updated like the bitbucket test repos? I ask because the testing will rely on dulwich git which does not talk at all with bitbucket (it seems the bitbucket git server works differently.. not that I blame it given the disfunction that is the git server protocol). Hence I need something useable for development of the testing scripts.
What do you mean by 'dulwich cannot talk with bitbucket' ? In my Debian GNU/Linux (sid) system, everything seems fine with git repository hosted on bitbucket using git protocol. What I tried was: 1. Install dulwich (using Debian package) 2. using the dulwich client: $ dulwich clone git@bitbucket.org:ryppl/boost.git 3. check the boost.git directory Best regards, -- Ryo IGARASHI, Ph.D. rigarash@gmail.com
On Tue, Feb 12, 2013 at 2:33 AM, Ryo IGARASHI <rigarash@gmail.com> wrote:
Hi,
On Mon, Feb 11, 2013 at 1:47 PM, Rene Rivera <grafikrobot@gmail.com> wrote:
Is that currently being updated like the bitbucket test repos? I ask because the testing will rely on dulwich git which does not talk at all with bitbucket (it seems the bitbucket git server works differently.. not that I blame it given the disfunction that is the git server protocol). Hence I need something useable for development of the testing scripts.
What do you mean by 'dulwich cannot talk with bitbucket' ? In my Debian GNU/Linux (sid) system, everything seems fine with git repository hosted on bitbucket using git protocol.
What I tried was: 1. Install dulwich (using Debian package) 2. using the dulwich client: $ dulwich clone git@bitbucket.org:ryppl/boost.git 3. check the boost.git directory
Try: === $ dulwich clone https://bitbucket.org/boostorg/boost.git Traceback (most recent call last): [...] File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 521, in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) urllib2.HTTPError: HTTP Error 404: NOT FOUND === -- -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo
On Sun, Feb 10, 2013 at 9:08 PM, Beman Dawes <bdawes@acm.org> wrote:
The "Git and Modular Boost" documentation has been updated:
* The GitHub mothership with be github.com/boostorg * Full history for the current trunk and branches/release will be available. * There are now active milestone and issue trackers for the conversion.
See https://svn.boost.org/trac/boost/wiki/WikiStart#GitandModularBoost
I'm a bit confused. Is https://github.com/boostorg the primary location of modularized Boost now? Is boost-lib deprecated? It's still mentioned on a few pages on the Wiki. And what is https://bitbucket.org/boostorg? Is it supposed to be a mirror? What repo boost devs should be using after modularization is complete?
2013/2/11 Andrey Semashev <andrey.semashev@gmail.com>:
On Sun, Feb 10, 2013 at 9:08 PM, Beman Dawes <bdawes@acm.org> wrote:
The "Git and Modular Boost" documentation has been updated:
* The GitHub mothership with be github.com/boostorg * Full history for the current trunk and branches/release will be available. * There are now active milestone and issue trackers for the conversion.
See https://svn.boost.org/trac/boost/wiki/WikiStart#GitandModularBoost
I'm a bit confused. Is https://github.com/boostorg the primary location of modularized Boost now?
Yes, we think that name is more appropriate. Our preference would be "boost", but that name is already taken.
Is boost-lib deprecated? It's still mentioned on a few pages on the Wiki.
And what is https://bitbucket.org/boostorg? Is it supposed to be a mirror? What repo boost devs should be using after modularization is complete?
That is a mirror and might go away in the future. We currently push there because of the gitk-like branch view. We think that is helpful for reviewing the modularization. The official place will be github.com/boostorg.
participants (10)
-
Andrey Semashev -
Beman Dawes -
Daniel James -
Daniel Pfeifer -
Dave Abrahams -
Jeffrey Lee Hellrung, Jr. -
Klaim - Joël Lamotte -
Mateusz Loskot -
Rene Rivera -
Ryo IGARASHI