[git] conversion stalled
Hi! It seems the conversion script is stalled on commit 515f5649bbfb25e2ac89958d700ff2a67fddc030 Date: Fri Sep 13 14:48:17 2013 +0000 There have been some Boost.Build commits in the meantime, can someone please re-enable the conversion again? Yours, Jürgen -- * Dipl.-Math. Jürgen Hunold ! * voice: ++49 4257 300 ! Fährstraße 1 * fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen * jhunold@gmx.eu ! Germany
Sorry, you've got the wrong Daniel.
On 4 October 2013 10:11, Jürgen Hunold
Hi!
It seems the conversion script is stalled on
commit 515f5649bbfb25e2ac89958d700ff2a67fddc030 Date: Fri Sep 13 14:48:17 2013 +0000
There have been some Boost.Build commits in the meantime, can someone please re-enable the conversion again?
Yours,
Jürgen -- * Dipl.-Math. Jürgen Hunold ! * voice: ++49 4257 300 ! Fährstraße 1 * fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen * jhunold@gmx.eu ! Germany
Hi, I hope I got the right Daniel this time. On Friday, 4. October 2013 11:11:07 Jürgen Hunold wrote:
Hi!
It seems the conversion script is stalled on
commit 515f5649bbfb25e2ac89958d700ff2a67fddc030 Date: Fri Sep 13 14:48:17 2013 +0000
There have been some Boost.Build commits in the meantime, can someone please re-enable the conversion again?
Ping? Any chance of the conversion starting again? Yours, Jürgen -- * Dipl.-Math. Jürgen Hunold ! * voice: ++49 4257 300 ! Fährstraße 1 * fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen * jhunold@gmx.eu ! Germany
2013/10/8 Jürgen Hunold
Hi,
I hope I got the right Daniel this time.
Ja.
On Friday, 4. October 2013 11:11:07 Jürgen Hunold wrote:
Hi!
It seems the conversion script is stalled on
commit 515f5649bbfb25e2ac89958d700ff2a67fddc030 Date: Fri Sep 13 14:48:17 2013 +0000
There have been some Boost.Build commits in the meantime, can someone please re-enable the conversion again?
Ping? Any chance of the conversion starting again?
It is already running for 9 hours, but has not completed yet.
Yours,
Jürgen -- * Dipl.-Math. Jürgen Hunold ! * voice: ++49 4257 300 ! Fährstraße 1 * fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen * jhunold@gmx.eu ! Germany
-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Dave Abrahams Sent: Tuesday, October 08, 2013 10:14 PM To: Daniel Pfeifer Cc: Boost Developers List Subject: Re: [boost] [git] conversion stalled
That's alarming. It never took as much as 2 hrs to complete in the past
Sent from my moss-covered three-handled family gradunza
<snip> It hat what we have in the long room ;-) Love P
On 8 Oct 2013 at 14:13, Dave Abrahams wrote:
That's alarming. It never took as much as 2 hrs to complete in the past
I see that .gitattributes now contains explicit eol and mime type mappings for every file in the repo. I advised you against that - I think it will make conversion performance slow from all the lookups, never mind poor old git trying to parse that map on every file checkout. Niall -- Currently unemployed and looking for work. Work Portfolio: http://careers.stackoverflow.com/nialldouglas/
2013/10/9 Niall Douglas
On 8 Oct 2013 at 14:13, Dave Abrahams wrote:
That's alarming. It never took as much as 2 hrs to complete in the past
I see that .gitattributes now contains explicit eol and mime type mappings for every file in the repo. I advised you against that - I think it will make conversion performance slow from all the lookups, never mind poor old git trying to parse that map on every file checkout.
The commit that added the file is labeled "Add Niall Douglas' .gitattributes file". Care to repeat what you did advise exactly? Cheers, Daniel
On 9 Oct 2013 at 22:49, Daniel Pfeifer wrote:
I see that .gitattributes now contains explicit eol and mime type mappings for every file in the repo. I advised you against that - I think it will make conversion performance slow from all the lookups, never mind poor old git trying to parse that map on every file checkout.
The commit that added the file is labeled "Add Niall Douglas' .gitattributes file". Care to repeat what you did advise exactly?
My original .gitattributes file was a set of file extension wildcards exactly matching the Boost wiki page describing SVN auto-props. This ensured that when checking out text formats, you got native EOLs or Unix EOLs as is correct for that type of text. The problem which then emerged is that many files have been historically committed to Boost years ago with all sorts of weird and unconventional EOLs and UTF variations e.g. there are some files in sandbox which have intentionally malformed EOLs as part of its unit tests, something which would break if committed as non-binary to git. There is also quite a bit of UTF-16 text, especially in ancient revisions before UTF-8 became popular. Git has no understanding of UTF-16 text, so text is either UTF-8 or it's a binary. My original advice was that unintentionally weird text formats needed fixing up for git correctness i.e. UTF-8 throughout with EOL 10. I hence sent a patch to Boost2Git which scanned text format files for bad EOLs, and it did repairs in flight. This seems to have worked, but it advertently repaired intentionally weird text formats. My original advice was that intentionally weird text formats need a better file extension than .txt e.g. .bin. That said, I accepted Dave's argument that this causes breakage in ancient revision checkouts, but I still would argue that if people really need ancient revisions working properly, go use a legacy SVN repo. Dave I think decided that every text file needed listing in .gitattributes with its EOL style, and the .gitattributes file needs permuting every commit because as commits pass some text files will change from UTF-16 EOL 13,10 through Latin1 EOL 13,10 to UTF-8 EOL 10 etc so a static .gitattributes would still introduce corruption. This is what is needed if you want any possible past revision checkout to be a perfect representation of the SVN repo. I'd imagine Boost2Git will also need a map of which text files are intentionally malformed and must be treated as binary over which range of SVN revision numbers. Building that map, I would imagine, will involve a lot of human hours. As this list knows, I think all this work being done for free is excessive. If people really, really want all possible ancient revisions to work, they should pay the contracting hourly rate to people to implement it. Expecting all this for free is in my opinion daft. A git conversion of Boost ought to work reasonably well for the past three years of checkouts, past that it should be for illustration of history only. Niall -- Currently unemployed and looking for work. Work Portfolio: http://careers.stackoverflow.com/nialldouglas/
On Wednesday, October 9, 2013 3:23:46 PM UTC-7, Niall Douglas wrote:
Dave I think decided that every text file needed listing in .gitattributes with its EOL style, and the .gitattributes file needs permuting every commit because as commits pass some text files will change from UTF-16 EOL 13,10 through Latin1 EOL 13,10 to UTF-8 EOL 10 etc so a static .gitattributes would still introduce corruption.
This doesn't sound reminiscent of anything I remember deciding.
Random thoughts and personal opinions: 1) It would have been better to have started this discussion on another thread. "Conversion stalled" isn't going to attract many of the stakeholders. 2) Trunk and branches/release (i.e. develop and master) matter. Sandbox doesn't matter. I'm a bit surprised we are even bothering with sandbox. Do what is best for trunk and branches/release. 3) As a library maintainer, I want all line endings in my libraries to be faithfully preserved by Git, both at conversion and into the future. I see it as quite a black eye for Git if that can't happen. --Beman
Beman Dawes
3) As a library maintainer, I want all line endings in my libraries to be faithfully preserved by Git, both at conversion and into the future. I see it as quite a black eye for Git if that can't happen.
Git is a "stupid content tracker," so at its lowest levels it doesn't know anything about text or line endings, and therefore they're preserved just like any other bits stored by Git. The only interesting question for Git is what happens to those line endings when a given file is checked out on a given system with a given set of user preferences and a given .gitattributes file. Subversion, on the other hand, has "smarts" about line endings that affect not only how they're checked out, but how they're stored on the server. It tries to remember whether a file is a text file (its whole MIME-type actually) and what kind of line endings it was stored with. So the question of what it means to "preserve" line endings stored in Subversion is a bit more complicated. Right now our translation pulls the bytes out of Subversion and stores them literally in Git, so by definition there's no lost information /except/ what may be given by SVN attributes like svn:mime-type and svn:eol-style. I dont' remember exactly what problem we're trying to solve here; I read https://groups.google.com/forum/#!topic/boost-devel-archive/guyHrDlW0Hc which documents the initial evolution of the issue. We may have concluded eventually that people were trying to correct mistakes in SVN history, like checking in files with the wrong attributes, and if that were the case, I would surely have decided that this should not be the role of the conversion tool.
Hi Daniel, On Tuesday, 8. October 2013 21:32:43 Daniel Pfeifer wrote:
2013/10/8 Jürgen Hunold
: Hi,
I hope I got the right Daniel this time.
Ja.
Sehr schön :-)
It is already running for 9 hours, but has not completed yet.
Thanks a lot, the conversion seems to have finished, but the modularized repository is missing the new "sync" library. Yours, Jürgen -- * Dipl.-Math. Jürgen Hunold ! * voice: ++49 4257 300 ! Fährstraße 1 * fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen * jhunold@gmx.eu ! Germany
2013/10/9 Jürgen Hunold
Hi Daniel,
On Tuesday, 8. October 2013 21:32:43 Daniel Pfeifer wrote:
2013/10/8 Jürgen Hunold
: Hi,
I hope I got the right Daniel this time.
Ja.
Sehr schön :-)
It is already running for 9 hours, but has not completed yet.
Thanks a lot, the conversion seems to have finished, but the modularized repository is missing the new "sync" library.
Was Boost.Sync accepted as a new library? I thoght it belongs to Boost.Thread. I can certainly turn it into its own repository. Cheers, Daniel
Hi Daniel, On Wednesday, 9. October 2013 20:13:42 Daniel Pfeifer wrote:
Thanks a lot, the conversion seems to have finished, but the modularized repository is missing the new "sync" library.
Was Boost.Sync accepted as a new library? I thoght it belongs to Boost.Thread. I can certainly turn it into its own repository.
I did not follow the long thread in detail, but we have "libs/sync" in parallel to "libs/thread" now. The basic idea was to factor out sync code etc. if I remember correctly. So a separate submodule seems the way to go. At least for (finally) setting up Boost.Build ... Yours, Jürgen -- * Dipl.-Math. Jürgen Hunold ! * voice: ++49 4257 300 ! Fährstraße 1 * fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen * jhunold@gmx.eu ! Germany
2013/10/9 Jürgen Hunold
Hi Daniel,
On Wednesday, 9. October 2013 20:13:42 Daniel Pfeifer wrote:
Thanks a lot, the conversion seems to have finished, but the modularized repository is missing the new "sync" library.
Was Boost.Sync accepted as a new library? I thoght it belongs to Boost.Thread. I can certainly turn it into its own repository.
I did not follow the long thread in detail, but we have "libs/sync" in parallel to "libs/thread" now. The basic idea was to factor out sync code etc. if I remember correctly. So a separate submodule seems the way to go. At least for (finally) setting up Boost.Build ...
Right. I have created the repositories on github and bitbucket and changed the ruleset. The buildjob is currently converting again.
participants (7)
-
Beman Dawes
-
Daniel James
-
Daniel Pfeifer
-
Dave Abrahams
-
Jürgen Hunold
-
Niall Douglas
-
Paul A. Bristow