I have just run this: svn diff http://svn.boost.org/svn/boost/branches/RC_1_34_0/boost/tools/build/v2 http://svn.boost.org/svn/boost/trunk/tools/build/v2 > RC_HEAD.diff and it took about 13 mins. On another machine (with another network), it took 15 mins. I've asked for confirmation on IRC, and two people are at 11 and 13 mins as a write this (and still running). Is there some network or setup issue with SVN server? Maybe the apache connection limit is set too low? - Volodya
Vladimir Prus wrote:
I have just run this:
svn diff http://svn.boost.org/svn/boost/branches/RC_1_34_0/boost/tools/build/v2 http://svn.boost.org/svn/boost/trunk/tools/build/v2 > RC_HEAD.diff
and it took about 13 mins. On another machine (with another network), it took 15 mins. I've asked for confirmation on IRC, and two people are at 11 and 13 mins as a write this (and still running).
Is there some network or setup issue with SVN server? Maybe the apache connection limit is set too low?
The time taken by this test didn't correlate significantly with the number of people participating. That could indicate that there are enough resources. It could also indicate there's an artifical limitation per connection, e.g. a bandwidth limit. Regards, m
Martin Wille wrote:
Vladimir Prus wrote:
I have just run this:
svn diff http://svn.boost.org/svn/boost/branches/RC_1_34_0/boost/tools/build/v2 http://svn.boost.org/svn/boost/trunk/tools/build/v2 > RC_HEAD.diff
and it took about 13 mins. On another machine (with another network), it took 15 mins. I've asked for confirmation on IRC, and two people are at 11 and 13 mins as a write this (and still running).
Is there some network or setup issue with SVN server? Maybe the apache connection limit is set too low?
The time taken by this test didn't correlate significantly with the number of people participating. That could indicate that there are enough resources. It could also indicate there's an artifical limitation per connection, e.g. a bandwidth limit.
It would correlate if the number of people doing this at the same time was above the number of CPUs the machine has. -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo
Vladimir Prus wrote:
I have just run this:
svn diff http://svn.boost.org/svn/boost/branches/RC_1_34_0/boost/tools/build/v2 http://svn.boost.org/svn/boost/trunk/tools/build/v2 > RC_HEAD.diff
and it took about 13 mins. On another machine (with another network), it took 15 mins. I've asked for confirmation on IRC, and two people are at 11 and 13 mins as a write this (and still running).
Is there some network or setup issue with SVN server? Maybe the apache connection limit is set too low?
From observing the CPU usage on the server while this was going on. It seems diff is a very expensive SVN operation. The httpd processes doing the diffs ran at 100% for most of the time. -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo
Rene Rivera wrote:
Vladimir Prus wrote:
I have just run this:
svn diff http://svn.boost.org/svn/boost/branches/RC_1_34_0/boost/tools/build/v2 http://svn.boost.org/svn/boost/trunk/tools/build/v2 > RC_HEAD.diff
and it took about 13 mins. On another machine (with another network), it took 15 mins. I've asked for confirmation on IRC, and two people are at 11 and 13 mins as a write this (and still running).
Is there some network or setup issue with SVN server? Maybe the apache connection limit is set too low?
From observing the CPU usage on the server while this was going on. It seems diff is a very expensive SVN operation. The httpd processes doing the diffs ran at 100% for most of the time.
Technically, diff should be: 1. Sending path A to client. 2. Sending binary delta from path A to path B to client. Both (1) and (2) should be rather fast, and (2) is what svn does all the time -- including during update or switch from path A to path B. I would suspect misconfiguration, not a genuine lack of processing resources. - Volodya
Vladimir Prus wrote:
I have just run this:
svn diff http://svn.boost.org/svn/boost/branches/RC_1_34_0/boost/tools/build/v2 http://svn.boost.org/svn/boost/trunk/tools/build/v2 > RC_HEAD.diff
and it took about 13 mins. On another machine (with another network), it took 15 mins. I've asked for confirmation on IRC, and two people are at 11 and 13 mins as a write this (and still running).
Is there some network or setup issue with SVN server? Maybe the apache connection limit is set too low?
Ping? The problem appears to persist. - Volodya
Vladimir Prus wrote:
Vladimir Prus wrote:
I have just run this:
svn diff http://svn.boost.org/svn/boost/branches/RC_1_34_0/boost/tools/build/v2 http://svn.boost.org/svn/boost/trunk/tools/build/v2 > RC_HEAD.diff
and it took about 13 mins. On another machine (with another network), it took 15 mins. I've asked for confirmation on IRC, and two people are at 11 and 13 mins as a write this (and still running).
Is there some network or setup issue with SVN server? Maybe the apache connection limit is set too low?
Ping? The problem appears to persist.
We'll see what we can tweak to improve performance... how long do you expect that this operation should take? I'm accustomed to large diffs on large trees taking 10 minutes or more, so the question is: is this a misconfiguration, or is it just Subversion? - Doug
Douglas Gregor wrote:
Vladimir Prus wrote:
Vladimir Prus wrote:
I have just run this:
svn diff http://svn.boost.org/svn/boost/branches/RC_1_34_0/boost/tools/build/v2 http://svn.boost.org/svn/boost/trunk/tools/build/v2 > RC_HEAD.diff
and it took about 13 mins. On another machine (with another network), it took 15 mins. I've asked for confirmation on IRC, and two people are at 11 and 13 mins as a write this (and still running).
Is there some network or setup issue with SVN server? Maybe the apache connection limit is set too low?
Ping? The problem appears to persist.
We'll see what we can tweak to improve performance... how long do you expect that this operation should take? I'm accustomed to large diffs on large trees taking 10 minutes or more, so the question is: is this a misconfiguration, or is it just Subversion?
For comparison, I've run diff of KDevelop between KDE 3.4 and KDE 3.5. It took 13 mins and the resulting diff was 11M in size. The size of (archived) source is 9M. In comparison, the size of Boost.Build is 1.4M (including huge generated docs, svn size is smaller), and the diff is 96K. Therefore, I'd suspect that if my commands takes more that 30 seconds, it's already a misconfiguration somewhere. Are you sure it's not connection limit? I remember that diff needs a 2 or 3 connections to server at the same time. - Volodya
On Sep 4, 2007, at 9:14 AM, Vladimir Prus wrote:
For comparison, I've run diff of KDevelop between KDE 3.4 and KDE 3.5. It took 13 mins and the resulting diff was 11M in size. The size of (archived) source is 9M.
In comparison, the size of Boost.Build is 1.4M (including huge generated docs, svn size is smaller), and the diff is 96K.
I misread your "svn diff" line... I thought you were diff'ing all of RC_1_34_0 against the trunk, for which 13 minutes didn't seem so horrible. But you're only diffing the build system...
Therefore, I'd suspect that if my commands takes more that 30 seconds, it's already a misconfiguration somewhere. Are you sure it's not connection limit? I remember that diff needs a 2 or 3 connections to server at the same time.
We've tweaked some things that helped performance on our end. Please try again to see how things are for you now. - Doug
Doug Gregor wrote:
On Sep 4, 2007, at 9:14 AM, Vladimir Prus wrote:
For comparison, I've run diff of KDevelop between KDE 3.4 and KDE 3.5. It took 13 mins and the resulting diff was 11M in size. The size of (archived) source is 9M.
In comparison, the size of Boost.Build is 1.4M (including huge generated docs, svn size is smaller), and the diff is 96K.
I misread your "svn diff" line... I thought you were diff'ing all of RC_1_34_0 against the trunk, for which 13 minutes didn't seem so horrible. But you're only diffing the build system...
Therefore, I'd suspect that if my commands takes more that 30 seconds, it's already a misconfiguration somewhere. Are you sure it's not connection limit? I remember that diff needs a 2 or 3 connections to server at the same time.
We've tweaked some things that helped performance on our end. Please try again to see how things are for you now.
I get 9 mins now -- still rather high. - Volodya
On a possible related note, when I checkout boost from SVN on my windows/xp it fails to complete with and error message - indicating - access denied. I'm managed to do it with a few cycles of update/clean-up etc But it doesn't seem that this should be necessary. I wonder if I'm the only one experiencing this. Robert Ramey
Joseph Gauterin wrote:
I wonder if I'm the only one experiencing this. You're not - it took me three or four update/cleanup cycles to get a checkout.
This might have something to do with your virus scanner, if you have one. Try disabling it for your working copy dir. Cheers, /Marcus
participants (8)
-
Doug Gregor -
Douglas Gregor
-
Joseph Gauterin -
Marcus Lindblom -
Martin Wille -
Rene Rivera -
Robert Ramey -
Vladimir Prus