Time for another bug sprint (after 1.41 ships)?
Earlier this year, we had a "bug sprint", where many people focused on fixing bugs in Boost. We managed to get the # of open tickets down from 800 to about 720. As of today, there are 820 open tickets in Trac. Is it time to focus on swotting bugs again? 1.41 is scheduled for release on 1-November; that's six weeks from yesterday. How about a bug sprint starting on 14-November, finishing on 22- November? That's two weekends - and finishes up before the American Thanksgiving holiday. -- Marshall P.S. This doesn't mean that people have to wait until November to fix bugs ;-) ;-)
Marshall Clow wrote:
Earlier this year, we had a "bug sprint", where many people focused on fixing bugs in Boost. We managed to get the # of open tickets down from 800 to about 720.
As of today, there are 820 open tickets in Trac.
Is it time to focus on swotting bugs again?
1.41 is scheduled for release on 1-November; that's six weeks from yesterday.
How about a bug sprint starting on 14-November, finishing on 22-November? That's two weekends - and finishes up before the American Thanksgiving holiday.
-- Marshall I'm all for it. But can't we sort out the tickets that's in fact website related thingy out of the mass ? Second, an interesting tidbits is how many of those 80 fixes get merged in 1.40 ?
-- ___________________________________________ Joel Falcou - Assistant Professor PARALL Team - LRI - Universite Paris Sud XI Tel : (+33)1 69 15 66 35
On Sep 21, 2009, at 11:18 AM, joel wrote:
Marshall Clow wrote:
Earlier this year, we had a "bug sprint", where many people focused on fixing bugs in Boost. We managed to get the # of open tickets down from 800 to about 720.
As of today, there are 820 open tickets in Trac.
Is it time to focus on swotting bugs again?
1.41 is scheduled for release on 1-November; that's six weeks from yesterday.
How about a bug sprint starting on 14-November, finishing on 22- November? That's two weekends - and finishes up before the American Thanksgiving holiday.
-- Marshall I'm all for it. But can't we sort out the tickets that's in fact website related thingy out of the mass ? Second, an interesting tidbits is how many of those 80 fixes get merged in 1.40 ?
Sure. See: https://svn.boost.org/trac/boost/report/19 - active tickets by component. "Website" is somwhere in the middle, with 15 tickets. Handy reports: Open ticket count by Owner https://svn.boost.org/trac/boost/report/18 Open ticket count by Component https://svn.boost.org/trac/boost/report/19 Open ticket count by Ticket Type https://svn.boost.org/trac/boost/report/20 Open ticket count by Milestone https://svn.boost.org/trac/boost/report/21
On Sep 21, 2009, at 9:53 AM, I wrote:
Earlier this year, we had a "bug sprint", where many people focused on fixing bugs in Boost. We managed to get the # of open tickets down from 800 to about 720.
As of today, there are 820 open tickets in Trac.
Is it time to focus on swotting bugs again?
1.41 is scheduled for release on 1-November; that's six weeks from yesterday.
How about a bug sprint starting on 14-November, finishing on 22- November? That's two weekends - and finishes up before the American Thanksgiving holiday.
-- Marshall
Who is up for this? -- Marshall
Who is up for this?
Good idea - but here's another suggestion - it would be great if someone could do an audit of release and trunk branches and see what hasn't been merged but should have been. I get the impression that some folks have committed fixes to Trunk and then forgotten all about them :-( Of course hopefully I'm wrong and there's nothing to actually do.... John.
John Maddock wrote:
Good idea - but here's another suggestion - it would be great if someone could do an audit of release and trunk branches and see what hasn't been merged but should have been. I get the impression that some folks have committed fixes to Trunk and then forgotten all about them :-(
A quick diff shows: 1336 files changed, 28506 insertions(+), 66896 deletions(-) A file by file summary is here: http://sodium.resophonic.com/boost/diffstat.txt Whitespace insensitive full diff here: http://sodium.resophonic.com/boost/difffull-ignoring-whitespace.txt Whitespace sensitive version: http://sodium.resophonic.com/boost/difffull-including-whitespace.txt Hopefully this is helpful. I've tried to generate reports like this with svn and have been consistently unsuccessful, as the network tends to time out before the diff has completed. -t
troy d. straszheim wrote:
Whitespace insensitive full diff here:
http://sodium.resophonic.com/boost/difffull-ignoring-whitespace.txt
Would it be possible to exclude "spirit" and "fusion" from the diff?
Thomas Klimpel wrote:
troy d. straszheim wrote:
Whitespace insensitive full diff here:
http://sodium.resophonic.com/boost/difffull-ignoring-whitespace.txt
Would it be possible to exclude "spirit" and "fusion" from the diff?
The git infrastructure that generates all this is set up and documented, all are welcome to use it to generate reports (or anything else). It is pretty easy to script this stuff however you like it. Say, a regex-only diff of release vs trunk (here called 'master'): % git diff --stat svn/master svn/release boost/regex libs/regex boost/regex/concepts.hpp | 94 ---- boost/regex/v4/basic_regex_parser.hpp | 7 +- boost/regex/v4/cpp_regex_traits.hpp | 6 +- [etc] libs/regex/src/fileiter.cpp | 21 +- libs/regex/src/static_mutex.cpp | 2 +- 45 files changed, 419 insertions(+), 1000 deletions(-) Personally I don't think another 'report' is a good long-term solution. This kind of report should be easy/fast for any developer to asynchronously generate for arbitrary pairs of trees. For instance, in our (separate) releases of Boost.CMake, it is both crucial and trivial to be able to verify that the only changes we've made to any given branch/release/tag/revision are strictly limited to the build system: % git diff --stat 1.40.0.cmake2 Boost_1_40_0 .gitignore | 5 - CMakeLists.txt | 3 +- libs/graph/module.cmake | 2 +- libs/mpi/src/CMakeLists.txt | 1 - libs/serialization/test/CMakeLists.txt | 2 +- [etc etc] tools/inspect/CMakeLists.txt | 3 +- tools/wave/CMakeLists.txt | 36 ++ 97 files changed, 79 insertions(+), 11627 deletions(-) The above runs almost instantaneously, and that is just one example of the many things we use. I don't think we would be able to do what we're doing at all without this capability. -t
I wrote a "compare directories" script this weekend. Here's the output for svn revision 57505. The counts are "lines", so "3 insertions" means "3 lines inserted". I'm actively taking suggestions for improvement at this point. ;-) Summaries by file type, maybe? Links from the directory table to the files table? -- Marshall
I wrote a "compare directories" script this weekend.
Here's the output for svn revision 57505. The counts are "lines", so "3 insertions" means "3 lines inserted".
I'm actively taking suggestions for improvement at this point. ;-) Summaries by file type, maybe? Links from the directory table to the files table?
Cool stuff, Marshal. This helped me to untangle some older issues! Would it make sense to add a link to each changed file displaying the unified diff? Regards Hartmut ------------------- Meet me at BoostCon http://boostcon.com
On Nov 9, 2009, at 7:53 AM, Hartmut Kaiser wrote:
I wrote a "compare directories" script this weekend.
Here's the output for svn revision 57505. The counts are "lines", so "3 insertions" means "3 lines inserted".
I'm actively taking suggestions for improvement at this point. ;-) Summaries by file type, maybe? Links from the directory table to the files table?
Cool stuff, Marshal. This helped me to untangle some older issues! Would it make sense to add a link to each changed file displaying the unified diff?
It certainly would - at the cost of making the HTML file bigger. How much bigger, I don't know. I'll look into it. [ I'm assuming that you mean to add the diff to the bottom of the file, and include a link from the table - unless you know a way to get svn to disgorge a diff from a URL ] -- Marshall
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Monday 09 November 2009, Marshall Clow wrote:
[ I'm assuming that you mean to add the diff to the bottom of the file, and include a link from the table - unless you know a way to get svn to disgorge a diff from a URL ]
I'm not following too closely, so I'm not sure exactly what kind of diff you are talking about, but would a link to the trac browser work? For example: https://svn.boost.org/trac/boost/changeset/57505/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAkr4QhMACgkQ5vihyNWuA4WKOwCdFnO0PLsfZdOP3hu2ZuFCuhdu tPYAoLt6MXWPWQzwT9ucPRV7y11mh64f =bYpI -----END PGP SIGNATURE-----
On Nov 9, 2009, at 8:23 AM, Frank Mori Hess wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Monday 09 November 2009, Marshall Clow wrote:
[ I'm assuming that you mean to add the diff to the bottom of the file, and include a link from the table - unless you know a way to get svn to disgorge a diff from a URL ]
I'm not following too closely, so I'm not sure exactly what kind of diff you are talking about, but would a link to the trac browser work? For example:
I'd like to do it with a link to svn.boost.org (either trac or svn), but that's not it. Given three paths, such as: trunk/ release/ boost/mpl/aux_/preprocessor/partial_spec_params.hpp and a revision #, say 57505, I would like to generate a URL that shows the difference of the files trunk/boost/mpl/aux_/preprocessor/partial_spec_params.hpp and release/boost/mpl/aux_/preprocessor/partial_spec_params.hpp as of revision #57505. I can do this locally, and append it to the HTML file that I am generating, but that will make the file significantly larger. -- Marshall
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Monday 09 November 2009, Marshall Clow wrote:
I'd like to do it with a link to svn.boost.org (either trac or svn), but that's not it.
Given three paths, such as: trunk/ release/ boost/mpl/aux_/preprocessor/partial_spec_params.hpp
and a revision #, say 57505, I would like to generate a URL that shows the difference of the files trunk/boost/mpl/aux_/preprocessor/partial_spec_params.hpp and release/boost/mpl/aux_/preprocessor/partial_spec_params.hpp
as of revision #57505.
I think you can do that through the trac browser: https://svn.boost.org/trac/boost/changeset?old_path=%2Ftrunk%2Fboost%2Fmpl%2Faux_%2Fpreprocessor%2Fpartial_spec_params.hpp&old=57505&new_path=%2Fbranches%2Frelease%2Fboost%2Fmpl%2Faux_%2Fpreprocessor%2Fpartial_spec_params.hpp&new=57505 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAkr4RwIACgkQ5vihyNWuA4XzbwCgpmUKFSePzDxTmJ7JKu4xHMjN s7gAn2DbFXoxg1U8nfCsfa2cmvMDNIWZ =WNQ3 -----END PGP SIGNATURE-----
AMDG Marshall Clow wrote:
[ I'm assuming that you mean to add the diff to the bottom of the file, and include a link from the table - unless you know a way to get svn to disgorge a diff from a URL ]
You can get it from trac https://svn.boost.org/trac/boost/changeset?new_path=/trunk/libs/units&old_path=/branches/release/libs/units In Christ, Steven Watanabe
[ I'm assuming that you mean to add the diff to the bottom of the file, and include a link from the table - unless you know a way to get svn to disgorge a diff from a URL ]
You can get it from trac
That's what I thought the URL should be, but trying that with libs/regex or boost/math yields nonsense (it claims every file has changed - when they haven't - and links to diffs yield garbage!) Confused yours, John.
On Nov 9, 2009, at 8:39 AM, Steven Watanabe wrote:
AMDG
Marshall Clow wrote:
[ I'm assuming that you mean to add the diff to the bottom of the file, and include a link from the table - unless you know a way to get svn to disgorge a diff from a URL ]
You can get it from trac
Thanks, Steven (and Frank!) Here's an updated HTML file. One note: I'm using "diff" to calculate the changes, and sometimes it reports changes that "svn diff" does not (things like changed "$Id: " lines, for instance). So sometimes when you click on the link, you'll get "no changes". -- Marshall
On Mon, 2009-11-09 at 10:22 -0800, Marshall Clow wrote:
One note: I'm using "diff" to calculate the changes, and sometimes it reports changes that "svn diff" does not (things like changed "$Id: " lines, for instance). So sometimes when you click on the link, you'll get "no changes".
Is it GNU diff? If so, you might want to try diff -b -I '\$Id[^$]*\$' -I '\$Date[^$]*\$' -I '\$Revision[^$]*\$' (assuming unix shell; some other form of quoting otherwise...) This isn't perfect - it won't detect other changes on a line with a $Id$, but that would be fairly unusual. You might need to extend this with other SVN keywords. Also, if there are other changes in the same diff-block as the SVN keyword changes, then the ignore pattern doesn't actually appear to trim things out. (I've tested this with mpl/accumulate.hpp, and it silences the diffs.) The "-b" is just to ignore changes of whitespace (in my case, differences in line endings). Phil -- Phil Richards, <news@derived-software.ltd.uk>
On Nov 1, 2009, at 9:05 AM, troy d. straszheim wrote:
John Maddock wrote:
Good idea - but here's another suggestion - it would be great if someone could do an audit of release and trunk branches and see what hasn't been merged but should have been. I get the impression that some folks have committed fixes to Trunk and then forgotten all about them :-(
A quick diff shows:
1336 files changed, 28506 insertions(+), 66896 deletions(-)
A file by file summary is here:
http://sodium.resophonic.com/boost/diffstat.txt
Whitespace insensitive full diff here:
http://sodium.resophonic.com/boost/difffull-ignoring-whitespace.txt
Whitespace sensitive version:
http://sodium.resophonic.com/boost/difffull-including-whitespace.txt
This is a great idea! Of course, it needs some organization ;-) I'm thinking that it could be something like the inspection report, with summary counts by folder and/or library. I'm pretty busy at work for the next week; if no one has tackled this by next weekend, I'll take a swing at it. -- Marshall
Marshall Clow wrote:
On Nov 1, 2009, at 9:05 AM, troy d. straszheim wrote:
John Maddock wrote:
Good idea - but here's another suggestion - it would be great if someone could do an audit of release and trunk branches and see what hasn't been merged but should have been. I get the impression that some folks have committed fixes to Trunk and then forgotten all about them :-(
A quick diff shows:
1336 files changed, 28506 insertions(+), 66896 deletions(-)
A file by file summary is here:
http://sodium.resophonic.com/boost/diffstat.txt
Whitespace insensitive full diff here:
http://sodium.resophonic.com/boost/difffull-ignoring-whitespace.txt
Whitespace sensitive version:
http://sodium.resophonic.com/boost/difffull-including-whitespace.txt
This is a great idea!
Of course, it needs some organization ;-) I'm thinking that it could be something like the inspection report, with summary counts by folder and/or library.
I'm pretty busy at work for the next week; if no one has tackled this by next weekend, I'll take a swing at it.
This is very useful. When improving this, consider including the date of the last change in each branch. e.g. serializaition library trunk for each file changed lines difference - date of change ... total # files changed date of most recent change release ... same as above robert Ramey
Whitespace sensitive version:
http://sodium.resophonic.com/boost/difffull-including-whitespace.txt
This is a great idea!
Of course, it needs some organization ;-) I'm thinking that it could be something like the inspection report, with summary counts by folder and/or library.
I'm pretty busy at work for the next week; if no one has tackled this by next weekend, I'll take a swing at it.
I'm not sure if this is going to be possible... but what would really matter is *how long* have unmerged changed been in Trunk? There will always be changes that have "just missed" the last release, those we want to filter out if possible, changes that have missed several releases are more worrying! Keep up the good work, John.
On Sun, Nov 1, 2009 at 12:05 PM, troy d. straszheim <troy@resophonic.com>wrote:
John Maddock wrote:
Good idea - but here's another suggestion - it would be great if someone could do an audit of release and trunk branches and see what hasn't been merged but should have been. I get the impression that some folks have committed fixes to Trunk and then forgotten all about them :-(
A quick diff shows:
1336 files changed, 28506 insertions(+), 66896 deletions(-)
A file by file summary is here:
http://sodium.resophonic.com/boost/diffstat.txt
Whitespace insensitive full diff here:
http://sodium.resophonic.com/boost/difffull-ignoring-whitespace.txt
Whitespace sensitive version:
http://sodium.resophonic.com/boost/difffull-including-whitespace.txt
Hopefully this is helpful. I've tried to generate reports like this with svn and have been consistently unsuccessful, as the network tends to time out before the diff has completed.
I've pecked away at generating similar reports via svn. Haven't had any timeouts or other problems. The design I had in mind was to generate a web page listing the names of all libraries with unmerged changes, with each name linked to a diff for that particular library. The web page would be generated and posted daily, similar to the inspection report. Your reports are very helpful; they are a clear demonstration of how serious the unmerged changes problem is. Thanks! --Beman
Beman Dawes wrote:
Your reports are very helpful; they are a clear demonstration of how serious the unmerged changes problem is.
I'm not convinced yet that this is a "serious ... problem" Lot's of the files are html files. It seems possible that these have been generated from quickbook/docbook. In such a case a small change in the original source might result it all the generated files being changed. filtering out the qbk, html, htm and tex files leaves 517 files with unmerged changes. It would be interesting to know when the date of the last modification was. Robert Ramey
"Robert Ramey" <ramey@rrsd.com> wrote in message news:hcn2d3$45d$1@ger.gmane.org...
Beman Dawes wrote: filtering out the qbk, html, htm and tex files leaves 517 files with unmerged changes. It would be interesting to know when the date of the last modification was.
Robert Ramey
The more visible the report on the boost web-site and the more clear (i.e., marked in red, yellow or whatever) are the changes made to the trunk that have not been merged for release, the greater will be the weight on library authors to merge said changes. I think unmerged code changes should become more and more visible as they age... Just my humble opinion, Michael Goldshteyn
On Sun, Nov 1, 2009 at 11:39 AM, John Maddock <john@johnmaddock.co.uk>wrote:
Good idea - but here's another suggestion - it would be great if someone could do an audit of release and trunk branches and see what hasn't been merged but should have been. I get the impression that some folks have committed fixes to Trunk and then forgotten all about them :-(
Of course hopefully I'm wrong and there's nothing to actually do....
Failing to merge fixes into release is a major problem. For 1.41.0, I contacted three or four maintainers where I was very sure there were unmerged changes. That resulted in those libraries getting merged. I was going to set up a procedure to report unmerged libraries, but got distracted and never did it. We really need a regular, preferably automatic, way to detect and report unmerged libraries. --Beman
participants (12)
-
Beman Dawes -
Frank Mori Hess -
Hartmut Kaiser -
joel -
John Maddock -
Marshall Clow -
Michael Goldshteyn -
Phil Richards -
Robert Ramey -
Steven Watanabe -
Thomas Klimpel -
troy d. straszheim