Improving review process
Recently, there were various comments about current review process, and its possible improvements. However, I wanted to start with a small point. Looking at: http://www.boost.org/community/review_schedule.html it seems that a few libraries have a review manager assigned, but there's no date. This means that: - It looks like our review schedule is full, because the item is there, and - It prevents anybody to volunteer as review manager for such library. I think there might be several reasons why a library does not have a review date: - The library is not actually ready. In that case, it should not be in that table at all. - The library author does not have the time for review. In that case, the library should also be removed from the list, because Boost is not responsible if the author is busy. - The review manager does not have the time for review. In that case, he should not be listed as assigned, and should not block others. Can we set a policy that: - A library can only be added in the review schedule if the author has time in near future to have a review, where near future is, say, 3 months. - A review manager is only assigned if a review date is set at the same time, where the date should be in near future -- say, 3 months again. I think such a policy might not improve our overall review speed too much, but surely will make the situation a bit clear, and allow to understand the real problems with the review system. Thanks, -- Vladimir Prus http://vladimir_prus.blogspot.com Boost.Build: http://boost.org/boost-build2
Vladimir Prus wrote:
Recently, there were various comments about current review process, and its possible improvements. However, I wanted to start with a small point.
Looking at:
http://www.boost.org/community/review_schedule.html
it seems that a few libraries have a review manager assigned, but there's no date. This means that:
- It looks like our review schedule is full, because the item is there, and - It prevents anybody to volunteer as review manager for such library.
If there's a review manager assigned, then there's no need for another volunteer. If the review manager is the bottleneck, the library author is free to solicit another.
I think there might be several reasons why a library does not have a review date:
- The library is not actually ready. In that case, it should not be in that table at all.
Agreed
- The library author does not have the time for review. In that case, the library should also be removed from the list, because Boost is not responsible if the author is busy.
I'm not certain I agree. The name of the first table on that page is "SCHEDULE," yet only one library in it has a date. It is reasonable to manage two tables, one for scheduled and one for not-yet-scheduled libraries. That would retain the information currently shown in the "SCHEDULE" table while not implying that such libraries are scheduled.
- The review manager does not have the time for review. In that case, he should not be listed as assigned, and should not block others.
I can almost agree with that. Perhaps the review manager could be flagged as "tentative" rather than removing the name altogether. Perhaps there should be several tables: scheduled, review manager needed, not yet scheduled. If a person is willing to be a review manager, but currently doesn't have time for a review, that library could be put in the "review manager needed" table which can include a "tentative review manager" column listing the current volunteer.
Can we set a policy that:
- A library can only be added in the review schedule if the author has time in near future to have a review, where near future is, say, 3 months.
I would agree to that if there was also a not-yet-scheduled table to which such libraries can be moved.
- A review manager is only assigned if a review date is set at the same time, where the date should be in near future -- say, 3 months again.
I disagree. A review manager should be assigned, even if marked tentative, as soon as a commitment has been made. It looks far better for a potential library that there is someone interested in promoting the library and its review. _____ Rob Stewart robert.stewart@sig.com Software Engineer, Core Software using std::disclaimer; Susquehanna International Group, LLP http://www.sig.com IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.
On Thu, Jan 13, 2011 at 8:57 AM, Stewart, Robert <Robert.Stewart@sig.com> wrote:
Vladimir Prus wrote:
Recently, there were various comments about current review process, and its possible improvements. However, I wanted to start with a small point.
Looking at:
http://www.boost.org/community/review_schedule.html
it seems that a few libraries have a review manager assigned, but there's no date. This means that:
- It looks like our review schedule is full, because the item is there, and - It prevents anybody to volunteer as review manager for such library.
If there's a review manager assigned, then there's no need for another volunteer. If the review manager is the bottleneck, the library author is free to solicit another.
I think I agree with Volodya here.
I think there might be several reasons why a library does not have a review date:
- The library is not actually ready. In that case, it should not be in that table at all.
Agreed
- The library author does not have the time for review. In that case, the library should also be removed from the list, because Boost is not responsible if the author is busy.
I'm not certain I agree. The name of the first table on that page is "SCHEDULE," yet only one library in it has a date. It is reasonable to manage two tables, one for scheduled and one for not-yet-scheduled libraries. That would retain the information currently shown in the "SCHEDULE" table while not implying that such libraries are scheduled.
I don't think information about not-yet-ready libraries has any public value in the context of the review queue. Boost and its review wizards should not be responsible for maintaining that, except perhaps to clean it up!
- The review manager does not have the time for review. In that case, he should not be listed as assigned, and should not block others.
I can almost agree with that. Perhaps the review manager could be flagged as "tentative" rather than removing the name altogether. Perhaps there should be several tables: scheduled, review manager needed, not yet scheduled. If a person is willing to be a review manager, but currently doesn't have time for a review, that library could be put in the "review manager needed" table which can include a "tentative review manager" column listing the current volunteer.
Again, I don't think that information has any public value in the context of a review queue.
Can we set a policy that:
- A library can only be added in the review schedule if the author has time in near future to have a review, where near future is, say, 3 months.
I would agree to that if there was also a not-yet-scheduled table to which such libraries can be moved.
Anybody who wants to can maintain such a table, but IMO it should *not* be associated with the review queue or maintained by the same people.
- A review manager is only assigned if a review date is set at the same time, where the date should be in near future -- say, 3 months again.
I disagree. A review manager should be assigned, even if marked tentative, as soon as a commitment has been made. It looks far better for a potential library that there is someone interested in promoting the library and its review.
I disagree; I don't think promoting submissions should be considered part of the role of the review queue. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
Dave Abrahams wrote:
On Thu, Jan 13, 2011 at 8:57 AM, Stewart, Robert <Robert.Stewart@sig.com> wrote:
Vladimir Prus wrote:
Recently, there were various comments about current review process, and its possible improvements. However, I wanted to start with a small point.
Looking at:
http://www.boost.org/community/review_schedule.html
it seems that a few libraries have a review manager assigned, but there's no date. This means that:
- It looks like our review schedule is full, because the item is there, and - It prevents anybody to volunteer as review manager for such library.
[snip my replies suggesting additional tables about tentative review managers, etc.]
I don't think that information has any public value in the context of a review queue.
The information I sought is captured, less succinctly, in <https://svn.boost.org/trac/boost/wiki/ReviewScheduleLibraries>. I hadn't realized there was such a page in the Wiki. You are right that the Review Wizards should only have to manage current and likely review candidates on the reviews queue. _____ Rob Stewart robert.stewart@sig.com Software Engineer, Core Software using std::disclaimer; Susquehanna International Group, LLP http://www.sig.com IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.
13.01.2011 17:25, Vladimir Prus пишет:
- The library is not actually ready. In that case, it should not be in that table at all. I agree. If the library is not ready - why should it be in that table? - The library author does not have the time for review. In that case, the library should also be removed from the list, because Boost is not responsible if the author is busy. I agree. Can we set a policy that:
- A library can only be added in the review schedule if the author has time in near future to have a review, where near future is, say, 3 months. Hmm... Why 3 months?
- Denis
----- Original Message ----- From: "Vladimir Prus" <ghost@cs.msu.su> To: <boost@lists.boost.org> Sent: Thursday, January 13, 2011 2:25 PM Subject: [boost] Improving review process
Recently, there were various comments about current review process, and its possible improvements. However, I wanted to start with a small point.
Looking at:
http://www.boost.org/community/review_schedule.html
it seems that a few libraries have a review manager assigned, but there's no date. This means that:
- It looks like our review schedule is full, because the item is there, and - It prevents anybody to volunteer as review manager for such library.
I think there might be several reasons why a library does not have a review date:
- The library is not actually ready. In that case, it should not be in that table at all.
+1. Another case could be that the library depends on another library not yet scheduled. I'm the review manager for Boost.Task (which was Boost.ThreadPool when I proposed myself). Boost.Task depends now on Boost.Fiber and Boost.Context (In addition these libraries depend as a implementation detail on Boost.Atomic which is not on the review Schedule. I don't know if these dependent libraries should be removed. Maybe it is worth to state cleary in the page why the review is not yet scheduled.
- The library author does not have the time for review. In that case, the library should also be removed from the list, because Boost is not responsible if the author is busy. - The review manager does not have the time for review. In that case, he should not be listed as assigned, and should not block others.
There is a particular case for the Boost.Log, which has a review manager but that links yet to the version that was rejected.
Can we set a policy that:
- A library can only be added in the review schedule if the author has time in near future to have a review, where near future is, say, 3 months.
IMO this was already the case. the author can be ready but without review manager.
- A review manager is only assigned if a review date is set at the same time, where the date should be in near future -- say, 3 months again.
I would say that the author and the review manager must set a date in the near future, let say 3 months and that this date must be in the near future, let say 3 months. IMO, a library that has no review manager before let me say 6 month should be removed from the list as there is no hard interest.
I think such a policy might not improve our overall review speed too much, but surely will make the situation a bit clear, and allow to understand the real problems with the review system.
I agree and will add that the best we can do is to know why the review is not scheduled or released, A library in the list could be in one of the following states: * locking for review manager (up to 6 months -- to force the author to probe the lib is interesting) * locking for a date and making it ready for review (up to 3 months -- needed as the review manager could have some specific requests to the author before the lib is ready for review) * scheduled (up to 3 months -- tomanage with author and review manager specific imperatives) * review ongoing * result pending (up to 1 month -- to force a rapid veredict) * accepted but not yet in trunk (up to 2 months -- the time to setup all the needed infrastructure) * moved to trunk (up to 3 months -- the time to make the library portable) * moved to release branch (up to 3 months -- the time of a release) Of course all the periods are subject to discussion, but the fact to have them should surely improve the time between the moment the author propose the library and the time the lib is released. Best, Vicente
On 13/01/2011 16:49, vicente.botet wrote:
- A review manager is only assigned if a review date is set at the same time, where the date should be in near future -- say, 3 months again.
I would say that the author and the review manager must set a date in the near future, let say 3 months and that this date must be in the near future, let say 3 months.
IMO, a library that has no review manager before let me say 6 month should be removed from the list as there is no hard interest.
Is that necessarily the case? There's a difference between interest and time available from those who consider themselves qualified to be review managers, and interest from end user of a library. Take an example of Xint. There were plenty of comments, suggestion and review the various time it was "beta'd" on this list asking for comments. And that's not counting the fact that a arbitrary precision integer library has traditionally been a perennial topic of suggestion for GSOC ideas. Yet, almost exactly 6 months to date, it still hasn't had a review manager assigned. Is that really a lack of interest in such a library, or more a lack of review managers? KTC
----- Original Message ----- From: "KTC" <ktc@ktchan.info> To: <boost@lists.boost.org> Sent: Thursday, January 13, 2011 6:13 PM Subject: Re: [boost] Improving review process
On 13/01/2011 16:49, vicente.botet wrote:
- A review manager is only assigned if a review date is set at the same time, where the date should be in near future -- say, 3 months again.
I would say that the author and the review manager must set a date in the near future, let say 3 months and that this date must be in the near future, let say 3 months.
IMO, a library that has no review manager before let me say 6 month should be removed from the list as there is no hard interest.
Is that necessarily the case? There's a difference between interest and time available from those who consider themselves qualified to be review managers, and interest from end user of a library.
Interest from the end users is not enoght. And 6 month is quite a lot of time to look for available time.
Take an example of Xint. There were plenty of comments, suggestion and review the various time it was "beta'd" on this list asking for comments. And that's not counting the fact that a arbitrary precision integer library has traditionally been a perennial topic of suggestion for GSOC ideas. Yet, almost exactly 6 months to date, it still hasn't had a review manager assigned. Is that really a lack of interest in such a library, or more a lack of review managers?
I would say a lack of interest of someone that could be a review manager :) IMO, one of the author's role is to look for a review manager between the people having an interest for the library. Vicente
On Thu, 13 Jan 2011 18:29:54 +0100 "vicente.botet" <vicente.botet@wanadoo.fr> wrote: [...]
Take an example of Xint. There were plenty of comments, suggestion and review the various time it was "beta'd" on this list asking for comments. And that's not counting the fact that a arbitrary precision integer library has traditionally been a perennial topic of suggestion for GSOC ideas. Yet, almost exactly 6 months to date, it still hasn't had a review manager assigned. Is that really a lack of interest in such a library, or more a lack of review managers?
I would say a lack of interest of someone that could be a review manager :) IMO, one of the author's role is to look for a review manager between the people having an interest for the library.
Yes, and I've been remiss in that department. Not for lack of desire, just lack of knowledge of how to go about it. So here's the request: would someone who's interested in XInt please step up and volunteer to be the review manager for it? I'm ready to help in any way I can. To refresh peoples' memory, XInt is the eXtended Integer library, a set of classes and functions for storing and manipulating numbers that can be much, *much* larger than an int, long, or even long long type can handle. It's designed for correctness and maximum portability, *not* maximum speed, so it's pure C++ targeted at any modern CPU and operating system (no GPU or assembly-language code, at least at this point). The library is very extensively documented, and the major parts of it require no specialized math knowledge -- it's pure arithmetic, though the details occasionally get complex. The only parts that might require some domain-specific knowledge to completely grok are the prime-number and (maybe) the modular math stuff. Boost has needed something like XInt for a long time now. It's general-purpose code that a sizable minority of programmers need at least occasionally; it's hard to do properly and all but impossible to do quickly; and the solutions that are available have licensing issues that make them unsuitable for many programs, and/or aren't C++ or aren't very well designed. That's the perfect recipe for a Boost library. At the risk of sounding like a military recruiting poster: step up and do your part for Boost! ;-) -- Chad Nelson Oak Circle Software, Inc. * * *
Chad Nelson wrote:
On Thu, 13 Jan 2011 18:29:54 +0100 "vicente.botet" <vicente.botet@wanadoo.fr> wrote:
[...]
Take an example of Xint. There were plenty of comments, suggestion and review the various time it was "beta'd" on this list asking for comments. And that's not counting the fact that a arbitrary precision integer library has traditionally been a perennial topic of suggestion for GSOC ideas. Yet, almost exactly 6 months to date, it still hasn't had a review manager assigned. Is that really a lack of interest in such a library, or more a lack of review managers?
I would say a lack of interest of someone that could be a review manager :) IMO, one of the author's role is to look for a review manager between the people having an interest for the library.
Yes, and I've been remiss in that department. Not for lack of desire, just lack of knowledge of how to go about it.
So here's the request: would someone who's interested in XInt please step up and volunteer to be the review manager for it? I'm ready to help in any way I can.
How about me? I don't pretend to be an expert in integers, and will gladly surrender to anybody else interested, but would be happy to help xint to be reviewed. And yes, I think we can set a date within next 3 months ;-) - Volodya -- Vladimir Prus Mentor Graphics +7 (812) 677-68-40
On Thu, 13 Jan 2011 21:47:46 +0300 Vladimir Prus <vladimir@codesourcery.com> wrote:
Chad Nelson wrote:
So here's the request: would someone who's interested in XInt please step up and volunteer to be the review manager for it? I'm ready to help in any way I can.
How about me? [...]
Looks like you're nominated. :-) Taking the discussion to e-mail. Thanks! -- Chad Nelson Oak Circle Software, Inc. * * *
On 1/13/2011 6:09 PM, Chad Nelson wrote:
On Thu, 13 Jan 2011 21:47:46 +0300 Vladimir Prus<vladimir@codesourcery.com> wrote:
Chad Nelson wrote:
So here's the request: would someone who's interested in XInt please step up and volunteer to be the review manager for it? I'm ready to help in any way I can.
How about me? [...]
Looks like you're nominated. :-) Taking the discussion to e-mail. Thanks!
Let Ron and I know when you have ideas for a start date. John
On Thu, 13 Jan 2011 21:22:37 -0500 John Phillips <phillips@delos.mps.ohio-state.edu> wrote:
Chad Nelson wrote:
So here's the request: would someone who's interested in XInt please step up and volunteer to be the review manager for it? I'm ready to help in any way I can.
How about me? [...]
Looks like you're nominated. :-) Taking the discussion to e-mail. Thanks!
Let Ron and I know when you have ideas for a start date.
Will do. We're discussing it now, but we probably won't have a date for at least another week. -- Chad Nelson Oak Circle Software, Inc. * * *
vicente.botet wrote:
IMO, one of the author's role is to look for a review manager
I'd just like to insert a small note of caution, that the review manager needs to be somewhat neutral about the library being reviewed. Having the library author choose their review manager makes it more likely that they will select someone who is predisposed in their favour. Ideally we would have multiple candidates for each post and the wizards would choose someone based on their domain expertise. In the absence of a queue of suitable candidates, of course it's fine for the library author to encourage people - but I would prefer for that to happen in public. Phil.
On 1/13/2011 2:20 PM, Phil Endecott wrote:
vicente.botet wrote:
IMO, one of the author's role is to look for a review manager
I'd just like to insert a small note of caution, that the review manager needs to be somewhat neutral about the library being reviewed. Having the library author choose their review manager makes it more likely that they will select someone who is predisposed in their favour. Ideally we would have multiple candidates for each post and the wizards would choose someone based on their domain expertise. In the absence of a queue of suitable candidates, of course it's fine for the library author to encourage people - but I would prefer for that to happen in public.
Phil.
In general, the Review Wizards discuss Manager candidates. On some occasions we have decided a potential Manager was not a good choice at that time. We discuss whether we think the potential manager has the technical skills to understand the issues in the review (The Manager does not need to be an expert in the field of the library, but does need to understand issues as they come up in the review.), and whether the manager is a good choice for providing a fair and active review period and evaluation. In specific, on the couple of instances where a conflict of interest seemed possible, there was an off line discussion with the potential Manager. We also encourage authors to beat the bushes for potential Managers, and we encourage interested parties to volunteer, but we view part of our role as quality control on the Manager selection, and we take that seriously. John
vicente.botet wrote:
- A review manager is only assigned if a review date is set at the same time, where the date should be in near future -- say, 3 months again.
I would say that the author and the review manager must set a date in the near future, let say 3 months and that this date must be in the near future, let say 3 months.
IMO, a library that has no review manager before let me say 6 month should be removed from the list as there is no hard interest.
Is that necessarily the case? There's a difference between interest and time available from those who consider themselves qualified to be review managers, and interest from end user of a library.
Interest from the end users is not enoght. And 6 month is quite a lot of time to look for available time.
In my opinion a library should not even be added to the review queue unless a review manager is found and assigned. vicente.botet wrote:
I would say a lack of interest of someone that could be a review manager :) IMO, one of the author's role is to look for a review manager between the people having an interest for the library.
I agree that it is the library author's responsibility to find a review manager, not the boost community. Based on what we see happening it is clear that the libraries that get reviewed and eventually accepted are the ones where the author is active and successful in finding a review manager for themselves. KTC wrote:
Take an example of Xint. There were plenty of comments, suggestion and review the various time it was "beta'd" on this list asking for comments. And that's not counting the fact that a arbitrary precision integer library has traditionally been a perennial topic of suggestion for GSOC ideas. Yet, almost exactly 6 months to date, it still hasn't had a review manager assigned. Is that really a lack of interest in such a library, or more a lack of review managers?
I have participated in many arbitrary precision integer library discussions on this list and am using gmp with my own library. Why have I never gotten an email asking me to be the review manager for Xint? Has Hartmut been asked? Has Thomas Klimpel been asked? Has Beman, has Berend? You have to ask individual people directly (preferably in person at boostcon) and you will find that they are often happy to be a review manager. Adding a library to the review queue and complaining you can't get a review manager isn't the same thing as asking a person to be your review manager and the differnce is this: one way works and the other way does not. Regards, Luke
Simonson, Lucanus J wrote:
I agree that it is the library author's responsibility to find a review manager, not the boost community. Based on what we see happening it is clear that the libraries that get reviewed and eventually accepted are the ones where the author is active and successful in finding a review manager for themselves.
There's a big deal of truth in that.
KTC wrote:
Take an example of Xint. There were plenty of comments, suggestion and review the various time it was "beta'd" on this list asking for comments. And that's not counting the fact that a arbitrary precision integer library has traditionally been a perennial topic of suggestion for GSOC ideas. Yet, almost exactly 6 months to date, it still hasn't had a review manager assigned. Is that really a lack of interest in such a library, or more a lack of review managers?
I have participated in many arbitrary precision integer library discussions on this list and am using gmp with my own library. Why have I never gotten an email asking me to be the review manager for Xint? Has Hartmut been asked? Has Thomas Klimpel been asked? Has Beman, has Berend? You have to ask individual people directly (preferably in person at boostcon) and you will find that they are often happy to be a review manager. Adding a library to the review queue and complaining you can't get a review manager isn't the same thing as asking a person to be your review manager and the differnce is this: one way works and the other way does not.
There are two ways: - Have it be author's responsibility to ask potential review managers - Have somebody else ask folks The latter approach might be better because "somebody else" might be more involved in Boost, and therefore know the best folks to review something. - Volodya -- Vladimir Prus Mentor Graphics +7 (812) 677-68-40
I agree that it is the library author's responsibility to find a review manager, not the boost community. Based on what we see happening it is clear that the libraries that get reviewed and eventually accepted are the ones where the author is active and successful in finding a review manager for themselves.
There's a big deal of truth in that.
Indeed, and looking at https://svn.boost.org/trac/boost/wiki/ReviewScheduleLibraries we might not be in such bad shape (no really!!), ignoring those reviewed and awaiting results we currently have: * 4 scheduled for review. So I guess the process is working for these :) * 5 stalled waiting on other reviews (more than a few of these may well become unblocked soon). I don't have a problem with this, it's very clear on the page why they're blocked, and it's up to these authors to chivvy along the blocking proposal when required :) * 9 stalled waiting for a review manager - I get the sense that most of these are small submissions - perhaps a few of questionable value - and a few that seem to overlap. Perhaps we should have a "review sprint" and get some of the smaller submissions out of the way en mass? Cheers, John.
On Jan 13, 2011, at 10:59 AM, John Maddock wrote:
I agree that it is the library author's responsibility to find a review manager, not the boost community. Based on what we see happening it is clear that the libraries that get reviewed and eventually accepted are the ones where the author is active and successful in finding a review manager for themselves.
There's a big deal of truth in that.
Indeed, and looking at https://svn.boost.org/trac/boost/wiki/ReviewScheduleLibraries we might not be in such bad shape (no really!!), ignoring those reviewed and awaiting results we currently have:
* 4 scheduled for review. So I guess the process is working for these :) * 5 stalled waiting on other reviews (more than a few of these may well become unblocked soon). I don't have a problem with this, it's very clear on the page why they're blocked, and it's up to these authors to chivvy along the blocking proposal when required :)
I've got one (Boost.Algorithm) that I should be posting RSN.... -- Marshall Marshall Clow Idio Software <mailto:mclow.lists@gmail.com> A.D. 1517: Martin Luther nails his 95 Theses to the church door and is promptly moderated down to (-1, Flamebait). -- Yu Suzuki
John Maddock wrote:
I agree that it is the library author's responsibility to find a review manager, not the boost community. Based on what we see happening it is clear that the libraries that get reviewed and eventually accepted are the ones where the author is active and successful in finding a review manager for themselves.
There's a big deal of truth in that.
Indeed, and looking at https://svn.boost.org/trac/boost/wiki/ReviewScheduleLibraries we might not be in such bad shape (no really!!), ignoring those reviewed and awaiting results we currently have:
* 4 scheduled for review. So I guess the process is working for these :) * 5 stalled waiting on other reviews (more than a few of these may well become unblocked soon). I don't have a problem with this, it's very clear on the page why they're blocked, and it's up to these authors to chivvy along the blocking proposal when required :) * 9 stalled waiting for a review manager - I get the sense that most of these are small submissions - perhaps a few of questionable value - and a few that seem to overlap. Perhaps we should have a "review sprint" and get some of the smaller submissions out of the way en mass?
Note that XInt is not listed on that page ;-) Do we need to have two pages with review schedule? - Volodya -- Vladimir Prus Mentor Graphics +7 (812) 677-68-40
Indeed, and looking at https://svn.boost.org/trac/boost/wiki/ReviewScheduleLibraries we might not be in such bad shape (no really!!), ignoring those reviewed and awaiting results we currently have:
* 4 scheduled for review. So I guess the process is working for these :) * 5 stalled waiting on other reviews (more than a few of these may well become unblocked soon). I don't have a problem with this, it's very clear on the page why they're blocked, and it's up to these authors to chivvy along the blocking proposal when required :) * 9 stalled waiting for a review manager - I get the sense that most of these are small submissions - perhaps a few of questionable value - and a few that seem to overlap. Perhaps we should have a "review sprint" and get some of the smaller submissions out of the way en mass?
Note that XInt is not listed on that page ;-) Do we need to have two pages with review schedule?
- Volodya
And Boost.Locale Artyom
On Thu, 13 Jan 2011 10:19:40 -0800 "Simonson, Lucanus J" <lucanus.j.simonson@intel.com> wrote:
KTC wrote:
Take an example of Xint. [...]
I have participated in many arbitrary precision integer library discussions on this list and am using gmp with my own library. Why have I never gotten an email asking me to be the review manager for Xint? [...]
Because I didn't know to ask, and didn't know who or even how until a couple months ago. But if you're interested, I'm asking now. :-) -- Chad Nelson Oak Circle Software, Inc. * * *
Chad Nelson wrote:
On Thu, 13 Jan 2011 10:19:40 -0800 "Simonson, Lucanus J" <lucanus.j.simonson@intel.com> wrote:
KTC wrote:
Take an example of Xint. [...]
I have participated in many arbitrary precision integer library discussions on this list and am using gmp with my own library. Why have I never gotten an email asking me to be the review manager for Xint? [...]
Because I didn't know to ask, and didn't know who or even how until a couple months ago. But if you're interested, I'm asking now. :-)
I think Vlad just volunteered, so consider me a backup. I will at the very least participate in the review. So, it is important for everyone to understand what the review process is for. It isn't just to decide if the code belongs in boost. It is also to decide whether the author is willing and able to maintain the library and contribute to the community over the long term. If you need help with a technical problem with your library you could ask the list, but you could also ask individuals (other boost authors) that you know have expertise in the area for advice or even direct help with you problem. If I know Steven Watanabe might be able to help me I post to the list and put his name in the post suggesting he might know the answer to some question. I know he reads the list and nothing gets a person's attention like seeing their own name. For other members of the boost community that I know don't follow the list closely (most don't, I don't) I email them directly so that I can be sure they see the email. No one comes to boost knowing the habits and background of various boost library authors, it takes time participating in the community to build relationships that you can draw on to be effective and it also takes time to build the experience working with the community to know how to draw on those relationships. There is no change, no fix to the review process, that we can make that will be a substitute for that time. Personally I think building relationships with our peers is more important and of more benefit to boost library authors than having a library that goes into the boost release. After my library was accepted and after it was released there was no big change. I wasn't different, boost wasn't different. What was a valuable result of it was what I learned along the way and the network of friends and colleagues I developed. My advice is to not be impatient and learn to value what is really important in all of this. Regards, Luke
vicente.botet wrote:
I don't know if these dependent libraries should be removed. Maybe it is worth to state cleary in the page why the review is not yet scheduled.
I think you're right.
- The library author does not have the time for review. In that case, the library should also be removed from the list, because Boost is not responsible if the author is busy. - The review manager does not have the time for review. In that case, he should not be listed as assigned, and should not block others.
There is a particular case for the Boost.Log, which has a review manager but that links yet to the version that was rejected.
Just to clarify -- you mean Boost.Logging? There's Boost.Log as well, which is accepted (provisionally). It would be nice to sort out Boost.Logging situation, indeed.
Can we set a policy that:
- A library can only be added in the review schedule if the author has time in near future to have a review, where near future is, say, 3 months.
IMO this was already the case. the author can be ready but without review manager.
- A review manager is only assigned if a review date is set at the same time, where the date should be in near future -- say, 3 months again.
I would say that the author and the review manager must set a date in the near future, let say 3 months and that this date must be in the near future, let say 3 months.
IMO, a library that has no review manager before let me say 6 month should be removed from the list as there is no hard interest.
This might be controversial. Maybe, some poking on the mailing list should be done before declaring that no review manager can be found?
I think such a policy might not improve our overall review speed too much, but surely will make the situation a bit clear, and allow to understand the real problems with the review system.
I agree and will add that the best we can do is to know why the review is not scheduled or released, A library in the list could be in one of the following states:
* locking for review manager (up to 6 months -- to force the author to probe the lib is interesting) * locking for a date and making it ready for review (up to 3 months -- needed as the review manager could have some specific requests to the author before the lib is ready for review) * scheduled (up to 3 months -- tomanage with author and review manager specific imperatives) * review ongoing * result pending (up to 1 month -- to force a rapid veredict) * accepted but not yet in trunk (up to 2 months -- the time to setup all the needed infrastructure) * moved to trunk (up to 3 months -- the time to make the library portable) * moved to release branch (up to 3 months -- the time of a release)
This might be a bit too many states for one table. How about having several tables: 1. Scheduled. Libraries that have review manager, were examined by review manager for obvious issues, and have a review date set (within next 3 months). 2. Ready. Libraries that were submitted for review, are considered ready by their authors, and only wait for a review manager. 3. Already reviewed. This set excludes your 'making it ready for review' state. It seems to me that if the library is not ready for review -- for example because it was submitted but somebody found serious problems with it -- should not be tracked by review wizards at all. - Volodya -- Vladimir Prus Mentor Graphics +7 (812) 677-68-40
On Thu, Jan 13, 2011 at 11:49 AM, vicente.botet <vicente.botet@wanadoo.fr> wrote:
Another case could be that the library depends on another library not yet scheduled.
That's actually no reason to hold up the review. Any dependencies can be recoded as implementation details for final submission until they're ready to be reviewed on their own. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
----- Original Message ----- From: "Dave Abrahams" <dave@boostpro.com> To: <boost@lists.boost.org> Sent: Thursday, January 13, 2011 10:29 PM Subject: Re: [boost] Improving review process
On Thu, Jan 13, 2011 at 11:49 AM, vicente.botet <vicente.botet@wanadoo.fr> wrote:
Another case could be that the library depends on another library not yet scheduled.
That's actually no reason to hold up the review. Any dependencies can be recoded as implementation details for final submission until they're ready to be reviewed on their own.
You are right if the dependency is an implementation detail, not if the dependency appears on the interface. Vicente
At Thu, 13 Jan 2011 23:26:32 +0100, vicente.botet wrote:
----- Original Message ----- From: "Dave Abrahams" <dave@boostpro.com> To: <boost@lists.boost.org> Sent: Thursday, January 13, 2011 10:29 PM Subject: Re: [boost] Improving review process
On Thu, Jan 13, 2011 at 11:49 AM, vicente.botet <vicente.botet@wanadoo.fr> wrote:
Another case could be that the library depends on another library not yet scheduled.
That's actually no reason to hold up the review. Any dependencies can be recoded as implementation details for final submission until they're ready to be reviewed on their own.
You are right if the dependency is an implementation detail, not if the dependency appears on the interface.
Meh. You can always alias the dependency into the library under review for the purposes of the interface. In other words, instead of using boost::bar::x in the interface of boost::foo::f, use boost::foo::bar::x. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On 1/13/2011 8:42 PM, Dave Abrahams wrote:
At Thu, 13 Jan 2011 23:26:32 +0100, vicente.botet wrote:
----- Original Message ----- From: "Dave Abrahams"<dave@boostpro.com> To:<boost@lists.boost.org> Sent: Thursday, January 13, 2011 10:29 PM Subject: Re: [boost] Improving review process
On Thu, Jan 13, 2011 at 11:49 AM, vicente.botet <vicente.botet@wanadoo.fr> wrote:
Another case could be that the library depends on another library not yet scheduled.
That's actually no reason to hold up the review. Any dependencies can be recoded as implementation details for final submission until they're ready to be reviewed on their own.
You are right if the dependency is an implementation detail, not if the dependency appears on the interface.
Meh. You can always alias the dependency into the library under review for the purposes of the interface. In other words, instead of using boost::bar::x in the interface of boost::foo::f, use boost::foo::bar::x.
I think the idea is that if library X depends on library Y, and library X gets reviewed before library Y, and library X gets accepted then: 1) Library X has to wait for library Y to be reviewed and Library Y has to then get accepted into Boost. or 2) Library X has to duplicate the functionality of whatever it uses in Library Y. This does create some problems especially if library X heavily depends on Library Y and Library Y is not accepted into Boost.
At Thu, 13 Jan 2011 21:49:10 -0500, Edward Diener wrote:
On 1/13/2011 8:42 PM, Dave Abrahams wrote:
At Thu, 13 Jan 2011 23:26:32 +0100, vicente.botet wrote:
You are right if the dependency is an implementation detail, not if the dependency appears on the interface.
Meh. You can always alias the dependency into the library under review for the purposes of the interface. In other words, instead of using boost::bar::x in the interface of boost::foo::f, use boost::foo::bar::x.
I think the idea is that...
I understand your point, Edward. I just think there are tractable (if slightly ugly) solutions available.
if library X depends on library Y, and library X gets reviewed before library Y, and library X gets accepted then:
1) Library X has to wait for library Y to be reviewed and Library Y has to then get accepted into Boost.
or
2) Library X has to duplicate the functionality of whatever it uses in Library Y.
I would rather say, "import" than "duplicate."
This does create some problems especially if library X heavily depends on Library Y and Library Y is not accepted into Boost.
Just do it the way Spirit/Fusion did it. When X is reviewed, make the parts of Y you need look like a sub-part of X. When Y comes up for review on its own, you adjust, implementing some backward-compatibility glue as needed. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On 1/14/2011 6:56 AM, Dave Abrahams wrote:
At Thu, 13 Jan 2011 21:49:10 -0500, Edward Diener wrote:
On 1/13/2011 8:42 PM, Dave Abrahams wrote:
At Thu, 13 Jan 2011 23:26:32 +0100, vicente.botet wrote:
You are right if the dependency is an implementation detail, not if the dependency appears on the interface.
Meh. You can always alias the dependency into the library under review for the purposes of the interface. In other words, instead of using boost::bar::x in the interface of boost::foo::f, use boost::foo::bar::x.
I think the idea is that...
I understand your point, Edward. I just think there are tractable (if slightly ugly) solutions available.
if library X depends on library Y, and library X gets reviewed before library Y, and library X gets accepted then:
1) Library X has to wait for library Y to be reviewed and Library Y has to then get accepted into Boost.
or
2) Library X has to duplicate the functionality of whatever it uses in Library Y.
I would rather say, "import" than "duplicate."
I do not understand the distinction you are making.
This does create some problems especially if library X heavily depends on Library Y and Library Y is not accepted into Boost.
Just do it the way Spirit/Fusion did it. When X is reviewed, make the parts of Y you need look like a sub-part of X. When Y comes up for review on its own, you adjust, implementing some backward-compatibility glue as needed.
I agree with that completely, and that's the way I design also. Still, in the scenario above, if library Y is not accepted there is work to be done in library X to implement under the hood whatever functionality from library Y it needed. But that's still not enough of a reason for library X to have to wait until library Y gets reviewed for it to get reviewed, and I think that was the point of your response.
At Fri, 14 Jan 2011 08:43:39 -0500, Edward Diener wrote:
2) Library X has to duplicate the functionality of whatever it uses in Library Y.
I would rather say, "import" than "duplicate."
I do not understand the distinction you are making.
It's not duplicate code if the code doesn't already exist in Boost.
This does create some problems especially if library X heavily depends on Library Y and Library Y is not accepted into Boost.
Just do it the way Spirit/Fusion did it. When X is reviewed, make the parts of Y you need look like a sub-part of X. When Y comes up for review on its own, you adjust, implementing some backward-compatibility glue as needed.
I agree with that completely, and that's the way I design also.
Still, in the scenario above, if library Y is not accepted there is work to be done in library X to implement under the hood whatever functionality from library Y it needed. But that's still not enough of a reason for library X to have to wait until library Y gets reviewed for it to get reviewed, and I think that was the point of your response.
Yes. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
- A library can only be added in the review schedule if the author has time in near future to have a review, where near future is, say, 3 months.
well, it would be great if the review would actually happen within a time frame of 3 months. unfortunately, it is a bit different in reality: the boost.lockfree library is on the review queue for more than a year. with 6 reviews per year (like in 2010) and 15 libraries earlier in the queue, i don't expect a review to happen before 2013. in a way, this is kind of nice since the dependency to boost.atomic (c++0x-atomics for c++-98) will hopefully be obsolete since compilers may already be reasonably c++0x-compliant. so a time frame of 3 months is probably more a time frame of 3 years :/ tim -- tim@klingt.org http://tim.klingt.org Music is the can opener of the soul. It makes you terribly quiet inside, makes you aware that there's a roof to your being. Henry Miller
Tim Blechmann wrote:
- A library can only be added in the review schedule if the author has time in near future to have a review, where near future is, say, 3 months.
well, it would be great if the review would actually happen within a time frame of 3 months. unfortunately, it is a bit different in reality: the boost.lockfree library is on the review queue for more than a year. with 6 reviews per year (like in 2010) and 15 libraries earlier in the queue, i don't expect a review to happen before 2013. in a way, this is kind of nice since the dependency to boost.atomic (c++0x-atomics for c++-98) will hopefully be obsolete since compilers may already be reasonably c++0x-compliant.
so a time frame of 3 months is probably more a time frame of 3 years :/
I am not suggesting that any newly submitted library can possibly be reviewed in 3 months. I am just suggesting that if a review manager is set, then the date is also set within next 3 months to avoid having a library in the limbo state. -- Vladimir Prus Mentor Graphics +7 (812) 677-68-40
On 1/13/2011 12:26 PM, Tim Blechmann wrote:
- A library can only be added in the review schedule if the author has time in near future to have a review, where near future is, say, 3 months.
well, it would be great if the review would actually happen within a time frame of 3 months. unfortunately, it is a bit different in reality: the boost.lockfree library is on the review queue for more than a year. with 6 reviews per year (like in 2010) and 15 libraries earlier in the queue, i don't expect a review to happen before 2013. in a way, this is kind of nice since the dependency to boost.atomic (c++0x-atomics for c++-98) will hopefully be obsolete since compilers may already be reasonably c++0x-compliant.
so a time frame of 3 months is probably more a time frame of 3 years :/
This is laughable if it were not so sad. Anyone with a finished library does not want to wait years for possible acceptance into Boost much less even more time before such an accepted library is added officially to a Boost distribution. Once again I will say it although I do not know how to get Boost to change the way it presently does things with reviews: More than one review should go on at any one time and the period for a review should be much longer ( I favor one month ) to give possible reviewers more time to look at and review seriously a library. Imagine 3 or 4 reviews during each month period. That should releive a few bottlenecks. Finally another GMane NG/mailing list for just reviews would give those interested in reviewing libraries a better focus on reviews and their responses. Call it the Boost Reviews mailing list and an appropriate gmane.comp.lib.boost.reviews NG. I will be glad to write up a short formal proposal for this, whether it is listened to ot not, if someone tells me what the format might be and what I do with it after I am finished.
On 1/13/2011 2:25 PM, Edward Diener wrote:
Once again I will say it although I do not know how to get Boost to change the way it presently does things with reviews: More than one review should go on at any one time and the period for a review should be much longer ( I favor one month ) to give possible reviewers more time to look at and review seriously a library. Imagine 3 or 4 reviews during each month period. That should releive a few bottlenecks.
+1
Finally another GMane NG/mailing list for just reviews would give those interested in reviewing libraries a better focus on reviews and their responses. Call it the Boost Reviews mailing list and an appropriate gmane.comp.lib.boost.reviews NG.
That might address one of the problems I had as a review manager, mentioned it briefly on IRC today. The biggest pain of the review was sorting out all the emails, it takes a lot of effort & time. Especially since it was not just the reviews themselves, but all the ensuing discussions. -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org (msn) - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim,yahoo,skype,efnet,gmail
On 1/13/2011 9:30 PM, Rene Rivera wrote:
On 1/13/2011 2:25 PM, Edward Diener wrote:
Once again I will say it although I do not know how to get Boost to change the way it presently does things with reviews: More than one review should go on at any one time and the period for a review should be much longer ( I favor one month ) to give possible reviewers more time to look at and review seriously a library. Imagine 3 or 4 reviews during each month period. That should relieve a few bottlenecks.
+1
Finally another GMane NG/mailing list for just reviews would give those interested in reviewing libraries a better focus on reviews and their responses. Call it the Boost Reviews mailing list and an appropriate gmane.comp.lib.boost.reviews NG.
That might address one of the problems I had as a review manager, mentioned it briefly on IRC today. The biggest pain of the review was sorting out all the emails, it takes a lot of effort & time. Especially since it was not just the reviews themselves, but all the ensuing discussions.
I think reviews tend to get lost amid the other issues brought up on this NG, and therefore a separate mailing list/GMane NG would make it easier to be aware, review, and respond to just reviews. This would especially be true if there were 3 or 4 reviews going on at the same time over a longer period.
At Thu, 13 Jan 2011 21:54:46 -0500, Edward Diener wrote:
On 1/13/2011 9:30 PM, Rene Rivera wrote:
On 1/13/2011 2:25 PM, Edward Diener wrote:
Once again I will say it although I do not know how to get Boost to change the way it presently does things with reviews: More than one review should go on at any one time and the period for a review should be much longer ( I favor one month ) to give possible reviewers more time to look at and review seriously a library. Imagine 3 or 4 reviews during each month period. That should relieve a few bottlenecks.
+1
Finally another GMane NG/mailing list for just reviews would give those interested in reviewing libraries a better focus on reviews and their responses. Call it the Boost Reviews mailing list and an appropriate gmane.comp.lib.boost.reviews NG.
That might address one of the problems I had as a review manager, mentioned it briefly on IRC today. The biggest pain of the review was sorting out all the emails, it takes a lot of effort & time. Especially since it was not just the reviews themselves, but all the ensuing discussions.
I think reviews tend to get lost amid the other issues brought up on this NG, and therefore a separate mailing list/GMane NG would make it easier to be aware, review, and respond to just reviews. This would especially be true if there were 3 or 4 reviews going on at the same time over a longer period.
In my vision, the reviews for a library are comments on a wordpress article, and the library's documentation links to the review article. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
Dave Abrahams wrote:
At Thu, 13 Jan 2011 21:54:46 -0500, Edward Diener wrote:
On 1/13/2011 2:25 PM, Edward Diener wrote:
Once again I will say it although I do not know how to get Boost to change the way it presently does things with reviews: More than one review should go on at any one time and the period for a review should be much longer ( I favor one month ) to give possible reviewers more time to look at and review seriously a library. Imagine 3 or 4 reviews during each month period. That should relieve a few bottlenecks. +1
Finally another GMane NG/mailing list for just reviews would give those interested in reviewing libraries a better focus on reviews and their responses. Call it the Boost Reviews mailing list and an appropriate gmane.comp.lib.boost.reviews NG. That might address one of the problems I had as a review manager, mentioned it briefly on IRC today. The biggest pain of the review was sorting out all the emails, it takes a lot of effort & time. Especially since it was not just the reviews themselves, but all the ensuing discussions. I think reviews tend to get lost amid the other issues brought up on
On 1/13/2011 9:30 PM, Rene Rivera wrote: this NG, and therefore a separate mailing list/GMane NG would make it easier to be aware, review, and respond to just reviews. This would especially be true if there were 3 or 4 reviews going on at the same time over a longer period.
In my vision, the reviews for a library are comments on a wordpress article, and the library's documentation links to the review article.
I'd actually like to suggest something that might be more direct. Based on a recommendation from fellow boosters at BoostCon last year we acquired and have been using a web based code review tool for the last 10 months. The impact of this tool has been a dramatic and radical increase in review quality over the email system we had been using -- in large part because all comments/discussion are attached to the actual source code directly for all to see. The tool provides supports the longer review model since someone looking in week 3 of the review can trivially look at all the review comment discussions to that point directly in context with the code. Authors can also update code during the review to address issues and the comment context (and prior versions are maintained). Registered reviewers can also receive email notification when other reviewers comment, etc, etc. Point is, it's a collaboration tool built for code reviews and it works well. The company that provides the tool allows for free use of the tool by open source projects -- so it's something should be possible for boost. That said, there will be work here to coordinate with the company, setup the boost space and review users, etc. The tool is also highly configurable so we'd have to establish some usage policies and such. All items that we would have to address, but I doubt any are a show stoppers. Course we likely would have to allow folks that want to provide email reviews to continue that way, but overall once you go down this road you won't go back to email. Note that I'm not mentioning the name of the tool just yet because I don't want to violate our 'advertising policies' on the list. If there's interest, I can make initial contact with company and get/post the details on how it would work. I was planning to propose this at BoostCon, but now that it's come up we should start the process now if folks agree. Jeff
At Fri, 14 Jan 2011 06:29:16 -0700, Jeff Garland wrote:
In my vision, the reviews for a library are comments on a wordpress article, and the library's documentation links to the review article.
I'd actually like to suggest something that might be more direct. Based on a recommendation from fellow boosters at BoostCon last year we acquired and have been using a web based code review tool for the last 10 months. The impact of this tool has been a dramatic and radical increase in review quality over the email system we had been using -- in large part because all comments/discussion are attached to the actual source code directly for all to see. The tool provides supports the longer review model since someone looking in week 3 of the review can trivially look at all the review comment discussions to that point directly in context with the code. Authors can also update code during the review to address issues and the comment context (and prior versions are maintained). Registered reviewers can also receive email notification when other reviewers comment, etc, etc. Point is, it's a collaboration tool built for code reviews and it works well.
The company that provides the tool allows for free use of the tool by open source projects -- so it's something should be possible for boost. That said, there will be work here to coordinate with the company, setup the boost space and review users, etc. The tool is also highly configurable so we'd have to establish some usage policies and such. All items that we would have to address, but I doubt any are a show stoppers. Course we likely would have to allow folks that want to provide email reviews to continue that way, but overall once you go down this road you won't go back to email.
Note that I'm not mentioning the name of the tool just yet because I don't want to violate our 'advertising policies' on the list.
I think you're being overly cautious. Atlassian?
If there's interest, I can make initial contact with company and get/post the details on how it would work. I was planning to propose this at BoostCon, but now that it's come up we should start the process now if folks agree.
Using a code review tool is an awesome idea. Many reviews are not attached to code, but you can put review comments in documentation just as well. A couple of things to consider: 1. We'd still need a place for overall assessments that don't pertain to specific details. 2. I know this is a bold predicition, but I think we will be transitioning to GitHub. It has an enormous momentum in the open source world, is responsive, and will continue to make a lot more sense as Boost is modularized. GitHub already supports code review. I think I'd rather go with a tool that requires absolutely no sysadmin on our part, is a known quantity to many already, etc. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
Dave Abrahams wrote:
At Fri, 14 Jan 2011 06:29:16 -0700, Jeff Garland wrote:
In my vision, the reviews for a library are comments on a wordpress article, and the library's documentation links to the review article.
I'd actually like to suggest something that might be more direct. Based on a recommendation from fellow boosters at BoostCon last year we acquired and have been using a web based code review tool for the last 10 months. The impact of this tool has been a dramatic and radical increase in review quality over the email system we had been using -- in large part because all comments/discussion are attached to the actual source code directly for all to see. The tool provides supports the longer review model since someone looking in week 3 of the review can trivially look at all the review comment discussions to that point directly in context with the code. Authors can also update code during the review to address issues and the comment context (and prior versions are maintained). Registered reviewers can also receive email notification when other reviewers comment, etc, etc. Point is, it's a collaboration tool built for code reviews and it works well.
The company that provides the tool allows for free use of the tool by open source projects -- so it's something should be possible for boost. That said, there will be work here to coordinate with the company, setup the boost space and review users, etc. The tool is also highly configurable so we'd have to establish some usage policies and such. All items that we would have to address, but I doubt any are a show stoppers. Course we likely would have to allow folks that want to provide email reviews to continue that way, but overall once you go down this road you won't go back to email.
Note that I'm not mentioning the name of the tool just yet because I don't want to violate our 'advertising policies' on the list.
I think you're being overly cautious. Atlassian?
Fair enough: http://smartbear.com/codecollab.php
If there's interest, I can make initial contact with company and get/post the details on how it would work. I was planning to propose this at BoostCon, but now that it's come up we should start the process now if folks agree.
Using a code review tool is an awesome idea. Many reviews are not attached to code, but you can put review comments in documentation just as well.
Documentation is a bit harder unless you are annotating the document source directly -- that is, I haven't see a mode in the tool to annotate against 'rendered html'.
A couple of things to consider:
1. We'd still need a place for overall assessments that don't pertain to specific details.
There's an 'overall comments' section at the top of each review for these kinds of comments.
2. I know this is a bold predicition, but I think we will be transitioning to GitHub. It has an enormous momentum in the open source world, is responsive, and will continue to make a lot more sense as Boost is modularized. GitHub already supports code review. I think I'd rather go with a tool that requires absolutely no sysadmin on our part, is a known quantity to many already, etc.
I haven't used the github review capabilities here -- so we'd have to evaluate what works best. As for the admin -- it's truly minimal -- basically the same as giving someone sandbox access today -- registering an email address so that comment discussions are tracked, etc. And the author has to upload code to the tool -- but a simple paragraph should be about enough to explain it. Jeff
At Fri, 14 Jan 2011 07:13:26 -0700, Jeff Garland wrote:
Note that I'm not mentioning the name of the tool just yet because I don't want to violate our 'advertising policies' on the list.
I think you're being overly cautious. Atlassian?
Fair enough:
I've heard very good things about that product.
If there's interest, I can make initial contact with company and get/post the details on how it would work. I was planning to propose this at BoostCon, but now that it's come up we should start the process now if folks agree.
Using a code review tool is an awesome idea. Many reviews are not attached to code, but you can put review comments in documentation just as well.
Documentation is a bit harder unless you are annotating the document source directly
Yes, that's what I was suggesting.
-- that is, I haven't see a mode in the tool to annotate against 'rendered html'.
That *would* be a nice feature, wouldn't it!? I don't think I've ever seen such a tool. Wait, that's not true. See http://djangobook.com/en/2.0/chapter01/ for an example.
A couple of things to consider:
1. We'd still need a place for overall assessments that don't pertain to specific details.
There's an 'overall comments' section at the top of each review for these kinds of comments.
Yeaaaaaah... do you think putting that kind of commentary in the same place as a code review would work for our review process, though? Maybe it would, but I have a hard time envisioning it.
2. I know this is a bold predicition, but I think we will be transitioning to GitHub. It has an enormous momentum in the open source world, is responsive, and will continue to make a lot more sense as Boost is modularized. GitHub already supports code review. I think I'd rather go with a tool that requires absolutely no sysadmin on our part, is a known quantity to many already, etc.
I haven't used the github review capabilities here -- so we'd have to evaluate what works best. As for the admin -- it's truly minimal -- basically the same as giving someone sandbox access today -- registering an email address so that comment discussions are tracked, etc. And the author has to upload code to the tool -- but a simple paragraph should be about enough to explain it.
I mean someone has to install the tool, administer the system on which it runs, ensure that there's always enough CPU power/bandwidth, manage upgrades, etc. Also I don't love the need to upload code. Having the tool built into a code repo/sharing system removes steps and *should* make things run more smoothly (no separate login, for example). One thing I do like very much about centering the process on a code review tool is that people can poke through making comments, etc., and then after they've taken a good look, consider voting and adding an overall write-up. That is, you can get into it incrementally. That's a lot harder with the current system. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
Dave Abrahams wrote:
-- that is, I haven't see a mode in the tool to annotate against 'rendered html'.
That *would* be a nice feature, wouldn't it!? I don't think I've ever seen such a tool. Wait, that's not true. See http://djangobook.com/en/2.0/chapter01/ for an example.
Ok, the example doesn't seem to be live, but looks interesting...
A couple of things to consider:
1. We'd still need a place for overall assessments that don't pertain to specific details. There's an 'overall comments' section at the top of each review for these kinds of comments.
Yeaaaaaah... do you think putting that kind of commentary in the same place as a code review would work for our review process, though? Maybe it would, but I have a hard time envisioning it.
I think it can work -- and yes, it helps to see it, but the review summary is part of the overall view of a review in code collab.
2. I know this is a bold predicition, but I think we will be transitioning to GitHub. It has an enormous momentum in the open source world, is responsive, and will continue to make a lot more sense as Boost is modularized. GitHub already supports code review. I think I'd rather go with a tool that requires absolutely no sysadmin on our part, is a known quantity to many already, etc. I haven't used the github review capabilities here -- so we'd have to evaluate what works best. As for the admin -- it's truly minimal -- basically the same as giving someone sandbox access today -- registering an email address so that comment discussions are tracked, etc. And the author has to upload code to the tool -- but a simple paragraph should be about enough to explain it.
I mean someone has to install the tool, administer the system on which it runs, ensure that there's always enough CPU power/bandwidth, manage upgrades, etc. Also I don't love the need to upload code. Having the tool built into a code repo/sharing system removes steps and *should* make things run more smoothly (no separate login, for example).
I *think* the way this works is SmartBear provides the computers/site and moderators/review managers admin things. Rather than speculate further though I'm going to contact them and find out the details of how their open source program works.
One thing I do like very much about centering the process on a code review tool is that people can poke through making comments, etc., and then after they've taken a good look, consider voting and adding an overall write-up. That is, you can get into it incrementally. That's a lot harder with the current system.
Precisely. The other very nice feature is that the tool keeps track of which comments each participant has read -- so as new comments are added if you relook at the review summary it highlights and provides links directly for the modified discussions -- since it's individualized you can catch up no matter how long or short a time you've been away from the review. Jeff
Dave Abrahams wrote:
At Fri, 14 Jan 2011 07:13:26 -0700, Jeff Garland wrote:
Dave Abrahams wrote:
Using a code review tool is an awesome idea. Many reviews are not attached to code, but you can put review comments in documentation just as well.
Documentation is a bit harder unless you are annotating the document source directly
Yes, that's what I was suggesting.
-- that is, I haven't see a mode in the tool to annotate against 'rendered html'.
http://smartbear.com/codecollab-new.php indicates that one can review URLs: "Review HTML, Wiki pages, or documents hosted in an Intranet or web-based document management system." This appears in the context of an image showing a PDF file in a view with comments alongside or overlaid. _____ Rob Stewart robert.stewart@sig.com Software Engineer, Core Software using std::disclaimer; Susquehanna International Group, LLP http://www.sig.com IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.
At Fri, 14 Jan 2011 15:23:44 -0500, Stewart, Robert wrote:
Documentation is a bit harder unless you are annotating the document source directly
Yes, that's what I was suggesting.
-- that is, I haven't see a mode in the tool to annotate against 'rendered html'.
http://smartbear.com/codecollab-new.php indicates that one can review URLs:
"Review HTML, Wiki pages, or documents hosted in an Intranet or web-based document management system."
This appears in the context of an image showing a PDF file in a view with comments alongside or overlaid.
That's something GitHub doesn't support... yet. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On 01/14/2011 06:13 AM, Jeff Garland wrote:
Dave Abrahams wrote:
At Fri, 14 Jan 2011 06:29:16 -0700, Jeff Garland wrote:
Note that I'm not mentioning the name of the tool just yet because I don't want to violate our 'advertising policies' on the list.
I think you're being overly cautious. Atlassian?
Fair enough:
Hi Jeff - I have also used their product and have had the same experience you articulated. I was not familiar with their open source usage. I think this is a great idea! michael -- Michael Caisse Object Modeling Designs www.objectmodelingdesigns.com
On 1/14/2011 6:58 AM, Dave Abrahams wrote:
At Thu, 13 Jan 2011 21:54:46 -0500, Edward Diener wrote:
On 1/13/2011 9:30 PM, Rene Rivera wrote:
On 1/13/2011 2:25 PM, Edward Diener wrote:
Once again I will say it although I do not know how to get Boost to change the way it presently does things with reviews: More than one review should go on at any one time and the period for a review should be much longer ( I favor one month ) to give possible reviewers more time to look at and review seriously a library. Imagine 3 or 4 reviews during each month period. That should relieve a few bottlenecks.
+1
Finally another GMane NG/mailing list for just reviews would give those interested in reviewing libraries a better focus on reviews and their responses. Call it the Boost Reviews mailing list and an appropriate gmane.comp.lib.boost.reviews NG.
That might address one of the problems I had as a review manager, mentioned it briefly on IRC today. The biggest pain of the review was sorting out all the emails, it takes a lot of effort& time. Especially since it was not just the reviews themselves, but all the ensuing discussions.
I think reviews tend to get lost amid the other issues brought up on this NG, and therefore a separate mailing list/GMane NG would make it easier to be aware, review, and respond to just reviews. This would especially be true if there were 3 or 4 reviews going on at the same time over a longer period.
In my vision, the reviews for a library are comments on a wordpress article, and the library's documentation links to the review article.
So the submitter of a library should write a wordpress article about it ? That sounds neat. Where is this article published ? On Boost's web site in an appropriate page ? I'm comfortable with that but I am pretty good with writing documentation and information since I have a literary background. Others may not like it as much.
At Fri, 14 Jan 2011 08:51:33 -0500, Edward Diener wrote:
So the submitter of a library should write a wordpress article about it ? That sounds neat. Where is this article published ? On Boost's web site in an appropriate page ? I'm comfortable with that but I am pretty good with writing documentation and information since I have a literary background.
I think that would work well.
Others may not like it as much.
Let's let them speak for themselves, shall we? :-) -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On 1/13/2011 12:26 PM, Tim Blechmann wrote:
- A library can only be added in the review schedule if the author has time in near future to have a review, where near future is, say, 3 months.
well, it would be great if the review would actually happen within a time frame of 3 months. unfortunately, it is a bit different in reality: the boost.lockfree library is on the review queue for more than a year. with 6 reviews per year (like in 2010) and 15 libraries earlier in the queue, i don't expect a review to happen before 2013. in a way, this is kind of nice since the dependency to boost.atomic (c++0x-atomics for c++-98) will hopefully be obsolete since compilers may already be reasonably c++0x-compliant.
so a time frame of 3 months is probably more a time frame of 3 years :/
tim
There is the potential for a couple of misunderstandings here that I want to make sure don't happen. First - The review queue is not a FIFO. It is a Priority Queue, where priority is established by having a review manager lined up and a time where you want the review to happen. You will never be told - "I'm sorry, but you have to wait for library X to be scheduled before you can schedule." In specific for Lockfree, this means that if there is a Manager for the review, then please contact Ron and I with some suggested start times, and you can be reviewing very soon. Second - We have no established quota for how many reviews should happen in a year. We try to avoid having multiple reviews happening at the same time because the experience has been that when it happens neither library gets much attention. The reason we wind up with ~ 6 reviews a year for the last few years is because that is how many reviews have a combination of library and manager ready to go in the year. With some care in scheduling, and allowing for every review to have a chance to extend for a third week if needed, we could run 17 in a year with no overlap, and have BoostCon week off from reviews. I don't know how long the community would sustain this pace before people got sick of writing reviews, but the review schedule is not the restriction here. John
On 1/13/2011 8:25 AM, Vladimir Prus wrote:
Recently, there were various comments about current review process, and its possible improvements. However, I wanted to start with a small point.
Looking at:
http://www.boost.org/community/review_schedule.html
it seems that a few libraries have a review manager assigned, but there's no date.
Where is the information on the responsibiliities of the Review Manager on the Boost web site ?
Edward Diener wrote:
On 1/13/2011 8:25 AM, Vladimir Prus wrote:
Recently, there were various comments about current review process, and its possible improvements. However, I wanted to start with a small point.
Looking at:
http://www.boost.org/community/review_schedule.html
it seems that a few libraries have a review manager assigned, but there's no date.
Where is the information on the responsibiliities of the Review Manager on the Boost web site ?
http://www.boost.org/community/reviews.html#Review_Manager -- Vladimir Prus Mentor Graphics +7 (812) 677-68-40
On 1/13/2011 3:36 PM, Vladimir Prus wrote:
Edward Diener wrote:
On 1/13/2011 8:25 AM, Vladimir Prus wrote:
Recently, there were various comments about current review process, and its possible improvements. However, I wanted to start with a small point.
Looking at:
http://www.boost.org/community/review_schedule.html
it seems that a few libraries have a review manager assigned, but there's no date.
Where is the information on the responsibiliities of the Review Manager on the Boost web site ?
Thanks for the link !
On 1/13/2011 8:25 AM, Vladimir Prus wrote:
Recently, there were various comments about current review process, and its possible improvements. However, I wanted to start with a small point.
<snip> ...
Can we set a policy that:
- A library can only be added in the review schedule if the author has time in near future to have a review, where near future is, say, 3 months. - A review manager is only assigned if a review date is set at the same time, where the date should be in near future -- say, 3 months again.
I think such a policy might not improve our overall review speed too much, but surely will make the situation a bit clear, and allow to understand the real problems with the review system.
Thanks,
I like the idea of having a clearly defined schedule table that really is the reviews that are on the schedule, and not other things. There are some other things the Wizards need to maintain, but keeping the separate appeals to me. As for assigning a review manager, the sequence usually runs the opposite way. A Manager attaches to a library, and then the manager and author come up with a few possible dates and make a request to the Wizards. We don't have a pool of potential managers sitting around from which we make assignments. The managers volunteer with specific libraries in mind, and we work with the combination of author and manager to get things moving from there. Also - Thank you to Volodya for starting this thread, and to the other participants for your opinions. Hopefully we, as a community can build enough of a consensus about how to improve the review process that we have a good basis for taking action and improving things for everyone. This topic comes up in various forms every few months, but building a real plan of action from the discussions is not usually easy. Ron and I make the day to day Wizard decisions without consulting the community, but we do not feel it is our role to make major changes to Boost policies without some fairly strong support from the active participants. John
participants (20)
-
Artyom -
Boris Schaeling -
Chad Nelson -
Dave Abrahams -
Denis Shevchenko -
Edward Diener -
Jeff Garland -
John Maddock -
John Phillips -
KTC -
Marshall Clow -
Michael Caisse -
Phil Endecott -
Rene Rivera -
Simonson, Lucanus J -
Stewart, Robert -
Tim Blechmann -
vicente.botet -
Vladimir Prus -
Vladimir Prus