[administrative] Formal review procedure
Hi, here is my next try at an improved formal review procedure. I did away with the additional mailing list. The outlined process tries to reduce the burden on all involved as far as possible. Procedure --------- * The author posts a prereview request to the list. Subject line must contain "[prereview request]". * The author posts review request to review wizard. Subject line must contain "[review request]" and the message should contain a link to the prereview request. The review request must be seconded by two boost members. To second a review request does not imply a positive review. A person who seconded a review request can act as a review manager. If nobody has volunteered so far the review wizard helps finding a review manager. * The review manager checks whether the library is suitable and ready for review. * The review manager sends a schedule request to the review wizard. The subject line must contain "[schedule request]". * The review wizard schedules the review. Reviews will be scheduled on a first come first serve basis, based on the schedule request. * The review manager announces the review. ... Comments? Questions? Thomas Boost Review Wizard
Thomas Witt wrote:
Hi,
here is my next try at an improved formal review procedure. I did away with the additional mailing list. The outlined process tries to reduce the burden on all involved as far as possible.
Procedure ---------
[snip]
Comments? Questions?
What about fast track reviews? How are those handled? -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com - 102708583/icq
On Tue, Feb 24, 2004 at 06:31:56PM -0800, Thomas Witt wrote:
here is my next try at an improved formal review procedure. I did away with the additional mailing list. The outlined process tries to reduce the burden on all involved as far as possible.
Nice. A couple comments/questions...
Procedure --------- * The author posts a prereview request to the list. Subject line must contain "[prereview request]".
* The author posts review request to review wizard. Subject line must contain "[review request]" and the message should contain a link to the prereview request. The review request must be seconded by two boost members. To second a review request does not imply a positive review. A person who seconded a review request can act as a review manager. If nobody has volunteered so far the review wizard helps finding a review manager.
I don't quite understand these two steps. What is the purpose of the "prereview request"? Just to obtain "seconds" (and possibly people willing to be managers)? I think so, but I'm not clear on the intent. I'm also not sure what you mean by "a link the to prereview request". If I understand the intent of the "prereview request", then perhaps what the wizard really wants is just a list of people/email-addresses who seconded.
* The review manager checks whether the library is suitable and ready for review.
* The review manager sends a schedule request to the review wizard. The subject line must contain "[schedule request]".
* The review wizard schedules the review. Reviews will be scheduled on a first come first serve basis, based on the schedule request.
I think this step (scheduling) needs to be more "interactive". The review manager needs a say in the scheduling too, as well as the author. (No sense having a review during a time when one of these parties is absent.) I guess the point is that the review wizard is just keeping up the FCFS order, so he says "we have other reviews going until May3, so find an agreeable time after that" and then the manager/author agree on a time and report back to the wizard to get it officially on the schedule.
* The review manager announces the review.
-- -Brian McNamara (lorgon@cc.gatech.edu)
Brian, Brian McNamara wrote:
On Tue, Feb 24, 2004 at 06:31:56PM -0800, Thomas Witt wrote:
Procedure --------- * The author posts a prereview request to the list. Subject line must contain "[prereview request]".
* The author posts review request to review wizard. Subject line must contain "[review request]" and the message should contain a link to the prereview request. The review request must be seconded by two boost members. To second a review request does not imply a positive review. A person who seconded a review request can act as a review manager. If nobody has volunteered so far the review wizard helps finding a review manager.
I don't quite understand these two steps. What is the purpose of the "prereview request"? Just to obtain "seconds" (and possibly people willing to be managers)? I think so, but I'm not clear on the intent.
Currently when a review request is made it results in a lot of comments on the lib. Sometimes the request is the first time boosters will be made aware of the lib. Depending on the comments authors sometimes want/need to go back to the drawingboard. The outlined two step process is just an attempt to formalize what's already happening.
I'm also not sure what you mean by "a link the to prereview request". If I understand the intent of the "prereview request", then perhaps what the wizard really wants is just a list of people/email-addresses who seconded.
I want to know that the list knows of the lib and that there are people who think it's ready/worth reviewing. The first issues is covered by the link (I will not always be able to monitor the list) the second by the list of people who second. The more general idea is that we have a lot to review and I want only those libs to go to review who are ready. Another thing connected to this goal is that the scheduling does not happen before the review manager says he is ready to go.
* The review manager checks whether the library is suitable and ready for review.
* The review manager sends a schedule request to the review wizard. The subject line must contain "[schedule request]".
* The review wizard schedules the review. Reviews will be scheduled on a first come first serve basis, based on the schedule request.
I think this step (scheduling) needs to be more "interactive". The review manager needs a say in the scheduling too, as well as the author. (No sense having a review during a time when one of these parties is absent.) I guess the point is that the review wizard is just keeping up the FCFS order, so he says "we have other reviews going until May3, so find an agreeable time after that" and then the manager/author agree on a time and report back to the wizard to get it officially on the schedule.
Agreed. That's what was meant by "The review wizard schedules the review" I was just to lazy to provide more detail. Thomas Boost Review Wizard
At 09:31 PM 2/24/2004, Thomas Witt wrote:
Hi,
here is my next try at an improved formal review procedure. I did away with the additional mailing list. The outlined process tries to reduce the burden on all involved as far as possible.
I've been thinking about the review scheduling process, and was about to propose we make it a bit more "open". The basic idea being that we keep enough current information available in CVS so that authors can round-up review managers themselves, and that review managers can handling the scheduling without the Review Wizard doing anything. The Review Wizard's job then becomes one of keeping an eye on the process to set policy, smooth out bumps, assist when needed, and make sure there are enough qualified Review Managers. This should eliminate the Review Wizard as a bottleneck for setting up most individual reviews. The basis for such a self-managing process would be two CVS files: * A queue of people willing to act as review managers, ordered so the people who have managed reviews least recently are at the top of the list. Only the Review Wizard should add people to the queue, but once on the queue Review Managers are responsible for keeping their position updated (by moving their name to the bottom whenever they accept a review to manage.) * The review schedule, including all future reviews that have review managers, and with dates if the review manager has approved a date for the review. Maintained on a first-come-first-served basis by the review managers. The procedure would be similar to Thomas' proposal, except who does what changes a bit:
Procedure ---------
* The author posts a prereview request to the list. Subject line must contain "[prereview request]".
Part of the request is a request for a volunteer as Review Manager
* The author posts review request to review wizard. Subject line must contain "[review request]" and the message should contain a link to the prereview request.
I think this message will no longer needed.
The review request must be seconded by two boost members. To second a review request does not imply a positive review. A person who seconded a review request can act as a review manager. If nobody has volunteered so far the review wizard helps finding a review manager.
If no one volunteers as review manager, let's make it the proposer's job to find a review manager from the top of the review manager's queue. The Review Wizard can privately twist arms if needed, but hopefully that won't be needed for most review requests.
* The review manager checks whether the library is suitable and ready for review.
Yes, as always.
* The review manager sends a schedule request to the review wizard. The subject line must contain "[schedule request]".
That step can be eliminated if the review manager schedules the review directly.
* The review wizard schedules the review. Reviews will be scheduled on a first come first serve basis, based on the schedule request.
The review manager schedules the review and updates the CVS reviews schedule accordingly.
* The review manager announces the review.
...
Comments? --Beman
On Wed, Feb 25, 2004 at 08:27:08PM -0500, Beman Dawes wrote: ...
The basis for such a self-managing process would be two CVS files: ... The procedure would be similar to Thomas' proposal, except who does what changes a bit: ... Comments?
Sounds fine to me. But I don't have a lot of experience with this, so don't let my opinion weigh too heavily. :) In any case, when there's an outcome to all this, someone should update: /more/submission_process.htm and /more/formal_review_process.htm accordingly. (If people tell me what to say, I would offer to write it up if no one else wants to.) Speaking of the process: where should libraries for review be posted? It says YahooGroups files section (and occasionally the old "vault" is mentioned), but I think these are currently held in disfavor. Can anyone get access to the sandbox? Is that appropriate? Is (what I did) just posting the zip of the web somewhere ok? Does it really matter at all? -- -Brian McNamara (lorgon@cc.gatech.edu)
On 2/26/04 3:18 AM, "Brian McNamara" <lorgon@cc.gatech.edu> wrote: [SNIP]
Speaking of the process: where should libraries for review be posted? It says YahooGroups files section (and occasionally the old "vault" is mentioned), but I think these are currently held in disfavor. Can anyone get access to the sandbox? Is that appropriate? Is (what I did) just posting the zip of the web somewhere ok? Does it really matter at all?
Yahoo now blocks anonymous access to the "vault". If the sandbox is used, the submission should still be in an archive file (Zip, Tar/GnuZip, Tar/Bzip, etc.), but in a separate part of the sandbox. It should not be lumped with all the miscellaneous code files. We could set up a new root-level directory and put the archives there. Is anonymous access activated for the sandbox? Maybe we could use some part of our SourceForge web space? For the sandbox or web space, the Review wizard should upload the file there, so we don't have give write-access to every submitter. -- Daryle Walker Mac, Internet, and Video Game Junkie darylew AT hotmail DOT com
Thomas Witt <witt@acm.org> writes:
Hi,
here is my next try at an improved formal review procedure. I did away with the additional mailing list. The outlined process tries to reduce the burden on all involved as far as possible.
Procedure ---------
* The author posts a prereview request to the list. Subject line must contain "[prereview request]".
You're missing something here. Does the list discuss the submission? Is the code made available somewhere?
* The author posts review request to review wizard. Subject line must contain "[review request]" and the message should contain a link to the prereview request. The review request must be seconded by two boost members. To second a review request does not imply a positive review. A person who seconded a review request can act as a review manager. If nobody has volunteered so far the review wizard helps finding a review manager.
I still like the idea that library authors have an obligation. There are 55 libraries at last count; it shouldn't be too much of a burden -- that's about one review every 2 years or more. -- Dave Abrahams Boost Consulting www.boost-consulting.com
David Abrahams wrote:
Thomas Witt <witt@acm.org> writes:
Hi,
here is my next try at an improved formal review procedure. I did away with the additional mailing list. The outlined process tries to reduce the burden on all involved as far as possible.
Procedure ---------
* The author posts a prereview request to the list. Subject line must contain "[prereview request]".
You're missing something here.
To brief again, sigh.
Does the list discuss the submission? Is the code made available somewhere?
Nearly every review request in the past has stirred discussion on the list. As I am see it the prereview request is a heads up to interested people. At least it has to generate enough interest to make people second the request. In order for this to work the code needs to be available. I don't have a strong opinion on requirements with regard to the place where the code is available (sandbox, ...). Thomas
participants (6)
-
Beman Dawes -
Brian McNamara -
Daryle Walker -
David Abrahams -
Rene Rivera -
Thomas Witt