Tracking Review Requests and RM Applications
Hi! For those who missed it, I suggested on the "Languishing review requests" thread that we keep track of Review Requests and Applications for Review Managers in the ticketing system. The goal being to avoid the "silent rejection" problem. A bonus side-effect is that each proposed library gets a unique place where it's progress along the Boost process can be tracked. Comments on library review requests would include links to relevant discussion on the list and rationale for decisions made. Each ticket would be closed upon final acceptance and merging of the code in the trunk, or rejection of the library. Below I offer some suggestions on how this could work, just to get the discussion rolling. * Ticket Type: we could use one or two more ticket types: (Library) Review Requests and a Review Manager Application. Perhaps a single generic "Review Request" would work just as well, to begin with. We could also reuse one of the existing ticket types (e.g. Tasks?), but I think a clearer separation would work better. * Reporter: The person submitting the review request or RM application * Status: Assigned would be used when a Review Manager has been assigned. He is responsible for "closing" the review ticket. * Owner: per above, this would be the Review Manager for a library submission and the Review Wizard for other types of requests. * Milestone: Perhaps a bit far-fetched, but this could be used to follow the library library submission process (http://boost.org/more/submission_process.htm). This allows one to check at a glance libraries in each stage. It could also clutter the Roadmap page. None of these Milestone Steps would advance: tickets would progress from one to another. * Version: Version of Boost with which the library was tested? Version for inclusion after acceptance? * Component: "Boost Process" And if you've got this far... Do you think this is a good idea in general? Do you think something like this adds more "process" with no or little gain? Is it any friendlier to potential submitters? How about potential review managers? ... All comments welcome. Best regards, João
João Abecasis wrote:
For those who missed it, I suggested on the "Languishing review requests" thread that we keep track of Review Requests and Applications for Review Managers in the ticketing system. The goal being to avoid the "silent rejection" problem.
I'm not convinces this problem is really a serious problem. Sometimes, there just isn't much interest in a given library, and the silence that comes along with that means that the library shouldn't move forward into the "formal" parts of the review process (finding a review manager, scheduling a review, etc.). Still, my view on this doesn't mean that I don't agree with your ideas below... because I do.
Below I offer some suggestions on how this could work, just to get the discussion rolling.
* Ticket Type: we could use one or two more ticket types: (Library) Review Requests and a Review Manager Application. Perhaps a single generic "Review Request" would work just as well, to begin with.
Let's just have one ticket type. I suggest calling it "Library Submission", because that seems to cover the whole process.
* Reporter: The person submitting the review request or RM application
* Status: Assigned would be used when a Review Manager has been assigned. He is responsible for "closing" the review ticket.
Sounds good. Actually, we might want to take this one step further: once a review manager is assigned, the ticket is passed to that person. When the formal review is completed (and assuming the library is accepted), the review manager could bounce the ticket back to the author. The ticket is closed, and milestone assigned, when the library is integrated in CVS/Subversion (and we therefore know the version in which the library will appear). That means one will be able to easily tell which version of Boost a library will appear in, just from the ticket system.
* Owner: per above, this would be the Review Manager for a library submission and the Review Wizard for other types of requests.
Ah, okay. So initially,. the review wizard would own the ticket, then pass it to the review manager, then finally to the library author before it's closed?
* Milestone: Perhaps a bit far-fetched, but this could be used to follow the library library submission process (http://boost.org/more/submission_process.htm). This allows one to check at a glance libraries in each stage. It could also clutter the Roadmap page. None of these Milestone Steps would advance: tickets would progress from one to another.
Let's not do this. Milestones should be reserved for Boost and tool releases. Anyway, we can easily create custom reports that would only look for library submissions, grouping them by milestone (which automagically shows which Boost release a library will appear in). Creating new reports in Trac is as simple as adding a new SQL query to http://svn.boost.org/trac/boost/report
* Version: Version of Boost with which the library was tested? Version for inclusion after acceptance?
"Milestone" seems a better match for this, but either one will do.
* Component: "Boost Process"
Okay.
And if you've got this far... Do you think this is a good idea in general? Do you think something like this adds more "process" with no or little gain? Is it any friendlier to potential submitters? How about potential review managers? ...
I think this is a good idea. It gives us a better "paper" trail to see where each submission is, and can help automate some of the tasks that the review wizards and review managers are already doing. Here's my suggestion: write up a page on the Trac describing how to use the ticket system to track the library submission process (which will probably just be called LibrarySubmissionProcess). Write it as if this were *the* submission process documentation, integrating all of the material from http://boost.org/more/submission_process.htm along with the instructions/links to the right places in the Trac system. If you like the names "Boost Process" (for the component) and "Library Submission" (for the ticket type), then drop me a line and I'll add those to the Trac. I've enabled the ability to create and modify reports on the Trac (assuming you already have access to the Subversion repository; if not, see http://svn.boost.org/trac/boost/wiki/BoostSubversion). I would suggest that you take some of the entries on http://boost.org/more/formal_review_schedule.html and create tickets for them retroactively (even for older, accepted libraries). Then, you could add a Trac report that shows the status of all of the library submissions. I like your idea to use the Trac ticket system to keep track of reviews. I'm also interested in migrating developer-centric content from the web site over to the Trac, where it can be more easily maintained, and in replacing hand-maintained web pages (like the formal review schedule) with auto-generated reports, where possible. In the end, however, Tom Brinkman and Ron Garcia manage the formal reviews, and these tools are meant to make their jobs easier along with helping others better understand the process. - Doug
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Douglas Gregor Sent: Friday, June 15, 2007 2:16 PM To: boost@lists.boost.org Subject: Re: [boost] Tracking Review Requests and RM Applications
João Abecasis wrote:
For those who missed it, I suggested on the "Languishing review requests" thread that we keep track of Review Requests and Applications for Review Managers in the ticketing system. The goal being to avoid the "silent rejection" problem.
I'm not convinces this problem is really a serious problem. Sometimes, there just isn't much interest in a given library, and the silence that comes along with that means that the library shouldn't move forward into the "formal" parts of the review process (finding a review manager, scheduling a review, etc.). Still, my view on this doesn't
I believe the "silent rejection" problem is related to the report of someone, that his _application as a review manager_ was three times silently rejected. Which might be much less acceptable, than silently rejecting a library proposal, IMHO. cheers, aa -- Andreas Ames | Programmer | Comergo GmbH | ames AT avaya DOT com Sitz der Gesellschaft: Stuttgart Registergericht: Amtsgericht Stuttgart - HRB 22107 Geschäftsführer: Andreas von Meyer zu Knonow, Udo Bühler, Thomas Kreikemeier
Douglas Gregor wrote:
João Abecasis wrote:
* Ticket Type: we could use one or two more ticket types: (Library) Review Requests and a Review Manager Application. Perhaps a single generic "Review Request" would work just as well, to begin with.
Let's just have one ticket type. I suggest calling it "Library Submission", because that seems to cover the whole process.
Sure. "Library submission" looks good to me. Better than my "Review Request".
* Reporter: The person submitting the review request or RM application
* Status: Assigned would be used when a Review Manager has been assigned. He is responsible for "closing" the review ticket.
Sounds good. Actually, we might want to take this one step further: once a review manager is assigned, the ticket is passed to that person. When the formal review is completed (and assuming the library is accepted), the review manager could bounce the ticket back to the author. The ticket is closed, and milestone assigned, when the library is integrated in CVS/Subversion (and we therefore know the version in which the library will appear). That means one will be able to easily tell which version of Boost a library will appear in, just from the ticket system.
Agree. I think being able to easily figure out where a candidate or accepted library currently stands is something we've been missing. And the "Formal Review Schedule" page doesn't really cut it because it must be maintained by hand and will lag behind, at times.
* Owner: per above, this would be the Review Manager for a library submission and the Review Wizard for other types of requests.
Ah, okay. So initially,. the review wizard would own the ticket, then pass it to the review manager, then finally to the library author before it's closed?
Yes, that's the idea. Actually, passing the ball to the library author is your suggestion. But it seems logical to be that way.
* Milestone: Perhaps a bit far-fetched, but this could be used to follow the library library submission process (http://boost.org/more/submission_process.htm). This allows one to check at a glance libraries in each stage. It could also clutter the Roadmap page. None of these Milestone Steps would advance: tickets would progress from one to another.
Let's not do this. Milestones should be reserved for Boost and tool releases.
Ok. I wasn't really comfortable with this one either. Anyway, I think we could put the status in a keyword: review requested, review scheduled, undergoing review, accepted, merged to trunk... or something along those lines.
Anyway, we can easily create custom reports that would only look for library submissions, grouping them by milestone (which automagically shows which Boost release a library will appear in). Creating new reports in Trac is as simple as adding a new SQL query to http://svn.boost.org/trac/boost/report
* Version: Version of Boost with which the library was tested? Version for inclusion after acceptance?
"Milestone" seems a better match for this, but either one will do.
* Component: "Boost Process"
Okay.
And if you've got this far... Do you think this is a good idea in general? Do you think something like this adds more "process" with no or little gain? Is it any friendlier to potential submitters? How about potential review managers? ...
I think this is a good idea. It gives us a better "paper" trail to see where each submission is, and can help automate some of the tasks that the review wizards and review managers are already doing.
Here's my suggestion: write up a page on the Trac describing how to use the ticket system to track the library submission process (which will probably just be called LibrarySubmissionProcess). Write it as if this were *the* submission process documentation, integrating all of the material from http://boost.org/more/submission_process.htm along with the instructions/links to the right places in the Trac system.
I'll look into this.
If you like the names "Boost Process" (for the component) and "Library Submission" (for the ticket type), then drop me a line and I'll add those to the Trac.
I'm happy with "Library Submission", not entirely with "Boost Process". Anyway, while we're assuming a single component for the new ticket type, we may as well use None. Unless, of course, we want to merge different types of tickets in a single report... Let me think some more of this. I'll start by formalizing the process on the wiki integrating the ideas discussed here and we'll go from there.
I've enabled the ability to create and modify reports on the Trac (assuming you already have access to the Subversion repository; if not, see http://svn.boost.org/trac/boost/wiki/BoostSubversion). I would suggest that you take some of the entries on http://boost.org/more/formal_review_schedule.html and create tickets for them retroactively (even for older, accepted libraries). Then, you could add a Trac report that shows the status of all of the library submissions.
I'm a bit uneasy about retroactively creating tickets for older requests/accepted libraries. It looks a bit like rewriting history... Anyway, I'll start integrating the Submission Process and Review Schedule in the Wiki and we'll see how that works. I may need to get some tickets in for testing, so perhaps there's no running from it...
I like your idea to use the Trac ticket system to keep track of reviews. I'm also interested in migrating developer-centric content from the web site over to the Trac, where it can be more easily maintained, and in replacing hand-maintained web pages (like the formal review schedule) with auto-generated reports, where possible. In the end, however, Tom Brinkman and Ron Garcia manage the formal reviews, and these tools are meant to make their jobs easier along with helping others better understand the process.
Agreed. Once I start the pages I'll be sure to explicitly mention that this is not yet the approved submission process, but still a work in progress. All the ideas I posted were meant to get the discussion started so I'm not attached to them in any way. While we're defining this, all can be changed. In the end, I'd like the process itself to be as streamlined as possible for everyone involved. We're all here for the code... ehm... and the docs, of course ;-) Best regards, João
On Jun 17, 2007, at 3:36 PM, João Abecasis wrote:
Douglas Gregor wrote:
João Abecasis wrote:
* Ticket Type: we could use one or two more ticket types: (Library) Review Requests and a Review Manager Application. Perhaps a single generic "Review Request" would work just as well, to begin with.
Let's just have one ticket type. I suggest calling it "Library Submission", because that seems to cover the whole process.
Sure. "Library submission" looks good to me. Better than my "Review Request". I'm happy with "Library Submission", not entirely with "Boost Process". Anyway, while we're assuming a single component for the new ticket type, we may as well use None. Unless, of course, we want to merge different types of tickets in a single report... Let me think some more of this.
I've added the "Library Submission" ticket type to the Trac.
I'll start by formalizing the process on the wiki integrating the ideas discussed here and we'll go from there.
Great!
I'm a bit uneasy about retroactively creating tickets for older requests/accepted libraries. It looks a bit like rewriting history... Anyway, I'll start integrating the Submission Process and Review Schedule in the Wiki and we'll see how that works. I may need to get some tickets in for testing, so perhaps there's no running from it...
So long as the end result in Trac matches what was documented on the review schedule, it's a Good Thing to have all of that information in one place, where it can be easily queried.
Agreed. Once I start the pages I'll be sure to explicitly mention that this is not yet the approved submission process, but still a work in progress.
Sure: just some blanket statement at the top should do fine. I'm a big fan of having prototypes that we can just drop in and run with... it makes it much easier to move forward, fast. - Doug
participants (4)
-
Ames, Andreas (Andreas) -
Douglas Gregor -
Douglas Gregor -
João Abecasis