Languishing review requests
Has anyone (other than Tom and Ron) looked at the review queue lately? There are some nice looking library ideas sitting there that can't be reviewed because no one has volunteered to manage the reviews. It has become a problem we shouldn't ignore. The lifeblood of boost is the time donated by the authors of new libraries. However, for some of these authors, nothing has been done with their submissions for months as they wait for someone to volunteer to run the review. This has to be addressed. I know that managing a review is time consuming. I know that many of the most experienced members of boost have been very busy with release issues. However, we need to address this problem. I'm working hard to find time to manage reviews on topics where I feel I have sufficient expertise, but we need people who understand the potential issues in these packages to step up and volunteer to run the languishing reviews. So, now the question. What can we do to improve the situation? Is anyone willing to sign on? Is there so little interest in the work the developers have done that we should say "Thanks for the effort, but we're not interested right now?" Some response is needed, in place of the silence we have seen so far. John Phillips
There is a [mis]conception that using boost is easy but contributing to boost is hard. I am not sure if there is any truth to it or not. I would like to be a review manager but I don't know all that is involved / required. Worse still I don't how! I have similar problems with contributing code. Is there any good tutorials out there and maybe boost should consider apprenticeships where there may be multiple review managers; an experienced one training an inexperienced one. Anyway to make this process, transfer of experience, as easy as possible and your shortage of review managers could be a thing of the past. -----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of John Phillips Sent: Tuesday, June 05, 2007 9:17 AM To: boost@lists.boost.org Subject: [boost] Languishing review requests Has anyone (other than Tom and Ron) looked at the review queue lately? There are some nice looking library ideas sitting there that can't be reviewed because no one has volunteered to manage the reviews. It has become a problem we shouldn't ignore. The lifeblood of boost is the time donated by the authors of new libraries. However, for some of these authors, nothing has been done with their submissions for months as they wait for someone to volunteer to run the review. This has to be addressed. I know that managing a review is time consuming. I know that many of the most experienced members of boost have been very busy with release issues. However, we need to address this problem. I'm working hard to find time to manage reviews on topics where I feel I have sufficient expertise, but we need people who understand the potential issues in these packages to step up and volunteer to run the languishing reviews. So, now the question. What can we do to improve the situation? Is anyone willing to sign on? Is there so little interest in the work the developers have done that we should say "Thanks for the effort, but we're not interested right now?" Some response is needed, in place of the silence we have seen so far. John Phillips _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
I would like to second the suggestion of trainee review managers. Perhaps that could be on a smaller library, not less important, but less in scope to look at. John Fletcher Jarrad Waterloo wrote:
There is a [mis]conception that using boost is easy but contributing to boost is hard. I am not sure if there is any truth to it or not. I would like to be a review manager but I don't know all that is involved / required. Worse still I don't how! I have similar problems with contributing code. Is there any good tutorials out there and maybe boost should consider apprenticeships where there may be multiple review managers; an experienced one training an inexperienced one. Anyway to make this process, transfer of experience, as easy as possible and your shortage of review managers could be a thing of the past.
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of John Phillips Sent: Tuesday, June 05, 2007 9:17 AM To: boost@lists.boost.org Subject: [boost] Languishing review requests
Has anyone (other than Tom and Ron) looked at the review queue lately?
There are some nice looking library ideas sitting there that can't be reviewed because no one has volunteered to manage the reviews. It has become a problem we shouldn't ignore.
The lifeblood of boost is the time donated by the authors of new libraries. However, for some of these authors, nothing has been done with their submissions for months as they wait for someone to volunteer to run the review. This has to be addressed.
I know that managing a review is time consuming. I know that many of the most experienced members of boost have been very busy with release issues. However, we need to address this problem. I'm working hard to find time to manage reviews on topics where I feel I have sufficient expertise, but we need people who understand the potential issues in these packages to step up and volunteer to run the languishing reviews.
So, now the question. What can we do to improve the situation? Is anyone willing to sign on? Is there so little interest in the work the developers have done that we should say "Thanks for the effort, but we're not interested right now?" Some response is needed, in place of the silence we have seen so far.
John Phillips
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
John Fletcher wrote:
I would like to second the suggestion of trainee review managers. Perhaps that could be on a smaller library, not less important, but less in scope to look at.
John Fletcher
Jarrad Waterloo wrote:
There is a [mis]conception that using boost is easy but contributing to boost is hard. I am not sure if there is any truth to it or not. I would like to be a review manager but I don't know all that is involved / required. Worse still I don't how! I have similar problems with contributing code. Is there any good tutorials out there and maybe boost should consider apprenticeships where there may be multiple review managers; an experienced one training an inexperienced one. Anyway to make this process, transfer of experience, as easy as possible and your shortage of review managers could be a thing of the past.
As Jake mentions in another post, one natural place to go for description and assistance is the Review Wizards. In my experience, they are very pleasant and helpful. Another easy source is the documentation of the role at the boost website. However, even after this, you may want another point of view. I'll happily volunteer to share my experiences with anyone who thinks they would be helpful. My first suggestion is that you participate in a review as a reviewer (if you haven't already) and try to follow the complete review discussion of the library closely. Identify which issues are important and which are secondary. Are there any show stoppers? How does the library author respond to constructive criticism? Are any of the reviewers resorting to non-constructive criticism? How could you sum up the issues in this review? Is the opinion of the reviewers positive, neutral or negative? A look at my review reports compared to some others will show that I am, if anything, over complete in my summations of the reviews. This is intentional on my part, as I like to have a single post to look through to understand the flow of the review and the issues raised for the library. However, it is not required. Some fine reviews have been summed up in far less text. Once you have participated a couple of times, and you understand the process you are ready to try managing. Always try to pick reviews where you understand the basic issues involved. For example, if you have never tried meta-programming, a library that uses metaprogramming to provide much of its functionality is probably not a good idea for you to manage. No one in boost is an expert in everything (though some people are very impressively broad), and the review manager needs to know enough about the topic to understand the plusses and minuses of the points that are discussed during the review. If need be, don't feel bad about checking a couple of references for some details, but know the topic well enough to read the references closely. For the review, and the week or so following set aside some time. Managing a review is no where near as time consuming as submitting a library, but it does take up some time. It will be far less of an issue if you budget for the time before you start. An active review may produce a few hundred posts with a wealth of details, so reading them closely enough to make an informed decision is time consuming. I hope that helps, and feel free to ask me (and everyone else on the list who wants to help) whatever questions come to mind. John
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of John Phillips Sent: Tuesday, June 05, 2007 9:17 AM To: boost@lists.boost.org Subject: [boost] Languishing review requests
Has anyone (other than Tom and Ron) looked at the review queue lately?
There are some nice looking library ideas sitting there that can't be reviewed because no one has volunteered to manage the reviews. It has become a problem we shouldn't ignore.
The lifeblood of boost is the time donated by the authors of new libraries. However, for some of these authors, nothing has been done with their submissions for months as they wait for someone to volunteer to run the review. This has to be addressed.
As a first time developer for the Boost community (I am working on the GSoC project "Visualization of STL Containers", it is disconcerting that the submission that I'm going to spend my entire summer working on may never get a chance to make it into Boost! And apparently this is not a concern that is unique to me: http://article.gmane.org/gmane.comp.lib.boost.devel/155019/match=summer+code Though there was some discussion on fixing this problem: http://article.gmane.org/gmane.comp.lib.boost.devel/155176/match=soc I haven't heard anything about the eventual fate of my project. When I get some more experience I wouldn't mind taking on a review now or then, but there's apparently no guarantee I'll ever get the experience. I am happy so far with the Boost community.. all of the people that I've dealt with so far are very kind and helpful, and I've had a lot of fun doing the work, which is not something many people can say nowadays. However, I'm worried that the work that me or the rest of my GSoCers do won't ever come to light. It looks to me like the "trainee review managers" idea is similar to the current practice, as per the following page: http://boost.org/more/formal_review_process.htm#Review_Manager It looks like the bar for being a review manager is fairly low so long as you are knowledgeable in the field. All of the steps in the duties of the review manager seem simple and straightforward (just time consuming!). The "Review Wizards" appear like they are in place to help out the managers with questions about process, etc. Jake
Jake Voytko wrote:
It looks to me like the "trainee review managers" idea is similar to the current practice, as per the following page: http://boost.org/more/formal_review_process.htm#Review_Manager
It looks like the bar for being a review manager is fairly low so long as you are knowledgeable in the field. All of the steps in the duties of the review manager seem simple and straightforward (just time consuming!). The "Review Wizards" appear like they are in place to help out the managers with questions about process, etc.
I think the operative term here is "active boost member". What is an active Boost member? Is it someone who has submitted and is maintaining a Boost library? Is it someone who actively helps out in diagnosing problems, enhancing libraries and so on? Or is it someone who simply participates in one or more of the mailing lists and has collected experience that way? For example, I've been on this list for a year and a half. I have participated in reviews, have helped people out with their problems. I have not fixed any bugs or contributed enhancements. Do I qualify as an "active boost member"? I have a small enhancement for the utility or iterator library half-done, though. (Documentation and test cases are missing.) If I complete this, submit it to the library owner and it gets integrated, does that make me an active boost member? Is there any place where the term is clarified? Sebastian Redl
On Tue, 05 Jun 2007 19:04:35 +0200 Sebastian Redl <sebastian.redl@getdesigned.at> wrote:
I think the operative term here is "active boost member". What is an active Boost member? Is it someone who has submitted and is maintaining
That's a good point. I've volunteered three times. I even volunteered to be a co-manager so I could learn the "behind the scenes" requirements. I never received a reason why I was not picked. I figured it was because I am not considered an "active boost member." Or maybe I have offended too many people in the past... or I have demonstrated too much ignorance on one or more topics... or, gasp, could it be, someone else was more qualified ;-) In any event, after the third silent rejection, with no explanation, I "got the message," and stopped volunteering. The review manager does have a fair amount of authority, especially in reviews which are not clear cut one way or another. So, it can't just be given to anyone who asks... though maybe there should be some explanation given to those who are not picked, and maybe some kind of "training roadmap" provided for those interested.
On Tue, 5 Jun 2007 13:16:43 -0400 Jody Hagins <jody-boost-011304@atdesk.com> wrote:
In any event, after the third silent rejection, with no explanation, I "got the message," and stopped volunteering.
That sounds a bit harsh, so I apologize to both Tom and Ron. I do not mean anything bad against the Review wizards at all. In two of my "applications" I specifically said that I had not done one before, and did not even know if I passed the basic qualifications of boost membership. I just meant to say that it would help if we could be a bit more specific about basic qualifications. Based on my personal experience, I simply assumed that only library authors/contributors could manage a review. I think this subject should get some authoritative clarification for the future, because until that comes, I will never again volunteer as a review manager since, based on my understanding, I do not meet the basic requirement.
Jody Hagins wrote:
On Tue, 5 Jun 2007 13:16:43 -0400 Jody Hagins <jody-boost-011304@atdesk.com> wrote:
In any event, after the third silent rejection, with no explanation, I "got the message," and stopped volunteering.
That sounds a bit harsh, so I apologize to both Tom and Ron. I do not mean anything bad against the Review wizards at all.
In two of my "applications" I specifically said that I had not done one before, and did not even know if I passed the basic qualifications of boost membership.
I just meant to say that it would help if we could be a bit more specific about basic qualifications. Based on my personal experience, I simply assumed that only library authors/contributors could manage a review.
I think this subject should get some authoritative clarification for the future, because until that comes, I will never again volunteer as a review manager since, based on my understanding, I do not meet the basic requirement.
I can't claim to be authoritative, but I can say that authoring a library is apparently not a requirement to be a review manager. I can say so because I have managed 2 reviews and authored no libraries to date. What I think is a reasonale requirement is some amount of experience/expertise in the domain of the library. Evidence of this expertise is best provided on the honor system, since it is not always easy to check up on things of this sort. However, it may be wise to have something like the apprentice system others have discussed, where an experienced manager agrees to keep a close eye on the progress of a novice before, during and after a review. That way, there is a safety net in case something goes wrong for the new person. I have no idea of the reason for the silence in response to your offers, but it might be nothing more than a symptom of the fact that the wizards are, like the rest of us, volunteering time and prone to sometimes losing something in the overload. I agree that some sort of response is always warranted for anyone who volunteers, though. John
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
on Tue Jun 05 2007, Jody Hagins <jody-boost-011304-AT-atdesk.com> wrote:
On Tue, 05 Jun 2007 19:04:35 +0200 Sebastian Redl <sebastian.redl@getdesigned.at> wrote:
I think the operative term here is "active boost member". What is an active Boost member? Is it someone who has submitted and is maintaining
That's a good point.
I've volunteered three times. I even volunteered to be a co-manager so I could learn the "behind the scenes" requirements. I never received a reason why I was not picked. I figured it was because I am not considered an "active boost member." Or maybe I have offended too many people in the past... or I have demonstrated too much ignorance on one or more topics... or, gasp, could it be, someone else was more qualified ;-)
In any event, after the third silent rejection, with no explanation, I "got the message," and stopped volunteering.
That's a real shame.
The review manager does have a fair amount of authority, especially in reviews which are not clear cut one way or another. So, it can't just be given to anyone who asks... though maybe there should be some explanation given to those who are not picked, and maybe some kind of "training roadmap" provided for those interested.
Great ideas. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
David Abrahams wrote:
on Tue Jun 05 2007, Jody Hagins <jody-boost-011304-AT-atdesk.com> wrote: ...
The review manager does have a fair amount of authority, especially in reviews which are not clear cut one way or another. So, it can't just be given to anyone who asks... though maybe there should be some explanation given to those who are not picked, and maybe some kind of "training roadmap" provided for those interested.
Great ideas.
As long as some people with real authority are part of the project, I'll volunteer to be in charge of making this roadmap happen. I think it is a great idea, as well, but I don't have the authority to generate it myself. Who else wants to be a part of this? John
on Wed Jun 06 2007, John Phillips <phillips-AT-delos.mps.ohio-state.edu> wrote:
David Abrahams wrote:
on Tue Jun 05 2007, Jody Hagins <jody-boost-011304-AT-atdesk.com> wrote: ...
The review manager does have a fair amount of authority, especially in reviews which are not clear cut one way or another. So, it can't just be given to anyone who asks... though maybe there should be some explanation given to those who are not picked, and maybe some kind of "training roadmap" provided for those interested.
Great ideas.
As long as some people with real authority are part of the project, I'll volunteer to be in charge of making this roadmap happen. I think it is a great idea, as well, but I don't have the authority to generate it myself.
Who else wants to be a part of this?
John, I think the Boost moderators are probably happy to grant the necessary authority to anyone with your credibility who wants to volunteer to be in charge of something. :) In general, y'all, this is *your* Boost. Please feel free to make stuff happen. Please! :) -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
David Abrahams wrote:
on Wed Jun 06 2007, John Phillips <phillips-AT-delos.mps.ohio-state.edu> wrote:
As long as some people with real authority are part of the project, I'll volunteer to be in charge of making this roadmap happen. I think it is a great idea, as well, but I don't have the authority to generate it myself.
Who else wants to be a part of this?
John, I think the Boost moderators are probably happy to grant the necessary authority to anyone with your credibility who wants to volunteer to be in charge of something. :)
In general, y'all, this is *your* Boost. Please feel free to make stuff happen. Please! :)
In which case, I'll try to generate something as a roadmap, and post it to the list as a proposal to start working from. It may be a few days before I get the chance, but I'll work on it. John
David Abrahams wrote:
on Tue Jun 05 2007, Jody Hagins <jody-boost-011304-AT-atdesk.com> wrote:
The review manager does have a fair amount of authority, especially in reviews which are not clear cut one way or another. So, it can't just be given to anyone who asks... though maybe there should be some explanation given to those who are not picked, and maybe some kind of "training roadmap" provided for those interested.
Great ideas.
It might be time to rethink this part of the Boost process as well. I'll start with one seemingly simple question: Why do we need a review manager at all?
Peter Dimov wrote:
David Abrahams wrote:
on Tue Jun 05 2007, Jody Hagins <jody-boost-011304-AT-atdesk.com> wrote:
The review manager does have a fair amount of authority, especially in reviews which are not clear cut one way or another. So, it can't just be given to anyone who asks... though maybe there should be some explanation given to those who are not picked, and maybe some kind of "training roadmap" provided for those interested. Great ideas.
It might be time to rethink this part of the Boost process as well. I'll start with one seemingly simple question:
Why do we need a review manager at all?
Primarily to avoid any questions or doubts about whether a library should be accepted or not. The review manager supposedly takes everybody's feedback into account, but makes the ultimate yes/no decision, and is even free to buck popular opinion. It's a representative democracy, twice removed: we elect the review wizard, who elects the review managers, who elect the libraries. Which would make the review managers the electoral college. ;-) Another key job of the review manager is to collect all the feedback and present a TODO list to the author of the new Boost library, but that's secondary. -- Eric Niebler Boost Consulting www.boost-consulting.com
Eric Niebler wrote:
Peter Dimov wrote:
Why do we need a review manager at all?
Primarily to avoid any questions or doubts about whether a library should be accepted or not. The review manager supposedly takes everybody's feedback into account, but makes the ultimate yes/no decision, and is even free to buck popular opinion.
Do so many of our reviews end in such a non-conclusive manner as to require a decision from a review manager?
Another key job of the review manager is to collect all the feedback and present a TODO list to the author of the new Boost library, but that's secondary.
I think that the author should be quite capable of doing that him/herself. This also applies to most of the other review manager duties. The rest can be handled by documentation. The final yes/no decision to accept can be done by the moderators. Going by the checklist one by one: * Checks the submission to make sure it really is complete enough to warrant formal review. Not really required. The reviewers will be quick to point that out. * Finalizes the schedule with the Review Wizard and the submitter. Not required. The submitter can do that. * Posts a notice of the review schedule on the regular boost mailing list, the boost-users mailing list, and the boost-announce mailing list. The submitter can do that. * Inspects the Boost library catalogue for libraries which may interact with the new submission. The submitter or the reviewers can do that. * Urges people to do reviews if they aren't forthcoming. The submitter has an incentive to do that. Lack of reviews leads to autoreject. * Follows review discussions regarding the library, moderating or answering questions as needed. Moderating rarely needed. Following/answering needs to be done by submitter. * Asks the review wizard for permission to extend the review schedule if it appears that too few reviews will be submitted during the review period. The review can take as long as necessary to gather a sufficient number of reviews. There is no need for a deadline. If we decide to keep the current scheme, the submitter can ask for the extension. * Decides if there is consensus to accept the library, and if there are any conditions attached. It is the responsibility of the submitter to prepare a summary of the reviews linking to them and to work with the reviewers to address their concerns. The summary is posted to the list and the moderators decide whether to accept the library. * Posts a notice of the review results on the regular boost mailing list, the boost-users mailing list, and the boost-announce mailing list. The moderators do that.
Peter Dimov wrote:
Eric Niebler wrote:
Peter Dimov wrote:
Why do we need a review manager at all?
Primarily to avoid any questions or doubts about whether a library should be accepted or not. The review manager supposedly takes everybody's feedback into account, but makes the ultimate yes/no decision, and is even free to buck popular opinion.
Do so many of our reviews end in such a non-conclusive manner as to require a decision from a review manager?
It's irrelevant that it doesn't happen often. If it happens EVER and we don't have one person designated to break the tie, there's the potential for a nasty situation. And that one person has to be qualified for his/her opinion to carry weight. You might say, "Well, if it's so close a call, we should just reject the library," but that doesn't help. It just moves the line. The question then becomes "Was there enough debate to bring the result into question?" If you don't have an adjudicator, you have no way to settle these disputes. -- Eric Niebler Boost Consulting www.boost-consulting.com
Eric Niebler wrote:
Peter Dimov wrote:
Eric Niebler wrote:
Peter Dimov wrote:
Why do we need a review manager at all?
Primarily to avoid any questions or doubts about whether a library should be accepted or not. The review manager supposedly takes everybody's feedback into account, but makes the ultimate yes/no decision, and is even free to buck popular opinion.
Do so many of our reviews end in such a non-conclusive manner as to require a decision from a review manager?
It's irrelevant that it doesn't happen often. If it happens EVER and we don't have one person designated to break the tie, there's the potential for a nasty situation. And that one person has to be qualified for his/her opinion to carry weight.
It is not irrelevant at all. If disputes only occur one time in a hundred, there is no need to recruit 100 volunteer review managers just because one of them might need to break a tie. We can just say "Eric Niebler breaks ties if they occur" and carry on.
Peter Dimov wrote:
Eric Niebler wrote:
Eric Niebler wrote:
Peter Dimov wrote:
Why do we need a review manager at all? Primarily to avoid any questions or doubts about whether a library should be accepted or not. The review manager supposedly takes everybody's feedback into account, but makes the ultimate yes/no decision, and is even free to buck popular opinion. Do so many of our reviews end in such a non-conclusive manner as to require a decision from a review manager? It's irrelevant that it doesn't happen often. If it happens EVER and we don't have one person designated to break the tie, there's the
Peter Dimov wrote: potential for a nasty situation. And that one person has to be qualified for his/her opinion to carry weight.
It is not irrelevant at all. If disputes only occur one time in a hundred, there is no need to recruit 100 volunteer review managers just because one of them might need to break a tie. We can just say "Eric Niebler breaks ties if they occur" and carry on.
That's a terrible system! I'd rather you do it. :-) I would have been a very bad person to break a tie in the ASIO review, for instance. I know very little about network programming. The idea is that the tie-breaker person for any review should have some domain knowledge and can make an informed decision, should it be necessary. There is no one person qualified to break ties in every possible domain. -- Eric Niebler Boost Consulting www.boost-consulting.com
Eric Niebler wrote:
Peter Dimov wrote:
It is not irrelevant at all. If disputes only occur one time in a hundred, there is no need to recruit 100 volunteer review managers just because one of them might need to break a tie. We can just say "Eric Niebler breaks ties if they occur" and carry on.
That's a terrible system! I'd rather you do it. :-) I would have been a very bad person to break a tie in the ASIO review, for instance. I know very little about network programming. The idea is that the tie-breaker person for any review should have some domain knowledge and can make an informed decision, should it be necessary. There is no one person qualified to break ties in every possible domain.
The idea is that the knowledge needs to come from the reviewers, not from the tie-breaker person. Anybody qualified and willing to act as a review manager will obviously be qualified and willing to write a review, but the converse is - as we are observing - not necessarily true. If the review process does not produce a sufficiently solid case for acceptance, the library is rejected. It is the responsibility of the submitter to present his/her case citing reviews as supporting material such that the busy tie-breaker person/group is able to make a quick decision.
Peter Dimov wrote:
The idea is that the knowledge needs to come from the reviewers, not from the tie-breaker person. Anybody qualified and willing to act as a review manager will obviously be qualified and willing to write a review, but the converse is - as we are observing - not necessarily true. If the review process does not produce a sufficiently solid case for acceptance, the library is rejected. It is the responsibility of the submitter to present his/her case citing reviews as supporting material such that the busy tie-breaker person/group is able to make a quick decision.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Who checks to see if the submitter is presenting a fair picture of the review? If it is no one, then there is the chance (hopefully rare) that the submitter will present a skewed version of what happened in the review. If it is one person, that person is the de-facto review manager, since that person has the single largest responsibility of the review manager (checking all the reviews and determining the best available recommendation). If it is the whole tie-breaker group, then there are many people doing the work that used to be done by one. If it is supposed to come from the participants, then the discussion of the submitter's version of the review becomes another review and little is gained (if anything). To me, the biggest responsibility of the review manager is to be someone who understands the domain but has no personal stake in whether the library passes or fails. That gives as good a chance at an objective and complete final report as is possible, while only taking up one person's time. John
on Thu Jun 07 2007, John Phillips <phillips-AT-delos.mps.ohio-state.edu> wrote:
To me, the biggest responsibility of the review manager is to be someone who understands the domain but has no personal stake in whether the library passes or fails. That gives as good a chance at an objective and complete final report as is possible, while only taking up one person's time.
I'd stress understanding the domain a little less than you do. It surely helps, but all that's really required is that you are able to evaluate the validity of the review commentary and the author's replies. That may mean learning a bit about the domain, but I don't think it necessarily implies pre-existing expertise. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
John Phillips wrote:
Who checks to see if the submitter is presenting a fair picture of the review? If it is no one, then there is the chance (hopefully rare) that the submitter will present a skewed version of what happened in the review. If it is one person, that person is the de-facto review manager, since that person has the single largest responsibility of the review manager (checking all the reviews and determining the best available recommendation). If it is the whole tie-breaker group, then there are many people doing the work that used to be done by one. If it is supposed to come from the participants, then the discussion of the submitter's version of the review becomes another review and little is gained (if anything).
I agree that it could be incredibly hard for a submitter to prepare an impartial summary of the opinions of the reviewers and that not everyone will want to do so. That said, if a submitter is willing and able to go through this experience, we might consider granting him/her the opportunity if a review manager hasn't turned up for, say, three months.
Peter Dimov wrote:
I agree that it could be incredibly hard for a submitter to prepare an impartial summary of the opinions of the reviewers and that not everyone will want to do so.
That said, if a submitter is willing and able to go through this experience, we might consider granting him/her the opportunity if a review manager hasn't turned up for, say, three months.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
This strikes me as one of those jobs where the people who would want it might include some of the people you would least want to give it to. The general high level of the conversations and professional responsibility on the boost developer list is a credit to the group. However, even here there have been cases of someone claiming to be reasonable and objective while actually being neither. There are people in this group I would trust to make the final calls on their own library, but not everyone is capable of that. I really don't want to have to make that choice, and I'm betting the moderators don't, either. John
John Phillips wrote:
Peter Dimov wrote:
I agree that it could be incredibly hard for a submitter to prepare an impartial summary of the opinions of the reviewers and that not everyone will want to do so.
That said, if a submitter is willing and able to go through this experience, we might consider granting him/her the opportunity if a review manager hasn't turned up for, say, three months.
This strikes me as one of those jobs where the people who would want it might include some of the people you would least want to give it to.
The general high level of the conversations and professional responsibility on the boost developer list is a credit to the group. However, even here there have been cases of someone claiming to be reasonable and objective while actually being neither.
The potential for abuse is not lost on me. However, the fact that the system can be abused does not necessarily mean that it will be abused in practice. If this happens, we can just go back to the traditional approach and we'll be no worse off, or we can try a hybrid approach allowing more people to qualify for the RM role. It would be nice if we could devise a system that does not suffer from the "silent rejection" problem: you input a review request (or a RM application) into it and receive no output back for months (or at all).
On 6/7/07, Peter Dimov <pdimov@mmltd.net> wrote:
The potential for abuse is not lost on me. However, the fact that the system can be abused does not necessarily mean that it will be abused in practice.
There are different forms of abuse other than direct manipulation of results. After working on these libraries, sometimes for months, I think you're going to find that even people who are striving to be as fair as they can about the negative reviews are going to try to present the negatives in a way so as to minimize impact. Developers are emotionally attached to projects that they spend a significant amount of time on.. they want to see them do well, and the developers want to be recognized positively for their work. It doesn't do the Boost users any good by allowing any kind of potential bias in the review results, and in fact could harm them in some cases by allowing inferior results slip through.
It would be nice if we could devise a system that does not suffer from the "silent rejection" problem: you input a review request (or a RM application) into it and receive no output back for months (or at all).
I absolutely agree. I feel that rapid feedback would be a necessary part of a solution. It would make potential helpers know what they can do while they are still in a helpful mood, rather than waiting until they have shifted their focus onto different projects and communities. Jake
on Thu Jun 07 2007, "Peter Dimov" <pdimov-AT-mmltd.net> wrote:
John Phillips wrote:
Peter Dimov wrote:
I agree that it could be incredibly hard for a submitter to prepare an impartial summary of the opinions of the reviewers and that not everyone will want to do so.
That said, if a submitter is willing and able to go through this experience, we might consider granting him/her the opportunity if a review manager hasn't turned up for, say, three months.
This strikes me as one of those jobs where the people who would want it might include some of the people you would least want to give it to.
The general high level of the conversations and professional responsibility on the boost developer list is a credit to the group. However, even here there have been cases of someone claiming to be reasonable and objective while actually being neither.
The potential for abuse is not lost on me. However, the fact that the system can be abused does not necessarily mean that it will be abused in practice. If this happens, we can just go back to the traditional approach and we'll be no worse off,
Not unless we can somehow undo the abuse. That's a lot harder than just changing the system. You have to be willing to say, "I know we told you it works like X, and you put all that work into a review, but we've retroactively changed our mind." Even getting the group to come to consensus about retroactively changing its mind on policy is hard (c.f. uBlas licensing).
or we can try a hybrid approach allowing more people to qualify for the RM role.
I hope more people can qualify.
It would be nice if we could devise a system that does not suffer from the "silent rejection" problem: you input a review request (or a RM application) into it and receive no output back for months (or at all).
Agreed. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
Peter Dimov wrote:
It would be nice if we could devise a system that does not suffer from the "silent rejection" problem: you input a review request (or a RM application) into it and receive no output back for months (or at all).
I think part of the "silent rejection" problem is the medium that is being used. In a mailing list it is just too easy to let a message slip by. Perhaps tracking review requests and RM applications in the ticketing system would help. Anyway, it's just an idea. Best regards, João
I would like to give my testimonial on this point... A few days ago I subscribed to this mailing list because I wanted to share with the boost community the results of my experiments with the random_on_sphere distribution. I uploaded a ready to read pdf file with the method, the results and a proposal to modify the current implementation (that is 20 times slower than it could be)... I expected a quick decision since I took the pain to make everything clear in the document. I did not understood anything to what happens here. I got some feedback from users of the mailing list and that's it! Nobody took the formal review in charge. First: Why should I subscribe to a mailing list to submit a proposal? Second Let's face it, there is very few users on this list in comparison to the interested people in boost. Third: People interested in boost do not have special interest to discuss on a mailing list. I think that you should setup a review process similar to scientific journal submissions. This mean: step 1: submit your proposal of new library to the Chief Editor (Tom or Ronald) step 2: the Chief Editor quickly look if the contribution worth a review and immediately feedback to the submitter step 3: If accepted for review, the Chief Editor delegate to an Associate Editor the managing of reviews step 4: The Associate Editor know trusted reviewers and personally ask if they want to review step 5: Each reviewers gives back their comments step 6: Based on the comments of the reviewers, the AE decide to accept, reject or ask modifications and feedback the submitter immediately Ok hope it helps... The current system with "Managers and Wizards" feel a lot of amateurism... Things should be changed and the formal review system of most scientific publications has proven to be efficient. It's easy to set up while the editors have a list of trusted potential reviewers (and there is hundreds of competent boost users around that could fulfil this). The key point is that: - The submitters and reviewers don't have to use the mailing list. (they don't have time for this anyway) - If the editor ask kindly for a review, chances are that the reviewer will accept it. Colas On 6/13/07, João Abecasis <jpabecasis@gmail.com> wrote:
It would be nice if we could devise a system that does not suffer from
Peter Dimov wrote: the
"silent rejection" problem: you input a review request (or a RM application) into it and receive no output back for months (or at all).
I think part of the "silent rejection" problem is the medium that is being used. In a mailing list it is just too easy to let a message slip by. Perhaps tracking review requests and RM applications in the ticketing system would help.
Anyway, it's just an idea.
Best regards,
João _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Hello more, Wednesday, June 13, 2007, 8:04:22 PM, you wrote:
I would like to give my testimonial on this point...
[snip]
I think that you should setup a review process similar to scientific journal submissions. This mean:
step 1: submit your proposal of new library to the Chief Editor (Tom or Ronald) step 2: the Chief Editor quickly look if the contribution worth a review and immediately feedback to the submitter step 3: If accepted for review, the Chief Editor delegate to an Associate Editor the managing of reviews step 4: The Associate Editor know trusted reviewers and personally ask if they want to review step 5: Each reviewers gives back their comments step 6: Based on the comments of the reviewers, the AE decide to accept, reject or ask modifications and feedback the submitter immediately
[snip] I have to say I disagree with you at the point where "trusted reviewers" appear. This means that the majority of potential users of the proposed library are not allowed to express their opinions, requests and questions about it. I think it's not the right way to develop libraries that are needed by _people_, not only by the _author_ and a group of trusted reviewers. As for the rest, I think it pretty much reflects the current order of library submission. -- Best regards, Andrey mailto:andysem@mail.ru
on Wed Jun 13 2007, "more effective thinking in the exceptional C++ programming language" <effective.thinking-AT-gmail.com> wrote:
I would like to give my testimonial on this point...
A few days ago I subscribed to this mailing list because I wanted to share with the boost community the results of my experiments with the random_on_sphere distribution. I uploaded a ready to read pdf file with the method, the results and a proposal to modify the current implementation (that is 20 times slower than it could be)...
I expected a quick decision since I took the pain to make everything clear in the document. I did not understood anything to what happens here. I got some feedback from users of the mailing list and that's it! Nobody took the formal review in charge.
Did you realize that it's too early to ask for a formal review? There are 4 steps in our process before formal review should be requested: http://www.boost.org/more/submission_process.htm -- Dave Abrahams Boost Consulting http://www.boost-consulting.com The Astoria Seminar ==> http://www.astoriaseminar.com
David Abrahams wrote:
on Wed Jun 13 2007, "more effective thinking in the exceptional C++ programming language" <effective.thinking-AT-gmail.com> wrote:
I would like to give my testimonial on this point...
A few days ago I subscribed to this mailing list because I wanted to share with the boost community the results of my experiments with the random_on_sphere distribution. I uploaded a ready to read pdf file with the method, the results and a proposal to modify the current implementation (that is 20 times slower than it could be)...
I expected a quick decision since I took the pain to make everything clear in the document. I did not understood anything to what happens here. I got some feedback from users of the mailing list and that's it! Nobody took the formal review in charge.
Did you realize that it's too early to ask for a formal review?
There are 4 steps in our process before formal review should be requested: http://www.boost.org/more/submission_process.htm
Does a new implementation of the uniform random distribution on a sphere really require a formal review? Couldn't just the maintainer of the Boost.Random library decide whether to incorporate this code into the library? -- Johan Råde
on Thu Jun 14 2007, Johan Råde <rade-AT-maths.lth.se> wrote:
Does a new implementation of the uniform random distribution on a sphere really require a formal review? Couldn't just the maintainer of the Boost.Random library decide whether to incorporate this code into the library?
In principle, yes. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com The Astoria Seminar ==> http://www.astoriaseminar.com
I would like to give my testimonial on this point...
A few days ago I subscribed to this mailing list because I wanted to share with the boost community the results of my experiments with the random_on_sphere distribution. I uploaded a ready to read pdf file with the method, the results and a proposal to modify the current implementation (that is 20 times slower than it could be)...
I expected a quick decision since I took the pain to make everything clear in the document. I did not understood anything to what happens here. I got some feedback from users of the mailing list and that's it! Nobody took the formal review in charge.
I'm sure many people on this list have a lot on their plate. I'm working 14 hour days for the past several weeks trying to push towards a milestone. In my spare time I like to get some r&r and I keep an eye on several interesting projects one of which is Boost. I saw your posts and followed the discussion but I haven't had time to evalute the work thoroughly enough to do it justice. And this is something that could potentially give us a noticeable speed improvement in the application I'm currently working on. At present I have some fixed-point code, proto, fusion, futures, join, and several other interesting things on my list of stuff to look at in addition to your contribution (and that's just in the scope of Boost). There are only so many hours in a day. Thanks, Michael Marcin
on Wed Jun 13 2007, João Abecasis <jpabecasis-AT-gmail.com> wrote:
Peter Dimov wrote:
It would be nice if we could devise a system that does not suffer from the "silent rejection" problem: you input a review request (or a RM application) into it and receive no output back for months (or at all).
I think part of the "silent rejection" problem is the medium that is being used. In a mailing list it is just too easy to let a message slip by. Perhaps tracking review requests and RM applications in the ticketing system would help.
Brilliant!
Anyway, it's just an idea.
A great one. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com The Astoria Seminar ==> http://www.astoriaseminar.com
David Abrahams wrote:
on Wed Jun 13 2007, João Abecasis <jpabecasis-AT-gmail.com> wrote:
Peter Dimov wrote:
It would be nice if we could devise a system that does not suffer from the "silent rejection" problem: you input a review request (or a RM application) into it and receive no output back for months (or at all). I think part of the "silent rejection" problem is the medium that is being used. In a mailing list it is just too easy to let a message slip by. Perhaps tracking review requests and RM applications in the ticketing system would help.
Brilliant!
Thanks! :-) I'll post a proposal with some more details shortly. Just to get the discussion rolling. Best regards, João
Eric Niebler wrote:
Peter Dimov wrote:
Eric Niebler wrote:
Peter Dimov wrote:
Why do we need a review manager at all? Primarily to avoid any questions or doubts about whether a library should be accepted or not. The review manager supposedly takes everybody's feedback into account, but makes the ultimate yes/no decision, and is even free to buck popular opinion. Do so many of our reviews end in such a non-conclusive manner as to require a decision from a review manager?
It's irrelevant that it doesn't happen often. If it happens EVER and we don't have one person designated to break the tie, there's the potential for a nasty situation. And that one person has to be qualified for his/her opinion to carry weight.
My (perhaps naive) impression is that while reviews need an expert manager, much of the day-to-day work of managing a review (coordinating with the review wizard and library author, posting announcements, soliciting reviews, etc) doesn't require a great deal of expertise. Perhaps in addition to a review manager, there could be a review administrator, who would handle such tasks. The review manager would still need to read the submitted reviews and follow discussions about the library, but his/her actual work would be limited to giving advice and expert opinions to the review administrator as needed, working with the administrator on the results/TODO list, and acting as the final authority in contentious cases. Lightening the manager's workload might increase the pool of available experts, while the administrator position would be a good way for aspiring review managers to gain experience and prove their ability handle such a job.
I feel that this idea has both its upsides and downsides. My understanding of the current situation is that there just aren't enough volunteers to begin with, and I don't think that there's any data saying that removing the administrative side of things will make more people volunteer in the long run. However, were there to be plenty of volunteers, it creates a clear hierarchy of people who are dedicated and interested to choose from for potential future review manager spots, as well as builds interest for people who could be future review managers Jake On 6/6/07, Beth Jacobson <bethj@bajac.com> wrote:
Eric Niebler wrote:
Peter Dimov wrote:
Eric Niebler wrote:
Peter Dimov wrote:
Why do we need a review manager at all? Primarily to avoid any questions or doubts about whether a library should be accepted or not. The review manager supposedly takes everybody's feedback into account, but makes the ultimate yes/no decision, and is even free to buck popular opinion. Do so many of our reviews end in such a non-conclusive manner as to require a decision from a review manager?
It's irrelevant that it doesn't happen often. If it happens EVER and we don't have one person designated to break the tie, there's the potential for a nasty situation. And that one person has to be qualified for his/her opinion to carry weight.
My (perhaps naive) impression is that while reviews need an expert manager, much of the day-to-day work of managing a review (coordinating with the review wizard and library author, posting announcements, soliciting reviews, etc) doesn't require a great deal of expertise. Perhaps in addition to a review manager, there could be a review administrator, who would handle such tasks. The review manager would still need to read the submitted reviews and follow discussions about the library, but his/her actual work would be limited to giving advice and expert opinions to the review administrator as needed, working with the administrator on the results/TODO list, and acting as the final authority in contentious cases.
Lightening the manager's workload might increase the pool of available experts, while the administrator position would be a good way for aspiring review managers to gain experience and prove their ability handle such a job.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Beth Jacobson wrote:
My (perhaps naive) impression is that while reviews need an expert manager, much of the day-to-day work of managing a review (coordinating with the review wizard and library author, posting announcements, soliciting reviews, etc) doesn't require a great deal of expertise. Perhaps in addition to a review manager, there could be a review administrator, who would handle such tasks. The review manager would still need to read the submitted reviews and follow discussions about the library, but his/her actual work would be limited to giving advice and expert opinions to the review administrator as needed, working with the administrator on the results/TODO list, and acting as the final authority in contentious cases.
Lightening the manager's workload might increase the pool of available experts, while the administrator position would be a good way for aspiring review managers to gain experience and prove their ability handle such a job.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
For me, the things you are listing as the administrator's job are a very small part of the work. The real work is making sure I understand the content of the review well enough to generate the recommendation and todo list. Setting the schedule is a matter of a couple of emails. Posting the announcement takes a couple of minutes, tops. On a big review, with dissenting opinions, reading the content and checking for the best available answers has taken me several hours. If I had to coordinate that with an administrator to generate a joint report, I would expect the time to increase markedly. I think looking for ways to encourage more new managers and provide a training and proving ground for those who want to be managers is a good thing. I just don't think this approach will work for that. John
I think looking for ways to encourage more new managers and provide a training and proving ground for those who want to be managers is a good thing. I just don't think this approach will work for that.
Maybe you can be named "Review managers manager" :P Out of joke, this position can really help to change the actual frozen review queue. Tomas has made several posts in the past with the title: "Review managers needed"... But hi is very busy with other boost stuff, this burden can be taken by other person. You have done three reviews, that is a lot of experience. You know what it is needed. IMO a person in this position can encourage active boost users to be review managers. For example, if another container library is presented into boost... he can gently ask some of the containers libs boost authors if he would be interested in managing the review. Best Regards Matias
John Phillips wrote:
Beth Jacobson wrote:
My (perhaps naive) impression is that while reviews need an expert manager, much of the day-to-day work of managing a review (coordinating with the review wizard and library author, posting announcements, soliciting reviews, etc) doesn't require a great deal of expertise. <snip> For me, the things you are listing as the administrator's job are a very small part of the work.
Yeah, judging from recent posts by other review managers, that seems to be the general consensus among those who know best. Oh well.
I think looking for ways to encourage more new managers and provide a training and proving ground for those who want to be managers is a good thing.
There was talk at one time about holding some sort of informal mini-review for all of the SOC libraries at the end of the SOC cycle. As well as giving the developers a chance to present their work, get some valuable feedback, and perhaps generate interest for an eventual formal submission, this might also provide a good opportunity for potential managers to gain some experience and demonstrate their ability to manage a review.
There was talk at one time about holding some sort of informal mini-review for all of the SOC libraries at the end of the SOC cycle. As well as giving the developers a chance to present their work, get some valuable feedback, and perhaps generate interest for an eventual formal submission, this might also provide a good opportunity for potential managers to gain some experience and demonstrate their ability to manage a review.
As a 2007 Summer of Coder, and as a person who is concerned about this issue, I think this is a great solution that kills two birds with one stone. Is there any way we can get a more official opinion on this? Jake
on Wed Jun 06 2007, "Peter Dimov" <pdimov-AT-mmltd.net> wrote:
Eric Niebler wrote:
Peter Dimov wrote:
Why do we need a review manager at all?
Primarily to avoid any questions or doubts about whether a library should be accepted or not. The review manager supposedly takes everybody's feedback into account, but makes the ultimate yes/no decision, and is even free to buck popular opinion.
Do so many of our reviews end in such a non-conclusive manner as to require a decision from a review manager?
IMO yes, enough do. Also, a cursory reading of the votes does not always give enough information to understand whether the library should be accepted. Some votes are made with less consideration than others. IMO it's important that somebody write up the decision and the rationale for the decision, so even if the consensus is apparent, the review manager has an important role to play.
Another key job of the review manager is to collect all the feedback and present a TODO list to the author of the new Boost library, but that's secondary.
I think that the author should be quite capable of doing that him/herself.
That would be nice, but IMO new authors will often be inclined to discount or discard things that would otherwise become important requirements.
This also applies to most of the other review manager duties. The rest can be handled by documentation. The final yes/no decision to accept can be done by the moderators.
That's definitely not a responsibility I want.
Going by the checklist one by one:
* Checks the submission to make sure it really is complete enough to warrant formal review.
Not really required. The reviewers will be quick to point that out.
Sure, but meanwhile we've wasted everyone's time and occupied a valuable review slot.
* Asks the review wizard for permission to extend the review schedule if it appears that too few reviews will be submitted during the review period.
The review can take as long as necessary to gather a sufficient number of reviews. There is no need for a deadline. If we decide to keep the current scheme, the submitter can ask for the extension.
A deadline is an incentive for people who want their vote counted to actually get on with doing a review.
* Decides if there is consensus to accept the library, and if there are any conditions attached.
It is the responsibility of the submitter to prepare a summary of the reviews linking to them and to work with the reviewers to address their concerns. The summary is posted to the list and the moderators decide whether to accept the library.
Puts too much burden on the moderators, IMO. How do we know that all the important concerns have been addressed? The review manager has to follow the review process in detail and make sure nothing important is being stepped over.
* Posts a notice of the review results on the regular boost mailing list, the boost-users mailing list, and the boost-announce mailing list.
The moderators do that.
Another responsibility I don't want. IMO, aside from not having enough volunteers at the moment, a condition I believe we can fix in any number of ways, the review manager role, and reviews in general, are a part of the Boost process that I think really works and doesn't need fixing. FWIW, I have to admit that, although I think it's good to re-examine everything we do from time to time, I am a bit uncomfortable with re-examining everything we do from the ground up all at once, which seems to be the direction in which we're now headed. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
David Abrahams wrote:
IMO, aside from not having enough volunteers at the moment, a condition I believe we can fix in any number of ways, the review manager role, and reviews in general, are a part of the Boost process that I think really works and doesn't need fixing.
Absolutely. My proposed alternative cannot be claimed to be better than the current scheme if experienced and knowledgeable volunteers are available to serve as review managers. It has been conceived with the initial assumption that they aren't. Not having enough RM volunteers has been a persistent problem for years now.
FWIW, I have to admit that, although I think it's good to re-examine everything we do from time to time, I am a bit uncomfortable with re-examining everything we do from the ground up all at once, which seems to be the direction in which we're now headed.
Coincidence. In addition, re-examination does not imply change. It only implies impartial reflection and critical thinking. It's OK to shoot down in flames a proposed change and reaffirm that the current process should stay.
on Thu Jun 07 2007, "Peter Dimov" <pdimov-AT-mmltd.net> wrote:
David Abrahams wrote:
FWIW, I have to admit that, although I think it's good to re-examine everything we do from time to time, I am a bit uncomfortable with re-examining everything we do from the ground up all at once, which seems to be the direction in which we're now headed.
Coincidence.
Relevance?
In addition, re-examination does not imply change.
I know. But I'm still uncomfortable.
It only implies impartial reflection and critical thinking.
Always a good thing.
It's OK to shoot down in flames a proposed change and reaffirm that the current process should stay.
Sure; I just like to think about no more than a few changes at once. In the end you need to make educated guesses about what will work, and it's hard to evaluate any one proposal while thinking, "oh, but we are reevaluating this other piece, so our base assumptions might not apply." -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
I am by no means an expert at the review process, but I have managed a few reviews, and I spent a fair amount of time doing some things that aren't mentioned in Eric's list. 1. Helping a new library author get a review started. While this isn't specifically listed in the role of the review manager, it can be helpful. 2. Helping to navigate some of the politics of the Boost community. It's not bad most of the time, but I've had to intervene more than once. In general, a review manager can make the fairly bureaucratic process less stressful and more pleasant for someone who hasn't already gone through the process several times. I suspect neither Eric nor Peter need this sort of help, but some others probably appreciate the help.
Why do we need a review manager at all?
Primarily to avoid any questions or doubts about whether a library should be accepted or not. The review manager supposedly takes everybody's feedback into account, but makes the ultimate yes/no decision, and is even free to buck popular opinion. It's a representative democracy, twice removed: we elect the review wizard, who elects the review managers, who elect the libraries. Which would make the review managers the electoral college. ;-)
Another key job of the review manager is to collect all the feedback and present a TODO list to the author of the new Boost library, but that's secondary.
Eric Niebler wrote:
Peter Dimov wrote:
David Abrahams wrote:
on Tue Jun 05 2007, Jody Hagins <jody-boost-011304-AT-atdesk.com> wrote:
The review manager does have a fair amount of authority, especially in reviews which are not clear cut one way or another. So, it can't just be given to anyone who asks... though maybe there should be some explanation given to those who are not picked, and maybe some kind of "training roadmap" provided for those interested. Great ideas.
It might be time to rethink this part of the Boost process as well. I'll start with one seemingly simple question:
Why do we need a review manager at all?
Primarily to avoid any questions or doubts about whether a library should be accepted or not. The review manager supposedly takes everybody's feedback into account, but makes the ultimate yes/no decision, and is even free to buck popular opinion. It's a representative democracy, twice removed: we elect the review wizard, who elects the review managers, who elect the libraries. Which would make the review managers the electoral college. ;-)
Actually, I liken the process to more like that of a judge (review manager) deciding a law suit. Advocates for both sides present their cases, arguments, counter arguments etc. When the smoke clears, the judge renders his decision. In cases concerning law, he includes a legal opinion. The reviewer is a mostly disinterested party charged with application of the previously established rules. None of the advocates can play that role with a conflict of interest.
Another key job of the review manager is to collect all the feedback and present a TODO list to the author of the new Boost library, but that's secondary.
Hmm - like a judge passing a "probationary sentence" subject to conditions such as enter rehab. To make the analogy complete, a conditionally accepted library shouldn't be checked until the review manager OK's it. FWIW - I think the boost review process - including the desiganation on one specific person to be responsabe for the decision is a masterful accomplishment. It is: a) totally devoid of any pretense to being a democratic process. Votes don't count. Argument/Rationale do. Imagine if our court decisions were decided by voting. b) avoids compromise for compromise sake - this helps promote conceptual integrity in libraries. If there is anything sacred in boost - this has to be it. Robert Ramey
on Tue Jun 05 2007, Sebastian Redl <sebastian.redl-AT-getdesigned.at> wrote:
Jake Voytko wrote:
It looks to me like the "trainee review managers" idea is similar to the current practice, as per the following page: http://boost.org/more/formal_review_process.htm#Review_Manager
It looks like the bar for being a review manager is fairly low so long as you are knowledgeable in the field. All of the steps in the duties of the review manager seem simple and straightforward (just time consuming!). The "Review Wizards" appear like they are in place to help out the managers with questions about process, etc.
I think the operative term here is "active boost member". What is an active Boost member? Is it someone who has submitted and is maintaining a Boost library? Is it someone who actively helps out in diagnosing problems, enhancing libraries and so on? Or is it someone who simply participates in one or more of the mailing lists and has collected experience that way?
For example, I've been on this list for a year and a half. I have participated in reviews, have helped people out with their problems. I have not fixed any bugs or contributed enhancements. Do I qualify as an "active boost member"?
It's intentionally vague, to give the review wizards some leeway to use their own best judgement. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
.... I'm going to spend my entire summer working on may never get a chance to make it into Boost! And apparently this is not a concern that is unique to me
Is the lack of a manager a sign of a lack of interest? Should managing a review be a prerequisite for having your own library reviewed? (boost credit ?)
It looks to me like the "trainee review managers" idea is similar to the current practice, as per the following page: http://boost.org/more/formal_review_process.htm#Review_Manager
It looks like the bar for being a review manager is fairly low so long as you are knowledgeable in the field. All of the steps in the duties of the review manager seem simple and straightforward (just time consuming!). The "Review Wizards" appear like they are in place to help out the managers with questions about process, etc.
I think time consuming would be a big issue, and maybe the way to gauge interest in a project (prior to putting it on the queue) should be based on people indicating whether they would be willing to review or manage the actual submission if asked. Perhaps there should be more structure recommended for the "Interest in a ....." emails, they should ask specific questions on level of interest, such as "Interest in a XXXX: I am writing a class Widget with code in the vault at http://XXXXX. Yada yada yada.... Is this project of interest to the boost community? Would you review this project during formal review? Would you be willing to manage a review for this project? .... Somebody" This approach is kind of actively soliciting people to volunteer to manage the review if they are actually interested in the project. John
John Phillips wrote:
Has anyone (other than Tom and Ron) looked at the review queue lately?
...
So, now the question. What can we do to improve the situation? Is anyone willing to sign on? Is there so little interest in the work the developers have done that we should say "Thanks for the effort, but we're not interested right now?" Some response is needed, in place of the silence we have seen so far.
John Phillips
Some of the responses so far convince me that we need to clear up confusion about what the qualifications are for a review manager. To some extent, this is a judgment call on the part of the Wizards, but it is also a reasonable topic of conversation for the whole community. I have no reason to consider my opinions definitive, but I'm the one who started this conversation so I shouldn't just lurk with my opinions. With that in mind, here are the qualifications that come to mind. This is not a complete list or proposal, since I haven't given the question enough thought for that, yet. Most of the points are self-evident enough that I don't think detailed explanations are needed. 1) Very familiar with the relevant parts of the language for the submission in question 2) At least solid working familiarity with the problem domain 3) Ability to draw technical content out of sometimes contentious discussions 4) Willingness to suspend personal preference enough to recognize the rationales and strengths of the arguments submitted by all reviewers 5) Time to commit to the review that is adequate for digesting and summarizing all the submissions 6) Solid written communications skills to prepare the summary 7) Familiarity with prior boost reviews, including submitting reviews and participating in the discussions about how to improve a library submission 8) Willingness to ask questions when the unusual arises 9) Ability to make a well reasoned decision at the end of the process, even in the cases where it will not make everyone happy That is a reasonable starting place from my perspective. What I do not think is necessary is that the manager have a history of successfully submitting libraries. While that is certainly a worthy qualification, many of the library authors just don't have time to also be the sole managers. Plus, managing a successful review is a different skill set from writing a successful library: especially in the case where the best available outcome for the review is a rejection that includes a clear critique and plan for how to improve the submission. John
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
participants (19)
-
Andrey Semashev -
Beth Jacobson -
David Abrahams -
Eric Niebler -
fred bertsch -
Jake Voytko -
Jarrad Waterloo -
Jody Hagins -
Johan Råde -
John Femiani -
John Fletcher -
John Phillips -
João Abecasis -
Matias Capeletto -
Michael Marcin -
more effective thinking in the exceptional C++ programming language -
Peter Dimov -
Robert Ramey -
Sebastian Redl