Hi All, Recently, we've seen a shortage of review managers: Joaquin has been looking for one for his bloom-filters library for a few weeks (now solved), and Dmitry recently withdrew his GDB debugging tools due to lack of review manager. Looking at the review schedule for the last few years[1] you'll find a number of repeated names and long-time contributors. In an effort to find new volunteers for review managers I would like to ask what are the showstoppers that keep you from wanting to be a review manager? Time, Training, Recognition, none of the above? I am genuinely curious in an effort to find new volunteers so we can continue to add great libraries to the project. Thank you, and Happy Easter to those that celebrate. Matt [1] https://www.boost.org/community/review_schedule.html
On Thu, Apr 17, 2025 at 10:41 AM Matt Borland via Boost < boost@lists.boost.org> wrote:
Recently, we've seen a shortage of review managers...
There might also be a related problem: libraries which have long since completed their review process, for which the review results have not been posted. Such as Boost.SQLite. Thanks
On 17 Apr 2025 20:40, Matt Borland via Boost wrote:
I would like to ask what are the showstoppers that keep you from wanting to be a review manager? Time, Training, Recognition, none of the above?
I think the requirement to be an expert or at least very knowledgeable in the problem domain is a big part of why. Personally, I do not see myself qualified to judge on the qualities of the recently proposed libraries. And, as always, time is limited, too.
On Thu, Apr 17, 2025 at 4:05 PM Andrey Semashev via Boost < boost@lists.boost.org> wrote:
On 17 Apr 2025 20:40, Matt Borland via Boost wrote:
I would like to ask what are the showstoppers that keep you from wanting
to be a review manager? Time, Training, Recognition, none of the above?
I think the requirement to be an expert or at least very knowledgeable in the problem domain is a big part of why. Personally, I do not see myself qualified to judge on the qualities of the recently proposed libraries.
I agree, and.. I've wondered about that requirement. Is it really needed? If an established Boost developer has a good amount of professional experience they are likely to be able to tackle most programming domains at the level needed for managing a review. Should we reconsider that requirement? For example I would consider myself to know enough to manage the Bloom review. But... And, as always, time is limited, too.
Yes, that aspect also gets in the way of me volunteering to manage or review anything. -- -- René Ferdinand Rivera Morell -- Don't Assume Anything -- No Supongas Nada -- Robot Dreams - http://robot-dreams.net
On 18 Apr 2025 01:01, René Ferdinand Rivera Morell wrote:
On Thu, Apr 17, 2025 at 4:05 PM Andrey Semashev via Boost <boost@lists.boost.org <mailto:boost@lists.boost.org>> wrote:
On 17 Apr 2025 20:40, Matt Borland via Boost wrote: > > I would like to ask what are the showstoppers that keep you from wanting to be a review manager? Time, Training, Recognition, none of the above?
I think the requirement to be an expert or at least very knowledgeable in the problem domain is a big part of why. Personally, I do not see myself qualified to judge on the qualities of the recently proposed libraries.
I agree, and.. I've wondered about that requirement. Is it really needed? If an established Boost developer has a good amount of professional experience they are likely to be able to tackle most programming domains at the level needed for managing a review. Should we reconsider that requirement? For example I would consider myself to know enough to manage the Bloom review. But...
Given that the review manager is the one solely responsible for deciding whether a library deserves acceptance, it would be strange not to require a certain, fairly high level of expertise.
Andrey Semashev wrote:
On 18 Apr 2025 01:01, René Ferdinand Rivera Morell wrote:
On Thu, Apr 17, 2025 at 4:05 PM Andrey Semashev via Boost <boost@lists.boost.org <mailto:boost@lists.boost.org>> wrote:
On 17 Apr 2025 20:40, Matt Borland via Boost wrote: > > I would like to ask what are the showstoppers that keep you from wanting to be a review manager? Time, Training, Recognition, none of the above?
I think the requirement to be an expert or at least very knowledgeable in the problem domain is a big part of why. Personally, I do not see myself qualified to judge on the qualities of the recently proposed libraries.
I agree, and.. I've wondered about that requirement. Is it really needed? If an established Boost developer has a good amount of professional experience they are likely to be able to tackle most programming domains at the level needed for managing a review. Should we reconsider that requirement? For example I would consider myself to know enough to manage the Bloom review. But...
Given that the review manager is the one solely responsible for deciding whether a library deserves acceptance, it would be strange not to require a certain, fairly high level of expertise.
The domain-specific expertise should come from the reviewers; the review manager should (minimally) just be qualified enough to evaluate the reviews. General C++ expertise is more important for the review manager because he is supposed to help the submitter prepare the library so that it's suitable for Boost.
On 18 Apr 2025 04:51, Peter Dimov via Boost wrote:
Andrey Semashev wrote:
On 18 Apr 2025 01:01, René Ferdinand Rivera Morell wrote:
On Thu, Apr 17, 2025 at 4:05 PM Andrey Semashev via Boost <boost@lists.boost.org <mailto:boost@lists.boost.org>> wrote:
On 17 Apr 2025 20:40, Matt Borland via Boost wrote: > > I would like to ask what are the showstoppers that keep you from wanting to be a review manager? Time, Training, Recognition, none of the above?
I think the requirement to be an expert or at least very knowledgeable in the problem domain is a big part of why. Personally, I do not see myself qualified to judge on the qualities of the recently proposed libraries.
I agree, and.. I've wondered about that requirement. Is it really needed? If an established Boost developer has a good amount of professional experience they are likely to be able to tackle most programming domains at the level needed for managing a review. Should we reconsider that requirement? For example I would consider myself to know enough to manage the Bloom review. But...
Given that the review manager is the one solely responsible for deciding whether a library deserves acceptance, it would be strange not to require a certain, fairly high level of expertise.
The domain-specific expertise should come from the reviewers; the review manager should (minimally) just be qualified enough to evaluate the reviews.
Reviewers are allowed to be less familiar with the domain, and in fact may not be familiar at all. There were plenty examples of such, including myself in the recent Boost.Hash2 review. The review manager's job is to weigh the quality and relevance of the reviews, and to be able to do that the review manager has to be at least as qualified as the reviewers, very preferably more. In fact, the review manager is within rights to discard a review from consideration if he believes it is inadequate. Reviews are supposed to indicate public opinion of the library and highlight pitfalls and strengths of the proposed solution. They *help* the review manager to reach a conclusion, but given the lack of useful reviews the review manager should be capable to reach it *on his own*. He may refuse to do so e.g. due to excessive responsibility, but technically it is a possibility. It is not a mere formality that a review manager volunteers and must be accepted by the library author. The acceptance means that the author trusts that the review manager is qualified enough to evaluate the library and the reviews and come up with a fair decision in the end.
General C++ expertise is more important for the review manager because he is supposed to help the submitter prepare the library so that it's suitable for Boost.
While important, I see this as a less important part of the review manager's work, compared to making the final decision. That is what matters most.
It is not a mere formality that a review manager volunteers and must be accepted by the library author. The acceptance means that the author trusts that the review manager is qualified enough to evaluate the library and the reviews and come up with a fair decision in the end.
Does the accomplishment of this task require someone to be an expert in the domain? How are we defining expert? I am with Peter here that I think this wording is too strong, and may scare people off. As long as the library author agrees thinks the review manager is competent and capable that should be sufficient. Matt
Andrey Semashev wrote:
On 18 Apr 2025 04:51, Peter Dimov via Boost wrote:
The domain-specific expertise should come from the reviewers; the review manager should (minimally) just be qualified enough to evaluate the reviews.
Reviewers are allowed to be less familiar with the domain, and in fact may not be familiar at all. There were plenty examples of such, including myself in the recent Boost.Hash2 review. The review manager's job is to weigh the quality and relevance of the reviews, and to be able to do that the review manager has to be at least as qualified as the reviewers, very preferably more.
For Hash2, we got a review from one of the authors of Blake2 (if we knew that in advance we'd certainly have included Blake2 in the initial submission.) I don't think it would have been realistic to expect or demand equal or higher level of expertise from the review manager in this case.
On Fri, Apr 18, 2025 at 4:47 PM Peter Dimov via Boost <boost@lists.boost.org> wrote:
On 18 Apr 2025 04:51, Peter Dimov via Boost wrote:
The domain-specific expertise should come from the reviewers; the review manager should (minimally) just be qualified enough to evaluate
Andrey Semashev wrote: the
reviews.
Reviewers are allowed to be less familiar with the domain, and in fact may not be familiar at all. There were plenty examples of such, including myself in the recent Boost.Hash2 review. The review manager's job is to weigh the quality and relevance of the reviews, and to be able to do that the review manager has to be at least as qualified as the reviewers, very preferably more.
For Hash2, we got a review from one of the authors of Blake2 (if we knew that in advance we'd certainly have included Blake2 in the initial submission.)
I don't think it would have been realistic to expect or demand equal or higher level of expertise from the review manager in this case.
I understand the difficulties of finding a review manager, but in this discussion I am mostly agreeing with Andrey point. To use your example: there was no guarantee that author of Blake2 would submit a review. And in my opinion at least one person who is judging the library needs to be a domain expert. Now I see two options: 1. keep the current requirement that manager is a domain expert 2. drop the requirement that manager needs to be an expert if we are confident expert(s) will submit reviews This way we have at least one domain expert involved with approval. Issue here is that this complicates the process, by requiring reviewers to agree beforehand to submit a review, compared to current one where it is much easier for reviewers to submit a review. Just to be clear I do understand how difficult is to find anybody to do unpaid work, not to mention highly skilled domain experts to do unpaid work, but I believe that beside author at least one person involved in review needs to be an expert in the domain, and that general C++ expert knowledge is not enough in general case(counter example would be variant2 that is "pure" C++ library without need for reviewers or manager to be experts in unicode or networking or coroutines or ...).
On Fri, Apr 18, 2025 at 1:56 PM Ivan Matek via Boost <boost@lists.boost.org> wrote:
On Fri, Apr 18, 2025 at 4:47 PM Peter Dimov via Boost < boost@lists.boost.org> wrote:
On 18 Apr 2025 04:51, Peter Dimov via Boost wrote:
The domain-specific expertise should come from the reviewers; the review manager should (minimally) just be qualified enough to
evaluate
Andrey Semashev wrote: the
reviews.
Reviewers are allowed to be less familiar with the domain, and in fact may not be familiar at all. There were plenty examples of such, including myself in the recent Boost.Hash2 review. The review manager's job is to weigh the quality and relevance of the reviews, and to be able to do that the review manager has to be at least as qualified as the reviewers, very preferably more.
For Hash2, we got a review from one of the authors of Blake2 (if we knew that in advance we'd certainly have included Blake2 in the initial submission.)
I don't think it would have been realistic to expect or demand equal or higher level of expertise from the review manager in this case.
I understand the difficulties of finding a review manager, but in this discussion I am mostly agreeing with Andrey point. To use your example: there was no guarantee that author of Blake2 would submit a review. And in my opinion at least one person who is judging the library needs to be a domain expert. Now I see two options:
1. keep the current requirement that manager is a domain expert 2. drop the requirement that manager needs to be an expert if we are confident expert(s) will submit reviews
This way we have at least one domain expert involved with approval. Issue here is that this complicates the process, by requiring reviewers to agree beforehand to submit a review, compared to current one where it is much easier for reviewers to submit a review.
Just to be clear I do understand how difficult is to find anybody to do unpaid work, not to mention highly skilled domain experts to do unpaid work, but I believe that beside author at least one person involved in review needs to be an expert in the domain, and that general C++ expert knowledge is not enough in general case(counter example would be variant2 that is "pure" C++ library without need for reviewers or manager to be experts in unicode or networking or coroutines or ...).
A problem with that thought I see is that we don't have any guarantee of domain experts doing reviews. And, honestly, I think being a domain expert gets in the way of being an impartial review manager. And I consider being an impartial manager is rather important. As domain experts often inject their own opinions (likes and dislikes) into the decision. Which can be counterproductive when it comes to evaluating what we hope are forward thinking modern library submissions. -- -- René Ferdinand Rivera Morell -- Don't Assume Anything -- No Supongas Nada -- Robot Dreams - http://robot-dreams.net
пт, 18 апр. 2025 г. в 21:56, Ivan Matek via Boost <boost@lists.boost.org>:
Now I see two options:
1. keep the current requirement that manager is a domain expert
Just to be clear, as far as I am aware, there is no actual *requirement* that the review manager is an expert in the reviewed library's domain. There is a certain level of understanding of the domain that is required to correctly assess the quality of the library and of the reviews, but I wouldn't call that being an expert. Like many things in Boost, this is kind of based on vibes. In my opinion this thread is about the fact that maybe potential review managers are hesitant to step up, because they are too hard on themselves. This on the one hand spares us of completely inadequate candidates, but on the other hand we have the same few people managing reviews.
On Fri, Apr 18, 2025 at 9:11 PM Дмитрий Архипов via Boost < boost@lists.boost.org> wrote:
пт, 18 апр. 2025 г. в 21:56, Ivan Matek via Boost <boost@lists.boost.org>: Just to be clear, as far as I am aware, there is no actual *requirement* that the review manager is an expert in the reviewed library's domain. There is a certain level of understanding of the domain that is required to correctly assess the quality of the library and of the reviews, but I wouldn't call that being an expert. Like many things in Boost, this is kind of based on vibes.
I was referring to this part. *In order for a review to proceed, a Boost member must volunteer to manage the review. This should be someone with experience with the review process and knowledge of the library's domain. * from https://www.boost.org/community/review_schedule.html but I agree that this is open to interpretation.
On 4/17/25 6:51 PM, Peter Dimov via Boost wrote:
I agree, and.. I've wondered about that requirement. Is it really needed? If an established Boost developer has a good amount of professional experience they are likely to be able to tackle most programming domains at the level needed for managing a review. Should we reconsider that requirement? For example I would consider myself to know enough to manage the Bloom review. But...
Given that the review manager is the one solely responsible for deciding whether a library deserves acceptance, it would be strange not to require a certain, fairly high level of expertise.
The domain-specific expertise should come from the reviewers; the review manager should (minimally) just be qualified enough to evaluate the reviews.
General C++ expertise is more important for the review manager because he is supposed to help the submitter prepare the library so that it's suitable for Boost.
I think it's pretty hard to set a specific policy here. Take a library like serialization. Most of us would be comfortable being review manager as we know what the domain is about and something about the features which different approaches. Take a library like safe_numerics. I think this would require a review manager with above average knowledge of numerical analysis. So I think the current process for selecting review managers is fine. Of course there will be issues from time to time. 1) Sometimes the review manager way underestimates the amount of time it will take to do the review. So the result is delayed. In at least one case I know of the review manager flaked out entirely. I'm sure there are more such chases. 2) It's hard to find a qualified reviewer for some libraries. No easy fix. The only thing that occurs to me that in our communications/announcements we increase the level of recognition for review manager - more or less to the level of that of author. This would be in the hope of making RM to be sufficiently prestigious that he might be motivated to note the fact that he was RM on his resume and/or other credentials. Personal experience note: I'm the author of the boost safe numerics library. I developed it incrementally over many years on my own until I felt it was ready to submit to boost. It had all the boost requirements: documentation, tests, etc. etc. I submitted it and it was accepted into Boost with conditions. I believe that Andrey Semashev was the review manager. I should know this for sure off the top of my head. The fact that I don't illustrates my point above. I was ecstatic to get this news and went right to work making modifications to full fill the conditions and other issues Andre had flagged on his own. In doing this task, I realized that the library was much, much lower quality than I had thought. In cleaning up these "nits" I found lots and lots of other issues : concepts not agreeing with code, undetected errors, etc, etc. The same thing happened with the serialization library ~24 years ago. At that time I had an excuse: I was still learning C++ (I'm forgetting it now). But this time ... I was honestly appalled and disheartened with my own work. With Andres help, I managed to fix everything. But I've learned so much in the meantime, that I'm tempted to make another pass over the library to make it more formally correct and eliminate unnecessary requirements. From feedback I get from users, and some statistics, I believe that the library has pretty good up take. I don't think this be so much the case without Andre's collaboration. This may not be true for other authors and their libraries, but it's a fact that it's true for mine. So maybe we should highlight/promote more of the contributions of reviewers and review manager in our documentation and promotional material (announcements of library acceptance, etc.). (do we have promotional material?) In fact we should spend a little more effort in promoting boost, libraries, authors and reviewers. Did you know that the boost serialization library is used in the software that runs "icecube" - huge project in antarctica which I believe is used to study neutrinos or something like that. I only found out about this by accident (no not that kind of accident). It's a huge motivator for me to be associated with a project like that - albeit peripherally. Robert Ramey
On 22 Apr 2025 22:57, Robert Ramey via Boost wrote:
Personal experience note: I'm the author of the boost safe numerics library. I developed it incrementally over many years on my own until I felt it was ready to submit to boost. It had all the boost requirements: documentation, tests, etc. etc. I submitted it and it was accepted into Boost with conditions. I believe that Andrey Semashev was the review manager.
I was not, it was Andrzej Krzemienski. https://www.boost.org/community/review_schedule.html
On 4/22/25 2:34 PM, Andrey Semashev via Boost wrote:
On 22 Apr 2025 22:57, Robert Ramey via Boost wrote:
Personal experience note: I'm the author of the boost safe numerics library. I developed it incrementally over many years on my own until I felt it was ready to submit to boost. It had all the boost requirements: documentation, tests, etc. etc. I submitted it and it was accepted into Boost with conditions. I believe that Andrey Semashev was the review manager.
I was not, it was Andrzej Krzemienski.
Damn!!! This is embarasing to me (which doesn't happen often!) Just proves my point that reviewers deserve higher visibility. Robert Ramey
вт, 22 апр. 2025 г. в 22:57, Robert Ramey via Boost <boost@lists.boost.org>:
So maybe we should highlight/promote more of the contributions of reviewers and review manager in our documentation and promotional material (announcements of library acceptance, etc.). (do we have promotional material?) In fact we should spend a little more effort in promoting boost, libraries, authors and reviewers.
The C++ Alliance tries to do things in this area. Vinnie had an idea for the new Boost web site to give badges to people for accomplishing various Boost-related tasks: filing issues, contributing code, reviewing and managing reviews. Maybe we can also increase visibility of review managers by displaying their names on library landing pages and on their documentation home pages?
The C++ Alliance tries to do things in this area. Vinnie had an idea for the new Boost web site to give badges to people for accomplishing various Boost-related tasks: filing issues, contributing code, reviewing and managing reviews. Maybe we can also increase visibility of review managers by displaying their names on library landing pages and on their documentation home pages?
On the new website we could revamp the Boost people page (https://www.boost.org/users/people.html). There could be a section for current leadership (FSC), authors, review managers, sustained contributors etc. It would be easy for someone to point to the website as a reference to their involvement in Boost. Matt
El 23/04/2025 a las 15:45, Matt Borland via Boost escribió:
[...]
On the new website we could revamp the Boost people page (https://www.boost.org/users/people.html). There could be a section for current leadership (FSC), authors, review managers, sustained contributors etc. It would be easy for someone to point to the website as a reference to their involvement in Boost.
This looks like an excellent idea to me. Joaquin M Lopez Munoz
Joaquin M López Muñoz wrote:
El 23/04/2025 a las 15:45, Matt Borland via Boost escribió:
[...]
On the new website we could revamp the Boost people page (https://www.boost.org/users/people.html). There could be a section for current leadership (FSC), authors, review managers, sustained contributors etc. It would be easy for someone to point to the website as a reference to their involvement in Boost.
This looks like an excellent idea to me.
I like it too.
On 4/22/25 11:58 PM, Дмитрий Архипов via Boost wrote:
вт, 22 апр. 2025 г. в 22:57, Robert Ramey via Boost <boost@lists.boost.org>:
So maybe we should highlight/promote more of the contributions of reviewers and review manager in our documentation and promotional material (announcements of library acceptance, etc.). (do we have promotional material?) In fact we should spend a little more effort in promoting boost, libraries, authors and reviewers.
The C++ Alliance tries to do things in this area. ...Maybe we can also increase visibility of review managers by displaying their names on library landing pages> and on their documentation home pages?
This is the kind of good idea that hadn't occurred to me but I hoped someone would come up with. Ideally it would be sufficiently prestigious that reviewers would note their service on their resumen - perhaps even a boost logo linking to the page of the library they reviewed. I think we're on a good track here. Same goes for library maintainers. I did create a "Boost Library Maintainers" group and a couple signed up, but anything like this needs more effort dedicated to promotion. Robert Ramey
I would like to ask what are the showstoppers>> that keep you from wanting to be a review>> manager? Time, Training, Recognition,>> none of the above? I think the requirement to be an expert or> at least very knowledgeable in the problem> domain is a big part of why. I sense that this may be a good point. Mandating such expertise can, in fact,be a discouragement. The review managershould be a moderate and fair reviewmanager, not some jock-expert in theextremely limited area of a particularreview. Being an expert in the domain may(at times) lead to hasty and biasedmoderator comments. These detract fromthe overall quality of the review. There is no perfect solution to this. But demanding expert domain knowledgefrom the review manager is probablyworse than it is better, raising thebar in a way is counter-productive. Christopher
On Thursday, April 17, 2025 at 11:05:03 PM GMT+2, Andrey Semashev via Boost <boost@lists.boost.org> wrote: On 17 Apr 2025 20:40, Matt Borland via Boost wrote:
I would like to ask what are the showstoppers that keep you from wanting to be a review manager? Time, Training, Recognition, none of the above?
I think the requirement to be an expert or at least very knowledgeable in the problem domain is a big part of why. Personally, I do not see myself qualified to judge on the qualities of the recently proposed libraries. And, as always, time is limited, too. _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On Thu, Apr 17, 2025, 12:41 PM Matt Borland via Boost <boost@lists.boost.org> wrote:
... and Dmitry recently withdrew his GDB debugging tools due to lack of review manager.
I am still willing to act as review manager for this library, if opinions change. I understand the reasoning for wanting someone else, but my offer stands regardless. Braden
Matt, I would like to volunteer to be review manager for bloom-filters, unless of course someone else would rather volunteer for it. Claudio
Matt,
I would like to volunteer to be review manager for bloom-filters, unless of course someone else would rather volunteer for it.
Claudio
Claudio, Thank you, but yesterday Arnaud Becheler volunteered to be the review manager. I can add you to my rolodex of people to contact when another review comes down the pipeline without a review manager (or should Arnaud need to back out). Matt
Thank you, but yesterday Arnaud Becheler volunteered to be the review manager. I can add you to my rolodex of people to contact when another review comes down the pipeline without a review manager (or should Arnaud need to back out)
Ah, no worries. I had offered, as I didn’t particularly wanted this library to languish for the lack of a review manager. I’m not opposed to be asked about any other library. Claudio
participants (12)
-
Andrey Semashev -
Braden Ganetsky -
Christopher Kormanyos -
Claudio DeSouza -
Ivan Matek -
Joaquin M López Muñoz -
Matt Borland -
Peter Dimov -
René Ferdinand Rivera Morell -
Robert Ramey -
Vinnie Falco -
Дмитрий Архипов