Thank your Vinnie for inviting/pushing me into this charming rabbit hole 3:) NO. The review is over with the announcement of the result. IF you add an

official discussion thread you will end up with a review of the review and no final decision will ever be accepted.

I agree, the decision of the Review Manager should be final. This is not incompatible with a Retrospective phase with the clearly established role of working towards improving the Review process for *future* reviews by gathering useful input from the Community about the *past* review in a transparent and structured discussion. The second guessing without serious reason needs to stop not be encouraged.

It hurts the review process.

My experience with communication would suggest that discouraging feedback and silencing frustrated voices can be detrimental. The more you try to suppress concerns or frustrations, the more tension builds, leading to eventual breaking point. For example, I may be a baby on this ML and yet I've already seen obviously heavily invested people quitting the ML quite rageously: from the answers that followed, this breaking point was clearly hurting both the process and the libraries and should be avoided.

Serious issues are to be brought up with the review wizards off-list.

Criticism can be send to the RM off-list as well. Agreed. This will inevitably raise the case of the process surrounding the Review Wizard designation and responsibilities: i) how are they selected, ii) how are conflicts of interests handled at this designation stage, iii) for how long do Wizards operate, iv) what is the process when they quit, v) what is the process in the case of conflict of interest brought up during a Review. It does not seem to be mentioned in the guidelines and should be made explicit to avoid future problems. If you want to discuss a past review it needs to happen after enough time

has passed, so it doesnt turn into a second round of reviews. We could maybe discuss the "reviews of the past year" every december or something

Yearly feedback could be valuable. It doesn’t address the issue of immediate feedback—both positive and negative—that naturally arises during the review process. Such feedback is a crucial part of the community experience and should be collected before it fades away. It often can’t (and maybe shouldn’t) be casted off some private channels, as it fosters transparency and shared learning. Also, let’s be honest: these conversations will likely surface elsewhere (mhhh lemme guess: slack?). It could be as simple as a 1-10 scale poll answering *How happy were you with this formal review?* Whatever the shape it takes, creating a space for immediate dialogue ensures feedback is acknowledged :) Best wishes, rainbow kitties etc, Arno

Vinnie Falco wrote:

Do note that this proposed "retrospective" took place anyway. The proposal would just make it a bit more formal.

There's no need to make it more formal. If people want to discuss it, they will.

Darryl Green wrote:

On Sun, 10 Nov 2024, 11:17 am Peter Dimov via Boost, wrote:

...

...

Do note that this proposed "retrospective" took place anyway. The

Vinnie Falco wrote: proposal

...

would just make it a bit more formal.

There's no need to make it more formal. If people want to discuss it, they will.

Will they? I still feel like I would be an interloper in these discussions even if I do belatedly look at slack because some hint was given that there was something going on there? I am a regular slack (the platform, not cpplang workspace) user but not in a context where just looking to see what channels have been created and adding my 5c with in them would be considered acceptable. You can take that, in itself, as feedback re a goal I've seen stated somewhere; to increase and extend engagement in Boost in general and reviews in particular.

I don't think that any of this will be fixed if we mandate "retrospective" post-review threads.

On Sat, Nov 9, 2024 at 3:33 PM Klemens Morgenstern via Boost < boost@lists.boost.org> wrote:

NO. The review is over with the announcement of the result. IF you add an official discussion thread you will end up with a review of the review and no final decision will ever be accepted.

The second guessing without serious reason needs to stop not be encouraged. It hurts the review process. Serious issues are to be brought up with the review wizards off-list. Criticism can be send to the RM off-list as well.

Nah. And I know this comment is exactly about me. The thing is, you're the one deciding that what I said wasn't "serious", Klemens. Look, it was just a poorly managed review. That's the point of these retrospectives and why we're having them. We need to have these open discussions to talk about what we could've done better and how we want future reviews to go. For example, we need to start mandating actual quality decorum from review managers. For example, if multiple people on the ML are discussing a design decision but the author isn't responding, it should be on the review manager to step in and encourage the author of the proposed lib to participate. This is something that didn't happen. Also, review managers should feel an onus to actually write down their reasoning for accepting a library, even in the face of open-ended design discussions. It seems like there was more criticism of my review for being "low-effort" than there was for the review manager putting in even less effort. But imagine if I actually had put in more effort only to just be ignored by the author anyway. The thing is, if we want Boost review to actually mean something, we have to actually take steps to ensure that we follow a good process. And a good process is certainly not what happened for this review. - Christian

For the review manager, it means that the review summary needs to contain all the information the manager has taken into account when coming to a decision. If there were discussions out of band, they need to be summarized. If there were discussions _on the list_ that haven't made their way into formal reviews, they need to be summarized.

A review manager should act like an impartial judge through the process,

not like the library’s lawyer.

To me, this reveals an implicit culture, not a formalized process. If the RM was given clear steps, responsibilities and instructions, these disagreements on diverse interpretations of "what the RM is supposed to do" would not be happening. To me, Peter's email is precious information, and should be somehow translated into guidelines, process, docs, templates ... for the RM so it can be agreed on and preserved :) Best wishes and kitties, Arno

On Sun, Nov 10, 2024 at 10:20 AM Peter Dimov via Boost < boost@lists.boost.org> wrote:

I didn't particularly agree then with that process - for me the review manager was, in fact, supposed to tally votes - but the process was what it was, and it worked.

Did the success come from the process rules, or was it the above-average skills of the review managers at the time? And how will this process perform in light of the changing demographics of the volunteer base? Thanks

Thanks Peter for your perspective. I appreciate that. I think the main question here is that some feel certain reviews were not given a fair shake, so they didn’t inform the process to begin with. How is the reviewer being a thorough observer when objections are not outright written off? Authors are not asked to reply, etc? Of course this is a subjective observation of mine, but it has been at the heart of this contention. But in general, yes, it is a position of trust and I understand the reasoning behind it. Maybe this is the nature of the review process and I am uninformed in my expectations, so I’ll defer to you judgement. Claudio.

We're between a rock and a hard place here; on one hand, we suffer from a lack of volunteers who would want to manage reviews, on the other, when Klemens volunteers, people don't like the way he manages the reviews.

It's not clear how we expect this dilemma to magically solve itself.

Thanks Peter, I guess I'm having the wrong expectation out of this process in general, which may be unfair to the people involved. - Claudio

Vinnie, excellent answer - it really clarified a lot. Combined with previous and detailed insights from authors (thanks peter, claudio, darryl, robert!), most of my questions around the unclear context and culture have been addressed (for now :p ). I am glad steps are being taken to freeze these cultural points into solid documentation that boostlings like me can use. Since I was down a GoogleDoc hole, I compiled some constructive feedback from this thread and others and drafted a template document that could (maybe/possibly/dontroastme) be a base for Review Manager reports: https://docs.google.com/document/d/1zHV1ygtC5x7vUWuJk2ib11NlRhgS20U4ELwft49R... It is far from perfect, it does not replace actual mentoring, but at least it gives a clear outline of what kind of report is expected from the Review Manager, and from the RM perspective it takes away the mental charge of putting together the structure and document. Hopefully, with enhancements from the ML, this can help in achieving GRP-level stuff (Great Reviews of the Past), I personally do not have more interrogations. I deeply appreciate how appeased and constructive the dialogue was. Thanks for having all chimed in! I would be irreverent I would say that this Retrospective Phase proved itself to be fruitful and useful, but I am no such thing <3 Have a great week (tomorrow is vacation day in France) 🥳 Best wishes and frenchy kitties, Arno

Christian Mazakas wrote:

The thing is, you're the one deciding that what I said wasn't "serious", Klemens.

It wasn't. If I were the review manager, I would have thought the same. I don't want to pick on you too much because, in principle, you didn't do anything wrong in your review, apart from the insistence that it ought to have been counted as "serious". You are fully within your rights to leave whatever review you like, with the awareness that it would be counted commensurately. Saying that you reject the library because it should have been designed "sans I/O" so that it could be used with any asynchronous backend, including your own (which has 0.2 users), requires justification. You need to explain what you think a sans-I/O library in this domain would look like. You need to give examples of existing libraries that can be used with any asynchronous backend, including your own. You need to explain why you think the value of the library lies in the I/O- independent part, rather than in the I/O management part.

Look, it was just a poorly managed review.

To an extent, yes. In principle, it's the review manager's job to give you the benefit of the doubt and to try to coax a serious review out of you, by for example asking the above questions. I feel like we've lost some implicit knowledge of what a review needs to look like, and what a review summary needs to accomplish. Stated concisely, this knowledge is: If, five years from now, someone would like to know why a library has been accepted or rejected, he _only_ needs to read the review posts and the review summary, and should come away with the correct understanding of what happened. For the reviewer, this means that your review needs to be self-contained. It's not a starting point for a discussion. Nobody is obligated to ask you clarifying questions, and there should be no missing parts that you fill in later in subsequent posts. This means that if you have questions about the library that you feel need to be answered by the author (or the review manager), you should ask these questions _before_ you write your review. For the review manager, it means that the review summary needs to contain all the information the manager has taken into account when coming to a decision. If there were discussions out of band, they need to be summarized. If there were discussions _on the list_ that haven't made their way into formal reviews, they need to be summarized.

On Sun, Nov 10, 2024 at 9:15 AM Peter Dimov via Boost wrote:

It wasn't. If I were the review manager, I would have thought the same.

I don't want to pick on you too much because, in principle, you didn't do anything wrong in your review, apart from the insistence that it ought to have been counted as "serious". You are fully within your rights to leave whatever review you like, with the awareness that it would be counted commensurately.

Of course. I think there's actually been a misunderstanding which has led to a mischaracterization of me. Which is my fault, in all actuality. I did a loose drive-by review which is fine. I didn't expect anyone to actually reject the library just because of my ideas. I was admittedly disappointed when the author didn't have anything to say, just thanking me for my review and then promptly moving on. But the inflection point was when other people on the mailing list said that I had brought up something worth discussing. This was when the review manager should've stepped in. And this was also when the author should've stepped in as well. The fact that they didn't is a poor reflection on them. This was the first strike. The second strike was when Darryl's review was also promptly ignored. If we're being honest here, only Darryl's review should really count, since we seem so insistent on grading others' reviews. The third strike was when the review manager made an abrupt acceptance of the library without a summary. The review manager also chose to specifically thank only the people who wrote acceptance reviews. It was then called out that one of the accept votes came from a reviewer who didn't even get an example compiling and running, which is simply a bad look when choosing to thank a subset of the reviewers.

To an extent, yes. In principle, it's the review manager's job to give you the benefit of the doubt and to try to coax a serious review out of you, by for example asking the above questions.

Yup. Thank you. Dialogues require participation from both sides and I was hoping the review manager would step up and do their job. But more importantly, this is where the author should've really participated. I've been told Boost review is equally about reviewing the author as well as the library. Boost thrives when it's a lively community of C++ developers sharing ideas and design trade-offs. This is what should've happened and the community even asked for it. To me, this review simply subverted the community to rush the library in. And that's where our process has failed us. The end-goal isn't to just accept libraries that come our way, but it's to build a community where people want to participate and great things result from it. In this case, I'm hesitant to say that happened. In the future, I can go into more detail though. I chose not to this time because I didn't want to do a 5000 word dump on the ML, and I instead was hoping the author would just ask me to elaborate on my ideas. For example, all the author had to do was ask me: "what would that look like?" and I would've gone to great lengths to explain it. It seems like in hindsight, this was a mistake on my part. I should've elaborated and made a stronger case, which I now recognize. Boost thrives when smart people write smart things and ultimately, I failed to do that and I own the mistake. - Christian