Re: [boost] [Boost-users] [afio] Formal review of Boost.AFIO
On 25 Aug 2015 at 8:17, Robert Ramey wrote:
Please answer the following questions:
1. Should Boost.AFIO be accepted into Boost? Please state all conditions for acceptance explicity.
Provisionally, yes.
Requirements: <snip>
This list of requirements constitutes much more than a "a few changes". It's impossible to know what what the library would like like after all these changes and and their repercussions were addressed. I don't think we can realistically expect the review manager to verify that and library with all these changes is sufficiently close the current one to take responsibility for signing off on the final version.
I read the provisional yes as meaning yes to how the library is roadmapped to look like in the future, and he recommends an additional mini-review at that point which could introduce yet another mini-review after again. In other words, he's saying fundamentally it's good, but needs a lot more work yet. Paul please do correct me if I am wrong in this interpretation. There is precedent for this: Boost.Fiber was provisionally approved here as a C++ 98 library with condition of a mini-review before entry. Boost.Fiber is now a C++ 14 library and sufficiently different from the library originally reviewed it may require a second mini-review if during its first mini-review it is felt still lacking. I think this pattern of repeated mini-reviews caused by changes to the common implementations of the C++ standard rather than the libraries themselves is going to be very common next few years. All these changes to AFIO are almost entirely driven by changes since 2012 to the various WG21 technical standards. If they hadn't changed and C++ compilers (specifically MSVC) hadn't changed, AFIO wouldn't have changed. It's very similar for Fiber, which had to be refactored in the face of substantial changes in C++ 14. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Niall Douglas Sent: 25 August 2015 18:28 To: boost-users@lists.boost.org; boost@lists.boost.org Subject: Re: [boost] [Boost-users] [afio] Formal review of Boost.AFIO
On 25 Aug 2015 at 8:17, Robert Ramey wrote:
Please answer the following questions:
1. Should Boost.AFIO be accepted into Boost? Please state all conditions for acceptance explicity.
Provisionally, yes.
Requirements: <snip>
I read the provisional yes as meaning yes to how the library is roadmapped to look like in the future, and he recommends an additional mini-review at that point which could introduce yet another mini- review after again. In other words, he's saying fundamentally it's good, but needs a lot more work yet. Paul please do correct me if I am wrong in this interpretation.
Correct - and more - "that we wish and trust Niall to get it right - eventually." but "we want a veto if we *really strongly* object".
There is precedent for this: Boost.Fiber was provisionally approved here as a C++ 98 library with condition of a mini-review before entry. Boost.Fiber is now a C++ 14 library and sufficiently different from the library originally reviewed it may require a second mini-review if during its first mini-review it is felt still lacking.
I think this pattern of repeated mini-reviews caused by changes to the common implementations of the C++ standard rather than the libraries themselves is going to be very common next few years. All these changes to AFIO are almost entirely driven by changes since 2012 to the various WG21 technical standards. If they hadn't changed and C++ compilers (specifically MSVC) hadn't changed, AFIO wouldn't have changed. It's very similar for Fiber, which had to be refactored in the face of substantial changes in C++ 14.
Concur. Paul --- Paul A. Bristow Prizet Farmhouse Kendal UK LA8 8AB +44 (0) 1539 561830
[Paul] Provisionally, yes.
[Niall] I read the provisional yes as meaning yes to how the library is roadmapped to look like in the future, and he recommends an additional mini-review at that point which could introduce yet another mini-review after again.
In other words, he's saying fundamentally it's good, but needs a lot more work yet.
Paul please do correct me if I am wrong in this interpretation.
[Paul] Correct - and more - "that we wish and trust Niall to get it right - eventually."
That did not sound like a strong "Accept" to me. It sounded more like "Not reject" and "Please resubmit for review [after getting it right, eventually]." Glen -- View this message in context: http://boost.2283326.n4.nabble.com/afio-Formal-review-of-Boost-AFIO-tp467911... Sent from the Boost - Dev mailing list archive at Nabble.com.
On 26 Aug 2015 at 5:31, Glen Fernandes wrote:
[Paul] Correct - and more - "that we wish and trust Niall to get it right - eventually."
That did not sound like a strong "Accept" to me. It sounded more like "Not reject" and "Please resubmit for review [after getting it right, eventually]."
The negative review responses so far have fallen into these categories: 1. Not all the APIs it uses are documented (because they are in other libraries). I will not look at this library until that is fixed. 2. Not all the APIs it uses are documented (because they are in other libraries). It therefore should be rejected. 3. The documentation has severe issues and needs a very great deal of additional work before it could be considered Boost-ready. 4. I think the fundamental design is flawed for these reasons X, Y and Z. Categories 1 and 2 are utterly useless to me. I appreciate the motives and where they are coming from, but let me be clear in return: if I bring AFIO back in twelve months time after lots more work, and those same people then say the design is fundamentally flawed for reasons X, Y and Z and should be rejected, I am going to be very upset with them indeed. I think anyone would understand where I would be coming from in that response. Category 3 has been an eye opener to put it mildly. I don't personally think severely flawed documentation is a reason to outright reject though. I do think that insufficient documentation is a reason to reject, but nobody can claim AFIO isn't well documented, it's just badly documented in a highly inaccessible structure with all the wrong information in the wrong places. That said, I think I have the fundamentals right e.g. race guarantees documented per API. There is a good base in there, I just hadn't realised where people get mentally stuck until now. Category 4 has been the most valuable, and that is exactly Thomas Heller's review. I personally believe AFIO's API design is the least worst possible given all the competing factors and pressures, else I'd present something better. However equally I may have chosen something wrong or misestimated the effect of some factor or pressure. Having to explain myself is immensely valuable - indeed right now it's 3.30am and I'm here typing this because I can't sleep thinking about how to explain my API design choices. Terrible on work life balance, hopefully great on a better API design if one is possible. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
On 2015-08-27 04:36, Niall Douglas wrote:
On 26 Aug 2015 at 5:31, Glen Fernandes wrote:
[Paul] Correct - and more - "that we wish and trust Niall to get it right - eventually." That did not sound like a strong "Accept" to me. It sounded more like "Not reject" and "Please resubmit for review [after getting it right, eventually]." The negative review responses so far have fallen into these categories:
1. Not all the APIs it uses are documented (because they are in other libraries). I will not look at this library until that is fixed.
2. Not all the APIs it uses are documented (because they are in other libraries). It therefore should be rejected.
3. The documentation has severe issues and needs a very great deal of additional work before it could be considered Boost-ready.
4. I think the fundamental design is flawed for these reasons X, Y and Z.
Categories 1 and 2 are utterly useless to me. I appreciate the motives and where they are coming from, but let me be clear in return: if I bring AFIO back in twelve months time after lots more work, and those same people then say the design is fundamentally flawed for reasons X, Y and Z and should be rejected, I am going to be very upset with them indeed. I think anyone would understand where I would be coming from in that response. So basically you are saying that anyone who votes against your library for reasons 1 or 2 has no right to vote against it ever again, and you will go to virtual war if they do?
My review will fall in categories 1, 2 and 3 (plus some additional ones). And because I do not have unlimited time on my hand, I simply cannot invest the time to find fundamental flaws. Doing that would require me to basically rectify the issues at least on my machine and then do the actual review. That is impossible. I am willing to attribute this mail of yours to the high amount of stress you feel, but please(!!) stop trying to put pressure on people who intend to write a negative review.
Category 3 has been an eye opener to put it mildly. I don't personally think severely flawed documentation is a reason to outright reject though. I do think that insufficient documentation is a reason to reject, but nobody can claim AFIO isn't well documented,
Again, stop pressurizing, please! I do claim that. How do you expect to get a reasonable review if you basically state as a fact that bad reviews on certain aspects are invalid by definition?
it's just badly documented in a highly inaccessible structure with all the wrong information in the wrong places. That said, I think I have the fundamentals right e.g. race guarantees documented per API. There is a good base in there, I just hadn't realised where people get mentally stuck until now.
Category 4 has been the most valuable, and that is exactly Thomas Heller's review. I personally believe AFIO's API design is the least worst possible given all the competing factors and pressures, else I'd present something better. However equally I may have chosen something wrong or misestimated the effect of some factor or pressure. Having to explain myself is immensely valuable - indeed right now it's 3.30am and I'm here typing this because I can't sleep thinking about how to explain my API design choices. Terrible on work life balance, hopefully great on a better API design if one is possible.
I feel you. And pretty much everybody here, to, I guess. But these constant hints on how hard it was and continues to be and how every additional hour hurts immensely? Please let go of that. This is adding pressure for the reviewers, too. Because if you worked that hard for such a long time under those circumstances, you have to be on the brink. Any negative review has to be devastating. I cannot remember a single review in which the author of the library under review constantly tried to influence reviewers in the way you do. For me personally, that means that I cannot be impartial, which is actually going to be the first item in my review (I finished writing it yesterday, btw, before reading this, I just wanted to check it again after a few hours of sleep). Sending the actual review will not happen until after a few more hours, since I am already late for work for writing this mail, but I had to get that off my chest. Regards, Roland
On 27/08/2015 17:40, Roland Bock wrote:
Categories 1 and 2 are utterly useless to me. I appreciate the motives and where they are coming from, but let me be clear in return: if I bring AFIO back in twelve months time after lots more work, and those same people then say the design is fundamentally flawed for reasons X, Y and Z and should be rejected, I am going to be very upset with them indeed. I think anyone would understand where I would be coming from in that response.
So basically you are saying that anyone who votes against your library for reasons 1 or 2 has no right to vote against it ever again, and you will go to virtual war if they do?
No, he's saying that while you're perfectly entitled to say the things in #1 or #2 if that's how you feel, he would prefer that you not just stop there, but also make comments from categories #3 and #4 as well. ie. don't use #1 or #2 as a justification to completely refuse to review the library in depth, but instead try to find other things "wrong" with it right now, so that they can be addressed before the next review, instead of first being raised *only then*. That's an ideal, of course, and people have limited time and may miss things (particularly if docs are incomplete or obscure -- but then *that* should be raised as an issue). And different people will spot different things, which is the whole point of a group peer review.
On 2015-08-27 08:28, Gavin Lambert wrote:
On 27/08/2015 17:40, Roland Bock wrote:
Categories 1 and 2 are utterly useless to me. I appreciate the motives and where they are coming from, but let me be clear in return: if I bring AFIO back in twelve months time after lots more work, and those same people then say the design is fundamentally flawed for reasons X, Y and Z and should be rejected, I am going to be very upset with them indeed. I think anyone would understand where I would be coming from in that response.
So basically you are saying that anyone who votes against your library for reasons 1 or 2 has no right to vote against it ever again, and you will go to virtual war if they do?
No, he's saying that while you're perfectly entitled to say the things in #1 or #2 if that's how you feel, he would prefer that you not just stop there, but also make comments from categories #3 and #4 as well. Thanks, Gavin. That is a very friendly interpretation. And maybe you are right. But even now, reading that statement from Niall again, I feel quite uneasy:
I intend to put items #1 and #2 in my review among other things. I am not happy about being told that this is "utterly useless". The current situation, time constraints and my current level of knowledge prevent me from finding critical flaws. In a potential future review, the situation will be different, as might my knowledge. Thus, I might find a critical flaw later on. I don't know. Maybe not. It is not my goal. But anyway, if I do, Niall is " going to be very upset with [me] indeed". Not cool.
ie. don't use #1 or #2 as a justification to completely refuse to review the library in depth, but instead try to find other things "wrong" with it right now, so that they can be addressed before the next review, instead of first being raised *only then*.
That's an ideal, of course, and people have limited time and may miss things (particularly if docs are incomplete or obscure -- but then *that* should be raised as an issue). And different people will spot different things, which is the whole point of a group peer review.
I agree with the idea of a group peer review of course. But with his last mail, Niall basically introduced a metric to decide which reviews are valid and which are not. I do not think that the author of a library under review is in the position to do that. Things would be totally different if this were a pre-review or any other informal discussion about AFIO. In that case, I would agree with Niall immediately in asking for concrete details instead of potential formal reasons for rejection. But since this is a formal review, I object to Niall trying to install a bias with statements like in his last mail. Best, Roland
Den 27-08-2015 kl. 10:31 skrev Roland Bock:
On 2015-08-27 08:28, Gavin Lambert wrote:
On 27/08/2015 17:40, Roland Bock wrote:
Categories 1 and 2 are utterly useless to me. I appreciate the motives and where they are coming from, but let me be clear in return: if I bring AFIO back in twelve months time after lots more work, and those same people then say the design is fundamentally flawed for reasons X, Y and Z and should be rejected, I am going to be very upset with them indeed. I think anyone would understand where I would be coming from in that response.
So basically you are saying that anyone who votes against your library for reasons 1 or 2 has no right to vote against it ever again, and you will go to virtual war if they do?
No, he's saying that while you're perfectly entitled to say the things in #1 or #2 if that's how you feel, he would prefer that you not just stop there, but also make comments from categories #3 and #4 as well. Thanks, Gavin. That is a very friendly interpretation.
Actually, I am very surprised that anyone interpreted it differently. And maybe you are
right. But even now, reading that statement from Niall again, I feel quite uneasy:
I intend to put items #1 and #2 in my review among other things. I am not happy about being told that this is "utterly useless".
However, if they were hidden implementation details, you wouldn't care?
The current situation, time constraints and my current level of knowledge prevent me from finding critical flaws. In a potential future review, the situation will be different, as might my knowledge.
Thus, I might find a critical flaw later on. I don't know. Maybe not. It is not my goal. But anyway, if I do, Niall is " going to be very upset with [me] indeed". Not cool.
Agreed to that point.
I agree with the idea of a group peer review of course. But with his last mail, Niall basically introduced a metric to decide which reviews are valid and which are not. I do not think that the author of a library under review is in the position to do that.
I don't think he did. He did not say that the first two metrics were invalid. He said they should not be reason *in themselves* for rejection. Obviously, that's not his call, I agree.
Things would be totally different if this were a pre-review or any other informal discussion about AFIO. In that case, I would agree with Niall immediately in asking for concrete details instead of potential formal reasons for rejection.
But since this is a formal review, I object to Niall trying to install a bias with statements like in his last mail.
Again, I think you misread him. I'm wondering at this point, whether the review shouldn't change status to a pre-review. Actual acceptance - without reservations - seems very unlikely at present, and seeing this as a pre-review might garner Niall the kind of feedback he is hoping for. Is the review manager present? /Brian
On 2015-08-27 12:19, Brian Ravnsgaard Riis wrote:
Den 27-08-2015 kl. 10:31 skrev Roland Bock:
On 2015-08-27 08:28, Gavin Lambert wrote:
On 27/08/2015 17:40, Roland Bock wrote:
Categories 1 and 2 are utterly useless to me. I appreciate the motives and where they are coming from, but let me be clear in return: if I bring AFIO back in twelve months time after lots more work, and those same people then say the design is fundamentally flawed for reasons X, Y and Z and should be rejected, I am going to be very upset with them indeed. I think anyone would understand where I would be coming from in that response.
So basically you are saying that anyone who votes against your library for reasons 1 or 2 has no right to vote against it ever again, and you will go to virtual war if they do?
No, he's saying that while you're perfectly entitled to say the things in #1 or #2 if that's how you feel, he would prefer that you not just stop there, but also make comments from categories #3 and #4 as well. Thanks, Gavin. That is a very friendly interpretation.
Actually, I am very surprised that anyone interpreted it differently.
After all this talk about vendettas and people using this review to take it out on him, maybe I am a bit sensible.
And maybe you are
right. But even now, reading that statement from Niall again, I feel quite uneasy:
I intend to put items #1 and #2 in my review among other things. I am not happy about being told that this is "utterly useless".
However, if they were hidden implementation details, you wouldn't care?
Monad at least is in plain sight. I consider it problematic that the scope of the review is unclear. That will make it hard to interpret the results.
The current situation, time constraints and my current level of knowledge prevent me from finding critical flaws. In a potential future review, the situation will be different, as might my knowledge.
Thus, I might find a critical flaw later on. I don't know. Maybe not. It is not my goal. But anyway, if I do, Niall is " going to be very upset with [me] indeed". Not cool.
Agreed to that point.
I agree with the idea of a group peer review of course. But with his last mail, Niall basically introduced a metric to decide which reviews are valid and which are not. I do not think that the author of a library under review is in the position to do that.
I don't think he did. He did not say that the first two metrics were invalid. He said they should not be reason *in themselves* for rejection. Obviously, that's not his call, I agree.
Things would be totally different if this were a pre-review or any other informal discussion about AFIO. In that case, I would agree with Niall immediately in asking for concrete details instead of potential formal reasons for rejection.
But since this is a formal review, I object to Niall trying to install a bias with statements like in his last mail.
Again, I think you misread him.
I would be surprised if I were the only person who feels uneasy when the author evaluates reviews in this way.
I'm wondering at this point, whether the review shouldn't change status to a pre-review. Actual acceptance - without reservations - seems very unlikely at present, and seeing this as a pre-review might garner Niall the kind of feedback he is hoping for. Is the review manager present?
+1 Just sent my review, btw.
Brian wrote:
I'm wondering at this point, whether the review shouldn't change status to a pre-review. Actual acceptance - without reservations - seems very unlikely at present, and seeing this as a pre-review might garner Niall the kind of feedback he is hoping for. Is the review manager present?
Is there sufficient time remaining? (I count five days, but it seems like Niall will not have electricity for some of them). The review should not be extended for more than three days because there is another review starting 09/04. Glen -- View this message in context: http://boost.2283326.n4.nabble.com/afio-Formal-review-of-Boost-AFIO-tp467911... Sent from the Boost - Dev mailing list archive at Nabble.com.
Den 27-08-2015 kl. 14:13 skrev Glen Fernandes:
Brian wrote:
I'm wondering at this point, whether the review shouldn't change status to a pre-review. Actual acceptance - without reservations - seems very unlikely at present, and seeing this as a pre-review might garner Niall the kind of feedback he is hoping for. Is the review manager present?
Is there sufficient time remaining? (I count five days, but it seems like Niall will not have electricity for some of them).
One of them, if I understood correctly. He should be back online Friday.
The review should not be extended for more than three days because there is another review starting 09/04.
Indeed. /Brian
I noticed that both Hartmut and Brian invoked the review manager. What was the response to both those issues? (If you don't mind making it public). Best, Glen -- View this message in context: http://boost.2283326.n4.nabble.com/afio-Formal-review-of-Boost-AFIO-tp467911... Sent from the Boost - Dev mailing list archive at Nabble.com.
... And because I do not have unlimited time on my hand, I simply cannot invest the time to find fundamental flaws.
This is an incredible important point. If the documentation does not allow us to *easily* understand the design and use cases it should be rejected. Because it may be bad, we just cannot tell. Have now followed a couple of reviews, it seems any largish library needs more than one team member, with different skills. Niall's last message is basically saying he wants a boost review not to rubber-stamp the library for acceptance, but to get some feedback. I.e. to use the boost brain power as a temporary team. Darren
On 2015-08-27 09:23, Darren Cook wrote:
... And because I do not have unlimited time on my hand, I simply cannot invest the time to find fundamental flaws. This is an incredible important point. If the documentation does not allow us to *easily* understand the design and use cases it should be rejected. Because it may be bad, we just cannot tell.
Have now followed a couple of reviews, it seems any largish library needs more than one team member, with different skills. Niall's last message is basically saying he wants a boost review not to rubber-stamp the library for acceptance, but to get some feedback. I.e. to use the boost brain power as a temporary team. And I would be totally fine with that if that intention was explicit, i.e. stop the official review, ask for feedback.
On 27 Aug 2015 at 8:23, Darren Cook wrote:
... And because I do not have unlimited time on my hand, I simply cannot invest the time to find fundamental flaws.
This is an incredible important point. If the documentation does not allow us to *easily* understand the design and use cases it should be rejected. Because it may be bad, we just cannot tell.
Even with excellent documentation, there may end up being so much of it that nobody can be reasonably expected to climb that hill for a Boost review. You may notice from the docs I really like code examples which solve real world rather than toy problems written optimally by an expert in the field. That makes them necessarily non-trivial and makes the learning curve steep, but in my opinion if you need to get up and running with a library quickly those examples to study are invaluable. It's my biggest bone to pick with ASIO's docs actually: there are only four or five of the example programs which approach real world solutions and are therefore useful when getting up to speed with ASIO quickly when you have a real world problem to solve. I still ended up searching ASIO's implementation code and examples of real world usage on the internet far more frequently than I should. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
On 27 Aug 2015 at 7:40, Roland Bock wrote:
Categories 1 and 2 are utterly useless to me. I appreciate the motives and where they are coming from, but let me be clear in return: if I bring AFIO back in twelve months time after lots more work, and those same people then say the design is fundamentally flawed for reasons X, Y and Z and should be rejected, I am going to be very upset with them indeed. I think anyone would understand where I would be coming from in that response. So basically you are saying that anyone who votes against your library for reasons 1 or 2 has no right to vote against it ever again, and you will go to virtual war if they do?
It was you who brought up "virtual war" by interpreting my statement this way. I said I would be very upset with people who choose to actively refuse to review this library now on a point of principle and in twelve months time then find fundamental design flaws in it because, well anybody would be.
My review will fall in categories 1, 2 and 3 (plus some additional ones). And because I do not have unlimited time on my hand, I simply cannot invest the time to find fundamental flaws. Doing that would require me to basically rectify the issues at least on my machine and then do the actual review. That is impossible.
I don't know where you are getting this from. Nobody asked you to rewrite code. Nobody even asked you personally to participate in this review.
Category 3 has been an eye opener to put it mildly. I don't personally think severely flawed documentation is a reason to outright reject though. I do think that insufficient documentation is a reason to reject, but nobody can claim AFIO isn't well documented, Again, stop pressurizing, please! I do claim that.
How do you expect to get a reasonable review if you basically state as a fact that bad reviews on certain aspects are invalid by definition?
I am allowed to have an opinion on how the review is going, which I shared. I don't think doing that is inappropriate. I stated that AFIO documentation has more detail than you could ever need on parts most people don't need nor want. It has insufficient detail on the parts people actually do need and want. Does this mean AFIO is well documented or not? I claimed and do claim yes. It's well documented on all the wrong things, that's all. I didn't know that till this week.
I feel you. And pretty much everybody here, to, I guess. But these constant hints on how hard it was and continues to be and how every additional hour hurts immensely? Please let go of that. This is adding pressure for the reviewers, too. Because if you worked that hard for such a long time under those circumstances, you have to be on the brink. Any negative review has to be devastating.
You appear to have a personal bone to pick. I would also worry less about whether I am "on the brink" or whether I would be "devastated", and more about whether any review you write was fair and impartial. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
On 27 Aug 2015 at 7:40, Roland Bock wrote:
I feel you. And pretty much everybody here, to, I guess. But these constant hints on how hard it was and continues to be and how every additional hour hurts immensely? Please let go of that. This is adding pressure for the reviewers, too. Because if you worked that hard for such a long time under those circumstances, you have to be on the brink. Any negative review has to be devastating. You appear to have a personal bone to pick. I don't. Maybe we run into each other at CppCon? I can explain
On 2015-08-28 02:55, Niall Douglas wrote: personally. That's easier and more efficient than mail, I guess.
participants (7)
-
Brian Ravnsgaard Riis -
Darren Cook -
Gavin Lambert -
Glen Fernandes -
Niall Douglas -
Paul A. Bristow -
Roland Bock