Re: [boost] Breaking existing libraries
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of boost-request@lists.boost.org Sent: Thursday, November 20, 2008 5:42 PM To: boost@lists.boost.org Subject: Boost Digest, Vol 2373, Issue 4
Yes, this is an unfortunate situation. If you indeed want to try to make sure this kind of silent breakage doesn't happen again, there's something you can do about it: write a page of library maintenance guidelines on the wiki at http://svn.boost.org. We are sorely missing a policy on this, and when less-experienced library authors come along, they often don't hew to the same standards that many of us take for granted. If you don't have write permission for the wiki, please let us know and we'll send you an invitation.
Hello Dave, This is a good answer but not the one I was looking for. It's a politician's answer, in that it doesn't really answer my original query. Let me just put a few of my thoughts here: What was described in my original email are bad practices in any project, not just boost. Given that the boost developers are presented frequently as "the best", it isn't unreasonable to expect that they would understand that causing silent breakage (and the other points I made) is bad. It takes a lot of work for a library to get accepted and the library has to meet very high standards. However, it seems that once a library is accepted, the maintainer is pretty much free to do what they please. I think it may be wise to apply the same level of rigour to the continued development. Do you think that a boost library that was broken should be rolled back? About the wiki: The desire to solve this problem has to come from within, not from without. Given how big some of the personalities are on this mailing list, do you think it's likely they would listen to someone who has not contributed a complete library to boost? Finally, I'd like to mention that within a large organisation, it's not possible to upgrade to the latest version of boost as soon as it comes out. The difficulty of rebuilding all of the internal systems just makes this impractical. As you are aware, a lot of people avoid the .0 releases. What does this say to you? I understand (and applaud) boost's efforts to try to to eliminate this notion with tighter release cycles etc but the problem is that it's much harder to lose people's confidence than to regain it. I like using boost and am very grateful to all the developers who make it happen. Having said that, SNAFUs such as the current Boost.Range or the MT problem with Boost.Function in 1.34 threaten to make it irrelevant in "serious" development and relegate it to the position of a great "toy library" you can see how C++ standard compliant your compiler is. Kind regards, Tom
on Fri Nov 21 2008, "Tomas Puverle" <Tomas.Puverle-AT-morganstanley.com> wrote:
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of boost-request@lists.boost.org Sent: Thursday, November 20, 2008 5:42 PM To: boost@lists.boost.org Subject: Boost Digest, Vol 2373, Issue 4
Yes, this is an unfortunate situation. If you indeed want to try to make sure this kind of silent breakage doesn't happen again, there's something you can do about it: write a page of library maintenance guidelines on the wiki at http://svn.boost.org. We are sorely missing a policy on this, and when less-experienced library authors come along, they often don't hew to the same standards that many of us take for granted. If you don't have write permission for the wiki, please let us know and we'll send you an invitation.
Hello Dave,
This is a good answer but not the one I was looking for.
Naturally, nobody likes to be asked to fix the problem they're complaining about. Such is the open-source world, I'm afraid.
It's a politician's answer, in that it doesn't really answer my original query.
Umm. Trying not to be insulted, here. I disagree. Your original query was not a "query" at all. You asked no questions. There was a list of (valid) complaints. The only actionable item I saw there (which you snipped out of the context of my quote) was the expression of a laudable desire "to try to make sure this kind of silent breakage doesn’t happen again," and *that* is what I responded to.
Let me just put a few of my thoughts here:
What was described in my original email are bad practices in any project, not just boost. Given that the boost developers are presented frequently as "the best", it isn't unreasonable to expect that they would understand that causing silent breakage (and the other points I made) is bad.
Maybe not. The truth is that not all of our developers can be "the best." Every time a library written by someone who doesn't already have a library in Boost is accepted, we gain a new developer. Of course, you also have some power to control the quality of Boost developers, by participating in the review process.
It takes a lot of work for a library to get accepted and the library has to meet very high standards.
Usually, I hope. There have been a few libraries --- which shall remain nameless --- that I didn't think met Boost standards when they were accepted. When I noticed that happening, I started to pay more attention to the review process.
However, it seems that once a library is accepted, the maintainer is pretty much free to do what they please.
Yes.
I think it may be wise to apply the same level of rigour to the continued development. Do you think that a boost library that was broken should be rolled back?
Quite possibly. Depends what you mean by "broken" and "rolled back."
About the wiki: The desire to solve this problem has to come from within, not from without.
I've got some bad news for you, Tom: you're "within." The Boost community is made up of more than just its volunteer developers, and they don't have time to do everything. I personally have the desire to write up such a page, but not the time right now.
Given how big some of the personalities are on this mailing list, do you think it's likely they would listen to someone who has not contributed a complete library to boost?
Yes, and so do you, or you wouldn't bother complaining about stability here, where nobody would listen. "Big personalities" have little to do with it; Boost library authors generally highly value the input of their users, and are eager to learn. That's part of why they participate here. You may think you're being asked to set Boost policy on your own, but that's not how it works. Boost is driven largely by consensus. We'll discuss and tweak the document together in this forum until we can arrive at a consensus that it should be adopted.
Finally, I'd like to mention that within a large organisation, it's not possible to upgrade to the latest version of boost as soon as it comes out.
I know that well.
The difficulty of rebuilding all of the internal systems just makes this impractical. As you are aware, a lot of people avoid the .0 releases. What does this say to you?
It says they are cautious. Cautious people often have that as a blanket policy; not just with Boost. What does it say to you?
I understand (and applaud) boost's efforts to try to to eliminate this notion with tighter release cycles etc but the problem is that it's much harder to lose people's confidence than to regain it.
I think you mean the opposite.
I like using boost and am very grateful to all the developers who make it happen. Having said that, SNAFUs such as the current Boost.Range or the MT problem with Boost.Function in 1.34 threaten to make it irrelevant in "serious" development and relegate it to the position of a great "toy library" you can see how C++ standard compliant your compiler is.
I know; it's a problem. If you still want to try to prevent this sort of thing in the future, I suggest you take up my earlier encouragement to write up a policy page. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
Hello Dave,
Naturally, nobody likes to be asked to fix the problem they're complaining about. Such is the open-source world, I'm afraid.
The question is whether "fix the problem" means the Boost.Range issue or the overarching problem of breaking changes. While the latter is something we could fix going forward, my current problem is with Boost.Range. So my post has two reasons: 1) Try to get a consensus whether the Boost.Range problem is actually a defect and should be rolled back 2) How to avoid such things in the future I am sure you understand I can't just waltz into the SVN repository and change Thorsten's code. However, I feel that if we could participate in a reasoned discussion and decide on the pros and cons of the change we may be able to come up with something that would resolve the problem going forward and perhaps find a transition path. Finding the best way to fix that problem is my priority, as this is currently a complete blocker on my project. My original post has also shown that there are quite a few people who feel similarly to me about such changes. Perhaps at this point it would be wise to break this thread into one about Boost.Range and another about boost development, even though I feel that they are very closely tied together, as the outcome of the Range discussion may or may not set a precedent for the latter.
It's a politician's answer, in that it doesn't really answer my original query.
Umm. Trying not to be insulted, here.
My apologies - I only used this as a figure of speech, with no intention to question your integrity. :)
Your original query was not a "query" at all. You asked no questions. There was a list of (valid) complaints.
My intention was to start a discussion.
The only actionable item I saw there (which you snipped out of the context of my quote)
No intention to misrepresent.
was the expression of a laudable desire "to try to make sure this kind of silent breakage doesn’t happen again," and *that* is what I responded to.
And that was a speakers way of beginning a discussion. I was only (respectfully) addressing an audience.
Of course, you also have some power to control the quality of Boost developers, by participating in the review process.
No, we would purely have input into the initial quality of the reviewed library. While this will have a high correlation with the submitters knowledge of C++ and their problem domain, it doesn't allow us to evaluate whether or not they have the experience in large scale software development and the foresight not to make changes which will break existing code. However, I also accept your point about more participation. My colleagues and I have at points been involved with library reviews and discussions. One of the reasons for the lack of participation (beyond shifting job responsibilities) is how far the head of boost is removed from the "stable" versions we use. It is hard to get excited or find the time get involved with something that has no immediate benefit and may not be available to us for another few years. In this respect, I think the idea of "stable" and "new" boost libraries would help in this respect.
However, it seems that once a library is accepted, the maintainer is pretty much free to do what they please.
Yes.
I hope behind this simple "Yes" there are alarm bells going off.
I think it may be wise to apply the same level of rigour to the continued development. Do you think that a boost library that was broken should be rolled back?
Quite possibly. Depends what you mean by "broken" and "rolled back."
For the sake of example, let's stay with the Boost.Range library. By "broken" I mean that the behaviour has changed drastically from one release to the next (not to mention the debug/release issue). IMHO, it has also "broken" the notion of what an empty range is. Steven Watanabe tried suggesting a workaround, which was broken in an insidious way. This in no way is a reflection on him, but on the fact that the behaviour of the library is unintuitive and leads to code with undefined behaviour. That's what I mean by "broken". By "rolled back" I mean that the original behaviour should be reinstated either in a patch or the next release, with either an alternative name for the new concept, a migration path, or both.
I've got some bad news for you, Tom: you're "within."
This isn't bad news - I like using boost and would like contribute more if my employer allowed it. (Having said that, I've made a few very minor additions)
Boost library authors generally highly value the input of their users, and are eager to learn.
Yes, this in general is the case. However, you can't deny that that people who critique your creation will frequently put you on the defensive. I am just as guilty of this behaviour as the developer next to me.
You may think you're being asked to set Boost policy on your own, but that's not how it works. Boost is driven largely by consensus. We'll discuss and tweak the document together in this forum until we can arrive at a consensus that it should be adopted.
In which case it's a shame that the Boost.Range changes just happened and were never discussed here. Again, please take this for what it is: A point made by an external observer.
The difficulty of rebuilding all of the internal systems just makes this impractical. As you are aware, a lot of people avoid the .0 releases. What does this say to you?
It says they are cautious. Cautious people often have that as a blanket policy; not just with Boost. What does it say to you?
It says to me that they've been burnt a few times and are not going to risk their neck again. At the end of the day, I think most people in this industry love what they do and love new toys: Boost is one of them. If we could come up with a way to prevent the toy from occasionally spontaneously bursting into flames, it would be much easier to sell to the parents (forgive the metaphor).
I think you mean the opposite.
Sorry, fat fingers.
I know; it's a problem. If you still want to try to prevent this sort of thing in the future, I suggest you take up my earlier encouragement to write up a policy page.
Before writing such a recommendation, let's have a discussion first about the proposal for the altered release schedule, which would have dramatic impact on the contents of the document. Best wishes, Tom
on Fri Nov 21 2008, Tomas Puverle <Tomas.Puverle-AT-morganstanley.com> wrote:
Hello Dave,
Naturally, nobody likes to be asked to fix the problem they're complaining about. Such is the open-source world, I'm afraid.
The question is whether "fix the problem" means the Boost.Range issue or the overarching problem of breaking changes. While the latter is something we could fix going forward, my current problem is with Boost.Range. So my post has two reasons: 1) Try to get a consensus whether the Boost.Range problem is actually a defect and should be rolled back 2) How to avoid such things in the future
I am sure you understand I can't just waltz into the SVN repository and change Thorsten's code.
Nobody can. Such a thing has to be done with great deliberation and after giving Thorsten a reasonable amount of time in which to respond.
However, I feel that if we could participate in a reasoned discussion and decide on the pros and cons of the change we may be able to come up with something that would resolve the problem going forward and perhaps find a transition path. Finding the best way to fix that problem is my priority, as this is currently a complete blocker on my project.
That's gonna be difficult unless we can engage the library maintainer. Not necessarily impossible, but difficult at best.
My original post has also shown that there are quite a few people who feel similarly to me about such changes. Perhaps at this point it would be wise to break this thread into one about Boost.Range and another about boost development, even though I feel that they are very closely tied together, as the outcome of the Range discussion may or may not set a precedent for the latter.
OK
It's a politician's answer, in that it doesn't really answer my original query.
Umm. Trying not to be insulted, here.
My apologies - I only used this as a figure of speech, with no intention to question your integrity. :)
Your original query was not a "query" at all. You asked no questions. There was a list of (valid) complaints.
My intention was to start a discussion.
The only actionable item I saw there (which you snipped out of the context of my quote)
No intention to misrepresent.
was the expression of a laudable desire "to try to make sure this kind of silent breakage doesn’t happen again," and *that* is what I responded to.
And that was a speakers way of beginning a discussion. I was only (respectfully) addressing an audience.
I have no problem with any of that. However, I would like you to acknowledge that I *did* in fact address what you wrote, whether or not it was what you wanted to hear.
Of course, you also have some power to control the quality of Boost developers, by participating in the review process.
No, we would purely have input into the initial quality of the reviewed library. While this will have a high correlation with the submitters knowledge of C++ and their problem domain, it doesn't allow us to evaluate whether or not they have the experience in large scale software development and the foresight not to make changes which will break existing code.
I don't know why you started out by saying "No" there, because it sounds like we're in agreement. You would have some (not total) power to control the quality of Boost developers.
However, I also accept your point about more participation. My colleagues and I have at points been involved with library reviews and discussions. One of the reasons for the lack of participation (beyond shifting job responsibilities) is how far the head of boost is removed from the "stable" versions we use. It is hard to get excited or find the time get involved with something that has no immediate benefit and may not be available to us for another few years. In this respect, I think the idea of "stable" and "new" boost libraries would help in this respect.
I'm not sure how. Would the "new" code that results from reviews have more immediate relevance to you than it does now, in such a scheme?
However, it seems that once a library is accepted, the maintainer is pretty much free to do what they please.
Yes.
I hope behind this simple "Yes" there are alarm bells going off.
Not really. I understand the risks and tradeoffs of that arrangement, and I think Boost made the right choices there. IMO what's needed is more structure to support the maintainer in making good decisions.
I think it may be wise to apply the same level of rigour to the continued development. Do you think that a boost library that was broken should be rolled back?
Quite possibly. Depends what you mean by "broken" and "rolled back."
For the sake of example, let's stay with the Boost.Range library. By "broken" I mean that the behaviour has changed drastically from one release to the next (not to mention the debug/release issue). IMHO, it has also "broken" the notion of what an empty range is. Steven Watanabe tried suggesting a workaround, which was broken in an insidious way. This in no way is a reflection on him, but on the fact that the behaviour of the library is unintuitive and leads to code with undefined behaviour. That's what I mean by "broken".
By "rolled back" I mean that the original behaviour should be reinstated either in a patch or the next release, with either an alternative name for the new concept, a migration path, or both.
It's certainly conceivable that we could form a consensus on something like that here.
I've got some bad news for you, Tom: you're "within."
This isn't bad news - I like using boost and would like contribute more if my employer allowed it. (Having said that, I've made a few very minor additions)
But the downside (again, referring to missing context) is that it puts the onus on you to do some of the work.
Boost library authors generally highly value the input of their users, and are eager to learn.
Yes, this in general is the case. However, you can't deny that that people who critique your creation will frequently put you on the defensive. I am just as guilty of this behaviour as the developer next to me.
Sure. Not to be too blunt (as I often am), but what's your point?
You may think you're being asked to set Boost policy on your own, but that's not how it works. Boost is driven largely by consensus. We'll discuss and tweak the document together in this forum until we can arrive at a consensus that it should be adopted.
In which case it's a shame that the Boost.Range changes just happened and were never discussed here.
Are you certain they were not discussed here? I know at least one instance where breaking changes were discussed (handling of NTBS's): http://www.nabble.com/-Range--Expected-complexity-constraints-tt4279749.html...
Again, please take this for what it is: A point made by an external observer.
Please don't imagine that there's any need to soften such a sentiment for my sake. It doesn't bother me at all to hear things like that.
The difficulty of rebuilding all of the internal systems just makes this impractical. As you are aware, a lot of people avoid the .0 releases. What does this say to you?
It says they are cautious. Cautious people often have that as a blanket policy; not just with Boost. What does it say to you?
It says to me that they've been burnt a few times and are not going to risk their neck again.
Yes. And as as I said, such policies often are used for *all* dependencies because people have been burnt by *something* that may or may not have been Boost in the past.
At the end of the day, I think most people in this industry love what they do and love new toys: Boost is one of them. If we could come up with a way to prevent the toy from occasionally spontaneously bursting into flames, it would be much easier to sell to the parents (forgive the metaphor).
Agreed.
I think you mean the opposite.
Sorry, fat fingers.
I know; it's a problem. If you still want to try to prevent this sort of thing in the future, I suggest you take up my earlier encouragement to write up a policy page.
Before writing such a recommendation, let's have a discussion first about the proposal for the altered release schedule,
<snaps head around> What proposal?
which would have dramatic impact on the contents of the document.
My advice: try to decouple these things as much as possible. When progress in one area is postponed in anticipation of progress in another, often you get no progress. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
participants (2)
-
David Abrahams -
Tomas Puverle