[Test] BOOST_TEST removed?
BOOST_TEST has been used for testing Boost since the dawn of time, but it's just been removed from CVS, breaking a whole lot of test cases both in Boost and elsewhere. Some warning would have been nice. Not removing it would have been nicer. Doug
It's been deprecated *since the dawn of time*. I planned post an announcement together with library update announcement. Gennadiy
BOOST_TEST has been used for testing Boost since the dawn of time, but it's just been removed from CVS, breaking a whole lot of test cases both in Boost and elsewhere. Some warning would have been nice. Not removing it would have been nicer.
Doug
"Gennadiy Rozental" <gennadiy.rozental@thomson.com> writes:
It's been deprecated *since the dawn of time*. I planned post an announcement together with library update announcement.
Still, developers like to avoid broken regression tests; they're upsetting to developers at best. A simple post with advance warning that you were going to remove BOOST_TEST would have given everyone an opportunity to prevent a large swath of red squares. -- Dave Abrahams Boost Consulting www.boost-consulting.com
It's been deprecated *since the dawn of time*. I planned post an announcement together with library update announcement.
Still, developers like to avoid broken regression tests; they're upsetting to developers at best. A simple post with advance warning that you were going to remove BOOST_TEST would have given everyone an opportunity to prevent a large swath of red squares.
What's wrong with red squares? Isn't it a basic paradigm of TDD? You make change - you fix all the test cases that failed. Failing test cases is your announcement that something needs to be done. Anyway 3 years should've been enough to get rid of tools that were announced deprecated. Gennadiy
Gennadiy Rozental wrote:
It's been deprecated *since the dawn of time*. I planned post an announcement together with library update announcement.
Still, developers like to avoid broken regression tests; they're upsetting to developers at best. A simple post with advance warning that you were going to remove BOOST_TEST would have given everyone an opportunity to prevent a large swath of red squares.
What's wrong with red squares? Isn't it a basic paradigm of TDD? You make change - you fix all the test cases that failed. Failing test cases is your announcement that something needs to be done. Anyway 3 years should've been enough to get rid of tools that were announced deprecated.
Well, one thing that is wrong with red squares is that they cost resources. Testing time significantly grows with the number of failing tests. Giving the developers a chance to phase out BOOST_TEST would have decreased the number of attempts to compile broken code. OTOH, if it has been deprecated for a long time, then developers should have phased it out already. Anyway, a heads-up would have been helpful. Regards, m
"Martin Wille" <mw8329@yahoo.com.au> wrote in message news:4201D178.60406@yahoo.com.au...
Gennadiy Rozental wrote:
It's been deprecated *since the dawn of time*. I planned post an announcement together with library update announcement.
Still, developers like to avoid broken regression tests; they're upsetting to developers at best. A simple post with advance warning that you were going to remove BOOST_TEST would have given everyone an opportunity to prevent a large swath of red squares.
What's wrong with red squares? Isn't it a basic paradigm of TDD? You make change - you fix all the test cases that failed. Failing test cases is your announcement that something needs to be done. Anyway 3 years should've been enough to get rid of tools that were announced deprecated.
Well, one thing that is wrong with red squares is that they cost resources.
Sure. Testing cost resources. But it brings real value instead. In this case, for example, it served as a change announcement. It general it brings a level of confidence so you could make any changes. If anything breakes test system tells you so. In extreme TDD you start with all test cases failing (some of them written even before implementation is done) and then fix them.
Testing time significantly grows with the number of failing tests.
Why is that? You need to redo any test case affected in any case, right? So test failing in compile time, for example, may actually decrease testing time (no linking, no running)
Giving the developers a chance to phase out BOOST_TEST would have decreased the number of attempts to compile broken code.
It's kinda unclear what change needs to be pre-anounced and which not. Any anouncement assumes some grace period (and what if some developers are unavailable or missed an anouncement?). It also assumes that something should be done before the change will actually happend. It all eventually leads to longer development time for one who make an anouncement. Amount of time/work for the rest of developers is the same (I would guess that it even less with known test cases that needs to be fixed and change already in place). In any case I do not see a failing test case as a something to be upset about. Rather I feel satisfaction that testing system again helped me. It much worse when all test cases are passing and you finding error only in users reports. This makes tests a most valuable and salvagable part of any project (IMO)
OTOH, if it has been deprecated for a long time, then developers should have phased it out already. Anyway, a heads-up would have been helpful.
Regards, m
Gennadiy P.S. Above sentiments of course assumes that all the developers making changes actually to improve the project, not to break one ;)
Gennadiy Rozental wrote:
"Martin Wille" wrote in message news:4201D178.60406@yahoo.com.au...
Well, one thing that is wrong with red squares is that they cost resources.
Sure. Testing cost resources. But it brings real value instead. In this case, for example, it served as a change announcement. It general it brings a level of confidence so you could make any changes. If anything breakes test system tells you so. In extreme TDD you start with all test cases failing (some of them written even before implementation is done) and then fix them.
Tests are never the right medium for a change announcement. Your "change announcement" costs hours of CPU time here. A simple grep would have been sufficient to find all places where the BOOST_TEST macro is still in use and it would have taken only a few minutes.
Testing time significantly grows with the number of failing tests.
Why is that? You need to redo any test case affected in any case, right? So test failing in compile time, for example, may actually decrease testing time (no linking, no running)
Yes. It's just that the compile time is significantly larger than linktime + runtime. Also, stuff that has been tested successfully won't be retested unless there is a change.
Giving the developers a chance to phase out BOOST_TEST would have decreased the number of attempts to compile broken code.
It's kinda unclear what change needs to be pre-anounced and which not. Any anouncement assumes some grace period (and what if some developers are unavailable or missed an anouncement?). It also assumes that something should be done before the change will actually happend. It all eventually leads to longer development time for one who make an anouncement. Amount of time/work for the rest of developers is the same (I would guess that it even less with known test cases that needs to be fixed and change already in place). In any case I do not see a failing test case as a something to be upset about. Rather I feel satisfaction that testing system again helped me. It much worse when all test cases are passing and you finding error only in users reports. This makes tests a most valuable and salvagable part of any project (IMO)
Right, there's no clear rule when to pre-announce and when not to. In that situation, it often is useful to make an educated guess about the possible consequences of a change before committing it. Simply searching for BOOST_TEST in the sources would have identified all relevant places in the code. If the number of those place is high then the consquences are presumably large and an announcement would be helpful. [...]
P.S. Above sentiments of course assumes that all the developers making changes actually to improve the project, not to break one ;)
Sure. I don't think anyone here would change anything willingly in a way that is not an improvement somehow. Gee, I hope you didn't misunderstand me. I never accused anyone of deliberately breaking stuff without good reason. Regards, m PS: grepping for BOOST_TEST[^_A-Z] in boost/libs yields 4637 hits, one of them in depecated.html and one in a comment.
Tests are never the right medium for a change announcement.
Well. I probably disagree here. But I am not gonna agrue that.
PS: grepping for BOOST_TEST[^_A-Z] in boost/libs yields 4637 hits, one of them in depecated.html and one in a comment.
Hmm. I should've check it myself. I didn't really expect to be that big. Ok. How about that: I could check in version that forward BOOST_TEST to BOOST_CHECK and remove it again in two weeks? Though I don't beleive it change anything: if 3 year were not enough two weeks wouldn't help either.
On Feb 2, 2005, at 11:22 PM, Gennadiy Rozental wrote:
It's been deprecated *since the dawn of time*. I planned post an announcement together with library update announcement.
Still, developers like to avoid broken regression tests; they're upsetting to developers at best. A simple post with advance warning that you were going to remove BOOST_TEST would have given everyone an opportunity to prevent a large swath of red squares.
What's wrong with red squares? Isn't it a basic paradigm of TDD? You make change - you fix all the test cases that failed. Failing test cases is your announcement that something needs to be done.
The problem is that a red square due to this change (that could have been avoided if you had given us warning) will mask other problems, so it slows development. For instance, John put in some type_traits changes that inevitably broke something, somewhere, but for him to find his errors in the sea of red will not be easy.
Anyway 3 years should've been enough to get rid of tools that were announced deprecated.
Sure. The problem is that the Test lib is one of those core libraries, like type_traits, where changes ripple through and can cause a lot of problems. We just need to know when deprecated bits are being removed, because it's very likely that some of us didn't even know they were deprecated until they were gone. Doug
The problem is that a red square due to this change (that could have been avoided if you had given us warning) will mask other problems, so it slows development. For instance, John put in some type_traits changes that inevitably broke something, somewhere, but for him to find his errors in the sea of red will not be easy.
Anyway 3 years should've been enough to get rid of tools that were announced deprecated.
Sure. The problem is that the Test lib is one of those core libraries, like type_traits, where changes ripple through and can cause a lot of problems. We just need to know when deprecated bits are being removed, because it's very likely that some of us didn't even know they were deprecated until they were gone.
So can someone undertake the search and replace (BOOST_TEST->BOOST_CHECK) if that will fix the problem? John.
It's been deprecated *since the dawn of time*. I planned post an announcement together with library update announcement.
Still, developers like to avoid broken regression tests; they're upsetting to developers at best. A simple post with advance warning that you were going to remove BOOST_TEST would have given everyone an opportunity to prevent a large swath of red squares.
What's wrong with red squares? Isn't it a basic paradigm of TDD? You make change - you fix all the test cases that failed. Failing test cases is your announcement that something needs to be done.
The problem is that a red square due to this change (that could have been avoided if you had given us warning) will mask other problems, so it slows development. For instance, John put in some type_traits changes that inevitably broke something, somewhere, but for him to find his errors in the sea of red will not be easy.
Anyway 3 years should've been enough to get rid of tools that were announced deprecated.
Sure. The problem is that the Test lib is one of those core libraries, like type_traits, where changes ripple through and can cause a lot of problems. We just need to know when deprecated bits are being removed, because it's very likely that some of us didn't even know they were deprecated until they were gone.
Agreed 100%. Let me say, that I'm quite upset about the rude way, how Gennadij decided to force a whole bunch of Boost developers (including me) to invest their time _now_, even if they had not planned to spent their time on this at this moment. I for my part didn't even know, that BOOST_TEST was depreciated (where is this documented?). Changing such a central part without further notice isn't the right way to go! We're all volunteering in Boost and it's unecessary to put additional burden onto everybody here, simply ignoring possible time constraints other people may have. Nevertheless I have some questions regarding this change: - Is there any documented migration path from BOOST_TEST? Perhaps a standard way to do it? - Is there a version number associated with the test library, which I can use to handle this change correctly in my code, even in between different Boost versions (we'll need some time until V1.33 get's out of the door, I assume)? I still have to support older versions of Boost for some time. All in all this incident forces me to think about moving away from using Boost.Test in my own code in the future. Regards Hartmut
Hartmut Kaiser wrote: [...]
- Is there any documented migration path from BOOST_TEST? Perhaps a standard way to do it?
BOOST_TEST --> BOOST_CHECK BOOST_CRITICAL_TEST --> BOOST_REQUIRE BOOST_CRITICAL_ERROR --> BOOST_FAIL
- Is there a version number associated with the test library, which I can use to handle this change correctly in my code, even in between different Boost versions (we'll need some time until V1.33 get's out of the door, I assume)? I still have to support older versions of Boost for some time.
Why can't you generally use the newer macros? They have been part of boost for some releases now. Stefan
On Thu, 3 Feb 2005 14:57:10 +0100 "Hartmut Kaiser" <hartmutkaiser@t-online.de> wrote:
Agreed 100%.
Let me say, that I'm quite upset about the rude way, how Gennadij decided to force a whole bunch of Boost developers (including me) to invest their time_now_, even if they had not planned to spent their time on this at this moment. I for my part didn't even know, that BOOST_TEST was depreciated(where is this documented?). Changing such a central part without further notice isn't the right way to go!
While I agree that an announcement should have been made, the tools have been deprecated for a very long time, and the documentation has also been around saying that they are deprecated for a very long time as well. So, I think your anger is a bit misdirected. What this actually does, is bring up a point about deprecating interfaces. We like to be "nice" and leave a deprecated interface around for a while to give users time to move. However, if the deprecation time is too long, people never move off, because they forget, or the notice is so old that new users never see it (though the test-tools reference has said these interfaces are deprecated for a while). So, for deprecated interfaces, it makes sense to make the proper announcement in the docs, and then when the author decides to act on the statement "Old deprecated names are still accepted but may be excluded in a future releases." another announcement should be made, saying that the removal is being planned, so everyone has time to properly move.
All in all this incident forces me to think about moving away from using Boost.Test in my own code in the future.
This seems a bit rash, since most libraries that have been around for a while have deprecated interfaces, and eventually, they will be removed as well. Maybe we should have authors place a message to this list any time stuff is being deprecated, so that it is easily searchable... or maybe there should be a common place in the boost documents that any deprecation announcements can stay so that it is easy to know what is planned for removal, or has been removed. Also, for core stuff, it proabably makes sense to make the change in a branch, announce the change, branch, and proposed date of merge. This gives developers a chance to make changes as they can relative to that branch, before the final commit.
Also, for core stuff, it proabably makes sense to make the change in a branch, announce the change, branch, and proposed date of merge. This gives developers a chance to make changes as they can relative to that branch, before the final commit.
I am all for making developement on branch. But! I may as well do my changes locally. The main reason I actually commit the changes it to go through regression testing. Some time ago I brough a proposal to allow specifing tag of files to use in testing. But AFAIKT it's undoable. Gennadiy.
Let me say, that I'm quite upset about the rude way, how Gennadij decided to force a whole bunch of Boost developers (including me) to invest their time _now_, even if they had not planned to spent their time on this at this moment.
I did not force you to do anything right now. IMO failing test case is just a reminder that something need to be done, preferably this release. This situation similar to MT locking: any sinchronization leeds to delays, but bring safety.
I for my part didn't even know, that BOOST_TEST was depreciated (where is this documented?). Changing such a central part without further notice isn't the right way to go!
We're all volunteering in Boost and it's unecessary to put additional burden onto everybody here, simply ignoring possible time constraints other
For a long time if was on front page of Boost.Test lib people
may have.
Yes. We all have our contraines (one of mine is that I should minimize period of time I making changes in Boost.Test). I chould've made an anouncement, and wait a week, but for some reason I think it may not leed to smaller number failures once I actually do the change. Or should I wait until everybody had time to make changes in their code?
Nevertheless I have some questions regarding this change:
- Is there any documented migration path from BOOST_TEST? Perhaps a standard way to do it?
Does the section "Depricated tools" here: http://www.boost.org/libs/test/doc/components/test_tools/reference/index.htm... is a what you want?
- Is there a version number associated with the test library, which I can use to handle this change correctly in my code, even in between different Boost versions (we'll need some time until V1.33 get's out of the door, I assume)? I still have to support older versions of Boost for some time.
I am not sure what you ask here. 1.32 had BOOST_TEST next one wouldn't Gennadiy
On Do, 3.02.2005, 13:48, Douglas Gregor wrote:
[...] My newsserver seems not to deliver messages (I try by mail now)...thus I'm very sorry for the late information: I've replaced all occurances of BOOST_[*_]TEST macros in the regressions and also tested the changes locally. So the next regression run (follows shortly for cw 9.4 which is very fast) should be 'green' again! Cheers, Stefan
Stefan Slapeta wrote:
On Do, 3.02.2005, 13:48, Douglas Gregor wrote:
[...]
My newsserver seems not to deliver messages (I try by mail now)...thus I'm very sorry for the late information:
I've replaced all occurances of BOOST_[*_]TEST macros in the regressions and also tested the changes locally.
So the next regression run (follows shortly for cw 9.4 which is very fast) should be 'green' again!
You just broke all bind and smart_ptr tests. They use their own BOOST_TEST.
Peter Dimov wrote:
Stefan Slapeta wrote:
On Do, 3.02.2005, 13:48, Douglas Gregor wrote:
[...]
My newsserver seems not to deliver messages (I try by mail now)...thus I'm very sorry for the late information:
I've replaced all occurances of BOOST_[*_]TEST macros in the regressions and also tested the changes locally.
So the next regression run (follows shortly for cw 9.4 which is very fast) should be 'green' again!
You just broke all bind and smart_ptr tests. They use their own BOOST_TEST.
Sorry, I spoke too soon. The new run isn't online yet and the previous failures are not caused by BOOST_TEST. There appears to be some configuration problem with CW 9.4.
Peter Dimov wrote:
Sorry, I spoke too soon. The new run isn't online yet and the previous failures are not caused by BOOST_TEST. There appears to be some configuration problem with CW 9.4.
This has been fixed yet. Nevertheless, I've to wait until the next tarball has been created. Stefan
On Feb 3, 2005, at 9:02 AM, Stefan Slapeta wrote:
On Do, 3.02.2005, 13:48, Douglas Gregor wrote:
[...]
My newsserver seems not to deliver messages (I try by mail now)...thus I'm very sorry for the late information:
I've replaced all occurances of BOOST_[*_]TEST macros in the regressions and also tested the changes locally.
So the next regression run (follows shortly for cw 9.4 which is very fast) should be 'green' again!
Thanks, Stefan! Doug
The problem is that a red square due to this change (that could have been avoided if you had given us warning) will mask other problems, so it slows development. For instance, John put in some type_traits changes that inevitably broke something, somewhere, but for him to find his errors in the sea of red will not be easy.
Ok we both made changes that inevitably broke something. Why don't you look on this other way around?
Anyway 3 years should've been enough to get rid of tools that were announced deprecated.
Sure. The problem is that the Test lib is one of those core libraries, like type_traits, where changes ripple through and can cause a lot of problems.
That's why I am have very limitted amount of time when I could make changes and do not want to wait for weeks after announcement making sure everybody had time/ heard about it.
We just need to know when deprecated bits are being removed, because it's very likely that some of us didn't even know they were deprecated until they were gone.
Ok. I am ready to make this a policy: One week notice for removed depricated interfaces.
Doug
Gennadiy
On Feb 3, 2005, at 12:47 PM, Gennadiy Rozental wrote:
The problem is that a red square due to this change (that could have been avoided if you had given us warning) will mask other problems, so it slows development. For instance, John put in some type_traits changes that inevitably broke something, somewhere, but for him to find his errors in the sea of red will not be easy.
Ok we both made changes that inevitably broke something. Why don't you look on this other way around?
Several reasons: 1) We had warning about the type_traits changes. 2) Few things actually broke, and AFAICT only on broken compilers. 3) The test lib change could be viewed as gratuitous, because the cost of not making the change is negligible (i.e., the macros could have stayed and nobody would have been hurt).
That's why I am have very limitted amount of time when I could make changes and do not want to wait for weeks after announcement making sure everybody had time/ heard about it.
Understood. Would waiting to remove 3 deprecated macros really have slowed you down any?
We just need to know when deprecated bits are being removed, because it's very likely that some of us didn't even know they were deprecated until they were gone.
Ok. I am ready to make this a policy: One week notice for removed depricated interfaces.
Thank you. If nobody is using the interfaces, pull 'em out for sure. What happened here was that the interface was heavily used by Boost, so the impact of pulling them out was huge, and we needed time to deal with that. Doug
That's why I am have very limitted amount of time when I could make changes and do not want to wait for weeks after announcement making sure everybody had time/ heard about it.
Understood. Would waiting to remove 3 deprecated macros really have slowed you down any?
It wouldn't slow me but extend a time I am making changes. Gennadiy
"Gennadiy Rozental" <gennadiy.rozental@thomson.com> wrote in message news:cttqqn$1l9$1@sea.gmane.org...
That's why I am have very limitted amount of time when I could make changes and do not want to wait for weeks after announcement making sure everybody had time/ heard about it.
Understood. Would waiting to remove 3 deprecated macros really have slowed you down any?
It wouldn't slow me but extend a time I am making changes.
Rather than make this a one-way critique of Gennadiy, I for one am glad that he can find the time to make changes in what is a very important part of Boost. Given that these macros were deprecated a long time ago and documented as such, the only critique I can level is that other boost library developers didn't act on this advice sooner. I'm grateful to everybody in boost of course, but sometimes I'm ashamed at the level of criticism being thrown at individuals with a proven track record in boost.
Doug Gregor wrote:
BOOST_TEST has been used for testing Boost since the dawn of time, but it's just been removed from CVS, breaking a whole lot of test cases both in Boost and elsewhere. Some warning would have been nice. Not removing it would have been nicer.
I hope nobody minds that I started replacing occurances of BOOST_[*_]TEST with newer equivalents. these are done and locally re-tested: - filesystem - function - graph - logic/tribool - math - multiarray - numeric/interval - program_options some others are still missing: - regex - signals - thread - utility - variant - config - integer - lambda - tuple I hope to get around to finish it during the next hour. Stefan
participants (11)
-
David Abrahams -
Doug Gregor -
Douglas Gregor
-
Gennadiy Rozental -
Hartmut Kaiser -
Jody Hagins -
John Maddock -
Martin Wille -
Paul Baxter -
Peter Dimov -
Stefan Slapeta