Re: [Boost-users] [Boost.Test] reporting tests with warnings

[Please do not mail me a copy of your followup] boost-users@lists.boost.org spake the secret code thusly:

Richard writes:

...

...

Boost.Test already support three "symbols":

BOOST_REQUIRE_* BOOST_CHECK_* BOOST_WARN_*

What you're seems to be implying is that BOOST_WARN_* is a bad idea.

Not just BOOST_WARN_*, but also BOOST_CHECK_* since they keep executing a failed test case.

Why so radical view?

I don't think its radical to want my tests to simply stop at the first point of failure.

This is TDD purist position and is not necessarily the only one.

I don't know if its "purist" or not and that's beside the point. I never said it was the "only" position.

...

If you keep executing a test case that's already failed, it is highly likely in C++ that the test runner will simply crash.

This is simply not true.

So if I do something like: BOOST_CHECK(p); BOOST_CHECK(p->accessor()); and p is a null pointer, what good does it do to continue after the first failed check? The pointer is null and anything else that is checked after that is useless. Dereferencing a null pointer is going to crash. Yes, on Windows that crash is turned into an exception that the test runner could intercept, but its still worthless. We don't even know if the whole heap has been corrupted at this point. The bottom line is that due to the low-level nature of C++, all kinds of things could go wrong in the test runner's environment that are outside the control of the test runner. Yes, you can guard against most kinds of controlled failures, but you simply can't guarantee that C++ test code (or the SUT) didn't horribly screw everything up. In managed languages like C# and Java, this simply isn't the case due to their different execution model. That's why for C++ code, I consider it a bad idea to use anything other than BOOS_REQUIRE. I want the test case to fail immediately and not continue any further at all once an expectation hasn't been met. Yes, its my opinion, but its not one that I grabbed onto randomly. Even in other environments (C#, for instance), I don't have any use for warnings or "mark a failure, but keep executing the test case". Whenever I've encountered this in other people's tests, its because the test is trying to test too many things instead of a single thing and so they are trying to report multiple different possible failures from a single test case. A single test should test one thing only. You may call this "TDD purism", but I simply call it good test design and it seems to be an opinion shared among other people I know that have been writing unit tests for a long time and have much more experience at it than I do. -- "The Direct3D Graphics Pipeline" -- DirectX 9 version available for download http://legalizeadulthood.wordpress.com/the-direct3d-graphics-pipeline/ Legalize Adulthood! http://legalizeadulthood.wordpress.com