[test] boost::test equivalent of CppUnit's "protectors" ?
I've used both CppUnit and boost::test for C++ unittesting. Generally I prefer boost::test, mainly because the auto-test macros minimise the effort to setup tests. But there's one thing I really miss from CppUnit: the ability to register your own custom "protectors", instances of which automatically wrap all the run tests. (Technically, you install a test "listener", and that can wrap each test run in a protector scope. Multiple listeners and nested protectors are possible). I've found these invaluable in the past for monitoring unittests for unexpected side effects (e.g checking code hasn't changed the floating-point unit state flags, or messed with other global state). I can't see any equivalent in the boost::test documentation, although BOOST_FIXTURE_TEST_CASE maybe comes closest. (However, whereas a CppUnit protector can return a bool to indicate pass/fail, it's less clear to me how a fixture destructor should indicate failure). Any suggestions for how to best achieve the same thing as CppUnit's protectors in boost::test (and without updating the body of each test) ? Thanks for any advice Tim
Tim Day
I've used both CppUnit and boost::test for C++ unittesting. Generally I prefer boost::test, mainly because the auto-test macros minimise the effort to setup tests. But there's one thing I really miss from CppUnit: the ability to register your own custom "protectors", instances of which automatically wrap all the run tests. (Technically, you install a test "listener", and that can wrap each test run in a protector scope. Multiple listeners and nested protectors are possible).
You can register test observers using framework::register_observer. See the header test_observer.hpp and framework.hpp for interface definition.
I've found these invaluable in the past for monitoring unittests for unexpected side effects (e.g checking code hasn't changed the floating-point unit state flags, or messed with other global state). I can't see any equivalent in the boost::test documentation, although
These are more on advanced usage side and are not coverred in current docs.
BOOST_FIXTURE_TEST_CASE maybe comes closest. (However, whereas a CppUnit protector can return a bool to indicate pass/fail, it's less clear to me how a fixture destructor should indicate failure).
What and where you want to indicate? How framework has to respond to this indication? Gennadiy
On Fri, 2009-05-08 at 08:19 +0000, Gennadiy Rozental wrote:
Tim Day
writes: But there's one thing I really miss from CppUnit: the ability to register your own custom "protectors", instances of which automatically wrap all the run tests.
You can register test observers using framework::register_observer. See the header test_observer.hpp and framework.hpp for interface definition. These are more on advanced usage side and are not coverred in current docs.
Thanks for the pointer; those look exactly like what I was looking for.
BOOST_FIXTURE_TEST_CASE maybe comes closest. (However, whereas a CppUnit protector can return a bool to indicate pass/fail, it's less clear to me how a fixture destructor should indicate failure).
What and where you want to indicate? How framework has to respond to this indication?
I was just concerned that if I went down the route of trying to indicate test(-suite) failure from BOOST_FIXTURE_TEST_CASE's destructor, pretty much the only option would seem to be be to throw (not a nice thing to do in a destructor) (hmmm... actually it occurs to me I have no idea how things like BOOST_CHECK indicate to the framework that a test has failed; it clearly isn't by throwing as, unlike BOOST_REQUIRE, execution of the test continues. I really should look at the source more closely). But failing a test within a test_observer::test_finish() call seems like it ought to be a cleaner place to do it. Tim
On Mon, 2009-05-11 at 11:50 +0100, Tim Day wrote:
But failing a test within a test_observer::test_finish() call seems like it ought to be a cleaner place to do it.
Hmmm having played with it a bit, I'm at a bit of a dead-end with regard to how or even whether it's possible for a test_observer to express the opinion that the test should actually be considered failed. It's easy enough for the observer to log out its opinion as text, but attempting to throw or use a BOOST_CHECK or similar (e.g direct call to framework::assertion_result or test_unit_aborted) within test_observer::test_finish() generally results in something like "Boost.Test framework internal error: unknown reason." or an access violation. Any suggestions ? (Previous experience suggests tests can spew logs and warnings all they like, but it takes an actual failure to really get attention). Tim
Tim Day
On Mon, 2009-05-11 at 11:50 +0100, Tim Day wrote:
But failing a test within a test_observer::test_finish() call seems like it ought to be a cleaner place to do it.
Hmmm having played with it a bit, I'm at a bit of a dead-end with regard to how or even whether it's possible for a test_observer to express the opinion that the test should actually be considered failed. It's easy
I can consider making this possible. For now try throwing exception from fixture. In general you should try to make your test cases self consistent, so that you would not need to test for side-effects outside of the test case.
On Tue, 2009-05-12 at 19:14 +0000, Gennadiy Rozental wrote:
Tim Day
writes: Hmmm having played with it a bit, I'm at a bit of a dead-end with regard to how or even whether it's possible for a test_observer to express the opinion that the test should actually be considered failed. It's easy
I can consider making this possible. For now try throwing exception from fixture.
In general you should try to make your test cases self consistent, so that you would not need to test for side-effects outside of the test case.
Yes I'm now coming round more to thinking that if a test passed by it's own rules, no external observer really has any business saying "no, actually you failed". It probably does make more sense for such observers to log their opinion "well you might think you passed, but I really don't like the fact that you changed <whatever state observer is monitoring>" somewhere sufficiently visible. Tim
Tim Day
BOOST_FIXTURE_TEST_CASE maybe comes closest. (However, whereas a CppUnit protector can return a bool to indicate pass/fail, it's less clear to me how a fixture destructor should indicate failure).
What and where you want to indicate? How framework has to respond to this indication?
I was just concerned that if I went down the route of trying to indicate test(-suite) failure from BOOST_FIXTURE_TEST_CASE's destructor, pretty much the only option would seem to be be to throw (not a nice thing to do in a destructor)
Yes. I think that you can do this. If you test stack unrolling using uncaught_exception().
closely). But failing a test within a test_observer::test_finish() call seems like it ought to be a cleaner place to do it.
It was not designed for this purpose. Maybe we can reconsider it. Gennadiy
participants (2)
-
Gennadiy Rozental -
Tim Day