-----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users- bounces@lists.boost.org] On Behalf Of Gennadiy Rozental Sent: Tuesday, May 12, 2009 8:54 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] [Boost.Test] BOOST_FAIL vs BOOST_ERROR
Greg Christopher wrote:
I think that _both_ BOOST_FAIL (or BOOST_REQUIRE) and BOOST_ERROR (or BOOST_CHECK) do indicate that there is a software under test failure (bug They can indicate whatever you want it to indicate. If you require some particular condition before proceeding with test you check for it using BOOST_REQUIRE on top of the test case. You can also use fixtures.
Right, but the point is there are two reasons why I might not be able to continue: 1)A feature of the software under test is broken, and therefore I won't be able to test further. Still a bug in the software I'm testing. 2)Something exceptional has happened outside the ability of my software under test to deal with. At this point, I want to abend without indicating a software "bug".
I do think fixtures are a great way to go. If the fixture initialize fails, don't run that test and report a problem in the testware or environment. But sometimes I may want to indicate that during the flow of the testing.
I believe, at the moment fixture execution is part of the test case and is treated accordingly.
I think that test reporting frameworks need at least three states they indicate- fail(bug), success, or "other". At a higher level, I want the system to _only_ contact the team executing the tests if "other" comes up so they can look at the testing system. As such systems over-report "failures" or bugs, trust in the testing system erodes.
Boost.Test tools already has 3 levels: CHECK, REQUIRE and WARN. You can use third level to report conditions which are not software bugs. You'll need to abort the test case manually though, since Boost.Test continues on warning. The warnings are reported accordingly in an output.
This is true- we can use warn to mean aborting test although it seems to go against what most "first readers" of the code would think it is. This actually has raised another question for me. I am looking at fixtures. I would like to abort if the setup step fails. Unfortunately, setup is a constructor. I take it that means to abort if setup fails, I need to throw? Is there a specific exception I can throw to make it abort the current test case vs abort the test framework? Maybe just using a BOOST_REQUIRE() statement in the constructor will work? Thanks!! Greg
Gennadiy
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users