Gennadiy Rozental wrote:
"Johan Nilsson" <r.johan.nilsson@gmail.com> wrote in message news:dnre20$vv1$1@sea.gmane.org...
Gennadiy Rozental wrote:
Hi,
This message should serve as a temporary source of information regarding self registration faculties of Boost.Test. I will include the complete docs in next update.
[snip]
One of my pet peeves about CppUnit (and most of the xUnit frameworks) is that there's usually no controllable way of indicating a fixture setup failure. I'm running hardware dependent tests in a couple of my projects (e.g. serial port communications). I need a _supported_ way of indicating a setup error, leading to no further tests being executed in the same suite. Is this possible using Boost.Test?
If I gather correctly you need one per suite fixture. It's last kind of fixture not yet supported and It's in my to do list for update 3. Once it's done you could just throw an exception to indicate init failure
That sounds about correct. I'll just have to wait and see.
My main point is that the framework should be prepared for an exception to be thrown from the fixture setup (in this case, during F's ctor), and report it as an setup failure (including the exceptions' "what()" information). And, for the BOOST_FIXTURE_TEST_SUITE, don't run any tests within the suite when setup fails.
BOOST_FIXTURE_TEST_SUITE doing different job - The fixture is per test case.
Even so (suite or not), if a fixture throws from the ctor, the framework should be prepared for that. I tried the following (1.33.1): --- namespace { struct Fixture { Fixture() { throw std::runtime_error("oops!"); } }; } BOOST_FIXTURE_TEST_CASE(Failure, Fixture) {} --- And the output was: --- unknown location(0): fatal error in "Failure": std::runtime_error: oops! --- Ideally, I'd like the framework to be able to differentiate between a Fixture construction failure (i.e. xUnit::TestXxx::setUp) and an error originating from e.g. BOOST_CHECK_xxx. Perhaps having a boost::test::setup_failure exception class that one could use for this purpose?
[snip]
---------------------------------------------------- BOOST_FIXTURE_TEST_SUITE( suite_name, F ) ----------------------------------------------------
Similar to the BOOST_AUTO_TEST_SUITE, but force all the test cases within this test suite to use struct F as a fixture.
BOOST_FIXTURE_TEST_SUITE( s, F )
// this test case use F as a fixture BOOST_AUTO_TEST_CASE( test1 ) { // body here }
BOOST_AUTO_TEST_SUITE_END()
Do all the test cases within the BOOST_FIXTURE_TEST_SUITE and BOOST_AUTO_TEST_SUITE_END macros share the same instance of F (I'd personally prefer that)?
No. I think that would defeat the purpose. You could always mimic what you want with plain Static instance:
struct F{} f;
TEST_CASE( t1 ) { f.i ... }
TEST_CASE( t2 ) { f.i ... }
And could you do this while using suites / fixtures - or is this part of the new functionality?
Also, how about having a BOOST_FIXTURE_TEST_SUITE_END macro for symmetry (simply resolving to BOOST_AUTO_TEST_SUITE_END)?
I don't see a need for this. BOOST_AUTO_TEST_SUITE_END ends any test suite.
I understood that (as you can see from my writing above). It's just that there exists BOOST_AUTO_TEST_SUITE and BOOST_AUTO_TEST_SUITE_END (which is symmetric and nice), but then there's also BOOST_FIXTURE_TEST_SUITE and it's not very intuitive to use BOOST_ *AUTO* _TEST_SUITE_END to end that suite. Perhaps use a plain BOOST_TEST_SUITE_END instead?
Another question, is there any possibility to implement hierarchical auto test suite registrations, e.g.:
<i/o test suite> - "io" <ip test suite> - "ip" <http test suite> - "http" <file test suite> - "file"
... etc ...
Yes. BOOST_AUTO_TEST_SUITE could be put inside one another. Check examples in libs/example directory.
I've searched, but couldn't find any examples with nested test suites. I only found one example with multiple test suites within the same file. Could you give me a pointer? Even if this is possible, does the current implementation require the suites to be defined within the same compilation unit? To be able to scale using this approach (I'm talking scaling within the context of maintaining large sets of unit tests), this should not be necessary.
With reference to the test runner, it would be great to be able to specify e.g.:
<runner> --run-tests "io" run all tests from "io","ip","http","file" test suites <runner> --run_tests "io/ip" run all tests from "ip","http" test suites
"Run test by name" is scheduled for update 3.
That's great - should that work with nested test suites as in my example above? // Johan