* Clear separation between components (execution monitor, from unit test call framework, from testing macros). Ideally each would be a separate mini
Isn't it already the case?
There's duplication of source between the different component libraries (execution monitor, test monitor, unit test). Plus the headers seem to pull in a whole lot of stuff I never use ;-)
library if that's possible, with the executable linking against just what it needs and no more.
Isn'r it already the case?
My gut feeling is that recent releases have got slower to compile and #include.
* Easy debugging: if I step into a test case in the debugger the first thing I should see is *my code*. As it is I have to step in and out of dozens of Boost.Test functions before I get to my code. This one really annoys me.
I am not sure I follow. Setup break point in your test case it will stop right there. Do you setup break point in first line of main? Or you mean something completely different?
No I mean that if I break on a test case and then hit "step into" in the debugger I have to step through your code before I get to mine, so for example if I break on a BOOST_CHECK_CLOSE_FRACTION and then step, I hit: scrap.exe!boost::unit_test::basic_cstring<char const >::basic_cstring<char const >() Line 163 C++ So I return from that, and step again and hit: scrap.exe!boost::unit_test::basic_cstring<char const >::basic_cstring<char const >(const char * s, unsigned int arg_size) Line 192 C++ Return and step and hit: scrap.exe!boost::unit_test::unit_test_log_t::set_checkpoint(boost::unit_test::basic_cstring<char const > file, unsigned int line_num, boost::unit_test::basic_cstring<char const > msg) Line 251 C++ Return and step and hit: scrap.exe!boost::unit_test::basic_cstring<char const >::basic_cstring<char const >(const char * s, unsigned int arg_size) Line 192 C++ Return and step and hit: scrap.exe!boost::unit_test::lazy_ostream::instance() Line 39 C++ Return and step and hit: scrap.exe!boost::unit_test::operator<<<char const [1]>(const boost::unit_test::lazy_ostream & prev, const char [1]& v) Line 83 C++ Return and step and hit: scrap.exe!boost::test_tools::tt_detail::check_frwd<boost::test_tools::check_is_close_t,double,double,double>(boost::test_tools::check_is_close_t P, const boost::unit_test::lazy_ostream & assertion_descr, boost::unit_test::basic_cstring<char const > file_name, unsigned int line_num, boost::test_tools::tt_detail::tool_level tl, boost::test_tools::tt_detail::check_type ct, const double & arg0, const char * arg0_descr, const double & arg1, const char * arg1_descr, const double & arg2, const char * arg2_descr) Line 293 C++ Return and step - and finally hit my code! So I have to step through 7 of your functions before I can finally debug a failing test case.
* Rapid execution of each test case, a BOOST_CHECK(no-op) should be as near to a no-op as possible. I was unable to use Boost.Test for a lot of the Math lib tests for this reason - looping over thousands of tests was simply impractical from a time point of view (maybe this has improved since then, I haven't checked).
BOOST_CHECK is noop (well to some degree). There is some overhead probably. There was indeed improvement couple years ago and now we only do minimal amount of work necessary to pass context information about.
OK, I rechecked this, and BOOST_CHECK_CLOSE_FRACTION appears to have next to no overhead now - excellent!
* Exemplary error messages when things fail - Boost.Test has improved in this area, but IMO not enough.
Specifically?
Testing: double a = 1; double b = 2; BOOST_CHECK_CLOSE_FRACTION(a, b, 0.0); Yields: m:/data/boost/trunk/ide/libraries/scrap/scrap.cpp(41): error: in "test_main_caller( argc, argv )": difference{0} between a{1} and b{2} exceeds 0 Leaving aside the obvious bug (the difference is not zero!!), I would have printed this as something like: m:/data/boost/trunk/ide/libraries/scrap/scrap.cpp(41): error: in "test_main_caller( argc, argv )": difference between a and b exceeds specified tolerance with: a = 1.0 b = 2.0 tolerance = 0.0 difference = 1.0 Which I'm sure will get mangled in email, but the idea is that the values are pretty printed so they all line up nicely - makes it much easier to see the problem compared to dumping them all on one line.
* An easy way to tell if the last test has failed, and/or an easy way to print auxiliary information when the last test has failed. This is primarily for testing in loops, when iterating over tabulated test data.
This is addressed with trunk improvements. There is several tools introduced to help with context specification.
Such as? Docs?
* Relatively simple C++ code, with no advanced/poorly supported compiler features used. This is one library that should be usable anywhere and everywhere.
I do not believe Boost.Test uses in it's core any advanced C++ features. I am looking to add new component which might use one, but it is always going to be an extension.
* Ultra stable code. Exempting bug fixes, I'd like to see a testing library almost never change, or only change after very careful consideration, for example if a new C++ language feature requires special testing support.
Test library as any other library has users, bugs, feature requests etc. It has a life on it's own. It does indeed need to be more carefully maintained in comparison with other libs, but:
1. Proper component dependency helps. Your library needs to be built against released version of Test library (even trunk one). This way Test library can do it's own development in parallel.
The way Boost works at present is that if Trunk breaks, then so does my stuff in Trunk: as a result I can no longer see whether I'm able to merge to release or not if Boost.Test is broken on Trunk. Unfortunately this has happened to me a few times now.
2. Test library might need to be released not that frequently (that's why I actually holding on to releasing my changes, cause there is still a chance something somewhere will break)
3. There should be a period of time (short one) when testing library in release branch is updated. If there are few regressions/conflicts these can be fixed. Otherwise the change is reverted.
And what I don't want:
* Breaking changes: Boost authors have absolutely no time to track breaking changes in their dependencies, since a successful testing library would be used universally by all of Boost, this is particularly important for this library.
Again: proper component dependency. Depending on trunk version of your dependencies is the root cause of the issue here. One library should depend on specific released version of another library A.deps = B:1.2.3
And again that's not how Boost testing currently works, no matter what you may wish. The issue here is that breaking changes should not be made to trunk without checking to see what else in Boost is depending on those features. As you know that didn't happen with the last major commit. Regards, John.