
Hi everybody I finished the first update to the 1.33 release of Boost.Test. This update includes several bug fixes and series on new features. Once this updates settles for next update I will invest my efforts exclusively in documentation making it up to date and more convenient to use. The last update should finalize runtime parameter framework and testing by name feature. I don't think most of below modification require any (semi-)formal review, but I would really appreciate any comments and reviews for new features. Here is an itemized list of modifications/innovations included in update 1 (in no particular order): * Autolinking support is implemented To employ autolinking with Program Execution Monitor and Test Execution Monitor one need to include new headers test_exec_monitor.hpp and prg_exec_monitor.hpp * DLL support is implemented Now it's possible to build Boost.Test as DLL. In addition this allows to use Boost.Test from within dll. See separate post on this topic for more details. * Auto-registration test tree facilities integration Starting next release auto registration facilities becoming a natural part of primary Boost.Test interface. Header auto_unit_test is deprecated and unit_test.hpp is now includes everything. All examples for unit test framework are reworked to advertise these facilities as easiest way to facilitate testing. I will post a separate message giving a short introduction for all self-registration facilities including new ones: BOOST_FIXTURE_TEST_SUITE, BOOST_GLOBAL_FIXTURE. * Mix of auto registered and explicitly registered test cases allowed Now within the same test module one may have both auto registered and explicitly registered test cases. * Interaction based testing is introduced This branch of testing was not covered by Boost.Test before. See separate post with more detailed on the topic. * Exception safety testing is introduced Based on interaction based testing these facilities mostly useful for generic components testing. See separate post with more details on this topic. * Existing init API twicks In existing API init function for unit test usually look like this: test_suite* init_unit_test_suite( int argc, char* argv[] ) { test_suite* test = BOOST_TEST_SUITE( "abc" ); .... return test; } If this function returns NULL test tree treated as non initialized. With this update master test suite is now always exist and test tree treated as non initialized if it doesn't have any test cases. NULL is not treated as failure anymore. To report initialization failure with existent API one should use an exceptions (preferably std::logic_error). It's recommended for init function to look like this: test_suite init_unit_test_suite( int argc, char* argv[] ) { framework::master_test_suite().p_name.value = "abc"; .... return 0; } * Alternative init API introduced There several drawbacks in existent init API for unit test: - argc. argv soon there wouldn't be the only source for runtime parameters (config file, environment, and other source one would be able to use) - it's unsafe: if user doesn't allocate resulting test suite in heap coredump will occur - there is no way to indicate now initialization failure without exceptions. NULL results could mean that we failed to initialize or that we don't have any non-auto registered test cases I've introduced an alternative init API: bool init_unit_test(); Here false return value designate init failure. TO access runtime parameters one would use special (separate) API. To employ this alternative API one should define BOOST_TEST_ALTERNATIVE_INIT_API both during library and test compilation. This API is required for any user willing to use DLL build of the library. * framework::deregister_observer is introduced Now one could use temporary observers * observers priority is introduced Framework invokes a test observers callback in order of observer priority * unit_test_log API slightly simplified File and line of log entry passed as part of entry begin processing. * BOOST_MESSAGE and BOOST_CHECKPOINT are deprecated I found these names too generic for testing framework. New names are BOOST_TEST_MESSAGE and BOOST_TEST_CHECKPOINT. In addition I moved this faculties in their proper place: from Test Tools to Unit Test Log since they only responsible for logging. * BOOST_TEST_PASSPOINT() is introduced In addition to BOOST_TEST_CHECKPOINT which require a message to mark a check point this new facility mark a place test is passing through without any message (just file,line - being there). All Test Tools now invoke BOOST_TEST_PASSPOINT() before tool is started. This allows to detect the failure point in case if exception or crash more effectively (I would use the same name BOOST_TEST_CHECKPOIINT, but this would require variable num of args in macro) * BOOST_<level>_CLOSE_FRACTION This update introduce an ability to test on closeness based on either percentage driven tolerance or fraction driven one. This new tool will test on closeness based on fraction tolerance. (Note: The second request in the same domain: printing actual difference in fractions is not implemented. It's not clear what needs to be printed) * Test collector and Test reporter are modified to report proper number of test cases that gets aborted in cases of fatal failure * output_test_stream now allows matching in binary mode * framework::master_test_suite() interface is introduced for access to the test tree master test suite. * framework::test_unit_aborted() modified to include a test_unit that is being aborted Regards, Gennadiy

Forgot one item * BOOST_AUTO_TEST_MAIN is depricated (though honored for now). Use BOOST_TEST_MAIN instead Gennadiy

Gennadiy Rozental wrote:
Hi everybody
I finished the first update to the 1.33 release of Boost.Test. This update includes several bug fixes and series on new features. Once this updates settles for next update I will invest my efforts exclusively in documentation making it up to date and more convenient to use. The last update should finalize runtime parameter framework and testing by name feature. I don't think most of below modification require any (semi-)formal review, but I would really appreciate any comments and reviews for new features.
Just a quick question - are these features only available at the current HEAD revision, or are some of them already available at version 1.33.1 (just being undocumented)? // Johan

* Autolinking support is implemented To employ autolinking with Program Execution Monitor and Test Execution Monitor one need to include new headers test_exec_monitor.hpp and prg_exec_monitor.hpp
Unfortunately this is causing a *huge* number of failures in the current test matrix - if you include a Boost.Test header now you get auto-linking which causes problems for all those tests that include the <boost/test/included/*.ipp> sources. Also the "new" header test_exec_monitor.hpp seems to get included as soon as you include *any* Boost.Test header. This isn't necessarily a bad thing, but does mean a breaking change. Even if the boost/test/included/*.ipp sources were to turn off auto-linking (and they should) that won't work if some other Boost.Test header has been included first. John.

* Autolinking support is implemented To employ autolinking with Program Execution Monitor and Test Execution Monitor one need to include new headers test_exec_monitor.hpp and prg_exec_monitor.hpp
Unfortunately this is causing a *huge* number of failures in the current test matrix - if you include a Boost.Test header now you get auto-linking which causes problems for all those tests that include the <boost/test/included/*.ipp> sources.
I am not quite sure what you mean. test_exec_monitor.hpp and prg_exec_monitor.hpp are brand new headers and not included from any other Boost.Test module. To disable auto linking one need to use regular means.
Also the "new" header test_exec_monitor.hpp seems to get included as soon as you include *any* Boost.Test header. This isn't necessarily a bad thing,
I don't think this is true.
but does mean a breaking change. Even if the boost/test/included/*.ipp sources were to turn off auto-linking (and they should) that won't work if some other Boost.Test header has been included first.
You need to give me more details or simple example indicating problem. Gennadiy

but does mean a breaking change. Even if the boost/test/included/*.ipp sources were to turn off auto-linking (and they should) that won't work if some other Boost.Test header has been included first.
You need to give me more details or simple example indicating problem.
#include <boost/test/test_tools.hpp> #include <boost/test/included/test_exec_monitor.hpp> Results in the new auto-linking code being brought in, it appears to come from the test_exec_monitor.hpp include. John.

#include <boost/test/test_tools.hpp> #include <boost/test/included/test_exec_monitor.hpp>
Results in the new auto-linking code being brought in, it appears to come from the test_exec_monitor.hpp include.
auto_link.hpp is included only in 3 headers: boost/test/unit_test.hpp boost/test/test_exec_monitor.hpp boost/test/prg_exec_monitor.hpp None it which is included by the 2 headers above. Are you sure you did not modify files locally? Gennadiy

None it which is included by the 2 headers above. Are you sure you did not modify files locally?
No! It's trivial to reproduce with: #include <boost/test/included/test_exec_monitor.hpp> int test_main(int, char*[]) { return 0; } which outputs: $ cl -GX -I. scrap.cpp Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3077 for 80x86 Copyright (C) Microsoft Corporation 1984-2002. All rights reserved. scrap.cpp Microsoft (R) Incremental Linker Version 7.10.3077 Copyright (C) Microsoft Corporation. All rights reserved. /out:scrap.exe scrap.obj LINK : fatal error LNK1104: cannot open file 'libboost_unit_test_framework-vc71- s-1_34.lib' The header inclusion backtrace is: #line 20 "c:\\data\\boost\\develop\\boost\\boost\\test\\included\\test_exec_monitor.hpp" #line 28 "c:\\data\\boost\\develop\\boost\\boost\\test\\impl\\framework.ipp" #line 1 "c:\\data\\boost\\develop\\boost\\boost\\test\\auto_unit_test.hpp" #line 34 "c:\\data\\boost\\develop\\boost\\boost\\test\\unit_test.hpp" #line 1 "c:\\data\\boost\\develop\\boost\\boost\\config\\auto_link.hpp" Clear now? John.

The header inclusion backtrace is:
#line 20 "c:\\data\\boost\\develop\\boost\\boost\\test\\included\\test_exec_monitor.hpp" #line 28 "c:\\data\\boost\\develop\\boost\\boost\\test\\impl\\framework.ipp" #line 1 "c:\\data\\boost\\develop\\boost\\boost\\test\\auto_unit_test.hpp"
Yes. This one is a mistake and should be included. Will fix today.
#line 34 "c:\\data\\boost\\develop\\boost\\boost\\test\\unit_test.hpp" #line 1 "c:\\data\\boost\\develop\\boost\\boost\\config\\auto_link.hpp"
Gennadiy

"c:\\data\\boost\\develop\\boost\\boost\\test\\included\\test_exec_monitor.hpp"
#line 28 "c:\\data\\boost\\develop\\boost\\boost\\test\\impl\\framework.ipp" #line 1 "c:\\data\\boost\\develop\\boost\\boost\\test\\auto_unit_test.hpp"
Yes. This one is a mistake and should be included. Will fix today.
This appears to be still broken, the following headers include <boost/test/unit_test.hpp> and thus activate auto-linking: C:\data\boost\develop\boost\boost\test\auto_unit_test.hpp(18):#include <boost/test/unit_test.hpp> C:\data\boost\develop\boost\boost\test\exception_safety.hpp(21):#include <boost/test/unit_test.hpp> C:\data\boost\develop\boost\boost\test\logged_expectations.hpp(22):#include <boost/test/unit_test.hpp> C:\data\boost\develop\boost\boost\test\impl\test_main.ipp(20):#include <boost/test/unit_test.hpp> So currently all the tests that include the one of the <boost/test/included/*.hpp> headers fail. John.

"John Maddock" <john@johnmaddock.co.uk> wrote in message news:004301c60593$db6ea230$47e40c52@fuji...
"c:\\data\\boost\\develop\\boost\\boost\\test\\included\\test_exec_monitor.hpp"
#line 28 "c:\\data\\boost\\develop\\boost\\boost\\test\\impl\\framework.ipp" #line 1 "c:\\data\\boost\\develop\\boost\\boost\\test\\auto_unit_test.hpp"
Yes. This one is a mistake and should be included. Will fix today.
This appears to be still broken, the following headers include <boost/test/unit_test.hpp> and thus activate auto-linking:
C:\data\boost\develop\boost\boost\test\auto_unit_test.hpp(18):#include <boost/test/unit_test.hpp> C:\data\boost\develop\boost\boost\test\exception_safety.hpp(21):#include <boost/test/unit_test.hpp> C:\data\boost\develop\boost\boost\test\logged_expectations.hpp(22):#include <boost/test/unit_test.hpp> C:\data\boost\develop\boost\boost\test\impl\test_main.ipp(20):#include <boost/test/unit_test.hpp>
So currently all the tests that include the one of the <boost/test/included/*.hpp> headers fail.
Ok. I will address this tonight. Gennadiy

Gennadiy Rozental wrote:
Hi everybody
I finished the first update to the 1.33 release of Boost.Test. This update includes several bug fixes and series on new features. Once this updates settles for next update I will invest my efforts exclusively in documentation making it up to date and more convenient to use. The last update should finalize runtime parameter framework and testing by name feature. I don't think most of below modification require any (semi-)formal review, but I would really appreciate any comments and reviews for new features.
I'd like to make a request for another new feature: BOOST_WARN/CHECK/REQUIRE_MATCH(<regex-string>, <string>) Example: std::string data("aha"); BOOST_REQUIRE_MATCH("^a", data); This is something I've found myself using a lot when working with Ruby (included with Ruby's unit test library - "assert_match"). The obvious problem is an added dependency on regular expression support, but this support could be made optional for those who require it (using e.g. Boost.Regex or Xpressive when available). I guess that it should be pretty easy to implement using Xpressive's header-only implementation when it is released. // Johan
participants (3)
-
Gennadiy Rozental
-
Johan Nilsson
-
John Maddock