[test] Support use of test tools and report generation in non-test programs - facilitating test/validation modes

An important use case that we have is the ability to run test scenarios on a production binary by specifying a 'test' mode. In test mode we essentially run through a series of different scenarios, asserting various conditions, comparing expected results and so on. In order to minimise the custom code we needed for this we decided to make use of Boost.Test. All the test tools are there, report generation and so on. In addition we have all the report analysis tools in place to interpret the output. The problem however, is that Boost.Test essentially requires a custom main() function. Either one of the ones provided by Boost.Test or one performing Boost.Test specific initialisation. In our production systems we have, like many, a fairly standard main() that ensures uniform behaviour across binaries and we do not want to pollute that with Boost.Test specific code. Notably we also do not want to pollute our command line parameters or configuration with 'global' Boost.Test arguments to control the test report output. Finally we want to control exactly when the tests are executed. I've created ticket #7419 to cover this use case and have also attached a small patch that facilitates the capabilities required to support it. https://svn.boost.org/trac/boost/ticket/7419 Essentially the patch does two things: * It allows boost::unit_test_framework::init() to be called more than once. The result is to reset the test status and master test suite so that new tests can be added and new reports generated through multiple calls. The CLA parsing is also reset. Recent changes to this code may make my call to clear() redundant. I'll let Gennadiy be the judge of that. * This first change then allows the framework setup and teardown to be encapsulated in a helper class (not provided in the patch but one could be). In our systems we use a helper class that exposes a simple API that allows the user to implement a method where test units can be added directly to the master test suite - see the ticket for an example. As I said the patch focuses solely on enabling the possibility of writing such a class, not the class itself. * It provides scoped Boost.Test arguments for the commandline when BOOST_TEST_USE_QUALIFIED_COMMANDLINE_ARGUMENTS is defined. All command line arguments are then called --boost.test.<argument> for example --boost.test.log_level instead of --log_level. This of course means that all Boost.Test related arguments will not clash with any program arguments. On its own this is a generally useful feature as it allows custom main() functions to pass arguments down to other components during simple unit testing. The patch attached is based on a patch against boost 1.49. The one attached to the ticket is for trunk. Again this is a small patch that is genuinely useful so I'd like to see it make its way into trunk. As far as I know there are no side-effects for adding these changes, tests all pass and so on. Jamie
participants (1)
-
Jamie Allsop