Hi,
I would like to take this moment to share with you finally the modifications
Boost.Test went through during last couple month. Most of Boost.Test
components undergo a major or significant change in both design and
implementation. I've strived to keep an interface unchanged as much as
possible. The primary goals for this update were:
* Make a unit test framework better suited for different 'runners'
* Make unit test framework simpler internally by eliminating test cases
hierarchy
* Make test tools more uniform
* Make auto unit test framework as usable as possible
Following are details of what updated/changed specifically in each area of
Boost.Test:
I. Execution Monitor
The execution monitor design changes from inheritance to delegation. If
previously one needed to inherit from the boost::execution monitor to do a
monitored run of a function, now one need to employ an instance of the
boost::execution_monitor:
boost::execution_monitor em;
// register translators for custom user exception
em.register_exception_translator<ExceptionType>( my_translator );
em.execute( function_to_execute );
function_to_execute should be any zero arity function or function object. To
implement the polymorphic function handling I employ my own class callback
(see utilities section). I did not want to introduce a dependency on
Boost.Function. But in a future I plan to switch to tr1::function.
In addition following features implemented:
* Debugger presence detected, in which case catching system errors (SEH) is
disabled unless explicitly enabled. For now only works for msvc family
* Automatic detection of memory leaks introduced. For now only works for
msvc family
Unit test framework turns this feature on by default. User could manage
it by specifying runtime parameter (in a form of command line argument or
environment variable). For example
test --detect_memory_leak=<leak_allocation_number>
set leak_allocation_number to integer number:
0 - to disable detection of memory leaks;
1 - to enable detection of memory leaks;
N>1 - to enable detection of memory leaks and set a system breakpoint at the
moment of Nth memory allocation. For more on how to find N and some issues
with usage of this feature see documentation
II Unit test framework (UTF)
This component of Boost.Test undergo a most dramatic changes in both
design and implementation. Here are main modifications:
1. UTF doesn't employ test cases hierarchy anymore. There is single class
test_case that model one testing unit and class test_suite that model a
collection of test units. Previously UTF employed compound test cases (for
example parameterized test case) which went through it's components (simple
subtests) during testing run time. Now compound test present itself as test
case generator which produce a collection if test_case instances during test
tree construction time. One consequence of this is that user doesn't need to
keep track of compound test case parameters is it used be. test_case now
employs a polymorphic callback to a actual test function. As a result of
this one could supply any zero arity function as an argument of
BOOST_TEST_CASE. SO no extra support needed to be able to create
boost::function or boost::bind based tests cases.
2 .Introduced generic mechanism to traverse the test tree. Anyone willing
to do so just need to implement test_tree_visitor interface and employ
traverse_test_tree. This allows to implement any pre/post processing over
test tree.
3. Introduced a notion of framework. The framework is responcible for:
a) initializing and providing access to master test suite
b) providing access to current test case
c) running testing from given test tree node (simple or compound). By
default it runs from root - master test suite. Supports both sequential and
random order.
d) One may register any test observer agent (implements test_observer
interface) and framework notifies it of all test events
e) providing access to any test case based to unique test unit id
Using the framework one could implement different test runners. UTF library
supplies one console runner (unit_test_main.cpp). But it's possible to do
one winmain based or anything else. For example GUI program that build test
tree and then run tests by request from any point in test tree.
4. Unit test log interface changed to single line (used to be begin/end
macros). Unit test log formatter interface changed. How it much simpler and
straightforward. Two supplied implementation are separated.
5. Notion of unit test result that was responcible for result collection
and reporting eliminated. Instead introduced independent notions:
results_collector and results_reporter. Results reporting formatter
interface made public. Also results_reporter now allows to change an output
stream.
6. Progress monitor functionality (which used to be activated in log
level log_progress) now is an independent test observer and implemented by
class progress_monitor.
7. Test case template interface changed. Now user doesn't need to
explicitly register meta class.
In addition couple more features implemented:
* Added support for test case timing
* UTF made __cdecl - safe
* Result report include skipped tests
* io saver facilities employed to guard against undesirable modification
in output streams formatting
* template test case automatically generate name that includes type name
* --detect_memory_leak and --random cla added
III Test Tools
The two major changes in Test Tools are:
* Test Tools interfaces to provide 3 version for each tool:
CHECK- reports an error if assertion failed
WARN - reports a warning if assertion failed
REQUIRE - reports an error if assertion failed and aborts test case
execution
* Test Tools implementation completely reworked. All tools implemented
now through single vararg function
In addition following modifications made:
* CHECK_COLLECTION interface changed: it now expect 4 arguments
* BITWISE_EQUAL renamed to CHECK_BITWISE_EQUAL; old name still provided,
but is deprecated
* CHECK_COLLECTION interface changed to use PP_SEQ and as a result
support arbitrary(actually maximum is fixed but configurable) number of
predicate arguments
* most of templates eliminated speeding up test modules compilation
* floating-point precision set to include all significant digits
(actually the formula may require some rework since it obviously doesn't do
a good job sometimes)
* BOOST_CHECK/WARN/REQUIRE_SMALL test tool added
* old test tools names are *deprecated*
IV Auto unit test framework
Interesting discussion with Noel Llopis and several user requests lead to
several additions to the auto_unit_test.hpp. Here is a short descriptions:
* BOOST_AUTO_UNIT_TEST renamed to BOOST_AUTO_TEST_CASE. Old name still
provided but deprecated
* BOOST_AUTO_TEST_CASE_TEMPLATE provides an ability to write a
self-registering test case template
* BOOST_FIXTURE_TEST_CASE provides an ability to write a self-registering
test case with fixture
* BOOST_AUTO_TEST_CASE_EXPECTED_FAILURES provides an ability to specify a
number of expected failures in any self-registering test case
* BOOST_AUTO_TEST_SUITE and BOOST_AUTO_TEST_SUITE_END provide an ability
to structure test cases into self-registering test suites
V Included components
Implementation moved from source files (.cpp) under lib/test/src into
implementation files (.ipp) under boost/test/impl to eliminate dependency of
included/minimal components on lib/test/src directory.
VI Utilities
Basic utilities that may have value outside of Boost.Test now separated into
utils subdirectory. If there is an interest to promote any of them onto
public boost level I am ready to discuss it. Here is a short descriptions
(some of them existed before, but most are new):
1) algorithm.hpp
Implements several variations of STL algorithms including 4-arg mismatch
2) assign_op.hpp
Hook for overloadable assignment
3) callback.hpp
Implements a polymorphic function callback facility (similar to
boost::function).
4) class_properties.hpp
Simple facility that mimic notion of read-only read-write properties in
C++ classes
5) custom_manip.hpp
Simple helpers for creating custom output manipulators
6) fixed_mapping.hpp
Fixed sized mapping (initialized in constructor ) with specified invalid
value.
7) foreach.hpp
This is an abridged version of BOOST_FOREACH. The only difference is
portability.
8) named_params.hpp
Facility for named function parameters support. Even though Dave A. did
not see a value in my suggestion during Boost.Parameter review period. I
couldn't justify for myself using that inconvenient interface (IMO) and
absence of optional parameters. This facility implements an alternative
solution with different set of tradeoffs and features. It used in several
places in Boost.Test.
9) nullstream.hpp
I should probably deprecate this one , but more-io was not accepted.
10) rtti.hpp
Simple alternative to compiler RTTI. Provides an access to unique type id
at runtime. I plan to extend this to implement type_descr similar to what is
described in "C++ template metaprogramming".
11) trivial_singleton.hpp
As the name suggest this simple singleton helper.
12) wrap_stringstream.hpp
Wraps strstream and stringstream (depends with one is present) to
provide the unified interface
13) basic_cstring/
class template basic_cstring wraps raw memory pointers and provide
std_string like interface
14) iterator/input_iterator_facade.hpp
Helps to build input iterators
15) iterator/token_iterator.hpp
Yet another token iterator
16) iterator/istream_line_iterator.hpp
Iterate over istream and produce lines
17) runtime/
This is long promised framework for parsing CLA handing environment
variable and parsing config files. This is an alternative solution to what
boost::program_options present. Unfortunately I did not have enough time to
incorporate it completely into Boost.Test main components, so the detailed
description will follow once this will happened.
Sorry, for babbling for so long. Any comments are greatly appreciated.
Regard,
Gennadiy
