I've tried out the suggestions Gennadiy had for streamlining the use of Boost.Test fixtures and suites, and I'm extremely pleased with the results. This is what I ended up with (compiled and tested with gcc under Linux). Feel free to adapt anything into Boost.Test: #define BOOST_TEST_SUITE_MODULE( name ) \ namespace { \ static boost::unit_test::test_suite* suite = BOOST_TEST_SUITE( #name ); \ static boost::unit_test::ut_detail::auto_unit_test_registrar \ BOOST_JOIN( test_registrar, __LINE__)( suite ); \ \ struct suite_registrar { \ suite_registrar( boost::unit_test::test_case* tc ) { suite->add( tc ); } \ }; \ } \ /**/ #define BOOST_AUTO_UNIT_TEST_WITH_FIXTURE_AND_SUITE( fixture, test_name ) \ struct fixture ## test_name : public fixture { \ void test_name(); \ }; \ \ void fixture ## test_name ## _invoker () \ { \ fixture ## test_name mt; \ mt.test_name(); \ } \ \ static suite_registrar \ BOOST_JOIN( suite_registrar, __LINE__) \ ( boost::unit_test::create_test_case( fixture ## test_name ## _invoker, #test_name " fixture " #fixture) ); \ void fixture ## test_name::test_name() \ /**/ #define BOOST_AUTO_UNIT_TEST_WITH_SUITE(test_name) \ static void test_name(); \ static suite_registrar \ BOOST_JOIN( suite_registrar, __LINE__) \ ( BOOST_TEST_CASE( test_name ) ); \ static void test_name() \ /**/ // Shortcuts to avoid lots of typing #define TEST_F(fixture,test_name) BOOST_AUTO_UNIT_TEST_WITH_FIXTURE_AND_SUITE(fixture,test_name) #define TEST(test_name) BOOST_AUTO_UNIT_TEST_WITH_SUITE(test_name) Now, if we can only have a unit_test_log_formatter that uses DebugOutputString, that would be fantastic. I'd volunteer to look into it, but I don't use Windows for development at home anymore. A couple of questions: What was the reason behind the choice of using the function init_unit_test_suite() as the initialization point? Why not let the user create main() and use a very simple object to accomplish the same thing. Something along these lines: int main ( int /* argc */, char* /* argv */ [] ) { test_suite * testsuite = BOOST_TEST_SUITE("Master test suite"); testrunner runner(testsuite); return runner.run(); } That way we can do whatever we want before and after the test runs much more easily. Also, if I wanted to add timings around each test, where would be the best place to hook them up? I'm also interested in the timings around the whole set of tests, but that's something I could accomplish easily if we had an exposed main() function. I think that with these changes, Boost.Test has tipped the balance as the favorite. I'm going to have to run a few more tests, but if all continues to look good, I'm going to put up another article on my web site with everything I've learned about Boost.Test Thanks. --Noel

Robert Ramey <ramey <at> rrsd.com> writes:

I'm not sure if this is what you're referring to but ... I added all my tests to the post-build stop according to instructions in the VC 7.1 IDE. Each time I rebuild, the tests are run - and the messages all show up in the "Outpu Window" of the IDE.

Hmm... I'm trying to think why I needed that in the past. Yes, redirecting the output to the output page will work fine as long as you're working with a "console" app. Unfortunately there are times that to initialize some of the libraries we need to create a DirectX surface, or at least a Window instance. In that case, we have to create a Win32 app, and std::out is not piped correctly to the output. I'll have to double-check this. One thing is for sure though, having control over main() is very important. Otherwise it might not even be possible to create anything other than a regular console app. I thought it would be relatively trivial to take over main, but then I realized that main() is part of the library right now, so the ony way to provide your own would be modifying the source code of the library and recompiling. Any thoughts on that? --Noel

Gennadiy Rozental wrote:

...

Yes, redirecting the output to the output page will work fine as long as you're working with a "console" app. Unfortunately there are times that to initialize some of the libraries we need to create a DirectX surface, or at least a Window instance. In that case, we have to create a Win32 app, and std::out is not piped correctly to the output. I'll have to double-check this.

Where is it piped to then?

I am not sure where it is piped to, but it is possible to get the output. Given: // winapp.exe WinMain(...) { std::cout << "Hello World!\n"; } On the console: winapp produces no output, while winapp | cat produces: Hello World! Regards, Reece

On Tuesday 11 January 2005 14:03, Gennadiy Rozental wrote:

...

What was the reason behind the choice of using the function init_unit_test_suite() as the initialization point? Why not let the user create main() and use a very simple object to accomplish the same thing.

...

Something along these lines:

int main ( int /* argc */, char* /* argv */ [] ) { test_suite * testsuite = BOOST_TEST_SUITE("Master test suite"); testrunner runner(testsuite); return runner.run(); }

1. If you take a look into unit_test_main.cpp you notice that there is quire a lot of things that needs to be done before test is initialized and run

Nothing that can't be easily wrapped in an object like the example above.

4. There may be other things in framework housekeeping that may needs to be done pre/post test execution

Right, but you can wrap all that inside a separate function instead of taking over main().

Anything you want to do pre/post your test is part of fixure, isn't it?

No. In this case I was thinking of tasks that I might have to do around the test run myself. For example, I don't think I could use Boost.Test with a Win32 application (as opposed to a DOS box). Also, I might want to set up some things conditionally depending on whether the debugger is attached or not. In general, it seems like a much better idea to let the user run the tests from anywhere they want as opposed to taking over main. For example, sometimes people like to embed the tests themselves into the program they're building, and running the tests if they pass a command-line parameter. This setup would allow them to do that without any problem. The ideal situation would be to wrap the contents of the current main in an object, and then provide a macro that gives you a main function calling that object. Those of us who need more control, we can write our own main and pass all the necessary parameters. That's one of the areas that I think CppUnit did a really good job. This is how one possible main looks for them: int main( int argc, char* argv[] ) { // Create the event manager and test controller CPPUNIT_NS::TestResult controller; // Add a listener that colllects test result CPPUNIT_NS::TestResultCollector result; controller.addListener( &result ); // Add the top suite to the test runner CPPUNIT_NS::TestRunner runner; runner.addTest( CPPUNIT_NS::TestFactoryRegistry::getRegistry().makeTest() ); runner.run( controller ); // Print test in a compiler compatible format. CPPUNIT_NS::CompilerOutputter outputter( &result, std::cerr ); outputter.write(); return result.wasSuccessful() ? 0 : 1; } It's very configurable, and it can be called from anywhere. It might be a bit long, but you're only going to write it once per test project, so it's not a big deal at all. --Noel

On Tuesday 11 January 2005 16:32, Beman Dawes wrote:

I beg to differ. Having to supply all that boilerplate would be a huge disincentive.

If that functionality were exposed separately, putting it together in one macro with all the options you want to would be absolutely trivial. Then all you would write is BOOST_TEST_MY_MAIN And if that's not good enough, then you get your hands dirty and write main.

If Gennadiy can provide addition functionality making it easier for those needing more startup/shutdown control, great. But it shouldn't come at any cost to those who don't need that functionality.

From my point of view, it's more important to provide that functionality than to keep the ease of use. Otherwise it might simply come a time that you can't use it (combining it with another library that also provides its own custom main). Fortunately, once that functionality is provided, wrapping it in your own macro with the exact options you want is quite trivial.

The more I think about it though, the more I see that the problem lies in the fact that Boost.Test is trying to do double-duty. On one hand, it's a unit-test framework to efficiently create and run unit tests. On the other hand, it's a part of the Boost.Test library, and tries to conform to certain outputs and command-line arguments, and that's why main is tightly controlled. Those two aspects should really be separated. The Boost.Test component can provide a main function that parses the command-line in a certain way, and calls the unit test framework accordingly. But people should also be free to use the unit test framework without being tied to that. For example, I might want to set different options (like what log to use) based on a command-line parameter, or set exception-handling on/off based on whether the debugger is present. Another way of separating those two aspects would be simply to create another library with what's right now in test_main.cpp. If you link to that library, you get the standard main. Otherwise you need to provide your own. Does that make sense? --Noel Games from Within http://www.gamesfromwithin.com