[Boost.Test] [1.38] Automated test suie registration with muti-file-test-modules

Greg Christopher

19 Jun 2009 19 Jun '09

10:50 p.m.

Hi, We are starting to expand the number of test modules and are having some growing pains. Originally we were using the "single header" variant included/unit_test.hpp file. Now we have switched to linking to the dynamic version of the boost test library and are including auto_unit_test.hpp. We have been counting on the Test Tools to automatically register suites and test cases. This still worked when we just included unit_test.hpp (instead of included/unit_test.hpp). However with multiple test modules, init_unit_test_suite is being defined in each module. I am looking for clear documentation on what to do. This seems like the closest thing: http://www.boost.org/doc/libs/1_31_0/libs/test/doc/components/unit_test_fram... However following those directions exactly when trying to use multiple test modules (in a single executable) and only one BOOST_AUTO_TEST_MAIN still results in init_unit_test_suite being multiply defined. I would like multiple test suites, although the documentation for this declares that there is a single global test suite. I apologize if this is well documented somewhere, but after an hour of googling I have to ask. Greg

Show replies by date

Gennadiy Rozental

20 Jun 20 Jun

12:53 a.m.

New subject: [Boost.Test] [1.38] Automated test suie registration with muti-file-test-modules

Greg Christopher writes:

...

Hi, We are starting to expand the number of test modules and are having some growing pains.

Originally we were using the "single header" variant included/unit_test.hpp file. Now we have switched to linking to the dynamic version of the boost test library and are including auto_unit_test.hpp.

Just unit_test.hpp is fine. Older name is deprecated.

...

We have been counting on the Test Tools to automatically register suites and test cases. This still worked when we just included unit_test.hpp (instead of included/unit_test.hpp).

However with multiple test modules, init_unit_test_suite is being defined in each module.

This should work as you expect. Please post simple example illustrating your issues.

...

I would like multiple test suites, although the documentation for this declares that there is a single global test suite.

You can have multiple test suite under the master test suite. Gennadiy

Greg Christopher

23 Jun 23 Jun

10:03 p.m.

New subject: [Boost.Test] [1.38] Automated test suie registration with muti-file-test-modules

...

...
Hi, We are starting to expand the number of test modules and are having some growing pains.

Originally we were using the "single header" variant included/unit_test.hpp file. Now we have switched to linking to the dynamic version of the boost test library and are including auto_unit_test.hpp.

Just unit_test.hpp is fine. Older name is deprecated.

Ok- hopefully that's true for the version mentioned in the subject line.

...

...
We have been counting on the Test Tools to automatically register suites and test cases. This still worked when we just included unit_test.hpp (instead of included/unit_test.hpp).

However with multiple test modules, init_unit_test_suite is being defined in each module.

This should work as you expect. Please post simple example illustrating your issues.

Test1.cpp: ----- #include "stdafx.h" #define BOOST_TEST_MODULE foo2 #include #include class DefaultFixture { public: DefaultFixture() { } ~DefaultFixture() {}; }; BOOST_AUTO_TEST_SUITE(foo2) BOOST_FIXTURE_TEST_CASE(foo2, DefaultFixture) { BOOST_CHECK(4 == 4) } ----- Test2.cpp: ----- #include "stdafx.h" #define BOOST_TEST_MODULE foo #include #include class DefaultFixture { public: DefaultFixture() { } ~DefaultFixture() {}; }; BOOST_AUTO_TEST_SUITE(foo) BOOST_FIXTURE_TEST_CASE(foo, DefaultFixture) { BOOST_CHECK(4 == 4); } BOOST_AUTO_TEST_SUITE_END() ----- And Link with boost library.... 1>TestCase2.obj : error LNK2005: "class boost::unit_test::test_suite * __cdecl init_unit_test_suite(int,char * * const)" (?init_unit_test_suite@@YAPAVtest_suite@unit_test@boost@@HQAPAD@Z) already defined in basic.obj Googling around, I discovered this cryptic define done in one of the files will fix it: ----- #define init_unit_test_suite init_auto_unit_test_suite ----- A strange observation as well.... Take this code: #include "stdafx.h" #define BOOST_TEST_MODULE foo #include #include class DefaultFixture { public: DefaultFixture() { } ~DefaultFixture() {}; }; BOOST_AUTO_TEST_SUITE(foo) BOOST_FIXTURE_TEST_CASE(foo, DefaultFixture) { BOOST_CHECK(4 == 4); } BOOST_AUTO_TEST_SUITE_END() And change BOOST_FIXTURE_TEST_CASE toBOOST_TEST_CASE(foo)... it won't compile: C:\basic\test2.cpp(21) : error C2882: 'foo' : illegal use of namespace identifier in expression c:\code\main\test\bora-vmsoft\svga\tests\bat\basic\basic\test2.cpp(22) : error C2448: 'boost::unit_test::make_test_case' : function-style initializer appears to be a function definition ----- Anyway, thanks for your help. The documentation right now seems to be lacking a little bit in giving clear examples of the different styles in which we are able to do test and suite registration. Greg

Steven Watanabe

10:32 p.m.

New subject: [Boost.Test] [1.38] Automated test suie registration with muti-file-test-modules

AMDG Greg Christopher wrote:

...

Test1.cpp: ----- #include "stdafx.h"

#define BOOST_TEST_MODULE foo2

<snip> ----- Test2.cpp: ----- #include "stdafx.h"

#define BOOST_TEST_MODULE foo <snip> ----- And Link with boost library....

Don't #define BOOST_TEST_MODULE in more than one translation unit. In Christ, Steven Watanabe

Gennadiy Rozental

24 Jun 24 Jun

1:27 a.m.

New subject: [Boost-users] [Boost.Test] [1.38] Automated test suie registration with muti-file-test-modules

Steven Watanabe writes:

...

Don't #define BOOST_TEST_MODULE in more than one translation unit.

This is covered in docs I believe. BOOST_TEST_MODULE has similar effect as BOOST_TEST_MAIN, but also define the name of the master test suite Gennadiy

Greg Christopher

1 Jul 1 Jul

6:20 p.m.

New subject: [Boost.Test] [1.38] Automated test suie registration with muti-file-test-modules

Thanks for clearing this up...

...

-----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users- bounces@lists.boost.org] On Behalf Of Gennadiy Rozental Sent: Tuesday, June 23, 2009 6:27 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] [Boost.Test] [1.38] Automated test suie registration with muti-file-test-modules

Steven Watanabe writes:

...
Don't #define BOOST_TEST_MODULE in more than one translation unit.

This is covered in docs I believe. BOOST_TEST_MODULE has similar effect as BOOST_TEST_MAIN, but also define the name of the master test suite

Since I don't really need to define the name of the master test suite, I will not use BOOST_TEST_MODULE. As it turns out, that still does not yield a working example. I then have no init_unit_test_suite() defined. By messing around, I find that defining BOOST_AUTO_TEST_MAIN fixes the problem. Looks like I need to do that in some module? I thought that the main program would be pulled out of the library if I didn't provide one. While that may be true, init_unit_test_suite is not defined. So may I assume that for a fully automated situation with multiple test modules, the right approach is simply to insure that BOOST_AUTO_TEST_MAIN is defined in one module? Thanks again, Greg

Gennadiy Rozental

4 Jul 4 Jul

6:47 a.m.

New subject: [Boost-users] [Boost.Test] [1.38] Automated test suie registration with muti-file-test-modules

Greg Christopher writes:

...

...
This is covered in docs I believe. BOOST_TEST_MODULE has similar effect as BOOST_TEST_MAIN, but also define the name of the master test suite

Since I don't really need to define the name of the master test suite, I will not use BOOST_TEST_MODULE.

As it turns out, that still does not yield a working example. I then have no init_unit_test_suite()

Please post an example and how you build it.

...

defined. By messing around, I find that defining BOOST_AUTO_TEST_MAIN fixes the problem. Looks like I need to do that in some module?

No. BOOST_AUTO_TEST_MAIN is deprecated and should not be required.

...

I thought that the main program would be pulled out of the library if I didn't provide one. While that may be true, init_unit_test_suite is not defined.

It may not have to be defined it you are using shared library variant of UTF. Gennadiy

Greg Christopher

8 Jul 8 Jul

5:01 p.m.

New subject: [Boost.Test] [1.38] Automated test suite registration with muti-file-test-modules

...

-----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users- bounces@lists.boost.org] On Behalf Of Gennadiy Rozental Sent: Friday, July 03, 2009 11:48 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] [Boost.Test] [1.38] Automated test suie registration with muti-file-test-modules

Greg Christopher writes:

...
...
This is covered in docs I believe. BOOST_TEST_MODULE has similar effect as BOOST_TEST_MAIN, but also define the name of the master test suite

Since I don't really need to define the name of the master test suite, I will not use BOOST_TEST_MODULE.

As it turns out, that still does not yield a working example. I then have no init_unit_test_suite()

Please post an example and how you build it.

The example is basically the same example that was posted earlier: Test1.cpp: #include "stdafx.h" #define BOOST_TEST_MAIN Yes // Remove this line and we stop working #include #include BOOST_AUTO_TEST_SUITE(foo1) BOOST_AUTO_TEST_CASE(foo1case) { BOOST_CHECK(4 == 4); } BOOST_AUTO_TEST_SUITE_END() Test2.cpp #include "stdafx.h" #include #include #define BOOST_TEST_MODULE Yeah BOOST_AUTO_TEST_SUITE(foo2) BOOST_AUTO_TEST_CASE(foo2case) { BOOST_CHECK(4 == 4); } BOOST_AUTO_TEST_SUITE_END() Now when you link to the static version of the lib you get this: 1>libboost_unit_test_framework-vc90-mt-gd-1_38.lib(unit_test_main.obj) : error LNK2019: unresolved external symbol "class boost::unit_test::test_suite * __cdecl init_unit_test_suite(int,char * * const)" (?init_unit_test_suite@@YAPAVtest_suite@unit_test@boost@@HQAPAD@Z) referenced in function _main Project is a simple Visual Studio 2008 one using static version of boost: http://www.easy-share.com/1906667710/basic.zip Thanks, Greg

Greg Christopher

27 Jul 27 Jul

7:58 p.m.

New subject: [Boost.Test] [1.38] Automated test suite registration with muti-file-test-modules

Ping :)

...

-----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users- bounces@lists.boost.org] On Behalf Of Greg Christopher Sent: Wednesday, July 08, 2009 10:02 AM To: boost-users@lists.boost.org Subject: Re: [Boost-users] [Boost.Test] [1.38] Automated test suite registration with muti-file-test-modules

...
-----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users- bounces@lists.boost.org] On Behalf Of Gennadiy Rozental Sent: Friday, July 03, 2009 11:48 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] [Boost.Test] [1.38] Automated test suie registration with muti-file-test-modules

Greg Christopher writes:

...
...
This is covered in docs I believe. BOOST_TEST_MODULE has similar effect as BOOST_TEST_MAIN, but also define the name of the master test suite

Since I don't really need to define the name of the master test suite, I will not use BOOST_TEST_MODULE.

As it turns out, that still does not yield a working example. I then have no init_unit_test_suite()

Please post an example and how you build it.

The example is basically the same example that was posted earlier: Test1.cpp: #include "stdafx.h"

#define BOOST_TEST_MAIN Yes // Remove this line and we stop working

#include #include

BOOST_AUTO_TEST_SUITE(foo1)

BOOST_AUTO_TEST_CASE(foo1case) { BOOST_CHECK(4 == 4); }

BOOST_AUTO_TEST_SUITE_END() Test2.cpp #include "stdafx.h"

#include #include

#define BOOST_TEST_MODULE Yeah

BOOST_AUTO_TEST_SUITE(foo2)

BOOST_AUTO_TEST_CASE(foo2case) { BOOST_CHECK(4 == 4); }

BOOST_AUTO_TEST_SUITE_END()

Now when you link to the static version of the lib you get this:

1>libboost_unit_test_framework-vc90-mt-gd-1_38.lib(unit_test_main.obj) : error LNK2019: unresolved external symbol "class boost::unit_test::test_suite * __cdecl init_unit_test_suite(int,char * * const)" (?init_unit_test_suite@@YAPAVtest_suite@unit_test@boost@@HQAPAD@Z) referenced in function _main

Project is a simple Visual Studio 2008 one using static version of boost:

http://www.easy-share.com/1906667710/basic.zip

Thanks, Greg

_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

Ahmed Badran

28 Jul 28 Jul

7:32 a.m.

New subject: [Boost.Test] [1.38] Automated test suite registration with muti-file-test-modules

...

...
-----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users- bounces@lists.boost.org] On Behalf Of Greg Christopher Sent: Wednesday, July 08, 2009 10:02 AM To: boost-users@lists.boost.org Subject: Re: [Boost-users] [Boost.Test] [1.38] Automated test suite registration with muti-file-test-modules

...
-----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users- bounces@lists.boost.org] On Behalf Of Gennadiy Rozental Sent: Friday, July 03, 2009 11:48 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] [Boost.Test] [1.38] Automated test suie registration with muti-file-test-modules

Greg Christopher writes:

...
...
This is covered in docs I believe. BOOST_TEST_MODULE has similar effect as BOOST_TEST_MAIN, but also define the name of the master test suite Since I don't really need to define the name of the master test suite, I will not use BOOST_TEST_MODULE.

As it turns out, that still does not yield a working example. I then have no init_unit_test_suite() Please post an example and how you build it. The example is basically the same example that was posted earlier: Test1.cpp: #include "stdafx.h"

#define BOOST_TEST_MAIN Yes // Remove this line and we stop working

#include #include

BOOST_AUTO_TEST_SUITE(foo1)

BOOST_AUTO_TEST_CASE(foo1case) { BOOST_CHECK(4 == 4); }

BOOST_AUTO_TEST_SUITE_END() Test2.cpp #include "stdafx.h"

#include #include

#define BOOST_TEST_MODULE Yeah

BOOST_AUTO_TEST_SUITE(foo2)

BOOST_AUTO_TEST_CASE(foo2case) { BOOST_CHECK(4 == 4); }

BOOST_AUTO_TEST_SUITE_END()

Now when you link to the static version of the lib you get this:

1>libboost_unit_test_framework-vc90-mt-gd-1_38.lib(unit_test_main.obj) : error LNK2019: unresolved external symbol "class boost::unit_test::test_suite * __cdecl init_unit_test_suite(int,char * * const)" (?init_unit_test_suite@@YAPAVtest_suite@unit_test@boost@@HQAPAD@Z) referenced in function _main

Project is a simple Visual Studio 2008 one using static version of boost:

http://www.easy-share.com/1906667710/basic.zip

Thanks, Greg This is how I usually do the automated test suite registration with multi-file test modules, I arrange my files as follows:

file1.cpp #include .... BOOST_AUTO_TEST_SUITE(file1_test_suite) ... tests ... BOOST_AUTO_TEST_SUITE_END() - end of file 1 - file2.cpp #include .... BOOST_AUTO_TEST_SUITE(file2_test_suite) ... tests ... BOOST_AUTO_TEST_SUITE_END() - end of file 2 - test_main.cpp #define BOOST_TEST_MODULE master_test_suite #include - end of test_main.cpp - This causes a main, a test runner and the automatic generation of the required functions in test_main.cpp. I'm not that experienced using boost.test, but I've found this solution to work nicely and fit my needs. Regards, Ahmed

Greg Christopher

13 Oct 13 Oct

10:56 p.m.

New subject: [Boost.Test] [1.38] --random runtime parameter

Hi folks, I am wondering what "non random" means (--random-0). We are seeing the test framework behave such that with --random-0, it seems to do the tests in an order and stick with that order, but I'm not sure how it's deciding the order. If we re-link the test program, it seems to do them in a different order. I would like to figure out how to make this deterministic without losing the benefit of not having to create my own test execution driver. I would think that non random would be in alphabetical order of test_case_name (we are doing auto registration) but that's not what we're seeing. Non-random seems to indicate maybe some kind of static initialization order, which is random. Thanks for any help. Greg

Steven Watanabe

14 Oct 14 Oct

3:16 a.m.

New subject: [Boost.Test] [1.38] --random runtime parameter

AMDG Greg Christopher wrote:

...

I am wondering what "non random" means (--random-0).

We are seeing the test framework behave such that with --random-0, it seems to do the tests in an order and stick with that order, but I'm not sure how it's deciding the order. If we re-link the test program, it seems to do them in a different order.

I would like to figure out how to make this deterministic without losing the benefit of not having to create my own test execution driver. I would think that non random would be in alphabetical order of test_case_name (we are doing auto registration) but that's not what we're seeing. Non-random seems to indicate maybe some kind of static initialization order, which is random.

You are correct that the order is effectively the static initialization order. This is not completely random. Test cases in the same translation unit should be executed in order. In Christ, Steven Watanabe P.S. Please start a new thread instead of replying to an existing thread.

Greg Christopher

16 Oct 16 Oct

6:40 p.m.

New subject: [Boost.Test] [1.38] Need way to have auto-registered tests remain consistant...

Hi folks, As mentioned in thread below, test cases on boost.test may be being run based on static initialization. However, we are seeing them run in different order with each new link- perhaps because Microsoft is intentionally changing the way things fall in memory? In the past I had created a test framework with auto-registration. In that framework, at static initialization time, objects registered themselves with a global parent object that had a container that was alphabetically sorted. Tests always ran in the same order, even if files were moved around in the project or makefile. At this point my tests aren't even running the same way when I recompile the same cpp file (there is only one cpp file in this test). I guess I'm down to two questions: 1) does this sound like a good feature request, and if so how do I file it? 2) for now what's the shortest path to what I want while retaining auto-registration. Thanks, Greg

...

-----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users- bounces@lists.boost.org] On Behalf Of Steven Watanabe Sent: Tuesday, October 13, 2009 8:16 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] [Boost.Test] [1.38] --random runtime parameter

AMDG

Greg Christopher wrote:

...
I am wondering what "non random" means (--random-0).

We are seeing the test framework behave such that with --random-0, it seems to do the tests in an order and stick with that order, but I'm not sure how it's deciding the order. If we re-link the test program, it seems to do them in a different order.

...
I would like to figure out how to make this deterministic without

losing the benefit of not having to create my own test execution driver. I would think that non random would be in alphabetical order of test_case_name (we are doing auto registration) but that's not what we're seeing. Non-random seems to indicate maybe some kind of static initialization order, which is random.

...
You are correct that the order is effectively the static initialization order. This is not completely random. Test cases in the same translation unit should be executed in order.

In Christ, Steven Watanabe

P.S. Please start a new thread instead of replying to an existing thread.

_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

Gennadiy Rozental

17 Oct 17 Oct

5:42 a.m.

New subject: [Boost.Test] [1.38] Need way to have auto-registered tests remain consistant...

Greg Christopher writes:

...

At this point my tests aren't even running the same way when I recompile the same cpp file (there is only one cpp file in this test).

These is something wrong going on here. The order should be well defined in this case. Are you sure you do not have 'random' set through either using CLA or environment? Can you create simple example illustrating the issues and provide details of your configuration? Gennadiy

Gennadiy Rozental

1 Aug 1 Aug

9:49 p.m.

New subject: [Boost.Test] [1.38] Automated test suite registration with muti-file-test-modules

Greg Christopher wrote:

...

#include #include

#define BOOST_TEST_MODULE Yeah

You obviously need to define the symbol *before* include, not after. Gennadiy

Gennadiy Rozental

24 Jun 24 Jun

12:25 p.m.

New subject: [Boost.Test] [1.38] Automated test suie registration with muti-file-test-modules

Greg Christopher wrote:

...

And change BOOST_FIXTURE_TEST_CASE toBOOST_TEST_CASE(foo)... it won't compile:

Correct name is BOOST_AUTO_TEST_CASE. BOOST_TEST_CASE is used with manual test case registration. Gennadiy

5503

Age (days ago)

5623

Last active (days ago)

List overview

Download

15 comments

4 participants

participants (4)

Ahmed Badran
Gennadiy Rozental
Greg Christopher
Steven Watanabe