[Please do not mail me a copy of your followup] boost-users@lists.boost.org spake the secret code thusly:

First off, is there a possibility to do a negative selection. i.e. tell boost.test to NOT run certain Testcases/testsuites?

There is only positive specification, although it does have a wildcard facility.

Further is it possible to run sets of Testcases(which may reside in different testsuites)?

Yes, you can specify multiple sets of test cases by using commas and identify suite levels using slash like a pathname. However, if you have your tests arranged into suites and you use a comma, things don't bind the way you'd expect because the matcher splits on slash first and then on comma. So commas don't separate completely seperate test specifications, they separate names at the same level of the test suite hierarchy.

Assume following test tree: MASTER suite1 case1 case2

suite2 testcase1 testcase2 suite3 subsuite testcase1

Now lets say I wanna run testcase case1, testcase2 and the subsuite testsuite.

I made a dummy test application with suites and test cases arranged as above and got this:

ConsoleApplication1.exe --report_level=detailed --run_test=*/case1,testcase2,subsuite Running 3 test cases...

Test suite "Master Test Suite" passed with: 3 assertions out of 3 passed 3 test cases out of 3 passed Test suite "suite1" passed with: 1 assertion out of 1 passed 1 test case out of 1 passed Test case "case1" passed with: 1 assertion out of 1 passed Test suite "suite2" passed with: 1 assertion out of 1 passed 1 test case out of 1 passed Test case "testcase2" passed with: 1 assertion out of 1 passed Test suite "suite3" passed with: 1 assertion out of 1 passed 1 test case out of 1 passed Test suite "subsuite" passed with: 1 assertion out of 1 passed 1 test case out of 1 passed Test case "testcase1" passed with: 1 assertion out of 1 passed However, things aren't as wonderful as this implies. Suppose I add another test case called "case1" to "subsuite" and rerun the same command:

ConsoleApplication1.exe --report_level=detailed --run_test=*/case1,testcase2,subsuite Running 4 test cases...

Test suite "Master Test Suite" passed with: 4 assertions out of 4 passed 4 test cases out of 4 passed Test suite "suite1" passed with: 1 assertion out of 1 passed 1 test case out of 1 passed Test case "case1" passed with: 1 assertion out of 1 passed Test suite "suite2" passed with: 1 assertion out of 1 passed 1 test case out of 1 passed Test case "testcase2" passed with: 1 assertion out of 1 passed Test suite "suite3" passed with: 2 assertions out of 2 passed 2 test cases out of 2 passed Test suite "subsuite" passed with: 2 assertions out of 2 passed 2 test cases out of 2 passed Test case "testcase1" passed with: 1 assertion out of 1 passed Test case "case1" passed with: 1 assertion out of 1 passed So you see the problem has to do with the backwards way in which slash and comma are used to parse the argument value. You'd expect that comma would specify a whole new test specification and not a variant at the level expressed between slashes. Since you have your tests in suites, you must specify the slash or your filter won't match any tests. There is another option which may be appealing if you have to run this subset long-term and that is to simply link the interesting subset into it's own executable and run everything in that executable. -- "The Direct3D Graphics Pipeline" free book http://tinyurl.com/d3d-pipeline The Computer Graphics Museum http://computergraphicsmuseum.org The Terminals Wiki http://terminals.classiccmp.org Legalize Adulthood! (my blog) http://legalizeadulthood.wordpress.com

[Please do not mail me a copy of your followup] boost-users@lists.boost.org spake the secret code thusly:

Is this even possible?

I consider the behavior I described in my other post to be a bug (and one that is easily fixed). https://svn.boost.org/trac/boost/ticket/9531 It would help if you commented agreement on this bug in the tracker. -- "The Direct3D Graphics Pipeline" free book http://tinyurl.com/d3d-pipeline The Computer Graphics Museum http://computergraphicsmuseum.org The Terminals Wiki http://terminals.classiccmp.org Legalize Adulthood! (my blog) http://legalizeadulthood.wordpress.com

Manuel Maier writes:

Hi I am looking for help considering running sets of Testcases/Testsuites.

Hi Manuel, Release version of Boost.Test does provide some functionality for selective test run. There are few issues with it and number of limitations. In trunk version I reworked this component significantly. Now you not only can have combination of compile time/runtime selection, there is an ability "exclude" commands, you can combine several run_test commands and you can also label your test cases and select/exclude based on a label.

First off, is there a possibility to do a negative selection.

In trunk.

i.e. tell boost.test to NOT run certain Testcases/testsuites?

Further is it possible to run sets of Testcases(which may reside in different testsuites)? Assume following test tree: MASTER suite1        case1        case2 suite2        testcase1        testcase2 suite3        subsuite                testcase1

Now lets say I wanna run testcase case1, testcase2 and the subsuite testsuite.

Is this even possible?

I think this should be possible even in release version. run_test comamnd will look a bit clumsy, but it should work. It will defnitely work in trunk version. Regards, Gennadiy