[Boost.Test] Test Ordering?
If I make unit*test*s in different source files, I need to declare a BOOST_AUTO_*TEST*_SUITEin each one, right? At link time, these appear in some undetermined order. How can I cause them to be run in the desired order instead?
Hi!
On Sun, Nov 4, 2012 at 4:50 PM, John M. Dlugosz
If I make unit*test*s in different source files, I need to declare a BOOST_AUTO_*TEST*_SUITEin each one, right?
No, you can have all test cases in the Master Test suite.
At link time, these appear in some undetermined order. How can I cause them to be run in the desired order instead?
I can come up with two ways: 1. Manual registration: http://www.boost.org/doc/libs/1_46_1/libs/test/doc/html/utf/user-guide/test-... 2. Put all test suites in a single cpp file, they will appear in the order they come in the cpp file. Thus you can write your tests in a header file and include them in the right order in to the single cpp-file. When dealing with manual registration you can also add dependencies to the TCs or test suites. At least it was possible in the previous version of Boost.Test (1.32 or so). Not sure if this is possible now. Regards, Ovanes
[Please do not mail me a copy of your followup] boost-users@lists.boost.org spake the secret code <24214-1352044228-124888@sneakemail.com> thusly:
At link time, these appear in some undetermined order. How can I cause them to be run in the desired order instead?
Good test design says that the tests are all independent of each other and can run in any order. Why do you want/need the tests to run in a certain order? -- "The Direct3D Graphics Pipeline" free book http://tinyurl.com/d3d-pipeline The Computer Graphics Museum http://computergraphicsmuseum.org The Terminals Wiki http://terminals.classiccmp.org Legalize Adulthood! (my blog) http://legalizeadulthood.wordpress.com
On Mon, Nov 5, 2012 at 10:43 PM, Richard
[Please do not mail me a copy of your followup]
boost-users@lists.boost.org spake the secret code <24214-1352044228-124888@sneakemail.com> thusly:
At link time, these appear in some undetermined order. How can I cause them to be run in the desired order instead?
Good test design says that the tests are all independent of each other and can run in any order.
Why do you want/need the tests to run in a certain order?
mmmm.... Test might be independent in their order and you can run them even randomly with Boost.Test by passing the random command line flag, but there is a big BUT here (as it is always the case in software development). This BUT is: I find such a feature very useful. Sometimes you can save lot's of time if using a dependency mechanism you can skip tests which would depend on critical tests which fail anyway. In that case it might make no sense to run smth that is obviously broken... Just mark it broken as well and don't run it. I really agree with you that otherwise putting the TCs in some special order and make them depend on some global state is very fragile probably not reproducible.
participants (3)
-
John M. Dlugosz -
legalize+jeeves＠mail.xmission.com -
Ovanes Markarian