[Regression Reports] Test case categorization

I'd like to draw library authors' attention to a recent generalization in the explicit markup scheme which allows one to control the grouping of a particular library's test cases in the Boost-wide regression reports, as demostrated here -- http://www.meta-comm.com/engineering/boost-regression/example/developer/libr... (http://tinyurl.com/bdd5u). The generalization is aimed at improving representation of library reports in at least two common situations: 1) "Duplicate" test cases: many libraries have two or more variations of the same test case for different configurations (DLL vs. static linking, char vs. wchar_t, etc.). 2) Test cases covering corner-case/optional/advanced functionality. In both of these cases, it's common to have an explicit markup (or actual failures) painting a group of test cases in a different color for one or more deficient toolsets -- see, for example, Boost.Thread page (http://tinyurl.com/8duf6). Test case categorization allows library authors to reduce the amount of visual clutter in the reports, giving them a more organized/structured look -- ultimately improving visitors' experience. Assigning a category to a test case or a group of test cases is just a matter of specifying the 'category' attribute to the corresponding '<test>' elements in "status/explicit-failures-markup.xml", e.g. <library name="my_library"> <test name="*_dll" category="DLL"/> </library> or <library name="iterator"> <test name="is_convertible_fail" category="Corner-case tests"> <mark-failure> <toolset name="borland"/> <toolset name="borland-5_6_4"/> <toolset name="msvc"/> <toolset name="vc7"/> <note refid="2"/> </mark-failure> ... As always, feedback is more than welcome. Enjoy, -- Aleksey Gurtovoy MetaCommunications Engineering
participants (1)
-
Aleksey Gurtovoy