explicite failures question
I'm struggling with the explicit-failures XML. About 40% of the serialization tests will fail on platforms which do not include support from wchar_t streams. As Understand the current schema (XML is not all that clear to me) I would have to specify one line for each pair of toolset-test that fails on this scenario. I my case this means 100 test * 6? Toolsets = ? 600 lines of xml added to this file. Robert Ramey
Robert Ramey wrote:
I'm struggling with the explicit-failures XML.
About 40% of the serialization tests will fail on platforms which do not include support from wchar_t streams. As Understand the current schema (XML is not all that clear to me) I would have to specify one line for each pair of toolset-test that fails on this scenario. I my case this means 100 test * 6? Toolsets = ? 600 lines of xml added to this file.
I'd like to second that question. On some toolsets, all *dll tests for program_options fail. To mark those failures as expected I need one entry for each test, and that's overly verbose. I'd much prefer something like <mark-expected-failures> <test name="cmdline_test_dll"/> <test name="parsers_test_dll"/> <toolset name="cw8-3"/> <toolset name="comeau"/> </mark-expected-failures> - Volodya
Vladimir Prus writes:
I'd much prefer something like
<mark-expected-failures> <test name="cmdline_test_dll"/> <test name="parsers_test_dll"/> <toolset name="cw8-3"/> <toolset name="comeau"/> </mark-expected-failures>
Implemented in CVS now. See MPL markup section in "status/explicit-failures-markup.xml" for specific examples. -- Aleksey Gurtovoy MetaCommunications Engineering
Aleksey Gurtovoy wrote:
Vladimir Prus writes:
I'd much prefer something like
<mark-expected-failures> <test name="cmdline_test_dll"/> <test name="parsers_test_dll"/> <toolset name="cw8-3"/> <toolset name="comeau"/> </mark-expected-failures>
Implemented in CVS now. See MPL markup section in "status/explicit-failures-markup.xml" for specific examples.
Thanks. However, I've noticed that all 'mark-expected-failures' elements must appear after all 'test' elements, otherwise http://apps.gotdotnet.com/xmltools/xsdvalidator/ will report validation error. Am I right that xs:sequence requires specific order of child elements? Is there any way around it? - Volodya
Vladimir Prus writes:
Aleksey Gurtovoy wrote:
Vladimir Prus writes:
I'd much prefer something like
<mark-expected-failures> <test name="cmdline_test_dll"/> <test name="parsers_test_dll"/> <toolset name="cw8-3"/> <toolset name="comeau"/> </mark-expected-failures>
Implemented in CVS now. See MPL markup section in "status/explicit-failures-markup.xml" for specific examples.
Thanks. However, I've noticed that all 'mark-expected-failures' elements must appear after all 'test' elements, otherwise http://apps.gotdotnet.com/xmltools/xsdvalidator/ will report validation error.
Am I right that xs:sequence requires specific order of child elements?
Yes.
Is there any way around it?
Sure: <xs:sequence> <xs:element ref="mark-unusable" minOccurs="0" maxOccurs="unbounded"/> <xs:sequence> <xs:choice minOccurs="0" maxOccurs="unbounded"> <xs:element ref="test"/> <xs:element ref="mark-expected-failures"/> </xs:choice> </xs:sequence> </xs:sequence> Fixed in the CVS. -- Aleksey Gurtovoy MetaCommunications Engineering
Aleksey Gurtovoy wrote:
Thanks. However, I've noticed that all 'mark-expected-failures' elements must appear after all 'test' elements, otherwise http://apps.gotdotnet.com/xmltools/xsdvalidator/ will report validation error.
Am I right that xs:sequence requires specific order of child elements?
Yes.
Is there any way around it?
Sure:
Thanks, there's another problem, though. The notes are now duplicated, see for example http://tinyurl.com/4cum8 - Volodya
Vladimir Prus writes:
Aleksey Gurtovoy wrote:
Thanks. However, I've noticed that all 'mark-expected-failures' elements must appear after all 'test' elements, otherwise http://apps.gotdotnet.com/xmltools/xsdvalidator/ will report validation error.
Am I right that xs:sequence requires specific order of child elements?
Yes.
Is there any way around it?
Sure:
Thanks, there's another problem, though. The notes are now duplicated, see for example
Fixed now. -- Aleksey Gurtovoy MetaCommunications Engineering
Robert Ramey writes:
I'm struggling with the explicit-failures XML.
About 40% of the serialization tests will fail on platforms which do not include support from wchar_t streams. As Understand the current schema (XML is not all that clear to me) I would have to specify one line for each pair of toolset-test that fails on this scenario. I my case this means 100 test * 6? Toolsets = ? 600 lines of xml added to this file.
Robert, we'll give you a more sane way to deal with this in a couple of days. -- Aleksey Gurtovoy MetaCommunications Engineering
Robert Ramey writes:
I'm struggling with the explicit-failures XML.
About 40% of the serialization tests will fail on platforms which do not include support from wchar_t streams. As Understand the current schema (XML is not all that clear to me) I would have to specify one line for each pair of toolset-test that fails on this scenario. I my case this means 100 test * 6? Toolsets = ? 600 lines of xml added to this file.
From this moment on, the regression tools implement support for a limited form of regular expressions in test case names, namely:
<test name="sometestname"> // exact match <test name="*"> // anything <test name="*sometestname*"> // contains "sometestname" <test name="sometestname*"> // starts with "sometestname" <test name="*sometestname"> // ends with "sometestname" Given this, your use case can be covered by something as simple as this: <test name="*_warchive"> <mark-failure> <toolset name="mingw-3.4.1"/> .... </mark-failure> HTH, -- Aleksey Gurtovoy MetaCommunications Engineering
participants (3)
-
Aleksey Gurtovoy -
Robert Ramey -
Vladimir Prus