On Monday, January 19, 2015 10:47 PM, Andrey Semashev wrote:
On Monday 19 January 2015 15:14:21 Andrzej Krzemienski wrote:
Hi Everyone, Could anyone give me some information on how regression test matrix is supposed to work? I can see that when testers run their tests they specify a toolset; this toolset is reported in test matrix, and based on this toolset name failures in tests can be labeled as expected. Did I get it right?
However, the fact that a tester turns C++11 support or not, is not necessarily reflected in the toolset name (some testers indicate it in the toolset name, others don't). On the other hand C++11 being turned on or off can affect the test results. In that case the toolset name alone is not sufficient to make a decision whether I expect a failure or not.
Is there some policy for addressing this problem? Am I the only person to be affected?
AFAIK, there is no formal property of a toolset that would differentiate testers between C++ versions. You basically rely on toolset names. Unfortunately, some testers reflect C++ version in the tester name rather than the toolset, and I don't think there's a way to use the tester name in the markup table. I would kindly ask those testers to change their settings.
If you just want to see the tester description you can read it through the link which is the tester name in the matrix. Some testers provide more detailed info there.
Should there be a C++ version in the list of build features? Would that help? http://www.boost.org/build/doc/html/bbv2/overview/builtins/features.html I'm more than happy to change anything with my configuration, although I think enough details are there, if any more are needed, or the formatting can be improved, feel free to make suggestions. Ben (BP x86_64 C++11) - the name is no longer correct, but changing it means somebody has to delete the old results file)