On Monday 19 January 2015 23:03:37 Ben Pope wrote:

On Monday, January 19, 2015 10:47 PM, Andrey Semashev wrote:

...

AFAIK, there is no formal property of a toolset that would differentiate testers between C++ versions. You basically rely on toolset names. Unfortunately, some testers reflect C++ version in the tester name rather than the toolset, and I don't think there's a way to use the tester name in the markup table. I would kindly ask those testers to change their settings.

If you just want to see the tester description you can read it through the link which is the tester name in the matrix. Some testers provide more detailed info there.

Should there be a C++ version in the list of build features? Would that help?

http://www.boost.org/build/doc/html/bbv2/overview/builtins/features.html

I'm more than happy to change anything with my configuration, although I think enough details are there, if any more are needed, or the formatting can be improved, feel free to make suggestions.

Boost.Build feature for C++ version would be nice, but I don't think this would help the markup. If you look at the markup file[1], you'll notice that one can only identify toolsets by their names, not even by tester names. So unless format of that file changes, it's preferable if the toolset name reflects its configuration. [1] https://github.com/boostorg/boost/blob/master/status/explicit-failures-marku...

Robert Ramey wrote:

When I run tests on my local system with gcc or clang I use the compile time switch -std=c++03 or -std=c++11 on my bjam command line which gets passed on to the gcc/clang command line.

This isn't very convenient, because you can't run the 03 and 11 tests at the same time (and they go into the same directory). What I use is this (in user-config.jam): import feature ; import toolset ; using gcc ; feature.subfeature toolset gcc : std : cxx0x cxx11 cxx1y cxx14 : optional composite propagated ; feature.compose toolset-gcc:stdcxx0x : <cxxflags>-std=c++0x ; feature.compose toolset-gcc:stdcxx11 : <cxxflags>-std=c++11 ; feature.compose toolset-gcc:stdcxx1y : <cxxflags>-std=c++1y ; feature.compose toolset-gcc:stdcxx14 : <cxxflags>-std=c++14 ; using clang : : : <cxxflags>-Wno-variadic-macros <cxxflags>-Wno-c99-extensions ; feature.subfeature toolset clang : std : cxx0x cxx11 cxx1y cxx14 : optional composite propagated ; feature.compose toolset-clang:stdcxx0x : <cxxflags>-std=c++0x ; feature.compose toolset-clang:stdcxx11 : <cxxflags>-std=c++11 ; feature.compose toolset-clang:stdcxx1y : <cxxflags>-std=c++1y ; feature.compose toolset-clang:stdcxx14 : <cxxflags>-std=c++14 ; This allows me to run b2 toolset=gcc,gcc-cxx11,clang,clang-cxx11 for example. -std=c++03 is, I believe, the default, but you can add it to the above if you like.

On 1/19/2015 11:14 AM, Robert Ramey wrote:

Andrey Semashev-2 wrote

...

AFAIK, there is no formal property of a toolset that would differentiate testers between C++ versions. You basically rely on toolset names.

When I run tests on my local system with gcc or clang I use the compile time switch -std=c++03 or -std=c++11 on my bjam command line which gets passed on to the gcc/clang command line. I started doing this when it occurred to me that I didn't no which version of C++ my library was being tested with. I still don't know what happens if you include no such switch at all.

Many testers are including this information on the test properties page others are either not including the information or are not specifying in which case it's a mystery what version of the compiler is being used.

In my test programs - I use boost config macros to detect and run/not run those parts of the test programs which are relevant to this or that compiler version.

I would like a method to condition an entire test program on the value of a boost config macro. I realize that this is a pain and god forbid I have to make my bjam script even more complex than it is or have to remember some more boost build syntax/rules. So far I've been able to avoid needing this.

Did you not see a previous post from John Maddock explaining how to use a "requires" rule invocation so that you can run a test depending on config macro features ? Does that do what you want ?

On 01/19/2015 09:34 PM, Robert Ramey wrote:

Edward Diener-3 wrote

...

On 1/19/2015 11:14 AM, Robert Ramey wrote:

...

I would like a method to condition an entire test program on the value of a boost config macro. I realize that this is a pain and god forbid I have to make my bjam script even more complex than it is or have to remember some more boost build syntax/rules. So far I've been able to avoid needing this.

Did you not see a previous post from John Maddock explaining how to use a "requires" rule invocation so that you can run a test depending on config macro features ?

No

...

Does that do what you want ?

Maybe.

What I really want is a window into the boost build documention which people can post this extra information to. Sort of like php - documentation or similar to the pages in the boost library incubator (www.blincubator). So that I don't have to spend hours trolling the whole world's internet to find information like this.

Robert, the standard way for projects on github to maintain such dynamic documentation is the wiki, so for Boost.Build, that would be: https://github.com/boostorg/build/wiki that page is empty now, but if anybody has content to add, it's one click. For example of a project that uses wiki for all documentation, see: https://github.com/mbostock/d3/wiki/API-Reference -- Vladimir Prus CodeSourcery / Mentor Embedded http://vladimirprus.com

2015-01-19 23:35 GMT+01:00 Edward Diener :

On 1/19/2015 1:34 PM, Robert Ramey wrote:

...

Edward Diener-3 wrote

...

On 1/19/2015 11:14 AM, Robert Ramey wrote:

...

I would like a method to condition an entire test program on the value of a boost config macro. I realize that this is a pain and god forbid I have to make my bjam script even more complex than it is or have to remember some more boost build syntax/rules. So far I've been able to avoid needing this.

Did you not see a previous post from John Maddock explaining how to use a "requires" rule invocation so that you can run a test depending on config macro features ?

No

Its title is:

"[Config] Announce: New Build time configuration feature"

Ok, got it. I have two questions, though. 1. Is any of the Boost libraries using it currently for regression tests, so that I can observe it in the wild? 2. Is this technique also recommended for my particular case of Optional references? The reason my tests fail on some (many) compilers is because different compilers have different bugs in the implementation of reference binding. Last time I was advised not to add macros into Boost.Config that only cover up for compiler bugs and are applicable to a single library. So I am not sure anymore what is the advice for the case of regression testing of Boost.Optional that is hit by reference binding bugs? Regards, Andrzej

2015-01-19 18:29 GMT+01:00 Rene Rivera :

Consider this my view as testing manager..

On Mon, Jan 19, 2015 at 8:14 AM, Andrzej Krzemienski wrote:

...

Hi Everyone, Could anyone give me some information on how regression test matrix is supposed to work? I can see that when testers run their tests they specify a toolset; this toolset is reported in test matrix, and based on this toolset name failures in tests can be labeled as expected. Did I get it right?

Correct.

...

necessarily reflected in the toolset name (some testers indicate it in

However, the fact that a tester turns C++11 support or not, is not the

...

toolset name, others don't). On the other hand C++11 being turned on or off can affect the test results. In that case the toolset name alone is not sufficient to make a decision whether I expect a failure or not.

Right.

Is there some policy for addressing this problem? Am I the only person to

...

be affected?

My view is that you should not use the toolset name as a basis for markup. In an ideal world the status markup would entirely go away. As it's an inferior mechanism for dealing with expected test variance. Instead please use the config method as previously (and below) mentioned.

I am sorry, I may be thinking slowly, but what config method are you referring to? Regards, &rzej