[Boost-users] "Re: [boost] Boost-1.62: How to get list of all dirs under 'run-tests libs:' target from status/Jamfile?"

This is nice idea, but unfortunately does not work for my purpose of calculatingpass rates for every test suite of regression testing. It would fit perfectly if instead of ---dumptest b2 will also have another option, something like: --dumptestdirs. I mean, the option which generates the whole list of dirs containing all jamfiles executed for full regression testing. Actually, I spent some time trying to generate such list of dirs based on the output produced by 'b2 -n -a -d0 --dump-tests', but was not able to find general algorithm [ wout using specific library names and whout parsing the content of jamfiles ] which transforms list of tests into the list of test dirs from which they are launched [ assuming that each test should be executed only once ]. Here just couple of examples which demonstarte those difficulties: 1. In 'wave' its 'test' subdir does not contain jamfile like many other libsdo, but rather wave's jamfile is located inside internal 'build' subdirectory, and it launches testslocated from 'testlexers' 'testwave' subdirs which are located on the same level as 'build' itself. When my script sees those wave tests from --dumptest output, how should it figure out that jamfile which launches them is located inside 'build' dir? 2. Inside 'unordered' library there are two subdirs with jamfiles: test/exception and test/unordered. All tests for 'unordered' may be run in two similar ways: either by executing both those jamfiles, or by executing jamfile which is located on higher level inside 'test' subdir. But to run every test once the script should choose one or another way. Now, in a similar situation for 'core' library, in order to run all its tests one time my script should run both jamfiles: internal one located inside core/test, and also another one located in core/test/swap. How the script should resolve such ambiguity for 'unordered' and core' libraries? So, I think that list of test dirs which was present in status/Jamfile.v2 in boost-1.61.0 was really useful to me. But, of course if it was created manually, and there is no easy way to do it automatically, then probably, I should also create it manually for Boost-1.62 and all later versions. Sergey

2016-09-04 11:45:41 Rene Rivera (/grafikrobot_at_[hidden]/)

wrote:

You can get a list of all the tests that get, or will get, run from b2 with the following:

% cd boost-root/status % b2 -n -a -d0 --dump-tests

That will produce, interspersed with other output, lines like the following:

boost-test(RUN) "accumulators/count" : "libs/accumulators/test/count.cpp"

One of those for each test that is tested. The format is:

boost-test(<TYPE>) "<LIBRARY>/<TEST>" : "<SOURCE>"

Where LIBRARY can be of the form "<library>" or "<library>/<sublibrary>". That should be sufficient to produce the results you want. And if you use just the above list of tests it's likely to be more accurate than your previous method. As the tests runs from the "boost-root/status" dir are not alway the same as those run from the corresponding library dirs.

-- -- Rene Rivera

...

On Sat, Sep 3, 2016 at 5:09 PM, Sergey Sprogis wrote:

...

I use latest Boost versions to test new C++ compiler in development stage.

It's important for me to know what would be "pass rate" for every library which Boost.org reports on its Regression Testing website. To calculate all those pass rates I created script which goes inside every library's 'test', 'example' or whatever subdirectory and launches bjam from there. It creates a separate logfile(s) for every library which I then analyze to calculate "pass rate".

But before launching my script I need to know the whole list of dirs which Boost.org uses for its Regression testing. In the previous Boost-1.61.0 version I was able to easily produce such list by extracting it from 'status/Jamfile.v2'. It looked like this:

run-tests libs : accumulators/test # test-suite accumulators algorithm/test # test-suite algorithm algorithm/minmax/test # test-suite algorith/minmax ... concept_check # test-suite concept_check config/test # test-suite config container/bench # test-suite container benchmarks container/example # test-suite container_example .....

As you may see this list contains not only 'test' subdirs, but also some others: 'example', 'bench', etc. But now the problem is that in Boost-1.62.0 'status/Jamfile.v2' does not have such full list anymore.

Could anyone suggest me some hack how to produce the full list of tested libraries for Boost-1.62.0, similar to one which Boost-1.61.0 has?

It would be nice if bjam will eventually have a special option which will allow to produce such list, but for now I'd appreciate just a hack which will help me to solve this issue.