AMDG On 06/29/2016 04:34 AM, Adam Wulkiewicz wrote:
Hi,
At Boost.Geometry we're using CircleCI for testing. They allow to run the tests concurently on N virtual machines. I'd like to alter my script to automatically redistribute the tests evenly across all of the machines, i.e. to run 1/N of the tests on each machine. The problem is that at Boost.Geometry we have a lot of nested project directories, i.e. b2 executed for some upper directory runs the tests in the test-suite in this directory and then follows nested projects defined by build-project directives, so runs the tests recursively. So e.g. if b2 was executed on some virtual machine for the most-top-level directory all of the tests would be built anyway.
Do you have an idea how this could be solved without altering the tests structure?
Is it currently somehow possible (though AFAIK it isn't) or would be possible to implement options allowing to e.g.: - run b2 only for a certain directory/Jamfile, e.g. --non-recursive
Boost.Build only builds subdirectories when you explicitly tell it to.
- ignore arbitrary Jamfile directive, e.g. --ignore-build-project
if ! ( --ignore-build-project in [ modules.peek : ARGV ] ) { build-project subdir1 ; }
- run only 1 per N tests, an option used together with -jX but running tests only for one thread, e.g. b2 -i2 -j4 running only 1 per 4 commands starting from 2nd command ?
Making something like this work would be really complex, for a number of reasons: - The order in which targets are updated is non-deterministic for parallel builds. - Simply partitioning targets won't work without a shared filesystem, because some targets depend on others. The partitioning needs to follow the dependency graph and some targets (such as libraries) may need to be included in more than one subset. - The partition algorithm cannot take into account how long each test takes to run, so it may still be suboptimal.
The last option would be the most convenient for me. The tests would be divided as evenly as possible. It wouldn't require to use some workarounds i.e. traversing test directories recursively and running b2 manually for them which could result in low-quality redistribution, e.g. if some directory contained many tests and other small number of tests.
It's actually possible to do something like this programatically at the virtual target layer. Basically, you can write a custom target that constructs a list of test targets and then filters the ones that you want. .group = [ MATCH --test-group=(.*) : [ modules.peek : ARGV ] ] ; rule filter-targets ( project name : property-set : sources * ) { # sources should be a list of test targets # return a subset of $(sources) } generate all-tests : # list of tests goes here : <generating-rule>@filter-targets ; Note: You can include test-suites and projects in the list of tests passed to generate. (If you use a project, just be careful to avoid recursion.) filter-targets will be called with an expanded list of the individual test-cases. In Christ, Steven Watanabe