Sent to the users list by mistake. Begin forwarded message:
From: "Belcourt, Kenneth" <kbelco@sandia.gov> Date: May 14, 2010 11:03:23 AM MDT To: "boost-users@lists.boost.org" <boost-users@lists.boost.org> Subject: [MPI] Test Processor Counts
Hi,
While running the trunk tests, we've found that occasionally the system load average jumps up over 100 and occasionally hits 200. As you can imagine for 8 to 32 core systems, this adversely impacts users on those systems.
The issue is that I'm running the trunk tests in parallel, usually -j8 to -j32 and when the MPI tests run, we run into the high load problem. Some of the MPI tests run up to 17 processors so the combination of parallel testing and large processor count tests cause this issue. Now the high load average may last for only 5 to 15 minutes or so but this is enough to trigger a sysadmin question asking what I'm up.
I'm looking for a solution that wouldn't require significantly reducing testing throughput. Possible solutions might include serializing the execution of the MPI tests using something like the link-semaphore in Boost.Build, reducing the processor counts of the MPI tests, only running the larger tests on unchecked or unknown MPI implementations, setting up an MPI machine file to run the MPI tests off of the local system that built the executable, etc...
Any other ideas on how to manage this?
Thanks.
-- Noel