Beman Dawes wrote:
On Wed, Jun 2, 2010 at 1:41 PM, Belcourt, Kenneth <kbelco@sandia.gov>
First of all, let me apologize for any inconvenience this may have caused.
bjam shouldn't stop entirely because a file isn't found. At worst, that library's tests should fail.
Can Boost.Build or bjam be modified to be more tolerant of errors during regression testing?
This is the perfect example of why I think trunk testing should be changed. Here is what I propose. For each library: - e.g. serialization a) Start with a machine which two separate release and trunk directory trees. Assume that these are named /boost_release and boost_trunk respectively. b) invoke the following script: mv boost_release/serialization boost_x/serialization mv boost_trunk/serialization boost_release/serialization mv boost_release/archive boost_x/archive mv boost_trunk/archive boost_release/archive mv boost_release/libs/serialization boost_x/libs/serialization mv boost_trunk//libs/serialization boost_release/libs/serialization We now have a boost_root tree which is the latest release but has the trunk for the one library being tested. c) run the test cd boost_release/libs/serialization/test bjam .... # I run ../../../tools/regression/src/library_test.sh ...which is equivalate to the above # but creates a report in the test directory c) run a script which reversed the changes made in a) above. The testing of all of boost would be defined as nothing more than the same procedure applied to each library. What would this give is? a) Problems such as the current one would be be limited only to the offending library. b) bugs in one library wouldn't show up as test failures in another library. c) Testing would be against the "next release". This would dimish surprises that might occur when libraries are merged into the release branch. d) Testing would be MUCH faster. Currently, if a header in library A changes in the trunk - then ALL the libraries which depend on A are rebuilt and tested. This costs a huge amount of time for almost no benefit. Under the new system, only the tests for library A would be re-run if library A changes. Tests for all other libraries which depend upon A would occur when library A is merged into the release branch. e) It takes a baby step in the direction of making boost libraries independently deployable. f) Certain libraries and tools such as bjam and boost.test are used in the testing process. When the developer of one of these tools checks in something that turns out to be an error - then it creates bogus test results when can hang around for weeks until they are cleared up. How can these developers work in such an environment? Finally, what would it take to implement this? Only a change in the test script. All current boost tools - bjam, boost test, etc would remain identical. I don't know how the test matrix generation works - but I suspect that it would change little if at all (I'm assuming it reviews the bin.v2 directory which would be the same under this proposal). Who sees anything wrong with this? Better yet, who can see any advantage in the current setup? Robert Ramey