David Abrahams wrote:
"Robert Ramey" <ramey@rrsd.com> writes:
That was my original point. If I'm not making any changes, which I haven't been, almost no failures have anything to do with the serialization code.
But we want to find out when changes to other Boost libraries break it. And we want to find out when other people break it.
Which was my other point. Using one library(1) as a test on another library(2) isn't an efficient usage of resources. If this is deemed necessary its really an indication that the tests for library (2) have insufficient coverage and this should be fixed. Of course, I see the whole thing being tested before "release" but this would be an special occasion.
Also, you could cut the testing time by 2/3 instantly if you'd just test all the archive types in a single executable, rather than building a separate executable for each one.
Hmmm - that was what I did originally but it was objected to in the first review.
On what grounds?
If I remember correctly. that the tests where very large and complex, and not sufficiently focused.
Also, more compilers failed with due to resource exhaustion.
On some weaker compilers you might have to break a few tests into multiple translation units.
ahh - more work
The current system also lets me run just one archive by adding a command line argument - extremely useful.
I noticed. It's not only useful, but necessary, to cut down on the number of targets the Jamfile generates.
All of that should be possible even with the tests built into a single executable.
at the cost of making the test program more complex. And to make tests run faster in order to generate results that aren't very useful to me anyhow.
The importance, to Boost as a whole, of having continuous testing of all libraries should not be underestimated.
or overestimated. I'm not really objecting to testing - heck my main sin is creating "too many" tests and testing every combination which could possibly fail. (An XP maxim). In my view - other libraries don't have enough tests. Actually, everytime a bug slips through it should occasion adding a new test to the suite. But, constant testing using one library to test other libraries is a poor substitute for not adding more tests in the library that actually has the bug. Another issue is that we're not really testing our libraries - we're testing other compilers and standard libraries with our own code so we can add workarounds, jamfile tweaks, etc. I understand that this has worked well in the past but it isn't scaling well. I'm going to guess that as the "release" system evolves along lines being discussed, its going to percipitate a parallel evolution in the test system as well - we'll see. Robert Ramey