
After having run two cycles of tests for Boost with the mingw-3_4_2-stlport-5_0 configuration and having it take more than 14 hours on a 2.2Ghz+1GB machine, most of that in the Boost.Serialization library[*]. And reading some of the recent discussion about the desire to expand testing to include cross-version compatibility and cross-compiler compatibility, and hence having the number of tests multiply possibly exponentially. I am seriously concerned that we are going in the wrong direction when it comes to structuring tests. From looking at the tests for serialization I think we are over-testing, and we are past the point of exhausting testing resources. Currently this library takes the approach of carpet bombing the testing space. The current tests follow this overall structure: [feature tests] x [archive types] x [char/wchar] x [DLL/not-DLL] Obviously this will never scale. My first observation is that it doesn't seem that those axis look like independent features to me. That is, for example, the char/wchar functionality doesn't depend on the feature getting tested, or at least it shouldn't. And I can't imagine the library is structure internally in that way. To me it doesn't make sense to test "array" saving with each of the 3 archive types since the code for serialization of the "array" is the same in all situations. Hence it would makes more sense to me to structure the tests as: [feature test] x [xml archive type] x [char] x [not-DLL] [text archive tests] x [char] x [non-DLL] [binary archive tests] x [non-DLL] [wchar tests] x [non-DLL] [DLL tests] Basically it's structured to test specific aspects of the library not to test each aspect against each other aspect. Some benefits as I see them: * Reduced number of tests means faster turn around on testing. * It's much easier to add tests for other aspects as one only has to concentrate on a few tests instead of many likely unrelated aspects. * The tests can be expanded to test the aspects more critically. For example the DLL tests can be very specific as to what aspect of DLL vs non-DLL they test. * It is easier to tell what parts of the library are breaking when the tests are specific. [*] That's just a CPU time observation, not to mention that it takes 2.15GB of disk space out of a total of 5.09GB. And it's only one compiler variation. -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - Grafik/jabber.org