Robert Ramey wrote:
This is something that my test suite should test but doesn't. Regardless of what happens with your portable_binary archive, It would be very helpful to have the test suite enhanced to explicitly test archive portability.
Since we're just talking, So I would like to see the following changes in the test suite:
a) That output of the tests go to a particular directory. There would be a directory for each column in the test matrix. b) There would be a set of directories for each boost release 1.32, 1.33, ... c) Tests would be run on the four combinations which result from: i) set of archives created by the previous version ii) set of archives created by other compilers (portable archives only)
This would result in a HUGE number of tests so it couldn't be run any more than on an occasional basis. But it would be helpful to run very occasionally.
This would require more investigation into boost test in order to find out what facilities are already there for helping out with ths.
Yeah, hmm. It's a lot of work, more tedious than difficult. Since we're just talking, here's some thinking out loud on the subject, for sanity-check purposes: There are a few different classes of tests. The compile-fail ones don't need modification of course. A great many contain the following or similar, one more more times: int test_main( int /* argc */, char* /* argv */[] ) { const char * testfile = boost::archive::tmpnam(NULL); BOOST_REQUIRE(NULL != testfile); BOOST_CHECKPOINT("something"); S s_out = create_some_serializable_data_structure(); { test_ostream os(testfile, TEST_STREAM_FLAGS); test_oarchive oa(os); oa << boost::serialization::make_nvp("S", s_out); } S s_in; { test_istream is(testfile, TEST_STREAM_FLAGS); test_iarchive ia(is); ia >> boost::serialization::make_nvp("S", s_in); } BOOST_CHECK(s_out == s_in); } and one would concievably want to factor these out so that the read-in-and-compare section would happen multiple times, once per boost version and per compiler. For portable archive types, it would happen for all boost versions, compilers *and* platforms. An additional check is that at the end, all portable archive types have the same checksum. tmpnam() would change to, say, persistent_testfile(). Many of the test modules create multiple archives. You could deal with this by keeping track of some index and appending this to the file name. So you would end up with files like $SERIALIZATION_TESTDATA_DIR/ BOOST_PLATFORM/ BOOST_VERSION/ BOOST_COMPILER/ __FILE__.INDEX.ARCHIVETYPE for instance (testdata)/BeOS/103300/Gnu_C++_version_4.0.1/test_something.cpp.2.xml which would be the third file written by tests inside test_something.cpp.
d) An alternative to the above might be just to make some specific tests for backward compatibility and for cross compiler compatibility. This is probably a better idea as it would result in tests that could be run more frequently.
I would think that you would in fact want to run the full battery of tests periodically. Or maybe constantly, on a couple machines dedicated to the purpose, but I agree that the average Joe who checks something out of CVS and runs the tests shouldn't be expected to sit through the whole lot. All that looks doable, to me. I wonder about storage. If you're really going to keep all that data around, the question is where, and how are you going to get at it. One could easily put together a little web interface, where the test suites use a little python script to post and retrieve test data. Let me see what I can come up with. -t