Jeff Garland writes:
On Fri, 21 May 2004 20:41:59 -0400, David Abrahams wrote
"Robert Ramey" <ramey@rrsd.com> writes:
Dave Abrahams wrote: I believe we will have to move to an "on demand" model for most testing while reserving "total coverage" testing for just prior to release.
I don't. You can get test results for any library on any compiler that's being tested daily within 24 hours. Some compilers are tested every 12 hours (see meta-comm). I don't see why that should be insufficient.
It's kind of spotty outside of the meta-comm guys: IBM Aix 11 days Mac OS today SGI Irix 2 weeks linux 4 days Sun Solaris 6 days Win32 4 weeks win32_metacomm today
And that's today.
IMO the only thing it indicates is that these tests are initiated manually.
Consider during the next couple months 3-4 new libraries are pending to be added.
No a problem, in general. Right now a *full rebuild* takes about 8 hours. If we switch to incremental model, we have plenty of reserve, here.
Serialization tests alone dramatically increase the length of the time to run the regression if we always run the full test.
Well, the dramatic cases need to be dealt with, and IMO a Jamfile that allows the library author to manage the level of "stressfulness" would be just enough.
What will happen in a year when we have say 10 new libraries?
Well, hopefully we'll also have more computing power. Surely a lot of organizations which use Boost libraries can afford to spare a middle-class machine for automatic testing?
Robert and I have believe something will need to be done. We've tried to start a discussion, but no one responded:
http://lists.boost.org/MailArchives/boost/msg64471.php http://lists.boost.org/MailArchives/boost/msg64491.php
Jeff
BTW I might be able to contribute to the Linux testing -- are there instructions on how to set this up somewhere?
For *nix systems, there is a shell script that is pretty much self-explanatory: http://cvs.sourceforge.net/viewcvs.py/boost/boost/tools/regression/run_tests... If you want something that requires even less maintenance, we can provide you with the Python-based regression system we use here at Meta. -- Aleksey Gurtovoy MetaCommunications Engineering