on Fri Aug 17 2007, Stefan Seefeld <seefeld-AT-sympatico.ca> wrote:
Would QMTest be used to drive multi-host testing across the internet (i.e. at different testers' sites), or more likely just within local networks? If the former, how do its facilities for that compare with BuildBot?
QMTest would typically be used to drive individual 'test runs', presumably only over local networks,
Why presumably? Is there a limitation that prevents it from going out to the web?
and can then be used during the aggregation of the results of such test runs into test reports.
As such, it is complementary to the facilities offered by buildbot.
Can you explain why it makes sense to use two systems?
Another important point is scalability: While some test suites are simple and small, we also deal with test suites that hold many thousands of tests (QMTest is used for some of the GCC test suites, for example). A test can mean to run a single (local) executable, or require a compilation, an upload of the resulting executable to a target board
Target board?
Yes (please note that 'target' here is not the same term used above). In the context here it refers to cross-compilation and cross-testing.
But what is it?
How could this be useful for boost ?
A good question, but I'm more interested in "how Boost might use it." That is, something like, "We'd set up a server with a test database. QMTest would run on the server and drive testing on each testers' machines, ..." etc.
Still looking for that.
I believe the hardest part is the connection between QMTest and boost.build. Since boost.build doesn't provide the level of introspection QMTest promises, a custom 'boost.build test database' implementation needs some special hooks from the build system. I discussed that quite a bit with Vladimir.
And what came of it?
I'm not sure. boost.build would need to be extended to allow QMTest to gain access to the database structure (the database already exists, conceptually, in terms of the directory layout...). Volodya ?
There's no a priori reason that Boost.Build needs to maintain the test database, is there? -- Dave Abrahams Boost Consulting http://www.boost-consulting.com The Astoria Seminar ==> http://www.astoriaseminar.com