"Beman Dawes" <bdawes@acm.org> wrote in message news:f47qsj$m3j$1@sea.gmane.org...
Thomas Witt wrote:
... The proposal seems to assume infinite resources in testing.
I must not have been clear. The proposal would do much less testing than is currently done. Under the current scheme, testing is done even when no one requests a test for a particular library and compiler. Under the proposal, tests only get run when someone requests a test. So for example, a Windows developer would only likely request tests on non-Windows platforms.
1. I don't think it's correct expectation. Even if I run the test locally on windows box, I still a) don't have access to all configurations b) want to run the tests in non-development environment. My local setup can be somehow different. I always prefer to run the test though regression test suite even if it passes locally. 2. The procedure for on-demand test request doesn't seem to be easy to implement and will require significant investment in both development and maintenance of the tools. 3. In many (most?) cases I don't have an idea what configurations exists. We can't force developers to send dozens of test requests for system they know nothing about. IMO regular "shotgun" testing is still going to be in big demand and we can't afford not to support it. In general I am not sure that investment in on-demand testing support will give us visible improvement. Gennadiy