Rush Manbert wrote:
Needless to say, this required a fairly extensive testing API, plus a notion within the objects that implemented the service that it could be running in "test mode".
Was the ability to put an object into test mode part of the client-visible API, or did you somehow have a testing API that clients could not access?
In fact, our system shipped with the test subsystem included. It was not readily accessible, of course, but was really useful in some cases where we needed to test something on an installed system. This sort of capability in the field can be a real saving grace in an embedded system.
This makes it sound like the testing API was in fact visible to clients, but, by convention, it was never used for non-test apps. Is this correct? If so, that's different from, for example, test-only APIs that exist only for debug builds, i.e., a truly separate API that non-test clients can't get at. (I'm not arguing that such an approach is better than what you did, I'm simply trying to understand what you did.) To clarify my interest, I recently sent this to somebody who send me private mail on this thread:
My interest here is not in testing, it's in good design, and good designs facilitate testing. Which means we need to be able to describe how testability affects other design desiderata, such as compile-time error detection, encapsulation, and overly general interfaces. There is a ton of recent literature on testing and testability, but virtually none of it addresses how making something more testable may be in tension with other characteristics we'd like. This thread in Boost is part of my attempt to figure out how the various pieces of the puzzle fit together -- or if they do at all.
Scott