Re: [Boost-users] Library Interface Design

18 Sep 2006

      Scott Meyers wrote:
...
Rush Manbert wrote:
...
Needless to say, this required a fairly extensive testing API, plus a 
notion within the objects that implemented the service that it could be 
running in "test mode".
Was the ability to put an object into test mode part of the 
client-visible API, or did you somehow have a testing API that clients 
could not access?
In this case, the only way to interact with the service was through a 
message interface. In essence, the service was passed a message object. 
It was also an embedded system, so we were not only the service 
provider, but we also wrote the client. (I realize that we have strayed 
from the original question of library design. I'm happy to take this off 
list if anyone is offended.) The messages that could put the object in 
test mode were part of the client visible API.

Additionally, the service always operated in test mode when it was built 
as part of the Windows executable, because then there was no choice but 
to simulate the events that were normally generated by the lower level 
code. This is why the management UI (our client) always thought that a 
copy process was real. It couldn't tell the difference between the real 
thing and the test mode behavior.
...
...
In fact, our system shipped with the test subsystem included. It was not 
readily accessible, of course, but was really useful in some cases where 
we needed to test something on an installed system. This sort of 
capability in the field can be a real saving grace in an embedded system.
This makes it sound like the testing API was in fact visible to clients, 
but, by convention, it was never used for non-test apps.  Is this 
correct?  If so, that's different from, for example, test-only APIs that 
  exist only for debug builds, i.e., a truly separate API that non-test 
clients can't get at.  (I'm not arguing that such an approach is better 
than what you did, I'm simply trying to understand what you did.)
To clarify my interest, I recently sent this to somebody who send me 
private mail on this thread:
...
My interest here is not in testing, it's in good design, and good designs facilitate testing.  Which means we need to be able to describe how testability affects other design desiderata, such as compile-time error detection, encapsulation, and overly general interfaces.  There is a ton of recent literature on testing and testability, but virtually none of it addresses how making something more testable may be in tension with other characteristics we'd like.  This thread in Boost is part of my attempt to figure out how the various pieces of the puzzle fit together -- or if they do at all.
I'm not sure how applicable this is to design of a library such as 
Boost, but my experience has been that having test interfaces available 
at the subsystem level in the release code version is a very useful 
thing, even if they are visible to clients. You need to be careful with 
this, and you really need to protect clients from getting into test mode 
accidentally, but when you are debugging a large, complex system that 
may have very limited debugging capabilities (I spent 25 years 
developing embedded systems, so that's where this viewpoint comes from) 
it can be very very useful to be able to isolate your subsystems. You 
can usually only do that if they have been designed so that they can be 
tested in isolation.

I also believe that you often need these sorts of interfaces so that you 
can force error conditions, especially in a heavily layered system. This 
lets me test that my subsystem handles errors correctly, but it also 
allows me to test the error propagation paths out of my subsystem and 
into the layers above it.

I made a little Xcode project that illustrates one way you could 
approach making objects that have a test mode. I have attached the code 
and header files to this email. In this case, MyObject has two 
constructors, each of which takes an initialization object as an 
argument. One of them takes a "normal" initializer object, while the 
other takes a "test" initializer. The object is in test mode if you 
construct it with the second form. There is also a public method that 
can force a test mode behavior, but only for a test mode object.

So my test API is visible to clients. However, if I don't distribute 
MyObjectInitializerForTest.h, then clients cannot construct a test 
initializer object, and therefore can't put the object in test mode. Of 
course, they can see how MyObjectInitializer was declared, so they could 
figure out how to declare and define a MyObjectInitializerForTest 
object, but it seems to me that the barrier is sufficiently high that 
there won't be much of that going on.

I took this approach in order to mimic the original case that I 
described. In that case, the initializer objects were the "normal" or 
"test" messages. I know that there are slicker ways to do this sort of 
thing, but this illustrates the basic idea.

- Rush

/*
 *  MyObjectInitializerForTest.h
 *
 */
#ifndef MyObjectInitializerForTest_H
#define MyObjectInitializerForTest_H

#include "MyObjectInitializerBase.h"

class MyObjectInitializerForTest: public MyObjectInitializerBase
{
public:
	MyObjectInitializerForTest (int a, bool b) : MyObjectInitializerBase (a,b) {};
	~MyObjectInitializerForTest (void) {};
};

#endif //MyObjectInitializerForTest_H

/*
 *  MyObject.cpp
 *
 */
#include <iostream>
#include "myObject.h"

#include "myObjectInitializerForTest.h"

MyObject::MyObject (MyObjectInitializer const initializer)
: m_testMode	(false)
, m_a			(initializer.m_a)
, m_b			(initializer.m_b)
{
	testingMemberDataInit ();
}

MyObject::MyObject (MyObjectInitializerForTest const initializer)
: m_testMode	(true)
, m_a			(initializer.m_a)
, m_b			(initializer.m_b)
{
	testingMemberDataInit ();
}

MyObject::~MyObject (void)
{
	return;
}

bool
MyObject::methodWithTestModeBehavior (void)
{
	if (m_testMode)
	{
		std::cout << "MyObject::methodWithTestModeBehavior: test mode is enabled!";
		if (m_doForceResultOfCallToMethodWithTestModeBehavior)
		{	// The return value for this call is forced this time only
		std::cout << " Forced return value: " << (m_forcedResultOfCallToMethodWithTestModeBehavior ? "true" : "false") << "\n";
			m_doForceResultOfCallToMethodWithTestModeBehavior = false;
			return m_forcedResultOfCallToMethodWithTestModeBehavior;
		}
		std::cout << "\n";
	}
	else
	{
		std::cout << "MyObject::methodWithTestModeBehavior: test mode is DISABLED.\n";
	}
	// Normal return here - whatever m_b contains
	return m_b;
}

void
MyObject::forTestingSetResultOfNextCallToMethodWithTestBehavior (bool result)
{
	if (m_testMode)
	{
		m_doForceResultOfCallToMethodWithTestModeBehavior	= true;
		m_forcedResultOfCallToMethodWithTestModeBehavior	= result;
	}
	else
	{
		std::cout << "MyObject::forTestingSetResultOfNextCallToMethodWithTestBehavior: Ignored in normal mode!\n";
	}
}

void
MyObject::testingMemberDataInit (void)
{
	m_doForceResultOfCallToMethodWithTestModeBehavior	= false;
	m_forcedResultOfCallToMethodWithTestModeBehavior	= false;
}

#include <iostream>

#include "MyObject.h"
#include "MyObjectInitializerForTest.h"

int main (int argc, char * const argv[]) {
    // insert code here...
    std::cout << "Hello, World!\n\n";

	MyObject normalModeObj (MyObjectInitializer::MyObjectInitializer (1,true));
	MyObject testModeObject (MyObjectInitializerForTest::MyObjectInitializerForTest (2, false));

	bool result;
	result = normalModeObj.methodWithTestModeBehavior();
	std::cout << "Call returned: " << (result ? "true" : "false") << "\n\n";

	result = testModeObject.methodWithTestModeBehavior();
	std::cout << "Call returned: " << (result ? "true" : "false") << "\n\n";

	// Try to force next return value on normalModeObject (fails)
	normalModeObj.forTestingSetResultOfNextCallToMethodWithTestBehavior(false);
	result = normalModeObj.methodWithTestModeBehavior();
	std::cout << "Call returned: " << (result ? "true" : "false") << "\n\n";

	// Force retuurn value on testModeObject for the next call to methodWithTestModeBehavior()
	testModeObject.forTestingSetResultOfNextCallToMethodWithTestBehavior(true);
	result = testModeObject.methodWithTestModeBehavior();
	std::cout << "Call returned: " << (result ? "true" : "false") << "\n\n";

	result = testModeObject.methodWithTestModeBehavior();
	std::cout << "Call returned: " << (result ? "true" : "false") << "\n\n";

    return 0;
}

/*
 *  MyObjectInitializerBase.h
 *
 */
#ifndef MyObjectInitializerBase_H
#define MyObjectInitializerBase_H

class MyObjectInitializerBase
{
public:
	MyObjectInitializerBase (int a, bool b) : m_a (a), m_b(b) {};
	~MyObjectInitializerBase (void) {};

	int m_a;
	int m_b;
};

#endif //MyObjectInitializerBase_H

/*
 *  MyObject.h
 *
 */
 #ifndef MyObject_H
 #define MyObject_H

 #include "MyObjectInitializerBase.h"

class MyObjectInitializer: public MyObjectInitializerBase
{
public:
	MyObjectInitializer (int a, bool b) : MyObjectInitializerBase (a,b) {};
	~MyObjectInitializer (void) {};
};

class MyObjectInitializerForTest;

class MyObject
{
public:
	// Normal constructor
	MyObject (MyObjectInitializer const initializer);
	// Test Mode constructor
	MyObject (MyObjectInitializerForTest const initializer);
	~MyObject (void);

	bool methodWithTestModeBehavior (void);

		// Testing API
	void forTestingSetResultOfNextCallToMethodWithTestBehavior (bool result);
private:
	void testingMemberDataInit (void);

	bool m_testMode;
	int m_a;
	bool m_b;

	// Member data used to control testing
	bool m_doForceResultOfCallToMethodWithTestModeBehavior;
	bool m_forcedResultOfCallToMethodWithTestModeBehavior;
};

#endif //MyObject_H