Hi Gennadiy, 2009/6/11 Gennadiy Rozental <rogeeff@gmail.com>
Can you please give specific usage examples you have in mind.
Mocking with the least amount of boilerplate code with the most clear error messages and the most usability. In general, applications consist of piles of classes with interactions, tied together with design patterns and architectural patterns and constraints. In the context of unit testing these classes are tied together too much so that testing them involves testing the underlying classes. If they are fragile (due to their nature or implementation) the upper tests are fragile as well. A project I recently joined at work has about a thousand test cases, about 200 of which break if something in the lower layers falls over. Tracing to the problem becomes hard at the minimal. Mocking out the bottom end classes resolves this as only the interface of the dependencies is tested, not the implementation. Creating tests with mocks using macros is tedious, using external programs is limited. Using this method they're versatile and quick. I must admit I'm a slight bit confused as to what kind of answer you're expecting.
It does this by creating an object that is "derived" from a given class at
runtime, and replacing the functions with functions that redirect to
What if I want to mock some concepts instead of interfaces?
I haven't spent any time on mocking concepts so far but I suspect it to be possible. The last time I used ConceptGCC it had a start-up time of 30 seconds and I haven't checked since. That was over a year and a half ago, so I suspect it's evolved since then. I need to check with it to see how to make it work. Can you show an example of how much effort is required mock something up? class IFoo { public: virtual int getLength(std::string what) = 0; }; int test_function() { MockRepository mocks; IFoo *foo = mocks.InterfaceMock<IFoo>(); mocks.ExpectCall(foo, IFoo::getLength).With("Hello World").Return(42); std::cout << foo->getLength("Hello World") << std::endl; } This is close to the lower limit for using a mock at all. The first line of the test can be put into the testing framework making the Boost.Test does have some support for interaction based testing already.
Including class mock_object <boost/test/mock_object.hpp>. That said I'd be happy to offload this part and support your efforts. I'd like to know though:
I've spent a bit of time searching the Boost mailing list and the Boost.Test docs beforehand to see if something like this was already implemented, but I failed to see it. I think it's completely absent in the docs. As far as I can tell, mock_object tests afterwards. I must admit that in the hour I took to look at it, I suspect I haven't quite figured out what it does. The three test cases I found that uses it (one which logs output and two that test exception safety) didn't seem to adequately explain the complexity of the code.
1. How does your solution compares with what I have in this header?
I think it is somewhat comparable, although less desirable for a developer of end-code. For testing within Boost it may well have an upper hand, being more general. Most people, however, do not develop Boost. Your mock_object requires inheriting from it and implementing all the functions using a default implementation, which increases the amount of code and reduces maintainability for the test code. It does work in the case of multiple inheritance and is easier to port to other compilers and platforms. My solution requires setting up expectations beforehand and it checks them with the fidelity that you choose. It requires you to put in your test what lower-level functions you expect it to call and what ordering relations between them you expect. It makes the actual creation of mock object classes implicit - nobody ever creates a full class that a compiler sees. This significantly reduces the chance of typos. It does require you to specify all functions that will be tested because otherwise they'll have no implementation. There are a few drawbacks, mainly in the category of multiple inheritance (currently MI doesn't work) and awkward compilers (at least the EDG-based GreenHills compiler has an anomaly in its virtual function tables that makes a single test case not work). 2. How you solution compares to google mock library? Google uses more macros and has much more code to do comparable things. Also, like most other libraries, Google's mocking library requires creating a mock class for each interface to be mocked and thereby requires you to mock the full interface each time. That creates a larger barrier to refactoring and code improvement as you need to adjust more code that isn't related to the change. Google mock does support multiple inheritance and has a default behaviour of ignoring function calls that some people will prefer over my default behavior of throwing an exception. Google Mock has more code and more complexity in setting it up, with more overhead in keeping it working on changes and more code to be created by the user for minimal advantages.
3. Did you see BoostCon presentation about mocks? How does your solution compares?
Sadly, I was unable to attend BoostCon. I've read his presentation but it is a bit hard to follow without his explanation, and I can't find the video of the presentation. I cannot find any release of BMock other than a request to email him for using it, which at least seems counter to the Boost license and target as I think it is formulated. 4. Will your solution support interaction-based testing facilities inside
the Boost.Test (exception safety testing, tests for logged interaction expectations)?
Testing a log that has been recorded is mostly identical to telling the log beforehand and testing based on that. The minor advantage is slightly less setup work, which likely translates in not having to think about the interaction of a test. Adjusting the code to include a default implementation (such as the one for recording a log) is very little work, in the order of minutes. The main downside is that functions that do not have a default implementation or return value, but by default throw an exception. Before they throw an exception, any code can be included but it does not know which function it is that is being called.
For Boost the main changes would be superficial in the naming and in using
more default libraries instead of a new implementation. For Hippo Mocks I've decided to make no assumptions in anything over C++98 so there's a full Tuple implementation and so on.
Why not use boost?
So far, to include mocking functionality, no paths have to be set up, no software installed, no configuration done, no prerequisites installed. Requiring boost would be a major step up from that. If it were to be integrated into boost, this of course falls flat. That would make for a lot more generic code in the implementation. There is definite interest on my part to move further interaction based
testing support and full featured mocks.
The mocks I currently support are much more like the mocks in C# (using Rhino.Mocks) and Java (JMock) but well-adjusted to regular C++ use (in the same way that RhinoMocks uses C# reflection, I use C++ argument matching and templates to give as much information about errors and to allow the user to type (and maintain) as little as possible. I'm definately interested in expanding interaction testing and mocking into a direction somebody is interested in but I myself see no more need. I'm interested in your reaction. Please upload your code and I can try to comment on it when I have time.
The most direct link to the code is at http://www.assembla.com/spaces/hippomocks/documents/c3poual4Or3Q9ueJe5afGb/d.... I'm getting married in three days somewhat limiting my time for adjusting it to Boost. I'll try to cook something up soon. I'm afraid this mail took a bit longer to type and compose than I expected. Some bits of logic may be missing, please ask if you can't follow a part of it. Kind regards and thanks for your interest, Peter Bindels