I'm reading this thread about doctest and I checked out the library.
The first thing I noticed was macros and odd syntax. For example:
TEST_CASE("testing the factorial function") {
CHECK(factorial(1) == 1);
CHECK(factorial(2) == 2);
CHECK(factorial(10) == 3628800);
}
This looks like non-standard C++ and heavy on macros which turned me
off to Boost.Test for the same reasons. I considered using Boost.Test
for yet another library I am about to release and I gave up when I saw
the baffling use of macros.
People seem to be lavishing praise on these test libraries so I am
asking sincerely, what am I missing here? I used my own unit test
framework which is header-only, customizable, doesn't require macros
in the tests, and has a test structure which is easy to generate XML
reports from. Here it is:
https://github.com/vinniefalco/Beast/blob/master/extras/beast/unit_test/suit...
To write a test you subclass from `beast::unit_test::suite` and
implement the `run()` override, and then call `expect()` for each
condition that should be true. I was able to get Beast to almost 97%
coverage using this simple system (see
https://codecov.io/gh/vinniefalco/Beast). Here's an example of a test:
https://github.com/vinniefalco/Beast/blob/master/test/core/basic_streambuf.c...
Can someone please explain what I am missing? What is the benefit of
macros to declare functions? What is my framework missing?
Thanks
On Sun, May 22, 2016 at 10:05 PM, Gavin Lambert
On 23/05/2016 00:12, Viktor Kirilov wrote:
I just released doctest - https://github.com/onqtam/doctest All the info about it can be found on github.
I like REQUIRE/CHECK/WARN levels.
I *really* like SUBCASEs.
Although one thing that seems a bit lacking with that syntax is support for TearDown that executes even on test failure / exception. I guess the equivalent is writing RAII guards, but these can be annoying. An execute-lambda-on-destructor class included in the framework can reduce the pain of this (or including explicit teardown syntax).
This allows the library to be used in more ways than any other - testscan be written directly in the production code!
I'm not convinced that this is useful beyond small utility functions (as demonstrated in the tutorial). "Real" tests often have other requirements (mocking frameworks or hand-made mocks, library substitutions [eg. in-memory db], alternate logging/output, helper methods, etc) that you'd typically want to keep out of the production code.
OTOH, it does make checking whether methods are covered by tests a bit easier if the test is right next to the method -- but [a] this is what coverage tools are for and [b] it's likely the test case wouldn't be close to the method implementation if the class requires complex setup (since it would be a subcase instead).
Once tests get sufficiently large it's easier to have them in a separate file anyway so you can toggle between the test and the code under test quickly. (Many editors are better at jumping between files rather than multiple points within a single file.)
I realise that these things are still possible with doctest, but if this wasn't a goal then I think it would simplify things or increase consistency, such as avoiding the issue with ensuring just one instantiation of the test runner / main.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
-- Follow me on Github: https://github.com/vinniefalco