On 18 Jun 2016 at 19:42, Gennadiy Rozental wrote:
I think that'd be great - even more so if you could integrate this with boost.test, so I could state an expectation of what my function calls. I.e. something like that:
void f(int) {g(); h(); g();};
BOOST_CHECK_CALLS(f(42), g, h, g);
This is specific case for interaction based testing which is usually being done based on mocks. We would have very hard time to present something more or less generic without mocks.
That being said, I'd be willing to work closely with anyone interested in bringing stacktrace to boost and integrate it into Boost.Test. It would be very useful to report stacks at the point of unexpected exceptions or fatal errors.
I have some buns in the oven on that front: https://github.com/ned14/boost.kerneltest. It's the "other half" of automated testing, the part unit test frameworks or dependency injection don't usually cover: the Monte Carlo proof of correctness part. Longer run i.e. not in 2016 it'll use LLVM to trace every edge of execution and flow of data between the public facing library API and any external functions/syscalls it makes. It will then automatically permute the inputs and effects from external functions/syscalls to formally prove a piece of C++ behaves as specified. Trunk clang and trunk GCC is currently required for the requisite compiler and runtime support, hence I won't land this till 2017 at the earliest, though I have a cool proof of concept test here locally which even works right in the MSVC ABI (well done clang!). KernelTest is a natural fit for other tricks too, like running libfuzzer, the AFL, all the sanitisers, valgrind etc. It multithreads and multiprocesses naturally, so you can fire all your CPUs at the problem for the many hours it takes to cover every permutation. You can see an example of usage in AFIO v2's very first unit test: https://github.com/ned14/boost.afio/blob/master/test/tests/file_handle _create_close/runner.cpp with an example of a test kernel at https://github.com/ned14/boost.afio/blob/master/test/tests/file_handle _create_close/kernel_async_file_handle.cpp.hpp. As you'll see, the simplest use case is to supply a kernel, an array of outcomes, kernel parameters, preconditions and postconditions. KernelTest will iterate the kernel and check everything is correct, notifying your unit test framework of choice of any failures. The new fast ringbuffer logging system will trace paths of execution in the kernel, optionally logging stacktraces as it goes, to aid future permutation solvers. I have yet to implement disk persistence for the logging ringbuffer, it's a todo. This stuff only started working for the first time the Thursday just past as I've wrestling with ICEs in clang or VS2015. I've been working on it exclusively since the ACCU conference in April as it would be embarassing to present AFIO v2 at CppCon without any unit test suite as I had to do at ACCU. I would stress in the strongest possible terms that nobody should remotely consider using any of this stuff any time in 2016, wait till 2017 or probably 2018. But interested people may wish to star my github repos and watch out for conference videos, it's highly likely KernelTest + the monadic outcomes at its very core will make an appearance during the 2017 conference circuit. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/