metatest interface update, better Boost.Test integration
Hi, Thank you for the feedback about my library so far. I've done changes to the interface of the metatest library (currently part of mpllibs - https://github.com/sabel83/mpllibs). The purpose of the changes was to improve compatibility with Boost.Test. They make it easy to embed assertions of compile-time predicates in runtime test functions. The result and pretty-printed reason of a compile-time predicate is represented by an object of type test_result (http://abel.web.elte.hu/mpllibs/metatest/test_result.html) at runtime. These objects can be built using a template function, test_result::build. It evaluates the predicate and does the pretty-printing. operator<< is overloaded for test_result, thus one can easily display these objects in output streams. This is useful for passing the results of compile-time tests to runtime unit testing frameworks. I've implemented the metaprogramming equivalent of the BOOST_<level>() macros as template functions (meta_<level>). One can use them the following way: meta_check< equal_to<int_<13>, metafunction_to_test<int, char> >
(MPLLIBS_HERE);
meta_warn< equal_to<int_<13>, metafunction_to_test<int, char> >
(MPLLIBS_HERE, "Some message");
// ... I've used a template function instead of a macro to improve the error messages about syntax errors in the predicates. The template function takes a location (http://abel.web.elte.hu/mpllibs/metatest/location.html) object as argument and metatest provides a macro, MPLLIBS_HERE, that constructs a location object with the current filename:line values. The template functions call BOOST_<level>_MESSAGE. A disadvantage of this solution is that the error reports contain two filename:line locations: the location of BOOST_<level>_MESSAGE in the integration function and the location of MPLLIBS_HERE (which is the one the user will be interested in). I've added a new example demonstrating these integration functions. You can find it in libs/metatest/example/boost_test_assertion in the source tree. I've tested the new interface with the same three compilers I've been testing the libraries with (gcc 4.5, clang 2.8, Visual C++ 10). I'm happy to receive feedback about the new interface. I'm interested if people find it useful. Regards, Abel
On Sat, Sep 24, 2011 at 3:17 AM, Ábel Sinkovics <abel@elte.hu> wrote:
Hi,
Thank you for the feedback about my library so far.
I've added a new example demonstrating these integration functions. You can find it in libs/metatest/example/boost_test_assertion in the source tree.
I would like to see more examples, collected in a single, easy to find
Abel, Your metatest library has some very interesting capabilities I really like. 1) An implementation detail, mpllibs::metatest::test_result has some very nice pretty printing capabilites, especially given its information is gleaned from static data and their types. 2) As a meta-test framework, I like the reporting capability, displaying the results of each of the unittests. Are these results able to be displayed alongside 'regular' BOOST_AUTO_TEST_CASE tests? 3) As a meta-test framework, the MPLLIBS_ADD_TEST convenience macro is very nice, as it is expressive, and reduces the syntactic complexity of instantiating a test. However, I am unclear exactly how it works, given that any BOOST_STATIC_ASSERTS, or BOOST_MPL_ASSERT_* in a library implementation will prevent the code from compiling when incorrect types are given. Perhaps I have missed something obvious. I would like to make a suggestion: Today, I am publishing some macros, which allow otherwise failing asserts, to instead throw an exception. These macros stand on their own, in the functionality they provide, and are independent of any unit testing framework, but room for improvement exists. 1) I would like to explore the possiblity of throwing your mpllibs:: metatest::test_result, or something with similiar capability. This would still be unit testing framework independent, but would add to the pretty printing value of the exception. 2) Could your framework be modified without too much effort, to catch metatest_exceptions, and then add them to your container class, for reporting? I imagine this framework element would become part of Boost Test. Users of metatest would then be motivated to use Boost Test, but not required. 3) Also the MPLLIBS_ADD_TEST macro could become part of Boost Test, simplifying the instantiation of test, but again, testing meta-programs could still be achieved using the assertion macros alone. There is much I like about your metatest library. I believe the macros I am submitting are complementary, and should probably be integrated into one submission, so I am sharing the same name metatest. I think your framework could wrap my framework-independent macros very nicely and provide great value. directory, which include both the input code, and the printed output together, if possible.
I'm happy to receive feedback about the new interface. I'm interested if people find it useful.
REMOVE_PARENTHESIS is a lifesaver! I have also created another macro which is similiar:
#define TYPEDEF_AWAY_COMMAS(type_possible_commas, type_no_commas) \ typedef typename REMOVE_PARENTHESIS(type_possible_commas) type_no_commas TYPEDEF_AWAY_COMMAS is useful when a nested macro passes in a type which contains commas in the templated type list. Receive the type surrounded parentheses, then typedef it into a comma-free-name for all subsequent use. Both these macros could possibly deserve a submission by themselves, provided I did not overlook their existence in the library already.
Regards, Abel
Thank you Abel,
Ben Robinson, Ph.D.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Hi Ben,
2) As a meta-test framework, I like the reporting capability, displaying the results of each of the unittests. Are these results able to be displayed alongside 'regular' BOOST_AUTO_TEST_CASE tests?
The short answer is yes. The meta_check, meta_warn, meta_require assertions can be used at the same places as BOOST_CHECK, BOOST_WARN and BOOST_REQUIRE and behave the same way. They are Boost.Test integration functions for the evaluated predicates. If you use the tree-hierarchy building capability of mpllibs/metatest (without writing Boost.Test test cases in your code) and use the <metatest/main_boost.hpp> header, it will add your compile-time test results to Boost.Test results.
However, I am unclear exactly how it works, given that any BOOST_STATIC_ASSERTS, or BOOST_MPL_ASSERT_* in a library implementation will prevent the code from compiling when incorrect types are given. Perhaps I have missed something obvious.
If your compile-time predicate fails to compile, you get a compilation error. I'm not aware of any solution preventing that, metatest can not do that either. However, for failed predicates it doesn't break the compilation but gives you a human readable/portable report at runtime.
2) Could your framework be modified without too much effort, to catch metatest_exceptions, and then add them to your container class, for reporting? I imagine this framework element would become part of Boost Test. Users of metatest would then be motivated to use Boost Test, but not required.
I don't understand why to throw a runtime exception - the data (was there a failed predicate or not) is static.
3) Also the MPLLIBS_ADD_TEST macro could become part of Boost Test, simplifying the instantiation of test, but again, testing meta-programs could still be achieved using the assertion macros alone.
Test metaprograms can be achieved by using static assertions, but based on my experience displaying a portable/human-readable report of what and why failed makes life easier.
I would like to see more examples, collected in a single, easy to find directory, which include both the input code, and the printed output together, if possible.
I'll implement more examples and upload the outputs somewhere as well shortly. However, you should be able to just download and build the code. Regards, Abel
Hi Ben, Am Montag, den 26.09.2011, 23:46 -0700 schrieb Ben Robinson:
I would like to see more examples, collected in a single, easy to find directory, which include both the input code, and the printed output together, if possible.
The examples are in different directories (one directory per example). They consist of multiple source files because it is easier to organise them this way. However, I've collected descriptions of the examples, generated outputs and links to the source codes in the documentation. They can be found here: http://abel.web.elte.hu/mpllibs/metatest/index.html#_examples Please let me know if you have difficulties with checking them. There are two outputs for the example called reverse: one of them is generated using metatest, the other one is generated using static assertions (and g++ 4.5.2). They can be found here: http://abel.web.elte.hu/mpllibs/metatest/index.html#_reverse Regards, Abel
participants (2)
-
Ben Robinson -
Ábel Sinkovics