[Test] Way to tell if a test has failed?
If I'm using #include <boost/test/included/test_exec_monitor.hpp> Is there a way to check if the last BOOST_CHECK_CLOSE passed or failed? I need to print out some additional information if it failed so I can track down which test case it was (the checks are in a loop that iterates over a table of data). Thanks, John.
On Tue, Oct 30, 2007 at 10:03:41AM -0000, John Maddock wrote:
Is there a way to check if the last BOOST_CHECK_CLOSE passed or failed?
I need to print out some additional information if it failed so I can track down which test case it was (the checks are in a loop that iterates over a table of data).
In this case you should probably change the test. To be honest I have the same problem that I need something similar to BOOST_CHECK_EQUAL_COLLECTIONS which checks a range and creates compressed output. I tried using custom predicate support (http://www.boost.org/libs/test/doc/components/test_tools/custom_predicate_su...) but it is complicated ... Using BOOST_CHECK_CLOSE in a loop sounds just wrong to me. Jens
Jens Seidel <jensseidel <at> users.sf.net> writes:
On Tue, Oct 30, 2007 at 10:03:41AM -0000, John Maddock wrote:
Is there a way to check if the last BOOST_CHECK_CLOSE passed or failed?
I need to print out some additional information if it failed so I can
track
down which test case it was (the checks are in a loop that iterates over a table of data).
In this case you should probably change the test. To be honest I have the same problem that I need something similar to BOOST_CHECK_EQUAL_COLLECTIONS which checks a range and creates compressed output. I tried using custom predicate support
(http://www.boost.org/libs/test/doc/components/test_tools/custom_predicate_su... ort.html)
but it is complicated ...
What problems do you have? Don;t use the page above. Try one unser http://www.patmedia.net/~rogeeff/html/ and let me know if you still having problems (or docs are not clear enough).
Using BOOST_CHECK_CLOSE in a loop sounds just wrong to me.
Hmm. What specifically is wrong with it? Gennadiy
On Tue, Oct 30, 2007 at 06:56:36PM +0000, Gennadiy Rozental wrote:
Jens Seidel <jensseidel <at> users.sf.net> writes:
In this case you should probably change the test. To be honest I have the same problem that I need something similar to BOOST_CHECK_EQUAL_COLLECTIONS which checks a range and creates compressed output. I tried using custom predicate support
(http://www.boost.org/libs/test/doc/components/test_tools/custom_predicate_su... ort.html)
but it is complicated ...
What problems do you have?
The problem was the not perfect documentation, no (working) example, I wasn't even sure it is what I need.
Don;t use the page above.
Can you please explain why you don't update the official documentation if you even suggest *not* to read docs on http://www.boost.org?
Try one unser http://www.patmedia.net/~rogeeff/html/ and let me know if you still having problems (or docs are not clear enough).
Thanks, will do so!
Using BOOST_CHECK_CLOSE in a loop sounds just wrong to me.
Hmm. What specifically is wrong with it?
I often need to test large data structures such as vectors or matrices. Using for (int i=0; i<n; ++i) BOOST_CHECK_CLOSE(x[i], expected[i], 1E-12); the output misses the failing indices and is too long (multiple lines instead a single one). If such a test happens multiple times it's a good idea to move the code into a separate compare(x, expected) function. But if the test fails it will display this function as location and not the previous callstack. Not optimal. I even used sometimes BOOST_REQUIRED_CLOSE in the loop just to reduce the output if the test failed but missed further tests this way ... Jens
On Tue, 2007-30-10 at 18:56 +0000, Gennadiy Rozental wrote:
(http://www.boost.org/libs/test/doc/components/test_tools/custom_predicate_su... ort.html)
but it is complicated ...
What problems do you have? Don;t use the page above. Try one unser http://www.patmedia.net/~rogeeff/html/ and let me know if you still having problems (or docs are not clear enough).
Would this be for the development version of Boost.Test then? I hope its not the release version (I've been using that!)
Hi Gennadiy, On Tue, Oct 30, 2007 at 06:56:36PM +0000, Gennadiy Rozental wrote:
Jens Seidel <jensseidel <at> users.sf.net> writes:
I tried using custom predicate support
(http://www.boost.org/libs/test/doc/components/test_tools/custom_predicate_su... ort.html)
but it is complicated ...
What problems do you have? Don;t use the page above. Try one unser http://www.patmedia.net/~rogeeff/html/ and let me know if you still having problems (or docs are not clear enough).
a few remarks: Example 32 (http://www.patmedia.net/~rogeeff/html/utf/testing-tools/custom-predicate.htm...) does not include boost/test/auto_unit_test.hpp which is required for BOOST_AUTO_TEST_CASE. You should also fix the typo comparizon (use s instead of z). Your example output doesn't display the full message but truncates it. Adapting your example using templates I get the following error (g++ 4.1, 4.2, ...): example31.cpp:25:47: error: macro "BOOST_CHECK" passed 2 arguments, but takes just 1 example31.cpp: In function ‘void test_list_comparizon()’: example31.cpp:25: error: ‘BOOST_CHECK’ was not declared in this scope If I use only a single template parameter instead of two it works! I attached the example file. Jens
a few remarks:
Example 32 (http://www.patmedia.net/~rogeeff/html/utf/testing-tools/custom-
Jens Seidel <jensseidel <at> users.sf.net> writes: predicate.html)
does not include boost/test/auto_unit_test.hpp which is required for BOOST_AUTO_TEST_CASE.
No it's not. In fact above header is depricated. Now you can mix both automatically and manually registrerred test units in a same test module and unit_test.hpp handles everything.
You should also fix the typo comparizon (use s instead of z).
Some people prefer comparizon. It sounds more cool ;) But I'll fix it.
Your example output doesn't display the full message but truncates it.
What do you mean?
Adapting your example using templates I get the following error (g++ 4.1, 4.2, ...):
What did you do?
example31.cpp:25:47: error: macro "BOOST_CHECK" passed 2 arguments, but takes just 1 example31.cpp: In function ‘void test_list_comparizon()’: example31.cpp:25: error: ‘BOOST_CHECK’ was not declared in this scope
This is easily fixed: BOOST_CHECK(( compare_lists( l1, l2 ) ));
If I use only a single template parameter instead of two it works!
I am not sure how templating the test function have anything to do with this test. Gennadiy
On Thu, Nov 01, 2007 at 05:13:49PM +0000, Gennadiy Rozental wrote:
a few remarks:
Example 32 (http://www.patmedia.net/~rogeeff/html/utf/testing-tools/custom-
Jens Seidel <jensseidel <at> users.sf.net> writes: predicate.html)
does not include boost/test/auto_unit_test.hpp which is required for BOOST_AUTO_TEST_CASE.
No it's not. In fact above header is depricated.
Right, I used older headers. Sorry.
You should also fix the typo comparizon (use s instead of z).
Some people prefer comparizon. It sounds more cool ;) But I'll fix it.
I know that there are some differences between British and American English related to s/z. But I still doubt that comparizon is valid. http://dict.leo.org/ende?lp=ende&p=eL4jU.&search=comparison mentions comparison is valid for British and American (see "color comparison" entry)!
Your example output doesn't display the full message but truncates it.
What do you mean?
Your help page custom-predicate.html contains a link "Show output". Clicking it I see a window with black background containing the text: <quote> Running 1 test case... test.cpp(28): error in "test_list_comparizon": check compare_list *** 1 failure detected in test suite "example" </quote> It seems that the text is too wide for the default font. In Iceweasel I see <quote> Running 1 test case... test.cpp(28): error in "test_list_comparizon": check compare_lists( l1, l2 ) failed. Diffe *** 1 failure detected in test suite "example" </quote> Do you see the difference?
Adapting your example using templates I get the following error (g++ 4.1, 4.2, ...):
What did you do?
I attached a sample program in my last mail. But the problem was indeed on my side: I tried to compile BOOST_CHECK( compare_lists<1,2>( l1, l2 ) ) but because of the comma I need to use an additional pair of commas.
example31.cpp:25:47: error: macro "BOOST_CHECK" passed 2 arguments, but takes just 1 example31.cpp: In function ‘void test_list_comparizon()’: example31.cpp:25: error: ‘BOOST_CHECK’ was not declared in this scope
This is easily fixed:
BOOST_CHECK(( compare_lists( l1, l2 ) ));
Right :-) Jens
John Maddock <john <at> johnmaddock.co.uk> writes:
If I'm using
#include <boost/test/included/test_exec_monitor.hpp>
Is there a way to check if the last BOOST_CHECK_CLOSE passed or failed?
You can do this by accessing number of failures in results collector: results_collector.results( framework::current_test_case().p_id ); Alternatively you can use check_is_close predicate directly, print whatever you need if it failes (including mismatched values) and report it to the framework using BOOST_ERROR of BOOST_CHECK call. Finally you can implement you own predicate on top of check_is_close, that returns class predicate_value. This will allow you to report arbitrary message in case of failure or success, but should print failed argument values itself. Gennadiy P.S. Yet another solution is to print what you need unconditionally. But do it on "message" level (using BOOST_TEST_MESSAGE). This way you can print what you need if you want, by setting proper log level
Gennadiy Rozental wrote:
John Maddock <john <at> johnmaddock.co.uk> writes:
If I'm using
#include <boost/test/included/test_exec_monitor.hpp>
Is there a way to check if the last BOOST_CHECK_CLOSE passed or failed?
You can do this by accessing number of failures in results collector:
results_collector.results( framework::current_test_case().p_id );
Thanks Gennadiy that did the job nicely. John.
participants (4)
-
Gennadiy Rozental -
Jens Seidel -
John Maddock -
Sohail Somani