
Hello! I have a question about timeouts implementation in Boost::Test. Please take a look at a simple example I wrote to demonstrate the problem: krivenok@develop2 16:50:52 /tmp/boost_test $ cat main.cpp #include <boost/test/unit_test.hpp> using namespace boost::unit_test; using boost::unit_test::framework::master_test_suite; //////////////////////////////////////////////////////////////////////////////// void some_test_case() { BOOST_TEST_CHECKPOINT("About to enter an infinite loop!"); while(1); } ////////////////////////////////////////////////////////////////////////////// void another_test_case() { BOOST_TEST_CHECKPOINT("About to enter an infinite loop 2!"); while(1); } ////////////////////////////////////////////////////////////////////////////// boost::unit_test::test_suite* create_test_suite() { boost::unit_test::test_suite *ts = new boost::unit_test::test_suite("Some test suite"); if(!ts) return 0; ts->add(BOOST_TEST_CASE(&some_test_case), 0, 5); ts->add(BOOST_TEST_CASE(&another_test_case), 0, 10); return ts; } /////////////////////////////////////////////////////////////////////////////// test_suite* TestSuitesInit(int argc, char* argv[]) { master_test_suite().add(create_test_suite(), 0, 3); return 0; } /////////////////////////////////////////////////////////////////////////////// int main(int argc, char** argv) { return unit_test_main(&TestSuitesInit, argc, argv); } /////////////////////////////////////////////////////////////////////////////// krivenok@develop2 16:50:53 /tmp/boost_test $ I compiled it as follows: krivenok@develop2 16:56:12 /tmp/boost_test $ g++-4.4.1 -Wall -I/usr/local/dev/boost-1.40.0/include main.cpp /usr/local/dev/boost-1.40.0/lib/libboost_unit_test_framework-mt.a krivenok@develop2 16:56:15 /tmp/boost_test $ And the run: krivenok@develop2 16:58:17 /tmp/boost_test $ time ./a.out Running 2 test cases... unknown location(0): fatal error in "some_test_case": signal: SIGALRM (timeout while executing function) main.cpp(10): last checkpoint: About to enter an infinite loop! unknown location(0): fatal error in "another_test_case": signal: SIGALRM (timeout while executing function) main.cpp(18): last checkpoint: About to enter an infinite loop 2! *** 2 failures detected in test suite "Master Test Suite" real 0m15.002s user 0m14.920s sys 0m0.030s krivenok@develop2 16:58:36 /tmp/boost_test $ According to timings measurement I concluded that Boost::Test supports timeouts on per-test-case basis. Test suite timeout (3 sec) is ignored. Is it a bug? P.S. I also found that timeouts are implemented using alarm system call: krivenok@develop2 16:58:36 /tmp/boost_test $ strace -fF ./a.out 2>&1 | grep alarm alarm(0) = 0 alarm(5) = 0 alarm(0) = 0 alarm(0) = 0 alarm(10) = 0 alarm(0) = 0 krivenok@develop2 17:02:03 /tmp/boost_test $ As I know you can't start two or more timers using alarm since alarm uses exactly one internal timer. The following simple program demonstrates the problem: krivenok@develop2 17:06:38 /tmp/boost_test/alarm $ cat main.c #include <unistd.h> int main() { alarm(2); alarm(5); while(1); return 0; } krivenok@develop2 17:06:42 /tmp/boost_test/alarm $ gcc-4.4.1 main.c krivenok@develop2 17:06:44 /tmp/boost_test/alarm $ time ./a.out Alarm clock real 0m5.001s user 0m4.990s sys 0m0.000s krivenok@develop2 17:06:51 /tmp/boost_test/alarm $ strace -fF ./a.out 2>&1 | grep alarm alarm(2) = 0 alarm(5) = 2 krivenok@develop2 17:07:00 /tmp/boost_test/alarm $ This makes the implementation of mixed timeouts difficult and unreliable. For example, consider the following test tree: MasterTestSuite (timeout = T0) -- TestSuite1 (timeout = T1) -- TestCase11(timeout = T11) -- TestCase12(timeout = T12) -- TestSuite2 (timeout = T2) -- TestCase21(timeout = T21) -- TestCase212(timeout = T22) Actual timeout of any test case depends on timeouts of all nodes in the path. -- Sincerely yours, Dmitry V. Krivenok Orange System Co., Ltd. Saint-Petersburg, Russia work phone: +7 812 332-32-40 cellular phone: +7 921 576-70-91 e-mail: krivenok@orangesystem.ru web: http://www.orangesystem.ru skype: krivenok_dmitry jabber: krivenok_dmitry@jabber.ru icq: 242-526-443

Dmitry V. Krivenok wrote:
Hello!
I have a question about timeouts implementation in Boost::Test. Please take a look at a simple example I wrote to demonstrate the problem:
Unfortunately it's unclear from interface, but timeout is only supported on test case level. We guard each test case execution separately. and we do not guard test suite as an entity. It might be possible to do so, so you if you have any ideas feel free to share. Gennadiy
participants (2)
-
Dmitry V. Krivenok
-
Gennadiy Rozental