Re: [boost] [1.72][Test][Math][Fiber] Recent breaking change in Boost.Test breaks fiber/math tests

-----Original Message----- From: Boost On Behalf Of Raffi Enficiaud via Boost Sent: 5 November 2019 18:49 To: boost@lists.boost.org Cc: Raffi Enficiaud Subject: Re: [boost] [1.72][Test][Math][Fiber] Recent breaking change in Boost.Test breaks fiber/math tests

On 05.11.19 17:03, Paul A Bristow via Boost wrote:

...

...

[snip]

Actually I like the change - did we need 'framework'?

"#me too" but this is not the right time to introduce any breaking changes, especially without notice. It is used nowhere in boost.test but I have not checked within boost (and certainly not outside) before making that change (that was also unrelated to the issue being solved). Some legacy code, a proper deprecation should be emitted, at least in the change logs.

I do not know if namespaces can have deprecation attributes, that would be handy.

...

...

Anyone using this is perhaps relying on a detail?

(The need for it in the Math test case was to ensure that the BOOST_TEST_MESSAGE("my_message") were actually visible. I think it would be better if messages were visible by default, not something one needs to ask for specially. I also was unconvinced the setting the environment variable BOOST_TEST_MESSAGE worked but I'll check this out and raise a bug report.)

I've finally found this exchange explaining the behaviour.

http://boost.2283326.n4.nabble.com/test-Problem-with-BOOST-TEST-MESSAG E-td467539 8.html

I think it is very counter-intuitive that messages only emerge to the normal output if there is an error or --log_level="message"

And that

#define BOOST_TEST_LOG_LEVEL "message"

does not appear to have the same effect as --log_level="message"

The current documentation never mentions the usage through a macro as you are using in the snippet above. Please see below about the relevant sections.

...

which is also surprising, and in my book a buglet?

I suggest that this behaviour should be changed so that BOOST_TEST_MESSAGE *always* produces output.

BOOST_AUTO_TEST_CASE(test_message) { BOOST_TEST_MESSAGE("\nTest BOOST_TEST_MESSAGE.\n"); } // BOOST_AUTO_TEST_CASE(test_message)

boost_test_message_test.vcxproj -> I:\Cpp\math\x64\Release\boost_test_message_test.exe

should output

Running 1 test case...

Test BOOST_TEST_MESSAGE.

*** No errors detected

I tend to disagree with that. What I am expecting from my tests is the report of "true positives", ie. assertions that fails for good reasons like regressions. All the rest I consider as information that distracts, or information printed only for non- regular/normal use like debugging. I would for instance put a message or a warning that I can easily activate if something goes wrong and needs my attention, until I fix the issue. Then I would go back to the normal mode.

This is also the approach taken by tools like ctest or b2: normal logging level prints only a very high level status, verbose logs are redirected to file for

There are times when eminently sensible suggestions (like yours and mine) are not the best thing to do. I agree that the benefit isn't worth causing downstream trouble. But I would still like some warning in the documentation. I've written notes-to-self: // #define BOOST_TEST_LOG_LEVEL "all" or /DBOOST_TEST_LOG_LEVEL // Do not output unless there is an error! // http://boost.2283326.n4.nabble.com/test-Problem-with-BOOST-TEST-MESSAGE-td46... 8.html // run with --log_level="message" always gives message. I use a VS IDE Custom build step "$(TargetDir)$(TargetName).exe" --log_level="message" for testing console apps. Paul potential

inspection, details are printed on error.

The behaviour of Boost.Test in that regard is documented in various places, mostly in the runtime arguments (command line) and in the logging part:

* https://www.boost.org/doc/libs/1_71_0/libs/test/doc/html/boost_test/test_out... ut/log_formats.html * https://www.boost.org/doc/libs/1_71_0/libs/test/doc/html/boost_test/test_out... ut/log_formats/test_log_output.html * https://www.boost.org/doc/libs/1_71_0/libs/test/doc/html/boost_test/test_out... ut/test_tools_support_for_logging/test_output_macro_message.html (even with its own section) * https://www.boost.org/doc/libs/1_71_0/libs/test/doc/html/boost_test/utf_refe... nce/rt_param_reference/log_level.html (indicates the environment variable as well) * https://www.boost.org/doc/libs/1_71_0/libs/test/doc/html/boost_test/runtime_... onfig.html although the section about environment variables can be clarified

...

Otherwise I am sure I will not be the only person confused.

At the very least the documentation should highlight that ones expectation may not match what actually happens.

I have to say that the current doc is sometimes more than complete with some redundant parts and confusing information. If you think of a better way to organize it or a better wording ("me not English native"), please let me know, I would be happy to improve it.

Thanks! Raffi

PS: and sorry about breaking your code again so close to the beta.

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost