Test failures on sanitize memory - are they caused by lightweight_test?
There is a test set on develop called BenPope x86_64 Ubuntu - phoenix - adapt_function / clang-linux-3.6~msan~c14_libc++ This runs this command line for example on the Phoenix test "adapt_function": "clang++-3.6" -c -x c++ -std=c++1y -stdlib=libc++ -fsanitize=memory -O0 -fno-inline -Wall -fPIC -m64 -DBOOST_ALL_NO_LIB=1 -I".." -o "/home/ben/development/boost/test/build/develop/results/boost/bin.v2/libs/phoenix/test/adapt_function.test/clang-linux-3.6~msan~c14_libc++/debug/address-model-64/architecture-x86/debug-symbols-off/function/adapt_function.o" "../libs/phoenix/test/function/adapt_function.cpp" There are numerous failures with this test set on Phoenix and I set out to find out why. Most of the failures are like this: SUMMARY: MemorySanitizer: use-of-uninitialized-value ??:0 std::__1::basic_ostream<char, std::__1::char_traits<char> >& std::__1::__put_character_sequence<char, std::__1::char_traits<char> >(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, char const*, unsigned long) Exiting I think this is coming from the boost::report_errors function defined in boost/core/lightweight_test.hpp I have put some temporary tests on to develop for Phoenix which contain only various calls to test the lightweight test. These are called aa_test0 to 3 so they will come at the top of the table. Unfortunately tests are not reporting at the moment so I don't have any results. There are also failures on the testing of Boost Core with the same test set. I thought I would report this without waiting for the results in the hope that it can be sorted out for the 1.58.0 release. John
On Wed, Mar 11, 2015 at 12:32 PM, Fletcher, John P <j.p.fletcher@aston.ac.uk> wrote:
There is a test set on develop called BenPope x86_64 Ubuntu - phoenix - adapt_function / clang-linux-3.6~msan~c14_libc++
This runs this command line for example on the Phoenix test "adapt_function":
"clang++-3.6" -c -x c++ -std=c++1y -stdlib=libc++ -fsanitize=memory -O0 -fno-inline -Wall -fPIC -m64 -DBOOST_ALL_NO_LIB=1 -I".." -o "/home/ben/development/boost/test/build/develop/results/boost/bin.v2/libs/phoenix/test/adapt_function.test/clang-linux-3.6~msan~c14_libc++/debug/address-model-64/architecture-x86/debug-symbols-off/function/adapt_function.o" "../libs/phoenix/test/function/adapt_function.cpp"
There are numerous failures with this test set on Phoenix and I set out to find out why.
Most of the failures are like this:
SUMMARY: MemorySanitizer: use-of-uninitialized-value ??:0 std::__1::basic_ostream<char, std::__1::char_traits<char> >& std::__1::__put_character_sequence<char, std::__1::char_traits<char> >(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, char const*, unsigned long) Exiting
I think this is coming from the boost::report_errors function defined in boost/core/lightweight_test.hpp
I have put some temporary tests on to develop for Phoenix which contain only various calls to test the lightweight test. These are called aa_test0 to 3 so they will come at the top of the table. Unfortunately tests are not reporting at the moment so I don't have any results.
There are also failures on the testing of Boost Core with the same test set.
I thought I would report this without waiting for the results in the hope that it can be sorted out for the 1.58.0 release.
This looks like a string insertion operator implementation. I don't see how the string memory could be uninitialized since there are only literals used in report_errors(), as well as other functions, unless you use BOOST_ERROR with uninitialized buffer as the message string. I suspect a false positive. Does MSan report an error for this code sample: #include <iostream> int main() { std::cerr << "Hello, world!" << std::endl; return 0; }
On Wednesday, March 11, 2015 05:43 PM, Andrey Semashev wrote:
On Wed, Mar 11, 2015 at 12:32 PM, Fletcher, John P <j.p.fletcher@aston.ac.uk> wrote:
There is a test set on develop called BenPope x86_64 Ubuntu - phoenix - adapt_function / clang-linux-3.6~msan~c14_libc++
This runs this command line for example on the Phoenix test "adapt_function":
"clang++-3.6" -c -x c++ -std=c++1y -stdlib=libc++ -fsanitize=memory -O0 -fno-inline -Wall -fPIC -m64 -DBOOST_ALL_NO_LIB=1 -I".." -o "/home/ben/development/boost/test/build/develop/results/boost/bin.v2/libs/phoenix/test/adapt_function.test/clang-linux-3.6~msan~c14_libc++/debug/address-model-64/architecture-x86/debug-symbols-off/function/adapt_function.o" "../libs/phoenix/test/function/adapt_function.cpp"
There are numerous failures with this test set on Phoenix and I set out to find out why.
Most of the failures are like this:
SUMMARY: MemorySanitizer: use-of-uninitialized-value ??:0 std::__1::basic_ostream<char, std::__1::char_traits<char> >& std::__1::__put_character_sequence<char, std::__1::char_traits<char> >(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, char const*, unsigned long) Exiting
I think this is coming from the boost::report_errors function defined in boost/core/lightweight_test.hpp
I have put some temporary tests on to develop for Phoenix which contain only various calls to test the lightweight test. These are called aa_test0 to 3 so they will come at the top of the table. Unfortunately tests are not reporting at the moment so I don't have any results.
There are also failures on the testing of Boost Core with the same test set.
I thought I would report this without waiting for the results in the hope that it can be sorted out for the 1.58.0 release.
This looks like a string insertion operator implementation. I don't see how the string memory could be uninitialized since there are only literals used in report_errors(), as well as other functions, unless you use BOOST_ERROR with uninitialized buffer as the message string. I suspect a false positive. Does MSan report an error for this code sample:
#include <iostream>
int main() { std::cerr << "Hello, world!" << std::endl; return 0; }
Yes. ben@yyls03:~/development/test$ ./a.out ==4752== WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x7f0103b09f3f in std::__1::basic_ostream<char, std::__1::char_traits<char> >& std::__1::__put_character_sequence<char, std::__1::char_traits<char> >(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, char const*, unsigned long) (/home/ben/development/test/a.out+0x8bf3f) #1 0x7f0103b09602 in std::__1::basic_ostream<char, std::__1::char_traits<char> >& std::__1::operator<< <std::__1::char_traits<char> >(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, char const*) (/home/ben/development/test/a.out+0x8b602) #2 0x7f0103b094d6 in main (/home/ben/development/test/a.out+0x8b4d6) #3 0x7f0102462ec4 in __libc_start_main /build/buildd/eglibc-2.19/csu/libc-start.c:287 #4 0x7f0103ab4f7e in _start (/home/ben/development/test/a.out+0x36f7e) SUMMARY: MemorySanitizer: use-of-uninitialized-value ??:0 std::__1::basic_ostream<char, std::__1::char_traits<char> >& std::__1::__put_character_sequence<char, std::__1::char_traits<char>
(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, char const*, unsigned long) Exiting
I've added it to the blacklist now, hopefully I have the correct incantation and significantly reduce the false positives. Ben
On Wednesday, March 11, 2015 07:18 PM, Ben Pope wrote:
On Wednesday, March 11, 2015 05:43 PM, Andrey Semashev wrote:
On Wed, Mar 11, 2015 at 12:32 PM, Fletcher, John P <j.p.fletcher@aston.ac.uk> wrote:
There is a test set on develop called BenPope x86_64 Ubuntu - phoenix - adapt_function / clang-linux-3.6~msan~c14_libc++
This runs this command line for example on the Phoenix test "adapt_function":
"clang++-3.6" -c -x c++ -std=c++1y -stdlib=libc++ -fsanitize=memory -O0 -fno-inline -Wall -fPIC -m64 -DBOOST_ALL_NO_LIB=1 -I".." -o "/home/ben/development/boost/test/build/develop/results/boost/bin.v2/libs/phoenix/test/adapt_function.test/clang-linux-3.6~msan~c14_libc++/debug/address-model-64/architecture-x86/debug-symbols-off/function/adapt_function.o" "../libs/phoenix/test/function/adapt_function.cpp"
There are numerous failures with this test set on Phoenix and I set out to find out why.
Most of the failures are like this:
SUMMARY: MemorySanitizer: use-of-uninitialized-value ??:0 std::__1::basic_ostream<char, std::__1::char_traits<char> >& std::__1::__put_character_sequence<char, std::__1::char_traits<char>
(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, char const*, unsigned long) Exiting
I think this is coming from the boost::report_errors function defined in boost/core/lightweight_test.hpp
I have put some temporary tests on to develop for Phoenix which contain only various calls to test the lightweight test. These are called aa_test0 to 3 so they will come at the top of the table. Unfortunately tests are not reporting at the moment so I don't have any results.
There are also failures on the testing of Boost Core with the same test set.
I thought I would report this without waiting for the results in the hope that it can be sorted out for the 1.58.0 release.
This looks like a string insertion operator implementation. I don't see how the string memory could be uninitialized since there are only literals used in report_errors(), as well as other functions, unless you use BOOST_ERROR with uninitialized buffer as the message string. I suspect a false positive. Does MSan report an error for this code sample:
#include <iostream>
int main() { std::cerr << "Hello, world!" << std::endl; return 0; }
Yes.
ben@yyls03:~/development/test$ ./a.out ==4752== WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x7f0103b09f3f in std::__1::basic_ostream<char, std::__1::char_traits<char> >& std::__1::__put_character_sequence<char, std::__1::char_traits<char> >(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, char const*, unsigned long) (/home/ben/development/test/a.out+0x8bf3f) #1 0x7f0103b09602 in std::__1::basic_ostream<char, std::__1::char_traits<char> >& std::__1::operator<< <std::__1::char_traits<char> >(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, char const*) (/home/ben/development/test/a.out+0x8b602) #2 0x7f0103b094d6 in main (/home/ben/development/test/a.out+0x8b4d6) #3 0x7f0102462ec4 in __libc_start_main /build/buildd/eglibc-2.19/csu/libc-start.c:287 #4 0x7f0103ab4f7e in _start (/home/ben/development/test/a.out+0x36f7e)
SUMMARY: MemorySanitizer: use-of-uninitialized-value ??:0 std::__1::basic_ostream<char, std::__1::char_traits<char> >& std::__1::__put_character_sequence<char, std::__1::char_traits<char>
(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, char const*, unsigned long) Exiting
I've added it to the blacklist now, hopefully I have the correct incantation and significantly reduce the false positives.
Hmmm, flags aren't being passed correctly: <test-log library="core" revision="dd17b0" test-name="explicit_operator_bool_noexcept" test-type="run" test-program="libs/core/test/explicit_operator_bool_noexcept.cpp" target-directory="boost/bin.v2/libs/core/test/explicit_operator_bool_noexcept.test/clang-linux-3.6~msan~c14_libc++/debug/address-model-64/architecture-x86" toolset="clang-linux-3.6~msan~c14_libc++" show-run-output="false"> <compile result="succeed" timestamp="2015-03-12 14:33:21 UTC"> "clang++-3.6" -c -x c++ -std=c++1y -stdlib=libc++ -fsanitize=memory -O0 -g -fno-inline -Wall -g -fPIC -m64 -DBOOST_ALL_NO_LIB=1 -I".." -o "/home/ben/development/boost/test/build/results/boost/bin.v2/libs/core/test/explicit_operator_bool_noexcept.test/clang-linux-3.6~msan~c14_libc++/debug/address-model-64/architecture-x86/explicit_operator_bool_noexcept.o" "../libs/core/test/explicit_operator_bool_noexcept.cpp" </compile> <link result="succeed" timestamp="2015-03-12 14:33:21 UTC"> "clang++-3.6" -o "/home/ben/development/boost/test/build/results/boost/bin.v2/libs/core/test/explicit_operator_bool_noexcept.test/clang-linux-3.6~msan~c14_libc++/debug/address-model-64/architecture-x86/explicit_operator_bool_noexcept" -Wl,--start-group "/home/ben/development/boost/test/build/results/boost/bin.v2/libs/core/test/explicit_operator_bool_noexcept.test/clang-linux-3.6~msan~c14_libc++/debug/address-model-64/architecture-x86/explicit_operator_bool_noexcept.o" -Wl,-Bstatic -Wl,-Bdynamic -Wl,--end-group -g -fsanitize=memory -fsanitize-blacklist=/home/ben/sanitize-blacklist.txt -lc++ -lc++abi -m64 </link> <run result="fail" timestamp="2015-03-12 14:33:21 UTC"> ==24502== WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x7fc24c95f9bf in std::__1::basic_ostream<char, std::__1::char_traits<char> >& std::__1::__put_character_sequence<char, std::__1::char_traits<char> >(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, char const*, unsigned long) /home/ben/development/llvm/3.6/install/release/bin/../include/c++/v1/ostream:752:13 #1 0x7fc24c95f082 in std::__1::basic_ostream<char, std::__1::char_traits<char> >& std::__1::operator<< <std::__1::char_traits<char> >(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, char const*) /home/ben/development/llvm/3.6/install/release/bin/../include/c++/v1/ostream:894:12 #2 0x7fc24c95eb31 in boost::report_errors() /home/ben/development/boost/test/build/boost_root/status/../boost/core/lightweight_test.hpp:133:11 #3 0x7fc24c95e952 in main /home/ben/development/boost/test/build/boost_root/status/../libs/core/test/explicit_operator_bool_noexcept.cpp:79:12 #4 0x7fc24afaaec4 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21ec4) #5 0x7fc24c90a3ee in _start (/home/ben/development/boost/test/build/results/boost/bin.v2/libs/core/test/explicit_operator_bool_noexcept.test/clang-linux-3.6~msan~c14_libc++/debug/address-model-64/architecture-x86/explicit_operator_bool_noexcept+0x363ee) SUMMARY: MemorySanitizer: use-of-uninitialized-value /home/ben/development/llvm/3.6/install/release/bin/../include/c++/v1/ostream:752 std::__1::basic_ostream<char, std::__1::char_traits<char> >& std::__1::__put_character_sequence<char, std::__1::char_traits<char> >(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, char const*, unsigned long) Exiting EXIT STATUS: 77 </run> </test-log> -fsanitize-blacklist needs passing at compile time, not link time. Ben
Le 12/03/15 15:48, Ben Pope a écrit :
On Thursday, March 12, 2015 10:45 PM, Ben Pope wrote:
-fsanitize-blacklist needs passing at compile time, not link time.
...which is entirely my fault.
I see a lot of reports that a similar to this one yet [1]. Uninitialized bytes in __interceptor_strlen at offset 88 inside [0x60c00000efa0, 89) ==7257== WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x7f46edfb1de2 in std::logic_error::logic_error(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) (/usr/lib/libc++.so.1+0x8dde2) #1 0x7f46eeddaf64 in boost::future_error::future_error(boost::system::error_code) /home/ben/development/boost/test/build/boost_root/status/../boost/thread/futures/future_error.hpp:28:7 #2 0x7f46eeddefc8 in boost::broken_promise::broken_promise() /home/ben/development/boost/test/build/boost_root/status/../boost/thread/futures/future_error.hpp:51:9 #3 0x7f46eedcdb02 in boost::promise<int>::~promise() /home/ben/development/boost/test/build/boost_root/status/../boost/thread/future.hpp:1951:63 #4 0x7f46eedc8120 in main /home/ben/development/boost/test/build/boost_root/status/../libs/thread/test/sync/futures/shared_future/move_assign_pass.cpp:39:3 #5 0x7f46ecce0ec4 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21ec4) #6 0x7f46eed7374c in _start (/home/ben/development/boost/test/build/results/boost/bin.v2/libs/thread/test/shared_future__move_asign_p.test/clang-linux-3.6~msan~c14_libc++/debug/address-model-64/architecture-x86/threading-multi/shared_future__move_asign_p+0x6974c) SUMMARY: MemorySanitizer: use-of-uninitialized-value ??:0 std::logic_error::logic_error(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) Exiting Does it corresponds to the same false positive? Have you reached to mask the PO false positive? Vicente [1] http://www.boost.org/development/tests/develop/developer/output/BenPope%20x8...
On Friday, March 13, 2015 03:09 PM, Vicente J. Botet Escriba wrote:
Le 12/03/15 15:48, Ben Pope a écrit :
On Thursday, March 12, 2015 10:45 PM, Ben Pope wrote:
-fsanitize-blacklist needs passing at compile time, not link time.
...which is entirely my fault.
I see a lot of reports that a similar to this one yet [1].
Uninitialized bytes in __interceptor_strlen at offset 88 inside [0x60c00000efa0, 89) ==7257== WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x7f46edfb1de2 in std::logic_error::logic_error(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) (/usr/lib/libc++.so.1+0x8dde2) #1 0x7f46eeddaf64 in boost::future_error::future_error(boost::system::error_code) /home/ben/development/boost/test/build/boost_root/status/../boost/thread/futures/future_error.hpp:28:7
#2 0x7f46eeddefc8 in boost::broken_promise::broken_promise() /home/ben/development/boost/test/build/boost_root/status/../boost/thread/futures/future_error.hpp:51:9
#3 0x7f46eedcdb02 in boost::promise<int>::~promise() /home/ben/development/boost/test/build/boost_root/status/../boost/thread/future.hpp:1951:63
#4 0x7f46eedc8120 in main /home/ben/development/boost/test/build/boost_root/status/../libs/thread/test/sync/futures/shared_future/move_assign_pass.cpp:39:3
#5 0x7f46ecce0ec4 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21ec4) #6 0x7f46eed7374c in _start (/home/ben/development/boost/test/build/results/boost/bin.v2/libs/thread/test/shared_future__move_asign_p.test/clang-linux-3.6~msan~c14_libc++/debug/address-model-64/architecture-x86/threading-multi/shared_future__move_asign_p+0x6974c)
SUMMARY: MemorySanitizer: use-of-uninitialized-value ??:0 std::logic_error::logic_error(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) Exiting
Does it corresponds to the same false positive? Have you reached to mask the PO false positive?
I haven't added that to the blacklist because I need to determine the mangled name and haven't got round to it. Also, since I'm not linking with a sanitized libc++, I'm not sure if it's a false positive or not. Hopefully I'll have some time this weekend to run checks on sanitized versions of libc++, as well as blacklist some of that stuff in the meanwhile for boost tests. Running the sanitized tests takes 6-8 hours per branch, just core atomic and thread takes over an hour, so it's slow going when I get it wrong ;) Ben
On Friday, March 13, 2015 03:09 PM, Vicente J. Botet Escriba wrote:
I see a lot of reports that a similar to this one yet [1].
Uninitialized bytes in __interceptor_strlen at offset 88 inside [0x60c00000efa0, 89) ==7257== WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x7f46edfb1de2 in std::logic_error::logic_error(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) (/usr/lib/libc++.so.1+0x8dde2) #1 0x7f46eeddaf64 in boost::future_error::future_error(boost::system::error_code) /home/ben/development/boost/test/build/boost_root/status/../boost/thread/futures/future_error.hpp:28:7
#2 0x7f46eeddefc8 in boost::broken_promise::broken_promise() /home/ben/development/boost/test/build/boost_root/status/../boost/thread/futures/future_error.hpp:51:9
#3 0x7f46eedcdb02 in boost::promise<int>::~promise() /home/ben/development/boost/test/build/boost_root/status/../boost/thread/future.hpp:1951:63
#4 0x7f46eedc8120 in main /home/ben/development/boost/test/build/boost_root/status/../libs/thread/test/sync/futures/shared_future/move_assign_pass.cpp:39:3
#5 0x7f46ecce0ec4 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21ec4) #6 0x7f46eed7374c in _start (/home/ben/development/boost/test/build/results/boost/bin.v2/libs/thread/test/shared_future__move_asign_p.test/clang-linux-3.6~msan~c14_libc++/debug/address-model-64/architecture-x86/threading-multi/shared_future__move_asign_p+0x6974c)
SUMMARY: MemorySanitizer: use-of-uninitialized-value ??:0 std::logic_error::logic_error(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) Exiting
Does it corresponds to the same false positive? Have you reached to mask the PO false positive?
New results are up, this one seems to be cleared up by using the sanitised libc++. I currently have no blacklist, I'll see what else needs looking at tomorrow, after the full run has completed. Apologies for the delay. Ben
Vicente
[1] http://www.boost.org/development/tests/develop/developer/output/BenPope%20x8...
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On Wednesday, March 11, 2015 05:32 PM, Fletcher, John P wrote:
There is a test set on develop called BenPope x86_64 Ubuntu - phoenix - adapt_function / clang-linux-3.6~msan~c14_libc++
This runs this command line for example on the Phoenix test "adapt_function":
"clang++-3.6" -c -x c++ -std=c++1y -stdlib=libc++ -fsanitize=memory -O0 -fno-inline -Wall -fPIC -m64 -DBOOST_ALL_NO_LIB=1 -I".." -o "/home/ben/development/boost/test/build/develop/results/boost/bin.v2/libs/phoenix/test/adapt_function.test/clang-linux-3.6~msan~c14_libc++/debug/address-model-64/architecture-x86/debug-symbols-off/function/adapt_function.o" "../libs/phoenix/test/function/adapt_function.cpp"
There are numerous failures with this test set on Phoenix and I set out to find out why.
All tests now pass. It was probably due to not running with the sanitised libc++. Ben
participants (4)
-
Andrey Semashev -
Ben Pope -
Fletcher, John P -
Vicente J. Botet Escriba