Re: [boost] [Boost-testing] Test Failures on develop
On Fri, Jun 21, 2019 at 5:11 AM Tom Kent <lists@teeks99.com> wrote:
On Thu, Jun 20, 2019, 10:35 PM Rene Rivera <grafikrobot@gmail.com> wrote:
On Thu, Jun 20, 2019 at 10:28 PM Tom Kent <lists@teeks99.com> wrote:
On Thu, Jun 20, 2019 at 7:15 AM Rene Rivera <grafikrobot@gmail.com> wrote:
Since it's not due to the b2 -j change..
On Fri, Jun 14, 2019 at 8:25 AM Tom Kent via Boost-Testing < boost-testing@lists.boost.org> wrote:
For the last few days (Since June 4?) I've been having problems with all my linux tests running against the develop branch.
GCC ends with this:
gcc.compile.c++ ../../bin/common/build/gcc-8/release/process_jam_log.o
"g++-8" -fPIC -O3 -finline-functions -Wno-inline -Wall -DBOOST_ALL_NO_LIB=1 -DNDEBUG -D_CRT_SECURE_NO_WARNINGS - I"/var/boost/run/boost_root" -c -o "../../bin/common/build/gcc-8/release/process_jam_log.o" "../../common/build/../src/p rocess_jam_log.cpp"
../../common/build/../src/process_jam_log.cpp: In function 'std::__cxx11::string {anonymous}::revision(const string&)': ../../common/build/../src/process_jam_log.cpp:359:16: warning: ignoring return value of 'int system(const char*)', decla red with attribute warn_unused_result [-Wunused-result] std::system("git rev-parse --short=6 HEAD >.short-sha"); ~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ...skipped <p../../bin/testing/build/gcc-8/release>process_jam_log for lack of <p/var/boost/run/boost_root/bin.v2/libs/f ilesystem/build/gcc-8/release/link-static/visibility-hidden>libboost_filesystem.a...
I guess you'd need to find out what it says for trying to build filesystem lib.
Here's what I'm seeing with the filesystem error:
ln -f -s "../../libs/detail/include/boost/detail/has_default_constructor.hpp" "../../../boost_root/boost/detail/has_ default_constructor.hpp"
gcc.compile.c++ /var/boost/run/boost_root/bin.v2/libs/filesystem/build/gcc-8/release/link-static/visibility-hidden/path_traits.o
"g++-8" -fvisibility-inlines-hidden -m64 -O3 -finline-functions -Wno-inline -Wall -fvisibility=hidden -DBOOST_ALL _NO_LIB=1 -DBOOST_FILESYSTEM_STATIC_LINK=1 -DNDEBUG -I"/var/boost/run/boost_root" -c -o "/var/boost/run/boost_root/bin. v2/libs/filesystem/build/gcc-8/release/link-static/visibility-hidden/path_traits.o" "/var/boost/run/boost_root/libs/file system/src/path_traits.cpp"
/var/boost/run/boost_root/libs/filesystem/src/path_traits.cpp:20:10: fatal error: boost/filesystem/config.hpp: No such f ile or directory #include <boost/filesystem/config.hpp> ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. ...failed gcc.compile.c++ /var/boost/run/boost_root/bin.v2/libs/filesystem/build/gcc-8/release/link-static/visibility-hidden/path_traits.o... gcc.compile.c++ /var/boost/run/boost_root/bin.v2/libs/filesystem/build/gcc-8/release/link-static/visibility-hidden/portability.o
"g++-8" -fvisibility-inlines-hidden -m64 -O3 -finline-functions -Wno-inline -Wall -fvisibility=hidden -DBOOST_ALL _NO_LIB=1 -DBOOST_FILESYSTEM_STATIC_LINK=1 -DNDEBUG -I"/var/boost/run/boost_root" -c -o "/var/boost/run/boost_root/bin. v2/libs/filesystem/build/gcc-8/release/link-static/visibility-hidden/portability.o" "/var/boost/run/boost_root/libs/file system/src/portability.cpp"
/var/boost/run/boost_root/libs/filesystem/src/portability.cpp:20:10: fatal error: boost/filesystem/config.hpp: No such f ile or directory #include <boost/filesystem/config.hpp> ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. ...failed gcc.compile.c++ /var/boost/run/boost_root/bin.v2/libs/filesystem/build/gcc-8/release/link-static/visibility-hidden/portability.o...
And several more like that.
However, I'm also suspicious of the timing with the processor change.
Looking back at the last successful builds of develop on linux. I have a few from linux/x86_64/docker machines. The last one I saw was building with gcc-8 on June 4 around 6:30am CDT. There should have been a successful clang run about 5hrs after that (11:30am CDT), and another successful gcc around 10hrs after the last one that was actually successful (4:30pm CDT).
This seems like it would correspond to the change to build here:
https://github.com/boostorg/build/commit/451059949d92b52b21f802562a50587f1ee...
The other failure I started seeing was only my linux/arm64 machine. This doesn't run docker, so might not be affected in the same way. The last successful run on there was at 4:50pm CDT on June 4...but it typically takes around 9hrs to run so it probably got the pre 451059949 commit. However here I can't rule out that it was impacted by a8ab76ef973.
I'm going to try to standup some normal VMs soon and run on them to see if they are affected in the same way. It would also be nice if we could try backing out the usage of _SC_NPROCESSORS_ONLN and see if that fixes things, is that feasible to try for a couple days?
Again.. It's not a b2 issue. Specifying the "-jX" option entirely obviates the changed code. It's almost certainly something else. Places to look would be in the filesystem lib, or the root "boost" project for changes.
Looking at regression.py it looks like the command that builds process_jam_log doesn't use the bjam options. Can you verify if this is/isn't the case?
Tom
Somewhere between 9:57am and 12:02pm CDT today this bug migrated to master. It can trivially be re-produced. On a bare linux vm...Azure F8s_v2, Ubuntu 18.04, standard SSD. $ sudo apt update && sudo apt install build-essential gcc-8 g++-8 python python3 git wget ... $ wget https://raw.githubusercontent.com/boostorg/regression/develop/testing/src/ru... $ echo using gcc : 8 : g++-8 : \; > ~/user-config.jam $ python run.py --runner=teeks99-test01 --toolsets=gcc-8 --tag=develop --bjam-options="-j8 address-model=64" ...fails with the above errors. You can also do the same thing to reproduce in a docker image (ubuntu:bionic). --bjam-options of -j1 -j8 and absent all fail the same. This does not correspond to any changes between the filesystem library that indicates the error. But it *does* correspond exactly with the release of the new B2. https://github.com/boostorg/build/commit/04efaac148671479b514db882b1c520dcaa... There have been no successful regression tests by any linux runners since that time. Tom
On Sun, Jun 23, 2019 at 7:37 PM Tom Kent <lists@teeks99.com> wrote:
On Fri, Jun 21, 2019 at 5:11 AM Tom Kent <lists@teeks99.com> wrote:
On Thu, Jun 20, 2019, 10:35 PM Rene Rivera <grafikrobot@gmail.com> wrote:
On Thu, Jun 20, 2019 at 10:28 PM Tom Kent <lists@teeks99.com> wrote:
On Thu, Jun 20, 2019 at 7:15 AM Rene Rivera <grafikrobot@gmail.com> wrote:
Since it's not due to the b2 -j change..
On Fri, Jun 14, 2019 at 8:25 AM Tom Kent via Boost-Testing < boost-testing@lists.boost.org> wrote:
For the last few days (Since June 4?) I've been having problems with all my linux tests running against the develop branch.
GCC ends with this:
gcc.compile.c++ ../../bin/common/build/gcc-8/release/process_jam_log.o
"g++-8" -fPIC -O3 -finline-functions -Wno-inline -Wall -DBOOST_ALL_NO_LIB=1 -DNDEBUG -D_CRT_SECURE_NO_WARNINGS - I"/var/boost/run/boost_root" -c -o "../../bin/common/build/gcc-8/release/process_jam_log.o" "../../common/build/../src/p rocess_jam_log.cpp"
../../common/build/../src/process_jam_log.cpp: In function 'std::__cxx11::string {anonymous}::revision(const string&)': ../../common/build/../src/process_jam_log.cpp:359:16: warning: ignoring return value of 'int system(const char*)', decla red with attribute warn_unused_result [-Wunused-result] std::system("git rev-parse --short=6 HEAD >.short-sha"); ~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ...skipped <p../../bin/testing/build/gcc-8/release>process_jam_log for lack of <p/var/boost/run/boost_root/bin.v2/libs/f ilesystem/build/gcc-8/release/link-static/visibility-hidden>libboost_filesystem.a...
I guess you'd need to find out what it says for trying to build filesystem lib.
Here's what I'm seeing with the filesystem error:
ln -f -s "../../libs/detail/include/boost/detail/has_default_constructor.hpp" "../../../boost_root/boost/detail/has_ default_constructor.hpp"
gcc.compile.c++ /var/boost/run/boost_root/bin.v2/libs/filesystem/build/gcc-8/release/link-static/visibility-hidden/path_traits.o
"g++-8" -fvisibility-inlines-hidden -m64 -O3 -finline-functions -Wno-inline -Wall -fvisibility=hidden -DBOOST_ALL _NO_LIB=1 -DBOOST_FILESYSTEM_STATIC_LINK=1 -DNDEBUG -I"/var/boost/run/boost_root" -c -o "/var/boost/run/boost_root/bin. v2/libs/filesystem/build/gcc-8/release/link-static/visibility-hidden/path_traits.o" "/var/boost/run/boost_root/libs/file system/src/path_traits.cpp"
/var/boost/run/boost_root/libs/filesystem/src/path_traits.cpp:20:10: fatal error: boost/filesystem/config.hpp: No such f ile or directory #include <boost/filesystem/config.hpp> ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. ...failed gcc.compile.c++ /var/boost/run/boost_root/bin.v2/libs/filesystem/build/gcc-8/release/link-static/visibility-hidden/path_traits.o... gcc.compile.c++ /var/boost/run/boost_root/bin.v2/libs/filesystem/build/gcc-8/release/link-static/visibility-hidden/portability.o
"g++-8" -fvisibility-inlines-hidden -m64 -O3 -finline-functions -Wno-inline -Wall -fvisibility=hidden -DBOOST_ALL _NO_LIB=1 -DBOOST_FILESYSTEM_STATIC_LINK=1 -DNDEBUG -I"/var/boost/run/boost_root" -c -o "/var/boost/run/boost_root/bin. v2/libs/filesystem/build/gcc-8/release/link-static/visibility-hidden/portability.o" "/var/boost/run/boost_root/libs/file system/src/portability.cpp"
/var/boost/run/boost_root/libs/filesystem/src/portability.cpp:20:10: fatal error: boost/filesystem/config.hpp: No such f ile or directory #include <boost/filesystem/config.hpp> ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. ...failed gcc.compile.c++ /var/boost/run/boost_root/bin.v2/libs/filesystem/build/gcc-8/release/link-static/visibility-hidden/portability.o...
And several more like that.
However, I'm also suspicious of the timing with the processor change.
Looking back at the last successful builds of develop on linux. I have a few from linux/x86_64/docker machines. The last one I saw was building with gcc-8 on June 4 around 6:30am CDT. There should have been a successful clang run about 5hrs after that (11:30am CDT), and another successful gcc around 10hrs after the last one that was actually successful (4:30pm CDT).
This seems like it would correspond to the change to build here:
https://github.com/boostorg/build/commit/451059949d92b52b21f802562a50587f1ee...
The other failure I started seeing was only my linux/arm64 machine. This doesn't run docker, so might not be affected in the same way. The last successful run on there was at 4:50pm CDT on June 4...but it typically takes around 9hrs to run so it probably got the pre 451059949 commit. However here I can't rule out that it was impacted by a8ab76ef973.
I'm going to try to standup some normal VMs soon and run on them to see if they are affected in the same way. It would also be nice if we could try backing out the usage of _SC_NPROCESSORS_ONLN and see if that fixes things, is that feasible to try for a couple days?
Again.. It's not a b2 issue. Specifying the "-jX" option entirely obviates the changed code. It's almost certainly something else. Places to look would be in the filesystem lib, or the root "boost" project for changes.
Looking at regression.py it looks like the command that builds process_jam_log doesn't use the bjam options. Can you verify if this is/isn't the case?
Tom
Somewhere between 9:57am and 12:02pm CDT today this bug migrated to master.
It can trivially be re-produced.
On a bare linux vm...Azure F8s_v2, Ubuntu 18.04, standard SSD. $ sudo apt update && sudo apt install build-essential gcc-8 g++-8 python python3 git wget ... $ wget https://raw.githubusercontent.com/boostorg/regression/develop/testing/src/ru... $ echo using gcc : 8 : g++-8 : \; > ~/user-config.jam $ python run.py --runner=teeks99-test01 --toolsets=gcc-8 --tag=develop --bjam-options="-j8 address-model=64" ...fails with the above errors.
You can also do the same thing to reproduce in a docker image (ubuntu:bionic). --bjam-options of -j1 -j8 and absent all fail the same.
This does not correspond to any changes between the filesystem library that indicates the error. But it *does* correspond exactly with the release of the new B2. https://github.com/boostorg/build/commit/04efaac148671479b514db882b1c520dcaa...
There have been no successful regression tests by any linux runners since that time.
This <https://github.com/boostorg/regression/pull/55> should fix it. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net
On Sun, Jun 23, 2019 at 8:34 PM Rene Rivera <grafikrobot@gmail.com> wrote:
This <https://github.com/boostorg/regression/pull/55> should fix it.
See also this <https://github.com/boostorg/admin/issues/170> for other relevant PRs. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net
participants (2)
-
Rene Rivera -
Tom Kent