On 8/3/2015 3:45 PM, Robert Ramey wrote:
On 8/3/15 11:58 AM, Edward Diener wrote:
I don't know how "to run the whole serialization test suite" but here is what I have done:
It's just running all the tests via the normal boost build/bjam system. I'm in the habit of mentioning this because it's quite common for someone to post a "fix" that is "too trivial" to require testing and it's helpful for me to avoid this.
Understood.
1) I have created PR "Changes to fix serialization for gcc and clang on Windows #19". This fixes the building of the serialization library for mingw(-64)/gcc, both regular and wide character.
I will consider this shortly. I've been trying to catch up on other stuff so I've got a few things pending with the serialization library maintenance. I'll look at the PR soon.
2) For clang I have been told that the system of exporting/importing individual member functions as opposed to entire class does not work, since clang will not export a class's RTTI unless the entire class is exported.
Hmmm - my problems showed up only when I enabled limited visibility. I managed to get all tests passing on gcc, clang, and msvc (with a little help from my friends). Then I got nailed by mingw which I couldn't figure out the visibility system.
My PR fixes mingw.
So while the build of the regular serialization library
succeeds it is probably not usable from other clang code.
hmmm - all the tests passed on my system with both static and shared library builds.
Did you yourself test clang on Windows targeting mingw(-64)/gcc ? Which version(s) of clang on Windows did you test ? Did you test the latest from source ? I am not speaking at all about clang on Linux, which works correctly with serialization. If I checkout serialization on 'master', go to the serialization build directory and execute "b2 toolset=clang-3.7" the final result is: clang-linux.link.dll ..\..\..\bin.v2\libs\serialization\build\clang-linux-3.7\debug\boost_wserialization-clang37-d-1_59.dll ..\..\..\bin.v2\libs\serialization\build\clang-linux-3.7\debug\basic_text_wiprimitive.obj: In function `ZN5boost13serialization15throw_exceptionINS_7archive17archive_exceptionEEEvRKT_': C:\Programming\VersionControl\modular-boost\libs\serialization\build/..\..\..\boost/serialization/throw_exception.hpp:36: undefined reference to `typeinfo for boost::archive::archive_exception' ..\..\..\bin.v2\libs\serialization\build\clang-linux-3.7\debug\basic_text_wiprimitive.obj: In function `archive_exception': C:\Programming\VersionControl\modular-boost\libs\serialization\build/..\..\..\boost/archive/archive_exception.hpp:42: undefined reference to `vtable for boost::archive::archive_exception' ..\..\..\bin.v2\libs\serialization\build\clang-linux-3.7\debug\text_wiarchive.obj:(.rdata[__ZTIN5boost7archive6detail15common_iarchiveINS0_14text_wiarchiveEEE]+0x10): undefined reference to `typeinfo for boost::archive::detail::basic_iarchive' ..\..\..\bin.v2\libs\serialization\build\clang-linux-3.7\debug\text_woarchive.obj:(.rdata[__ZTIN5boost7archive6detail15common_oarchiveINS0_14text_woarchiveEEE]+0x10): undefined reference to `typeinfo for boost::archive::detail::basic_oarchive' ..\..\..\bin.v2\libs\serialization\build\clang-linux-3.7\debug\xml_wgrammar.obj:(.rdata[__ZTCN5boost7archive21xml_archive_exceptionE4_NS0_17archive_exceptionE]+0x10): undefined reference to `typeinfo for boost::archive::archive_exception' ..\..\..\bin.v2\libs\serialization\build\clang-linux-3.7\debug\xml_wgrammar.obj:(.rdata[__ZTCN5boost7archive21xml_archive_exceptionE4_NS0_17archive_exceptionE]+0x2c): undefined reference to `typeinfo for boost::archive::archive_exception' ..\..\..\bin.v2\libs\serialization\build\clang-linux-3.7\debug\xml_wgrammar.obj:(.rdata[__ZTIN5boost7archive21xml_archive_exceptionE]+0x10): undefined reference to `typeinfo for boost::archive::archive_exception' ..\..\..\bin.v2\libs\serialization\build\clang-linux-3.7\debug\xml_wiarchive.obj:(.rdata[__ZTIN5boost7archive6detail15common_iarchiveINS0_13xml_wiarchiveEEE]+0x10): undefined reference to `typeinfo for boost::archive::detail::basic_iarchive' ..\..\..\bin.v2\libs\serialization\build\clang-linux-3.7\debug\xml_woarchive.obj:(.rdata[__ZTIN5boost7archive6detail15common_oarchiveINS0_13xml_woarchiveEEE]+0x10): undefined reference to `typeinfo for boost::archive::detail::basic_oarchive' clang++.exe: error: linker command failed with exit code 1 (use -v to see invocation) clang++.exe: error: linker command failed with exit code 1 (use -v to see invocation) "C:/Programming/VersionControl/bninja_installed_clang/bin/clang++.exe" -o "..\..\..\bin.v2\libs\serialization\build\clang-linux-3.7\debug\boost_wserialization-clang37-d-1_59.dll" -Wl,-soname -Wl,boost_wserialization-clang37-d-1_59.dll -shared -Wl,--start-group "..\..\..\bin.v2\libs\serialization\build\clang-linux-3.7\debug\basic_text_wiprimitive.obj" "..\..\..\bin.v2\libs\serialization\build\clang-linux-3.7\debug\basic_text_woprimitive.obj" "..\..\..\bin.v2\libs\serialization\build\clang-linux-3.7\debug\text_wiarchive.obj" "..\..\..\bin.v2\libs\serialization\build\clang-linux-3.7\debug\text_woarchive.obj" "..\..\..\bin.v2\libs\serialization\build\clang-linux-3.7\debug\utf8_codecvt_facet.obj" "..\..\..\bin.v2\libs\serialization\build\clang-linux-3.7\debug\xml_wgrammar.obj" "..\..\..\bin.v2\libs\serialization\build\clang-linux-3.7\debug\xml_wiarchive.obj" "..\..\..\bin.v2\libs\serialization\build\clang-linux-3.7\debug\xml_woarchive.obj" "..\..\..\bin.v2\libs\serialization\build\clang-linux-3.7\debug\codecvt_null.obj" "..\..\..\bin.v2\libs\serialization\build\clang-linux-3.7\debug\boost_serialization-clang37-d-1_59.dll" -Wl,-Bstatic -Wl,-Bdynamic -Wl,--end-group -g -march=i686 -m32 ...failed clang-linux.link.dll ..\..\..\bin.v2\libs\serialization\build\clang-linux-3.7\debug\boost_wserialization-clang37-d-1_59.dll... ...failed updating 1 target... The same thing happens when I build serialization using clang-3.4.1, clang-3.5.2, and clang-3.6.1. If I switch serialization to 'develop' the result is the same. Notice that the regular serialization build succeeds and it is only the wide character serialization build that fails with the above messages. However the regular serialization library built for clang has some problems ( see below ). If I run the serialization tests using clang ( the latest 3.7 or any of the earlier versions I can run ) I get errors because some things are not really exported/imported as one would assume it should be. Here is the result: ...failed updating 167 targets... ...skipped 676 targets... ...updated 1127 targets...
This can be
seen in the fact that using your current system of exporting/importing individual member functions of a visible class I can not build the wide character serialization library using clang because it refuses to link with the dependent regular character serialization library.
Again, I'm not seeing this - I've only had problems with mingw
So unless
you want to change the way that export/import works for clang on Windows clang on Windows will not work with serialization.
I can't remember right now whether I originally had been exporting the whole class or just function by function. It's been 6 months.
One reason that it's more complicated with the serialization library is that there are two DLLS the core one and the wide character add-in. The second exports functions of it's own while importing functions from the core. This is doable, but confusing.
I can see how it is working and it is understandable to me. There is nothing intrinsically unusual about one shared library depending on another shared library. It certainly should work.
3) I have tested my changes with gcc and clang on Linux and with gcc, VC++, and clang on Windows. As explained above everything succeeds except clang on Windows, which also failed before my changes.
Hmmm - I'm not sure how this is different than the current status. As I said, my pending issues were with mingw. Since mingw is also a relative of windows - it wouldn't surprise me of clang for windows also has issues.
I was able to solve the mingw problems by e-mails on the mingw-64 mailing list. The basic problem was getting mingw-64/gcc to correctly export the RTTI for certain classes. It turns out that for a class with exported virtual functions even certain implicit compiler generated member functions must be explicitly specified and exported also. I could never get the rules for this explained to me but by trial and error I managed to explicitly export the right functions. The two problematic classes in the srialization library were archive_exception and xml_archive_exception. In my PR you will see the changes I had to make to get these two classes to export/import correctly in mingw(-64)/gcc.
I am interested in getting this sorted out since I want to keep the serialization library as widely applicable as possible and I appreciate your help with this (and other people too). On the other hand, I'm beginning to wonder. The problem with mingw has been there since 1.58 and no one has complained. Same goes for clang on windows. It's hard to get motivated to spend time making things work for platforms which it doesn't seem anyone is using.
My PR fixes mingw, but not clang. In my discussion about this on the clang developer mailing list I was told very explicitly that clang is incapable of exporting just specific member functions of certain classes because it can not export the RTTI information for a class unless the class is exported as a whole. The problem seems to occur with classes which are exporting virtual functions. The culprits again in serialization are archive_exception and xml_archive_exception. There are very unfortunately no regression tests run on Windows for anything other than VC++. I have the setup but I just don't have the resources to run the regression tests for various versions of mingw(-64)/gcc and clang/mingw(-64) on Windows.
If you could look at this PR and try it with your "whole serialization test suite" it would be appreciated.
I absolutely will, and soon.
As I said, the whole test suite is just the normal boost build test. One thing that I do is run all the combinations of shared/static/debug/release for gcc and clang - I don't have a windows development system any more. This takes a while sucks up an incredible amount of disk space.
If you are willing to give the go ahead I can try changing clang on Windows so that the entire class is exported rather than individual member functions for that one-off case. It is pretty easy to test for clang on Windows using gcc at the preprocessor level. I do realize exporting the entire class in the one-off clang case changes your work in carefully honing visibility, for this single case, but I cannot change the way clang on Windows works. Changing for clang on Windows will not change any other compiler implementation but will make your code slightly messier.
LOL - you mean the serialization library is messy?
<g> No, of course not. What I of course meant is that a one-off system will make the code a little more difficult to follow as far as the visibility and importing/exporting is concerned. Actually it should not be so bad. I can probably manipulate detail/decl.hpp to do it for clang on Windows.
PS. I also created PR "Delete recreated files. #18" which deletes some files from Git which were being regenerated by serialization tests each time.
Fine.
Again I much appreciate your efforts. I just need to find the time to give them attention they deserve.