questions about visibility/dlls under mingw
MINGW is the platform which runs gcc compilers under winsoze. I include all the visibility attribute in the "right" places with serialization build and tests. This is looking like it's passing on all platforms EXCEPT MINGW. I think MINGW uses the windows attributes, but I'm wonder if our boost setup uses the gcc macros. This would explain the bahavior I'm getting now. But honestly I really have no idea. here's the link to the appveyor results: https://ci.appveyor.com/project/RobertRamey/serialization/builds/20309631 Perhaps this is a config issue? or I'm doing wrong? I'm loath to much with this just for this one platform. Robert Ramey
On 15/11/2018 00:04, Robert Ramey via Boost-users wrote:
MINGW is the platform which runs gcc compilers under winsoze.
I include all the visibility attribute in the "right" places with serialization build and tests. This is looking like it's passing on all platforms EXCEPT MINGW. I think MINGW uses the windows attributes, but I'm wonder if our boost setup uses the gcc macros. This would explain the bahavior I'm getting now. But honestly I really have no idea. here's the link to the appveyor results:
https://ci.appveyor.com/project/RobertRamey/serialization/builds/20309631
Perhaps this is a config issue? or I'm doing wrong? I'm loath to much with this just for this one platform.
I had a quick look, but I'm stuck. * I can reproduce with mingw-gcc-8.1 * Focussing on just one failing test - test_no_rtti_polymorphic_text_archive I see: ..\..\..\bin.v2\libs\serialization\test\test_no_rtti_polymorphic_text_archive.test\gcc-8.1.0s17\debug\test_no_rtti.o: In function `save_derived(char const*)': D:\data\boost\boost\libs\serialization\test/test_no_rtti.cpp:54: undefined reference to `__imp__ZN20polymorphic_derived2C1Ev' And others, but focussing on the first error, __imp__ZN20polymorphic_derived2C1Ev is an exported default constructor for polymorphic_derived2. * If I build with b2 -d2 and capture the command lines, I can preprocess polymorphic_derived2.cpp and check the declarations, and indeed I see: class polymorphic_derived2 : public polymorphic_base { friend class boost::serialization::access; template<class Archive> __attribute__((__dllexport__)) void serialize( Archive &ar, const unsigned int ); __attribute__((__dllexport__)) const char * get_key() const; public: __attribute__((__dllexport__)) polymorphic_derived2(); __attribute__((__dllexport__)) ~polymorphic_derived2(); }; and __attribute__((__dllexport__)) polymorphic_derived2::polymorphic_derived2(){} __attribute__((__dllexport__)) polymorphic_derived2::~polymorphic_derived2(){} So that all looks OK. But, * If I examine the object files with nm, I see symbols for polymorphic_derived2 *without* the leading "__imp_" - ie regular non-exported symbols. At this point I'm stumped. Anyone else? John. --- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus
On 11/15/18 3:57 AM, John Maddock via Boost-users wrote:
On 15/11/2018 00:04, Robert Ramey via Boost-users wrote:
MINGW is the platform which runs gcc compilers under winsoze.
d2(){}
So that all looks OK. But,
* If I examine the object files with nm, I see symbols for polymorphic_derived2 *without* the leading "__imp_" - ie regular non-exported symbols.
At this point I'm stumped. Anyone else?
Your analysis is much better than my own - Thanks for investing time for this. It looks like the MING compiler is not actually processing the "export" attribute. I guess we'll just have to accept it as something we just can't fix.
John.
--- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
On 16/11/2018 04:46, Robert Ramey wrote:
It looks like the MING compiler is not actually processing the "export" attribute. I guess we'll just have to accept it as something we just can't fix.
Are you using vanilla MinGW or Mingw-w64? Because the latter is better in just about every possible way. (And despite the name, it's compatible with 32-bit as well, if that's important to you.)
On 11/15/18 3:57 AM, John Maddock via Boost-users wrote:
On 15/11/2018 00:04, Robert Ramey via Boost-users wrote:
MINGW is the platform which runs gcc compilers under winsoze.
I include all the visibility attribute in the "right" places with serialization build and tests. This is looking like it's passing on all platforms EXCEPT MINGW. I think MINGW uses the windows attributes, but I'm wonder if our boost setup uses the gcc macros. This would explain the bahavior I'm getting now. But honestly I really have no idea. here's the link to the appveyor results:
https://ci.appveyor.com/project/RobertRamey/serialization/builds/20309631
Perhaps this is a config issue? or I'm doing wrong? I'm loath to much with this just for this one platform.
I had a quick look, but I'm stuck.
* I can reproduce with mingw-gcc-8.1
* Focussing on just one failing test - test_no_rtti_polymorphic_text_archive I see:
..\..\..\bin.v2\libs\serialization\test\test_no_rtti_polymorphic_text_archive.test\gcc-8.1.0s17\debug\test_no_rtti.o: In function `save_derived(char const*)': D:\data\boost\boost\libs\serialization\test/test_no_rtti.cpp:54: undefined reference to `__imp__ZN20polymorphic_derived2C1Ev'
And others, but focussing on the first error, __imp__ZN20polymorphic_derived2C1Ev is an exported default constructor for polymorphic_derived2.
* If I build with b2 -d2 and capture the command lines, I can preprocess polymorphic_derived2.cpp and check the declarations, and indeed I see:
class polymorphic_derived2 : public polymorphic_base { friend class boost::serialization::access; template<class Archive> __attribute__((__dllexport__)) void serialize( Archive &ar, const unsigned int ); __attribute__((__dllexport__)) const char * get_key() const; public: __attribute__((__dllexport__)) polymorphic_derived2(); __attribute__((__dllexport__)) ~polymorphic_derived2(); };
and
__attribute__((__dllexport__)) polymorphic_derived2::polymorphic_derived2(){} __attribute__((__dllexport__)) polymorphic_derived2::~polymorphic_derived2(){}
So that all looks OK. But,
* If I examine the object files with nm, I see symbols for polymorphic_derived2 *without* the leading "__imp_" - ie regular non-exported symbols.
At this point I'm stumped. Anyone else?
OK - due to some sleuthing on the part of one of the auxiliary maintainers of the boost serialization library, it's seems that the problem is that BOOST_SYMBOL_IMPORT is not defined to be __attribute__((__dllimport__)) under MINGW and that is the source of the problem. It's totally plausible since gcc visibility attributes are only applied to exports and it seems that in moving to windows this rule is carried over. So I'm thinking that BOOST_SYMBOL_EXPORT needs to be updated for a special case of MINGW. Of course, this is subject to confirmation. Hope this is helpful. It's also been mentioned that the rules for application of these attributes might be different for functions and variables. The thought of this makes me ill. I don't know if it's true, but it's one more thing to keep things interesting. Robert Ramey
John.
--- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
OK - due to some sleuthing on the part of one of the auxiliary maintainers of the boost serialization library, it's seems that the problem is that
BOOST_SYMBOL_IMPORT is not defined to be __attribute__((__dllimport__)) under MINGW and that is the source of the problem.
That's not true, gcc.hpp has: # define BOOST_SYMBOL_EXPORT __attribute__((__dllexport__)) # define BOOST_SYMBOL_IMPORT __attribute__((__dllimport__)) when _Win32 is defined. And if I modify config_info.cpp to print these out then I see: BOOST_SYMBOL_EXPORT =__attribute__((__dllexport__)) BOOST_SYMBOL_IMPORT =__attribute__((__dllimport__)) BOOST_SYMBOL_VISIBLE =__attribute__((__visibility__("default"))) Which looks good to me. John.
It's totally plausible since gcc visibility attributes are only applied to exports and it seems that in moving to windows this rule is carried over. So I'm thinking that BOOST_SYMBOL_EXPORT needs to be updated for a special case of MINGW.
Of course, this is subject to confirmation. Hope this is helpful.
It's also been mentioned that the rules for application of these attributes might be different for functions and variables. The thought of this makes me ill. I don't know if it's true, but it's one more thing to keep things interesting.
Robert Ramey
John.
--- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
On 20/11/2018 08:07, John Maddock wrote:
BOOST_SYMBOL_IMPORT is not defined to be __attribute__((__dllimport__)) under MINGW and that is the source of the problem.
That's not true, gcc.hpp has:
# define BOOST_SYMBOL_EXPORT __attribute__((__dllexport__)) # define BOOST_SYMBOL_IMPORT __attribute__((__dllimport__))
when _Win32 is defined.
Only for GCC >= 4.0 though. Perhaps they have an older gcc?
Mere moments ago, quoth I:
On 20/11/2018 08:07, John Maddock wrote:
BOOST_SYMBOL_IMPORT is not defined to be __attribute__((__dllimport__)) under MINGW and that is the source of the problem.
That's not true, gcc.hpp has:
# define BOOST_SYMBOL_EXPORT __attribute__((__dllexport__)) # define BOOST_SYMBOL_IMPORT __attribute__((__dllimport__))
when _Win32 is defined.
Only for GCC >= 4.0 though. Perhaps they have an older gcc?
Also, why does that check use __GNUC__ instead of BOOST_GCC_VERSION?
Also, why does that check use __GNUC__ instead of BOOST_GCC_VERSION?
It's overkill, since we only need to major version number. --- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus
On 20/11/2018 03:52, Gavin Lambert via Boost-users wrote:
On 20/11/2018 08:07, John Maddock wrote:
BOOST_SYMBOL_IMPORT is not defined to be __attribute__((__dllimport__)) under MINGW and that is the source of the problem.
That's not true, gcc.hpp has:
# define BOOST_SYMBOL_EXPORT __attribute__((__dllexport__)) # define BOOST_SYMBOL_IMPORT __attribute__((__dllimport__))
when _Win32 is defined.
Only for GCC >= 4.0 though. Perhaps they have an older gcc?
No I can reproduce with gcc-8. See my other post for an explanation. John. --- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus
On 19/11/2018 19:07, John Maddock wrote:
OK - due to some sleuthing on the part of one of the auxiliary maintainers of the boost serialization library, it's seems that the problem is that
BOOST_SYMBOL_IMPORT is not defined to be __attribute__((__dllimport__)) under MINGW and that is the source of the problem.
That's not true, gcc.hpp has:
# define BOOST_SYMBOL_EXPORT __attribute__((__dllexport__)) # define BOOST_SYMBOL_IMPORT __attribute__((__dllimport__))
when _Win32 is defined.
And if I modify config_info.cpp to print these out then I see:
BOOST_SYMBOL_EXPORT =__attribute__((__dllexport__)) BOOST_SYMBOL_IMPORT =__attribute__((__dllimport__)) BOOST_SYMBOL_VISIBLE =__attribute__((__visibility__("default")))
Which looks good to me.
I *think* I may have twigged on to what's happening here. To recap, msvc when exporting a symbol from a dll mangles the name with a __imp_ prefix. Further it performs "early name mangling" - ie the compiler does it and the mangled name for the export ends up in the object file. GCC normally does not mangle names when changing visibility, instead it stores "something" in the object file to tell the linker to make the symbol visible. On windows it's the *linker* that takes this information and creates an alias with the appropriately mangled name when building the dll. That means that the object files do not contain records with the __imp_ prefix, only the finished dll. However, when importing, since dllimport suggests import from a dll, gcc mangles the name immediately so that the linker knows to look for a __imp_ prefixed name. Now look at what you're doing in these failing tests: you're mixing object files *in the same executable* some of which are using dllexport and some using dllimport and now we have a mismatch with dllimport implying the __imp_ prefix but there are no matching __imp_ prefixed records in the object files. I think this is a bug in your tests, albeit a very obscure one. To fix you would need to either: * Use dllexport consistently throughout an executable: ie if a class is dllexport in one object file in the dll or exe, then it should be dllexport in *all the object files in that dll/exe*, including in object files that only use those symbols not define them. * Don't export these symbols: so far as I can tell, the exported symbols are never imported from anything (since we never build a dll with them) so exporting is completely redundant in this case. You may still need BOOST_SYMBOL_VISIBLE on them on for *nix systems, not sure about that though. * Or, put these exported symbols in a dll and link the test program to it. Hopefully that makes sense, John. --- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus
On 11/20/18 9:35 AM, John Maddock via Boost-users wrote:
On 19/11/2018 19:07, John Maddock wrote:
OK - due to some sleuthing on the part of one of the auxiliary maintainers of the boost serialization library, it's seems that the problem is that
BOOST_SYMBOL_IMPORT is not defined to be __attribute__((__dllimport__)) under MINGW and that is the source of the problem.
That's not true, gcc.hpp has:
# define BOOST_SYMBOL_EXPORT __attribute__((__dllexport__)) # define BOOST_SYMBOL_IMPORT __attribute__((__dllimport__))
when _Win32 is defined.
And if I modify config_info.cpp to print these out then I see:
BOOST_SYMBOL_EXPORT =__attribute__((__dllexport__)) BOOST_SYMBOL_IMPORT =__attribute__((__dllimport__)) BOOST_SYMBOL_VISIBLE =__attribute__((__visibility__("default")))
Which looks good to me.
I *think* I may have twigged on to what's happening here.
To recap, msvc when exporting a symbol from a dll mangles the name with a __imp_ prefix. Further it performs "early name mangling" - ie the compiler does it and the mangled name for the export ends up in the object file.
GCC normally does not mangle names when changing visibility, instead it stores "something" in the object file to tell the linker to make the symbol visible. On windows it's the *linker* that takes this information and creates an alias with the appropriately mangled name when building the dll. That means that the object files do not contain records with the __imp_ prefix, only the finished dll.
However, when importing, since dllimport suggests import from a dll, gcc mangles the name immediately so that the linker knows to look for a __imp_ prefixed name.
Now look at what you're doing in these failing tests: you're mixing object files *in the same executable* some of which are using dllexport and some using dllimport and now we have a mismatch with dllimport implying the __imp_ prefix but there are no matching __imp_ prefixed records in the object files.
I think this is a bug in your tests, albeit a very obscure one. To fix you would need to either:
* Use dllexport consistently throughout an executable: ie if a class is dllexport in one object file in the dll or exe, then it should be dllexport in *all the object files in that dll/exe*, including in object files that only use those symbols not define them.
Right. I don't think I'm doing this - but I'll double check.
* Don't export these symbols: so far as I can tell, the exported symbols are never imported from anything (since we never build a dll with them) so exporting is completely redundant in this case. You may still need BOOST_SYMBOL_VISIBLE on them on for *nix systems, not sure about that though.
* Or, put these exported symbols in a dll and link the test program to it.
I believe that that is what I'm doing. Again, we'll double check. This is the test test_dll_exported...
Hopefully that makes sense, John.
I'm sure it does. But not to me yet. I'll spend some time investigating this with the information you've provided. Note that in the test there is dll_polymorphic_derived2. This exports a couple of functions. But it depends on dll_polymorphic_base and imports some symbols from that. In C++, the class derived2 inherits from the class base. So within the same module we're both exporting and importing albeit different symbols. The normal windows tests seem to be fine with this. I'll try to figure out what mingw does differently. Thanks for your help with this. I couldn't do it by myself. Robert Ramey
Note that in the test there is dll_polymorphic_derived2. This exports a couple of functions. But it depends on dll_polymorphic_base and imports some symbols from that. In C++, the class derived2 inherits from the class base. So within the same module we're both exporting and importing albeit different symbols. The normal windows tests seem to be fine with this. I'll try to figure out what mingw does differently.
To take a concrete example, in the failing "test_no_rtti_polymorphic_text_archive", both polymorphic_derived2.o and test_no_rtti.o are linked directly into the executable (no dll's involved), but in the former you are dllexport'ing the class and the latter dllimport'ing, and since it's all one big exectutable with no dll's the import/export is redundant and ideally you would be doing neither, but if you must mark them up, then it should be dllexport all the way. John. --- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus
participants (3)
-
Gavin Lambert -
John Maddock -
Robert Ramey