Boost and Microsoft's _SECURE_SCL Long story short of this post: Are there any plans (left) to add additional name mangling to the boost libs on Windows to address the problem of mismatched _SECURE_SCL setting? The Microsoft (Dinkumware) _SECURE_SCL=1 macro [1],[2] adds additional checks to standard library iterators/containers, rendering them ABI incompatible with _SECURE_SCL setting 0 vs. 1. This in turn leads to crashes when using a wrong combination of this flag. For example: [3] This has been discussed numerous times before: [4], [5], [6] -- see [5] for a list of further links to more threads. A trac issue [7] to add a specific build option has been closed as invalid. What I have been unable to determine is what the status is on supporting an additional *name mangling scheme* for the _SECURE_SCL option. From what I can discern from the current Getting Started page [8], not only do we not support any such name mangling, but the _SECURE_SCL macro (or the new _ITERATOR_DEBUG_LEVEL macro in VS2010) is not even mentioned on the Getting Started page. Since Visual Studio 2005 defaulted to _SECURE_SCL=1 and boost just uses the default, many people hit this when they used _SECURE_SCL=0 to improve performance in their release builds. (Which, as an aside, doesn't yield too much, as the real showstopper is the - independent! - _HAS_ITERATOR_DEBUGGING macro and that is only active in debug builds anyway.) In Visual Studio 2010, the situation is reversed, as Microsoft decided to now default to _SECURE_SCL=0. That means the people who change it to 1 in their release build for additional security will be bitten by the problem. (Or not, as it seems that they added some link-time checks in VS2010 to detect mismatches regarding the _ITERATOR_DEBUG_LEVEL. [9]) OK, so now again: Are there any plans (left) to add additional name mangling to the boost libs on Windows to address the problem of mismatched _SECURE_SCL setting? -- Or are we just going to ignore the issue since VS2010 defaults to the "boost preferred" - off - option now anyway? By "boost preferred" I mean that from some of the discussions it seemed to me that there was doubts w/rgd to complexity issues etc. when _SECURE_SCL is on. cheers, Martin p.s.: FWIW, here's how /I/ compile my release version of boost 1.44 on VS2005: bjam.exe --build-type=complete variant=release toolset=msvc-8.0 threading=multi define=_SECURE_SCL=0 --with-regex --with-program_options --with-test --with-serialization stage [1] Visual Studio 2005 - _SECURE_SCL : http://msdn.microsoft.com/en-us/library/aa985896%28VS.80%29.aspx [2] Visual Studio 2010 - _SECURE SCL : http://msdn.microsoft.com/en-us/library/aa985896%28v=VS.100%29.aspx [3] access violation with _SECURE_SCL turned off : http://lists.boost.org/boost-users/2009/02/45137.php [4] _SECURE_SCL for msvc (2009) : http://thread.gmane.org/gmane.comp.lib.boost.devel/185898/focus=185987 [5] MSVC _SECURE_SCL causes ABI change, ODR violation - mangle library names? : http://lists.boost.org/boost-users/2008/10/41253.php [6] Re: Bug/Incompatibility: Visual C++ 2005 checkediteratorslead to major crashes : http://thread.gmane.org/gmane.comp.lib.boost.devel/144082/focus=144089 [7] Add build option to define _secure_scl=0 + add runtime check to ensure no ODR violation : https://svn.boost.org/trac/boost/ticket/2086 [8] Boost 1.44 Getting Started on Windows : http://www.boost.org/doc/libs/1_44_0/more/getting_started/windows.html [9] Visual STuduio 2010 - detect_mismatch : http://msdn.microsoft.com/en-us/library/ee956429.aspx
[Martin B.]
(Which, as an aside, doesn't yield too much, as the real showstopper is the - independent! - _HAS_ITERATOR_DEBUGGING macro and that is only active in debug builds anyway.)
HID (which, among other things, takes locks) is much more expensive than SCL, but HID is also much more useful than SCL. That's why in VC10, HID (now named IDL=2) is still enabled by default in debug mode (and still cannot be enabled in release mode).
In Visual Studio 2010, the situation is reversed, as Microsoft decided to now default to _SECURE_SCL=0. That means the people who change it to 1 in their release build for additional security will be bitten by the problem.
I can confidently predict that almost nobody will do that.
(Or not, as it seems that they added some link-time checks in VS2010 to detect mismatches regarding the _ITERATOR_DEBUG_LEVEL. [9])
Yes. VC10's linker deterministically detects IDL mismatch and emits hard errors, naming the offending object files. (This machinery, #pragma detect_mismatch, is available for anyone to use. It will also detect when object files using the C++ Standard Library and compiled with different major versions of VC are mixed together, as long as both are VC10+; unfortunately we can't detect mixing when VC9 or earlier is involved.) Note that there are 5 possible modes: Release SCL=0 (now IDL=0, default in VC10) Release SCL=1 (now IDL=1, default in VC8/9) Debug HID=0 SCL=0 (now IDL=0) Debug HID=0 SCL=1 (now IDL=1) Debug HID=1 SCL=1 (now IDL=2, default in VC8/9/10) (The HID and SCL macros are still supported for backwards compatibility, but they are immediately mapped to IDL, which is what the rest of our headers now refer to.) Stephan T. Lavavej Visual C++ Libraries Developer
Yes. VC10's linker deterministically detects IDL mismatch and emits hard errors, naming the offending object files. (This machinery, #pragma detect_mismatch, is available for anyone to use. It will also detect when object files using the C++ Standard Library and compiled with different major versions of VC are mixed together, as long as both are VC10+; unfortunately we can't detect mixing when VC9 or earlier is involved.)
Cool, I'll investigate adding support for that to our auto-linking header.
Note that there are 5 possible modes:
Release SCL=0 (now IDL=0, default in VC10) Release SCL=1 (now IDL=1, default in VC8/9)
Debug HID=0 SCL=0 (now IDL=0) Debug HID=0 SCL=1 (now IDL=1) Debug HID=1 SCL=1 (now IDL=2, default in VC8/9/10)
(The HID and SCL macros are still supported for backwards compatibility, but they are immediately mapped to IDL, which is what the rest of our headers now refer to.)
Is HID=1 SCL=0 ever possible in past versions? Thanks, John.
[John Maddock]
Is HID=1 SCL=0 ever possible in past versions?
That's the case that I refer to as "the oddball". It can be requested, but it behaves like HID=1 SCL=1 as far as binary compatibility is concerned. They both perform exhaustive correctness checks and trigger debug assertion dialogs. They may differ on what happens if those dialogs are ignored. STL
Yes. VC10's linker deterministically detects IDL mismatch and emits hard errors, naming the offending object files. (This machinery, #pragma detect_mismatch, is available for anyone to use. It will also detect when object files using the C++ Standard Library and compiled with different major versions of VC are mixed together, as long as both are VC10+; unfortunately we can't detect mixing when VC9 or earlier is involved.)
I've been testing this, and it works well for object files being linked into one application (and presumably for static libraries), but appears not to detect mismatches between an application and a DLL, which basically makes it next to useless... unless I'm missing something? Thanks, John.
[STL]
Yes. VC10's linker deterministically detects IDL mismatch and emits hard errors, naming the offending object files. (This machinery, #pragma detect_mismatch, is available for anyone to use. It will also detect when object files using the C++ Standard Library and compiled with different major versions of VC are mixed together, as long as both are VC10+; unfortunately we can't detect mixing when VC9 or earlier is involved.)
[John Maddock]
I've been testing this, and it works well for object files being linked into one application (and presumably for static libraries), but appears not to detect mismatches between an application and a DLL, which basically makes it next to useless... unless I'm missing something?
If you're using import libs, that *should* work. If you're using LoadLibrary(), there's no way to detect mismatch in that case. STL
[STL]
Yes. VC10's linker deterministically detects IDL mismatch and emits hard errors, naming the offending object files. (This machinery, , is available for anyone to use. It will also detect when object files using the C++ Standard Library and compiled with different major versions of VC are mixed together, as long as both are VC10+; unfortunately we can't detect mixing when VC9 or earlier is involved.)
[John Maddock]
I've been testing this, and it works well for object files being linked into one application (and presumably for static libraries), but appears not to detect mismatches between an application and a DLL, which basically makes it next to useless... unless I'm missing something?
If you're using import libs, that *should* work.
I'm using import libraries - my test project is attached - no errors are generated even though they have hardcoded mismatched #pragma detect_mismatch's. Cheers, John.
I've been testing this, and it works well for object files being linked into one application (and presumably for static libraries), but appears not to detect mismatches between an application and a DLL, which basically makes it next to useless... unless I'm missing something?
If you're using import libs, that *should* work.
I'm using import libraries - my test project is attached - no errors are generated even though they have hardcoded mismatched #pragma detect_mismatch's.
That doesn't work across PE images out of the box -- and arguably it's not necessarily a good idea to flag these mismatches as an error. So long as DLL & EXE have an interface that doesn't rely on STL constructs, there's no problem. Effectively, the #pragma detect_mismatch causes the corresponding /FAILIFMISMATCH linker option to be emitted in a .drectve section. When the linker encounters an object file it will process the contents of .drectve section as command line switches (there's some additional logic, some switches don't work in directive sections, others including /FAILIFMISMATCH only work in directive sections). In your case, the object files with the directives do not contribute to the same image. Hence link.exe won't complain. If your interface relies on STL constructs, you could add an object file to the import library, make sure it's linked in by the consuming side and so trigger validation. E.g. Lib_interface.hpp: // ... #pragma comment( linker, "/INCLUDE:_dll_impl_interface_mismatch_check") Then add an object file into your import lib (*not* the DLL) which has a public symbol with that name. You can add to an import lib, with the lib command. E.g.: extern "C" const char dll_impl_interface_mismatch_check=0; /* x86 is the only platform to add a leading underscore for C linkage names */ cl /c /Zl foo.cpp lib import.lib foo.obj Now on the consuming side, when a compilation unit sees a comment linker pragma it will emit the /INCLUDE switch in the object file's directive section. The linker will then resolve the symbol by selecting the object file from the hybrid import library. It will then process that object file's directive section which includes the /FAILIFMISMATCH switch, which in turn should trigger a linker error. dumpbin /DIRECTIVES shows linker options embedded in an object file dumpbin /LINKERMEMBER shows the archive symbol table of a .lib dumpbin /ARCHIVEMEMBERS shows object files (including the special import objects) in a .lib -hg
[John Maddock]
I'm using import libraries - my test project is attached - no errors are generated even though they have hardcoded mismatched #pragma detect_mismatch's.
[Holger Grund]
That doesn't work across PE images out of the box -- and arguably it's not necessarily a good idea to flag these mismatches as an error. So long as DLL & EXE have an interface that doesn't rely on STL constructs, there's no problem.
If your interface relies on STL constructs, you could add an object file to the import library, make sure it's linked in by the consuming side and so trigger validation.
I naively expected this to work automatically, but Holger's excellent explanation reveals why it doesn't. Our enabled-by-default validation for static linking is very obviously the correct default. This disabled-by-default validation for dynamic linking is a tougher call, but I tend to agree that it's desirable. STL
Long story short of this post: Are there any plans (left) to add additional name mangling to the boost libs on Windows to address the problem of mismatched _SECURE_SCL setting?
Long story short: I'm happy to add this to the auto-linking header if we all agree on a mangling scheme - how about "idl#" where # is the IDL level? However, there really isn't much point in this unless we also add this to Boost.Build: since it seems that now that this has been rationalized in VC10 I guess "idl=0/1/2" would work well? Unless it's possible to have HID=1 SCL=0 as an option in any VC versions? John.
On 03.11.2010 19:01, John Maddock wrote:
Long story short of this post: Are there any plans (left) to add additional name mangling to the boost libs on Windows to address the problem of mismatched _SECURE_SCL setting?
Long story short: I'm happy to add this to the auto-linking header if we all agree on a mangling scheme - how about "idl#" where # is the IDL level?
However, there really isn't much point in this unless we also add this to Boost.Build: since it seems that now that this has been rationalized in VC10 I guess "idl=0/1/2" would work well? Unless it's possible to have HID=1 SCL=0 as an option in any VC versions?
Yes. VC10's linker deterministically detects IDL mismatch and emits hard errors, naming the offending object files.
Cool, I'll investigate adding support for that to our auto-linking
Given your own comment in the other post: (03.11.2010 19:03, John Maddock) header. and Stephan's comment: (03.11.2010 18:02, Stephan T. Lavavej)
In Visual Studio 2010, the situation is reversed, as Microsoft decided to now default to _SECURE_SCL=0. That means the people who change it to 1 in their release build for additional security will be bitten by the problem.
I can confidently predict that almost nobody will do that.
Adding the name mangling may not be necessary for VS2010: I agree that very few people will actually enable _SECURE_SCL in release mode and if #pragma detect_mismatch works with the boost autolink machinery, then we'll get linker errors anyway. cheers, Martin
On Thu, 2010-11-04 at 10:13 +0100, Martin B. wrote:
On 03.11.2010 19:01, John Maddock wrote:
However, there really isn't much point in this unless we also add this to Boost.Build: since it seems that now that this has been rationalized in VC10 I guess "idl=0/1/2" would work well? Unless it's possible to have HID=1 SCL=0 as an option in any VC versions? [...] Adding the name mangling may not be necessary for VS2010: I agree that very few people will actually enable _SECURE_SCL in release mode and if #pragma detect_mismatch works with the boost autolink machinery, then we'll get linker errors anyway.
The generation of link errors is one thing: the ability to pre-build boost for a Windows platform so that the correct libraries are auto-linked is something different. You won't be able to do that unless the idl# is added to the variant name, so I reckon the change is still desirable. Phil -- Phil Richards, <news@derived-software.ltd.uk>
participants (5)
-
Holger Grund -
John Maddock -
Martin B. -
Phil Richards -
Stephan T. Lavavej