C++03 and C++11 ABI compatibility for compiled libraries
Hi, I would like to raise the issue of binary compatibility of compiled Boost libraries between C++03 and C++11. I faced this problem with gcc on Linux, but other compilers and platforms may be affected as well. To summarize the problem: * Most, if not all Linux distributions build packages in C++03 mode. This is also the default in gcc. * Some Boost libraries conditionally use C++11 features in their binary interfaces (see [1] as an example). This makes C++03 and C++11 versions of the library binary incompatible. * Given that, users are unable to use C++11 in their code that uses Boost because of linker failures. I don't have the complete list of the offending C++11 features that break ABI, but it would seem that at least scoped enums and rvalue references are among them. Are there any more? I would like to hear opinions on the possible course of action to tackle this problem. Currently, I see 2 choices: 1. Avoid using C++11 features in binary interfaces. This may also mean avoiding STL types in binary interfaces, if they differ between C++03 and C++11 (I didn't verify that). This is my suggested hotfix solution for Boost.Filesystem [1] and the approach I took in Boost.Log. Obviously, this cripples code and error prone, but it keeps a single binary that can be used from any flavor of C++ on the user's side. 2. Compile different versions of Boost libraries, for each supported C++ version. The C++ version should be encoded into the library file names, so that different versions can coexist. This would require changes in Boost.Build and autolinking support code, but I think this would be a better long-term solution. Opinions? [1] https://svn.boost.org/trac/boost/ticket/6779
On 13/05/13 14:08, Andrey Semashev wrote:
I would like to hear opinions on the possible course of action to tackle this problem. Currently, I see 2 choices:
1. Avoid using C++11 features in binary interfaces. This may also mean avoiding STL types in binary interfaces, if they differ between C++03 and C++11 (I didn't verify that). This is my suggested hotfix solution for Boost.Filesystem [1] and the approach I took in Boost.Log. Obviously, this cripples code and error prone, but it keeps a single binary that can be used from any flavor of C++ on the user's side. 2. Compile different versions of Boost libraries, for each supported C++ version. The C++ version should be encoded into the library file names, so that different versions can coexist. This would require changes in Boost.Build and autolinking support code, but I think this would be a better long-term solution.
Opinions?
I prefer [1].
On 5/13/2013 10:09 AM, Mathias Gaunard wrote:
On 13/05/13 14:08, Andrey Semashev wrote:
I would like to hear opinions on the possible course of action to tackle this problem. Currently, I see 2 choices:
1. Avoid using C++11 features in binary interfaces. This may also mean avoiding STL types in binary interfaces, if they differ between C++03 and C++11 (I didn't verify that). This is my suggested hotfix solution for Boost.Filesystem [1] and the approach I took in Boost.Log. Obviously, this cripples code and error prone, but it keeps a single binary that can be used from any flavor of C++ on the user's side. 2. Compile different versions of Boost libraries, for each supported C++ version. The C++ version should be encoded into the library file names, so that different versions can coexist. This would require changes in Boost.Build and autolinking support code, but I think this would be a better long-term solution.
Opinions?
I prefer [1].
Does boost ever claim support for abi compatibility with different compiler settings? I thought not. If there needs to be a change beyond users just building the version they actually need, I would prefer 2. Binding Boost to C++03 seems counter to Boost's stated goals.
On Mon, May 13, 2013 at 5:29 PM, Michael Marcin <mike.marcin@gmail.com>wrote:
Does boost ever claim support for abi compatibility with different compiler settings? I thought not.
I thought boost explicitely, somewhere, stated that binary compatibility is not a requirement for libraries. I remember it being the main argument for developing "booster" which is a "fixed" set of boost libraries used in CPPCMS. Joel Lamotte
On Monday 13 May 2013 18:26:38 Klaim - Joël Lamotte wrote:
On Mon, May 13, 2013 at 5:29 PM, Michael Marcin <mike.marcin@gmail.com>wrote:
Does boost ever claim support for abi compatibility with different compiler settings? I thought not.
I thought boost explicitely, somewhere, stated that binary compatibility is not a requirement for libraries. I remember it being the main argument for developing "booster" which is a "fixed" set of boost libraries used in CPPCMS.
I think that was about binary compatibility between different releases of Boost or its libraries. It's a bit different from compatibility between single release builds for different C++ versions. Speaking of different releases, Boost does mangle library names with the release version, so linked applications will always use the correct binaries. This counts in favor for the solution (2), with mangling library names differently in C++03 and C++11.
On 5/13/13 12:21 PM, Andrey Semashev wrote:
On Monday 13 May 2013 18:26:38 Klaim - Joël Lamotte wrote:
On Mon, May 13, 2013 at 5:29 PM, Michael Marcin <mike.marcin@gmail.com>wrote:
Does boost ever claim support for abi compatibility with different compiler settings? I thought not.
I thought boost explicitely, somewhere, stated that binary compatibility is not a requirement for libraries. I remember it being the main argument for developing "booster" which is a "fixed" set of boost libraries used in CPPCMS.
I think that was about binary compatibility between different releases of Boost or its libraries. It's a bit different from compatibility between single release builds for different C++ versions.
Speaking of different releases, Boost does mangle library names with the release version, so linked applications will always use the correct binaries. This counts in favor for the solution (2), with mangling library names differently in C++03 and C++11.
There are also parameters where the names do not encoding important settings. If the code is compiled for x86 or x64 for instance. The user is just expected to build and use the libraries they need.
On Monday 13 May 2013 15:35:06 Michael Marcin wrote:
On 5/13/13 12:21 PM, Andrey Semashev wrote:
On Monday 13 May 2013 18:26:38 Klaim - Joël Lamotte wrote:
On Mon, May 13, 2013 at 5:29 PM, Michael Marcin
<mike.marcin@gmail.com>wrote:
Does boost ever claim support for abi compatibility with different compiler settings? I thought not.
I thought boost explicitely, somewhere, stated that binary compatibility is not a requirement for libraries. I remember it being the main argument for developing "booster" which is a "fixed" set of boost libraries used in CPPCMS.
I think that was about binary compatibility between different releases of Boost or its libraries. It's a bit different from compatibility between single release builds for different C++ versions.
Speaking of different releases, Boost does mangle library names with the release version, so linked applications will always use the correct binaries. This counts in favor for the solution (2), with mangling library names differently in C++03 and C++11.
There are also parameters where the names do not encoding important settings. If the code is compiled for x86 or x64 for instance. The user is just expected to build and use the libraries they need.
Libraries for different architectures are installed into different system folders, so it's not a problem. Surely there are compiler options that affect ABI and do not get encoded in the file names. Things like stack alignment and calling conventions are far from being commonly changed in projects, and if they are people know the caveats. C++ flavor is another kind of option. You should typically _expect_ more and more people enable C++11, despite the fact it's not the default on current systems. And, IMHO, Boost should honor and help that trend.
On Monday 13 May 2013 19:14:16 Michael Marcin wrote:
On 5/13/13 3:47 PM, Andrey Semashev wrote:
Libraries for different architectures are installed into different system folders, so it's not a problem.
Not on windows.
There is no common place for libraries on Windows, so you can arrange them the way you like for your project.
On 14/05/13 05:21, Andrey Semashev wrote:
On Monday 13 May 2013 19:14:16 Michael Marcin wrote:
On 5/13/13 3:47 PM, Andrey Semashev wrote:
Libraries for different architectures are installed into different system folders, so it's not a problem.
Not on windows.
There is no common place for libraries on Windows, so you can arrange them the way you like for your project.
There is a common place for dlls, and it is indeed two different directories for 32 and 64 bit.
On Tue, May 14, 2013 at 11:06:34AM +0200, Mathias Gaunard wrote:
On 14/05/13 05:21, Andrey Semashev wrote:
On Monday 13 May 2013 19:14:16 Michael Marcin wrote:
On 5/13/13 3:47 PM, Andrey Semashev wrote:
Libraries for different architectures are installed into different system folders, so it's not a problem.
Not on windows.
There is no common place for libraries on Windows, so you can arrange them the way you like for your project.
There is a common place for dlls, and it is indeed two different directories for 32 and 64 bit.
Noting of course that system32 and syswow64 are completely off limits to end users and end user applications. Anyone who puts anything in there in this day and age I will personally keelhaul for polluting system directories. It's bad enough that people pollute the %PATH% with DLLs, keep them out of the system directories. The recommended way to deploy third party libraries is to bundle them in the application directory and keep them out of %PATH%. -- Lars Viklund | zao@acc.umu.se
On Tue, May 14, 2013 at 1:06 PM, Mathias Gaunard < mathias.gaunard@ens-lyon.org> wrote:
On 14/05/13 05:21, Andrey Semashev wrote:
On Monday 13 May 2013 19:14:16 Michael Marcin wrote:
On 5/13/13 3:47 PM, Andrey Semashev wrote:
Libraries for different architectures are installed into different system folders, so it's not a problem.
Not on windows.
There is no common place for libraries on Windows, so you can arrange them the way you like for your project.
There is a common place for dlls, and it is indeed two different directories for 32 and 64 bit.
I was referring to the libraries you would link your application with (i.e. *.lib). Regarding built dlls I mentioned earlier that on Windows it is typical to distribute applications with their own set of libraries. So on Windows the ABI incompatibility between C++03 and C++11 is mostly non-issue. The exception is Cygwin, which still needs different library names.
On 13/05/13 22:35, Michael Marcin wrote:
There are also parameters where the names do not encoding important settings. If the code is compiled for x86 or x64 for instance. The user is just expected to build and use the libraries they need.
A linker will automatically reject linking together code that was built for two different incompatible architectures
-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Klaim - Joël Lamotte Sent: Monday, May 13, 2013 5:27 PM To: Boost Developers List Subject: Re: [boost] C++03 and C++11 ABI compatibility for compiled libraries
On Mon, May 13, 2013 at 5:29 PM, Michael Marcin <mike.marcin@gmail.com>wrote:
Does boost ever claim support for abi compatibility with different compiler settings? I thought not.
I thought boost explicitely, somewhere, stated that binary compatibility is not a requirement for libraries.
Boost has *never* sought binary compatibility - it is too much of a brake on change. Paul --- Paul A. Bristow, Prizet Farmhouse, Kendal LA8 8AB UK +44 1539 561830 07714330204 pbristow@hetp.u-net.com
On Monday 13 May 2013 10:29:47 Michael Marcin wrote:
Does boost ever claim support for abi compatibility with different compiler settings? I thought not.
If there needs to be a change beyond users just building the version they actually need, I would prefer 2.
Binding Boost to C++03 seems counter to Boost's stated goals.
Building and distributing custom builds of libraries with applications is typical in Windows but not in Linux. And with the current state of affair applications cannot use Boost libraries in C++11 mode, which also contradicts Boost goals. I'm not advocating for solution (1) here, I'm just saying that custom builds are not the answer in this case, IMHO.
Hi Andrey, I jsut focus on the Boost.Build issues. On Monday, 13. May 2013 16:08:31 Andrey Semashev wrote:
2. Compile different versions of Boost libraries, for each supported C++ version. The C++ version should be encoded into the library file names, so that different versions can coexist.
Theoretically, yes.
This would require changes in Boost.Build
This is mostly a gcc (and compatible compiler, aka clang, icc) question. Technically, you have to provide a Boost.Build feature (e.g. " feature std : c++98 c++11 ;") to distinguish between the standards. Then you can change library names in boostcpp.jam easily. The tricky part is to figure out which gcc enables c++11 support with which switches and hardcode this depending on the real gcc version into gcc.jam. There are discussion about this on the ML, too.
and autolinking support code Autolinking is a msvc (and compatible, aka intel) solution. (Un-)fortunately, msvc just enables c++11 features by default without any switches. But as the msvc runtimes are ABI-incompatible from version to version, this should be a non-issue for msvc users. mingw is another story, sadly.
, but I think this would be a better long-term solution.
The long-term solution is to get gcc (or the distros) to switch to c++11 as default. Your solution is a mid-term solution. But I fear that similar problems will occur with the next standard version, too. I am just not sure whether this justifies the work needed here. Yours, Jürgen -- * Dipl.-Math. Jürgen Hunold ! * voice: ++49 4257 300 ! Fährstraße 1 * fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen * jhunold@gmx.eu ! Germany
[Jürgen Hunold]
(Un-)fortunately, msvc just enables c++11 features by default without any switches.
That's definitely a "fortunately". If VC's compiler or STL had C++03 modes, that'd be yet another thing for you to worry about. This is a deliberate design choice.
But as the msvc runtimes are ABI-incompatible from version to version, this should be a non-issue for msvc users.
Yep. VC's STL absolutely forbids mixing different major versions. Stephan T. Lavavej Visual C++ Libraries Developer
Le 13/05/13 14:08, Andrey Semashev a écrit :
Hi,
I would like to raise the issue of binary compatibility of compiled Boost libraries between C++03 and C++11. I faced this problem with gcc on Linux, but other compilers and platforms may be affected as well.
To summarize the problem:
* Most, if not all Linux distributions build packages in C++03 mode. This is also the default in gcc. * Some Boost libraries conditionally use C++11 features in their binary interfaces (see [1] as an example). This makes C++03 and C++11 versions of the library binary incompatible. * Given that, users are unable to use C++11 in their code that uses Boost because of linker failures.
I don't have the complete list of the offending C++11 features that break ABI, but it would seem that at least scoped enums and rvalue references are among them. Are there any more?
I would like to hear opinions on the possible course of action to tackle this problem. Currently, I see 2 choices:
1. Avoid using C++11 features in binary interfaces. This may also mean avoiding STL types in binary interfaces, if they differ between C++03 and C++11 (I didn't verify that). This is my suggested hotfix solution for Boost.Filesystem [1] and the approach I took in Boost.Log. Obviously, this cripples code and error prone, but it keeps a single binary that can be used from any flavor of C++ on the user's side. 2. Compile different versions of Boost libraries, for each supported C++ version. The C++ version should be encoded into the library file names, so that different versions can coexist. This would require changes in Boost.Build and autolinking support code, but I think this would be a better long-term solution.
Opinions?
I'm completely against option 1, it would kill Boost. Best, Vicente
On 13/05/13 23:26, Vicente J. Botet Escriba wrote:
I'm completely against option 1, it would kill Boost.
Why would it kill Boost? Binary interfaces are purely implementation details. There is no problem lowering them to C++03 while still exposing C++11 features to the user or using C++11 internally.
Le 14/05/13 00:01, Mathias Gaunard a écrit :
On 13/05/13 23:26, Vicente J. Botet Escriba wrote:
I'm completely against option 1, it would kill Boost.
Why would it kill Boost? Binary interfaces are purely implementation details. There is no problem lowering them to C++03 while still exposing C++11 features to the user or using C++11 internally.
Maybe kill Boost was too strong. The binaries not only can not use c++11 features in their interface, them can not use any class or function that can have a different form on C++11 or C++03. This seems to me more than an implementation detail. I would say that it could be good if a library author follows this advice, but I would not require it. Best, Vicente
On 14/05/13 00:23, Vicente J. Botet Escriba wrote:
The binaries not only can not use c++11 features in their interface, them can not use any class or function that can have a different form on C++11 or C++03.
They can as long as it's not on the ABI boundary.
I would say that it could be good if a library author follows this advice, but I would not require it.
Of course this would not need to apply to libraries that are C++11-only.
I would like to hear opinions on the possible course of action to tackle this problem. Currently, I see 2 choices:
1. Avoid using C++11 features in binary interfaces.
2. Compile different versions of Boost libraries, for each supported C++ version.
Opinions?
We use option 2 and rely on the names being well-formed. Charles Wilson Senior Software Development Engineer Dell | Enterprise Solutions Group
I maintain Boost in Fedora, and C++03/11 incompatibility has been on my list of things to look into for some time now. Andrey Semashev <andrey.semashev@gmail.com> writes:
1. Avoid using C++11 features in binary interfaces.
This is attractive from the downstream maintainer point of view, but would likely be fragile. Also, avoiding use of ABI-incompatible STL types seems like a serious limitation. This doesn't seem like it could work.
2. Compile different versions of Boost libraries
I'm not sure how this fares with respect to autotools (by any name). We might need to update some automation to adapt to the changes in soname mangling, and to take any C++11-enabling switches into account. But I didn't look closely. Covering autotools and cmake would take us a long way towards the goal, which seems doable, so maybe #2 is the way to go. Thanks, PM
On Mon, May 20, 2013 at 3:44 PM, Petr Machata <pmachata@redhat.com> wrote:
I maintain Boost in Fedora, and C++03/11 incompatibility has been on my list of things to look into for some time now.
2. Compile different versions of Boost libraries
I'm not sure how this fares with respect to autotools (by any name). We might need to update some automation to adapt to the changes in soname mangling, and to take any C++11-enabling switches into account. But I didn't look closely. Covering autotools and cmake would take us a long way towards the goal, which seems doable, so maybe #2 is the way to go.
Great, nice to hear from package maintainers. I got the same impression from the discussion, so maybe I'll try to implement a Boost.Build property to select C++ flavor. However, I don't know much about autotools or cmake internals, so I can't tell how it will affect them.
Did you ever get anywhere with this? This issue has bitten me now, and I would really like to avoid having to build custom boost versions on Linux. The ease of installation when using the distro repos is *so* nice. Cheers Lars On Mon, May 20, 2013 at 2:15 PM, Andrey Semashev <andrey.semashev@gmail.com> wrote:
On Mon, May 20, 2013 at 3:44 PM, Petr Machata <pmachata@redhat.com> wrote:
I maintain Boost in Fedora, and C++03/11 incompatibility has been on my list of things to look into for some time now.
2. Compile different versions of Boost libraries
I'm not sure how this fares with respect to autotools (by any name). We might need to update some automation to adapt to the changes in soname mangling, and to take any C++11-enabling switches into account. But I didn't look closely. Covering autotools and cmake would take us a long way towards the goal, which seems doable, so maybe #2 is the way to go.
Great, nice to hear from package maintainers. I got the same impression from the discussion, so maybe I'll try to implement a Boost.Build property to select C++ flavor.
However, I don't know much about autotools or cmake internals, so I can't tell how it will affect them.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On Wed, Jan 22, 2014 at 12:16 PM, Lars Hagström <lars@foldspace.nu> wrote:
On Mon, May 20, 2013 at 2:15 PM, Andrey Semashev <andrey.semashev@gmail.com> wrote:
On Mon, May 20, 2013 at 3:44 PM, Petr Machata <pmachata@redhat.com> wrote:
I maintain Boost in Fedora, and C++03/11 incompatibility has been on my list of things to look into for some time now.
2. Compile different versions of Boost libraries
I'm not sure how this fares with respect to autotools (by any name). We might need to update some automation to adapt to the changes in soname mangling, and to take any C++11-enabling switches into account. But I didn't look closely. Covering autotools and cmake would take us a long way towards the goal, which seems doable, so maybe #2 is the way to go.
Great, nice to hear from package maintainers. I got the same impression from the discussion, so maybe I'll try to implement a Boost.Build property to select C++ flavor.
However, I don't know much about autotools or cmake internals, so I can't tell how it will affect them.
Did you ever get anywhere with this? This issue has bitten me now, and I would really like to avoid having to build custom boost versions on Linux. The ease of installation when using the distro repos is *so* nice.
No, not really. I had a few quick attempts that failed because of my incompetence with Boost.Build and then I got distracted.
Ok, I didn't really think so, since there is a distinct lack of further discussions of the issue on the mailing lists. Unfortunately this issue probably stops me from using C++11 at all, at least until GCC changes their default and some of the larger distros have started using that version of GCC. So C++11 is at least a few years away for me, it seems :-( /Lars On Wed, Jan 22, 2014 at 9:22 AM, Andrey Semashev <andrey.semashev@gmail.com> wrote:
On Wed, Jan 22, 2014 at 12:16 PM, Lars Hagström <lars@foldspace.nu> wrote:
On Mon, May 20, 2013 at 2:15 PM, Andrey Semashev <andrey.semashev@gmail.com> wrote:
On Mon, May 20, 2013 at 3:44 PM, Petr Machata <pmachata@redhat.com> wrote:
I maintain Boost in Fedora, and C++03/11 incompatibility has been on my list of things to look into for some time now.
2. Compile different versions of Boost libraries
I'm not sure how this fares with respect to autotools (by any name). We might need to update some automation to adapt to the changes in soname mangling, and to take any C++11-enabling switches into account. But I didn't look closely. Covering autotools and cmake would take us a long way towards the goal, which seems doable, so maybe #2 is the way to go.
Great, nice to hear from package maintainers. I got the same impression from the discussion, so maybe I'll try to implement a Boost.Build property to select C++ flavor.
However, I don't know much about autotools or cmake internals, so I can't tell how it will affect them.
Did you ever get anywhere with this? This issue has bitten me now, and I would really like to avoid having to build custom boost versions on Linux. The ease of installation when using the distro repos is *so* nice.
No, not really. I had a few quick attempts that failed because of my incompetence with Boost.Build and then I got distracted.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Lars Hagström Sent: Friday, January 24, 2014 8:12 AM To: boost@lists.boost.org Subject: Re: [boost] C++03 and C++11 ABI compatibility for compiled libraries
Ok, I didn't really think so, since there is a distinct lack of further discussions of the issue on the mailing lists. Unfortunately this issue probably stops me from using C++11 at all, at least until GCC changes their default and some of the larger distros have started using that version of GCC. So C++11 is at least a few years away for me, it seems.
Well the compiler boys are making good progress with C++11: http://clang-developers.42468.n3.nabble.com/Heads-up-C-11-switch-mid-week-td... so you may not have that long to wait? Paul --- Paul A. Bristow, Prizet Farmhouse, Kendal LA8 8AB UK +44 1539 561830 07714330204 pbristow@hetp.u-net.com
participants (12)
-
Andrey Semashev -
Charles_J_Wilson＠Dell.com -
Jürgen Hunold -
Klaim - Joël Lamotte -
Lars Hagström -
Lars Viklund -
Mathias Gaunard -
Michael Marcin -
Paul A. Bristow -
Petr Machata -
Stephan T. Lavavej -
Vicente J. Botet Escriba