[Config] Multiple versions of Boost
Recently, I had the pleasure of integrating a statically linked library that used a version of Boost different from the one I was using. All relevant compiler flags matched. Only the Boost version was the problem. Fortunately, I was able to solve the problem by getting the provider of the statically linked library to rename the Boost namespace by passing in -Dboost=boost_133_1_provider to their compiler. This worked fine for them. Obviously part of this solution involved them rebuilding the part of boost that was important to them in the same manner, but we always build Boost ourselves don't we? Anyway, I thought it might be a good idea to do the same on the larger code base so I attempted to do the same. Failed miserably. The problem is that some of the Boost code contains some variant of: #if defined(SOME_CONDITION) # define HEADER <boost/some/header.hpp> #else # define HEADER <boost/some/other/header.hpp> #endif #include HEADER Reading 16.2/4, it seems that the behaviour in this case is not defined. Specifically, an implementation may or may not replace the boost text with boost_133_1_provider. This is true whether or not you have <boost/some/header.hpp> or "boost/some/header.hpp". So on one compiler, you may end up with: #include <boost_133_1_provider/some/header.hpp> And on another: #include <boost/some/header.hpp> My immediate problem solved, I made do. However, the problem still bugs me. There should be /some/ way to do this. So... What if Boost.Config did something like: #if !defined(BOOST) # define BOOST boost #endif And we replace all uses of the boost namespace with BOOST? Would such a patch against trunk be acceptable? What are the problems? I know it isn't aesthetically pleasing but it might have made this a bit easier. -- Sohail Somani http://uint32t.blogspot.com
What if Boost.Config did something like:
#if !defined(BOOST) # define BOOST boost #endif
And we replace all uses of the boost namespace with BOOST?
Not all boost headers include boost/config.hpp. Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode
Emil Dotchevski wrote:
What if Boost.Config did something like:
#if !defined(BOOST) # define BOOST boost #endif
And we replace all uses of the boost namespace with BOOST?
Not all boost headers include boost/config.hpp.
Is that really a problem? #include <boost/config/namespace.hpp> -- Sohail Somani http://uint32t.blogspot.com
On Fri, Sep 12, 2008 at 5:03 PM, Sohail Somani <sohail@taggedtype.net> wrote:
Emil Dotchevski wrote:
What if Boost.Config did something like:
#if !defined(BOOST) # define BOOST boost #endif
And we replace all uses of the boost namespace with BOOST?
Not all boost headers include boost/config.hpp.
Is that really a problem?
#include <boost/config/namespace.hpp>
In principle, personally, I'm against making any header mandatory. But also, if someone asks you to integrate a statically linked library that used a version of the standard library different from the one you were using, what would you do? :) Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode
Emil Dotchevski wrote:
On Fri, Sep 12, 2008 at 5:03 PM, Sohail Somani <sohail@taggedtype.net> wrote:
Emil Dotchevski wrote:
What if Boost.Config did something like:
#if !defined(BOOST) # define BOOST boost #endif
And we replace all uses of the boost namespace with BOOST? Not all boost headers include boost/config.hpp. Is that really a problem?
#include <boost/config/namespace.hpp>
In principle, personally, I'm against making any header mandatory.
I can understand that but this isn't a problem that affects users. Besides if there is a policy that says you can use multiple versions of Boost if you do XYZ, then it might be OK even in your world?
But also, if someone asks you to integrate a statically linked library that used a version of the standard library different from the one you were using, what would you do? :)
I would say that you find a new job ;-) But seriously, there are N different standard libraries with multiple versions whereas there is only one Boost with multiple versions. Fixing the problem for Boost solves it on all compilers. There is nothing you could do to solve the problem for the standard library that wouldn't require you to repeat the same steps in the next update of the compiler. Not only that but I don't normally build the C++ standard library as part of my build. Boost, I do. Boost is a position to be able to solve the problem in a useful manner without making life overly difficult for developer or user. Sohail
On Fri, Sep 12, 2008 at 6:38 PM, Sohail Somani <sohail@taggedtype.net> wrote:
Emil Dotchevski wrote:
But also, if someone asks you to integrate a statically linked library that used a version of the standard library different from the one you were using, what would you do? :)
But seriously, there are N different standard libraries with multiple versions whereas there is only one Boost with multiple versions. Fixing the problem for Boost solves it on all compilers. There is nothing you could do to solve the problem for the standard library that wouldn't require you to repeat the same steps in the next update of the compiler. Not only that but I don't normally build the C++ standard library as part of my build. Boost, I do.
My point was that besides different Boost versions, there are too many other reasons why a pre-built library won't link with your code. Isn't this type of problem solved by distributing N builds of the library -or- distributing the source code? Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode
Emil Dotchevski wrote:
On Fri, Sep 12, 2008 at 6:38 PM, Sohail Somani <sohail@taggedtype.net> wrote:
But also, if someone asks you to integrate a statically linked library that used a version of the standard library different from the one you were using, what would you do? :) But seriously, there are N different standard libraries with multiple versions whereas there is only one Boost with multiple versions. Fixing
Emil Dotchevski wrote: the problem for Boost solves it on all compilers. There is nothing you could do to solve the problem for the standard library that wouldn't require you to repeat the same steps in the next update of the compiler. Not only that but I don't normally build the C++ standard library as part of my build. Boost, I do.
My point was that besides different Boost versions, there are too many other reasons why a pre-built library won't link with your code. Isn't this type of problem solved by distributing N builds of the library -or- distributing the source code?
Sure, but as I said previously, all other flags matched. It isn't too hard to match up the relevant compiler flags. And even though we know how having the source avoids these problems, it isn't always possible. How would you distribute N builds of the library? Would you create N different builds with N different versions of Boost? I am not sure what you are suggesting here. This is a testing nightmare! Besides, what if you patch Boost? It is quite sensible to build N variants for compilers that have multiple C++ standard libraries or debug/release settings for any given release as that is a known quantity. It is more difficult to build, support and test against different versions of Boost. The funny thing is, it isn't that much incremental work to support it once the ability is there. There just has to be a concious decision to make it so. -- Sohail Somani http://uint32t.blogspot.com
Sohail Somani wrote: [snip]
What if Boost.Config did something like:
#if !defined(BOOST) # define BOOST boost #endif
And we replace all uses of the boost namespace with BOOST?
Would such a patch against trunk be acceptable? What are the problems?
I know it isn't aesthetically pleasing but it might have made this a bit easier.
In case it isn't clear, what I meant by this is that I am willing to do the work to make it possible. -- Sohail Somani http://uint32t.blogspot.com
AMDG Sohail Somani wrote:
Sohail Somani wrote: [snip]
What if Boost.Config did something like:
#if !defined(BOOST) # define BOOST boost #endif
And we replace all uses of the boost namespace with BOOST?
Would such a patch against trunk be acceptable? What are the problems?
I know it isn't aesthetically pleasing but it might have made this a bit easier.
In case it isn't clear, what I meant by this is that I am willing to do the work to make it possible.
In order to make this work we need to make sure that there is a namespace alias where appropriate: #if !defined(BOOST_NAMESPACE) #define BOOST_NAMESPACE boost #endif #define BOOST_NAMESPACE_boost 0 #if BOOST_PP_CAT(BOOST_NAMESPACE_, BOOST_NAMESPACE) namespace BOOST_NAMESPACE {} namespace boost = BOOST_NAMESPACE; #endif #undef BOOST_NAMESPACE_boost In Christ, Steven Watanabe
Steven Watanabe wrote:
AMDG
Sohail Somani wrote:
Sohail Somani wrote: [snip]
What if Boost.Config did something like:
#if !defined(BOOST) # define BOOST boost #endif
And we replace all uses of the boost namespace with BOOST?
Would such a patch against trunk be acceptable? What are the problems?
In order to make this work we need to make sure that there is a namespace alias where appropriate:
#if !defined(BOOST_NAMESPACE) #define BOOST_NAMESPACE boost #endif
#define BOOST_NAMESPACE_boost 0
#if BOOST_PP_CAT(BOOST_NAMESPACE_, BOOST_NAMESPACE) namespace BOOST_NAMESPACE {} namespace boost = BOOST_NAMESPACE; #endif
#undef BOOST_NAMESPACE_boost
Indeed, you are right. Clever too :-) There is one obvious problem I see with the -DBOOST_NAMESPACE=foo approach: there is no way to include two versions of Boost within the same translation unit. Unfortunately, I don't think there would be any acceptable way to do this. Though I'm sure you will come back with a way to do it! By the way, I've received a message on IRC telling me that Xalan and Xerces do something very similar so it wouldn't be totally unprecedented. -- Sohail Somani http://uint32t.blogspot.com
Sohail Somani wrote:
By the way, I've received a message on IRC telling me that Xalan and Xerces do something very similar so it wouldn't be totally unprecedented.
STLport also uses different namespace for encoding the build variant. -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org (msn) - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim,yahoo,skype,efnet,gmail
AMDG Sohail Somani wrote:
There is one obvious problem I see with the -DBOOST_NAMESPACE=foo approach: there is no way to include two versions of Boost within the same translation unit. Unfortunately, I don't think there would be any acceptable way to do this. Though I'm sure you will come back with a way to do it!
There is actually a worse problem. Any solution involving a namespace alias will break code like: namespace boost { namespace fusion { namespace extension { template<> struct begin_impl<...> { ... }; } } } In Christ, Steven Watanabe
On Sat, Sep 13, 2008 at 9:37 AM, Steven Watanabe <watanabesj@gmail.com> wrote:
AMDG Sohail Somani wrote:
There is one obvious problem I see with the -DBOOST_NAMESPACE=foo approach: there is no way to include two versions of Boost within the same translation unit. Unfortunately, I don't think there would be any acceptable way to do this. Though I'm sure you will come back with a way to do it!
There is actually a worse problem. Any solution involving a namespace alias will break code like:
namespace boost { namespace fusion { namespace extension { template<> struct begin_impl<...> { ... }; } } }
It would also break forward declarations found in user code, which are perhaps not officially supported but I've done my share of: namespace boost { template <class> class function; } and others, in my own headers. Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode
Emil Dotchevski wrote:
On Sat, Sep 13, 2008 at 9:37 AM, Steven Watanabe <watanabesj@gmail.com> wrote:
There is actually a worse problem. Any solution involving a namespace alias will break code like:
namespace boost { namespace fusion { namespace extension { template<> struct begin_impl<...> { ... }; } } }
It would also break forward declarations found in user code, which are perhaps not officially supported but I've done my share of:
namespace boost { template <class> class function; }
Ok, aliasing the boost namespace is out. That only leaves the preprocessor unless I'm missing something. Which, unless Boost radically changes the way headers are included or someone finds a conforming workaround, is not possible... Doh. -- Sohail Somani http://uint32t.blogspot.com
Steven Watanabe wrote:
AMDG
Sohail Somani wrote:
There is one obvious problem I see with the -DBOOST_NAMESPACE=foo approach: there is no way to include two versions of Boost within the same translation unit. Unfortunately, I don't think there would be any acceptable way to do this. Though I'm sure you will come back with a way to do it!
There is actually a worse problem. Any solution involving a namespace alias will break code like:
namespace boost { namespace fusion { namespace extension { template<> struct begin_impl<...> { ... }; } } }
Hm, that is definitely an extension point. STLport gets around this problem because they never have to #include INCLUDE something that has "std" in it. Similarly for Xalan and Xerces. So they can just #define std my_special_std. Is there any conforming way to prevent the following from replacing the boost token? #define boost boost_1 #define INCLUDE <boost/foo/bar.hpp> #include INCLUDE Still, I wonder if this is the only problem once solved. -- Sohail Somani http://uint32t.blogspot.com
on Fri Sep 12 2008, Sohail Somani <sohail-AT-taggedtype.net> wrote:
#undef BOOST_NAMESPACE_boost
Indeed, you are right. Clever too :-)
There is one obvious problem I see with the -DBOOST_NAMESPACE=foo approach: there is no way to include two versions of Boost within the same translation unit. Unfortunately, I don't think there would be any acceptable way to do this. Though I'm sure you will come back with a way to do it!
I think you do that by sticking each version of boost in its own namespace, turning off the global namespace alias, and using explicit qualification to select the version in question... If you want to be ambitious, there's also the issue that some libraries (or parts thereof) might want to maintain binary compatibility across releases. Suppose that nothing in Boost.Signals changes between 1.37 and 1.38. Should that generate two sets of symbols with non-interchangeable types? -- Dave Abrahams BoostPro Computing http://www.boostpro.com
David Abrahams wrote:
on Fri Sep 12 2008, Sohail Somani <sohail-AT-taggedtype.net> wrote:
#undef BOOST_NAMESPACE_boost Indeed, you are right. Clever too :-)
There is one obvious problem I see with the -DBOOST_NAMESPACE=foo approach: there is no way to include two versions of Boost within the same translation unit. Unfortunately, I don't think there would be any acceptable way to do this. Though I'm sure you will come back with a way to do it!
I think you do that by sticking each version of boost in its own namespace, turning off the global namespace alias, and using explicit qualification to select the version in question...
This still has the problem that all the Boost source code needs to be changed to use version-specific macros and version specific includes. So a version 1.34 header might look like: #ifndef INCLUDED_BOOST_FUNCTION_HPP_134 ... #include <boost_134/function/detail/base.hpp> // or whatever I think you can guess what the 1.35 header would look like. I personally think that two versions of Boost in the same TU can /only/ be solved in an acceptable way by bcp.
If you want to be ambitious, there's also the issue that some libraries (or parts thereof) might want to maintain binary compatibility across releases. Suppose that nothing in Boost.Signals changes between 1.37 and 1.38. Should that generate two sets of symbols with non-interchangeable types?
Without a totally comprehensive policy for ABI compatibility (good luck with that one, I'm told!), I have to say yes. Ideally, no, but practically yes. -- Sohail Somani http://uint32t.blogspot.com
Sohail Somani skrev:
Steven Watanabe wrote:
AMDG
Sohail Somani wrote:
Sohail Somani wrote: [snip]
What if Boost.Config did something like:
#if !defined(BOOST) # define BOOST boost #endif
And we replace all uses of the boost namespace with BOOST?
Would such a patch against trunk be acceptable? What are the problems?
In order to make this work we need to make sure that there is a namespace alias where appropriate:
#if !defined(BOOST_NAMESPACE) #define BOOST_NAMESPACE boost #endif
#define BOOST_NAMESPACE_boost 0
#if BOOST_PP_CAT(BOOST_NAMESPACE_, BOOST_NAMESPACE) namespace BOOST_NAMESPACE {} namespace boost = BOOST_NAMESPACE; #endif
#undef BOOST_NAMESPACE_boost
Indeed, you are right. Clever too :-)
There is one obvious problem I see with the -DBOOST_NAMESPACE=foo approach: there is no way to include two versions of Boost within the same translation unit. Unfortunately, I don't think there would be any acceptable way to do this. Though I'm sure you will come back with a way to do it!
Even if you solve this for the namespace, you can not safely have side-by-side headers from two boost versions in the same translation unit. You still have the issues of include paths and all the other BOOST_ preprocessor symbol definitions. There are probably other issues as well that is off my radar screen. Namespace variations only solve part of the issues, but if you don't mix boost versions in the same translation units you may be Ok. However, if you need to use boost types in interfaces used across boost versions -- then you need a different solution. The types need adapters, or they need to be aliased and binary compatible.
By the way, I've received a message on IRC telling me that Xalan and Xerces do something very similar so it wouldn't be totally unprecedented
I was never pleased with the limitation of the Xerces method. So, some years ago I did some testing with a different approach to solve this. I used a patched version of the bcp tool to manipulate names of preprocessor symbol names, include paths, and namespace names in a copy boost (or of selected boost libriaries) copied with bcp. I did some promising tests and posted a patch to the list. I think Eric Niebler reported some success in using this approach. -- Bjørn Roald
Bjørn Roald wrote:
Even if you solve this for the namespace, you can not safely have side-by-side headers from two boost versions in the same translation unit. You still have the issues of include paths and all the other BOOST_ preprocessor symbol definitions. There are probably other issues as well that is off my radar screen. Namespace variations only solve part of the issues, but if you don't mix boost versions in the same translation units you may be Ok. However, if you need to use boost types in interfaces used across boost versions -- then you need a different solution. The types need adapters, or they need to be aliased and binary compatible.
I am of the opinion that the best and most portable C++ ABI is C. So I hope the number of occurrences of Boost types in a C++ binary-only interface is close to 0! But yeah, two versions of Boost in one TU is asking for it.
By the way, I've received a message on IRC telling me that Xalan and Xerces do something very similar so it wouldn't be totally unprecedented
I was never pleased with the limitation of the Xerces method. So, some years ago I did some testing with a different approach to solve this. I used a patched version of the bcp tool to manipulate names of preprocessor symbol names, include paths, and namespace names in a copy boost (or of selected boost libriaries) copied with bcp. I did some promising tests and posted a patch to the list. I think Eric Niebler reported some success in using this approach.
I think your bcp patch is overkill just for compiling with a different namespace. However, it is the best solution for multiple versions in a single TU. I don't envy that guy's job though... -- Sohail Somani http://uint32t.blogspot.com
on Fri Sep 12 2008, Sohail Somani <sohail-AT-taggedtype.net> wrote:
Anyway, I thought it might be a good idea to do the same on the larger code base so I attempted to do the same. Failed miserably. The problem is that some of the Boost code contains some variant of:
#if defined(SOME_CONDITION) # define HEADER <boost/some/header.hpp> #else # define HEADER <boost/some/other/header.hpp> #endif
#include HEADER
Reading 16.2/4, it seems that the behaviour in this case is not defined. Specifically, an implementation may or may not replace the boost text with boost_133_1_provider. This is true whether or not you have <boost/some/header.hpp> or "boost/some/header.hpp". So on one compiler, you may end up with:
#include <boost_133_1_provider/some/header.hpp>
And on another:
#include <boost/some/header.hpp>
My immediate problem solved, I made do. However, the problem still bugs me. There should be /some/ way to do this. So...
What if Boost.Config did something like:
#if !defined(BOOST) # define BOOST boost #endif
And we replace all uses of the boost namespace with BOOST?
Would such a patch against trunk be acceptable? What are the problems?
I know it isn't aesthetically pleasing but it might have made this a bit easier.
This is a longstanding issue for C++ libraries in general, and for Boost in particular. I know for a fact there are several companies out there that struggle with it. We already do something like the above for Boost.Signals because of potential conflicts with Qt. I think it's worth taking a look at how Boost.TR1 does its business, and we should probably also be thinking about how to leverage namespace aliases. I don't know if anyone at Boost has taken the time to solve the problem in general, so I'm really glad to see the issue raised. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
David Abrahams wrote:
I don't know if anyone at Boost has taken the time to solve the problem in general, so I'm really glad to see the issue raised.
That may not exactly be the solution you have in mind, but I can't resist pointing out that a somewhat different approach to backward compatibility might have helped alleviating the effects that ultimately cause people to be looking for such alternative solutions. Regards, Stefan -- ...ich hab' noch einen Koffer in Berlin...
on Sat Sep 13 2008, Stefan Seefeld <seefeld-AT-sympatico.ca> wrote:
David Abrahams wrote:
I don't know if anyone at Boost has taken the time to solve the problem in general, so I'm really glad to see the issue raised.
That may not exactly be the solution you have in mind, but I can't resist pointing out that a somewhat different approach to backward compatibility might have helped alleviating the effects that ultimately cause people to be looking for such alternative solutions.
I guess you'd have to be more specific if you want me to understand what you're driving at. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
David Abrahams wrote:
on Sat Sep 13 2008, Stefan Seefeld <seefeld-AT-sympatico.ca> wrote:
David Abrahams wrote:
I don't know if anyone at Boost has taken the time to solve the problem in general, so I'm really glad to see the issue raised.
That may not exactly be the solution you have in mind, but I can't resist pointing out that a somewhat different approach to backward compatibility might have helped alleviating the effects that ultimately cause people to be looking for such alternative solutions.
I guess you'd have to be more specific if you want me to understand what you're driving at.
I figure the main reason people are urged to use multiple boost versions side-by-side is because of API incompatibilities between them. Regards, Stefan -- ...ich hab' noch einen Koffer in Berlin...
Stefan Seefeld wrote:
I figure the main reason people are urged to use multiple boost versions side-by-side is because of API incompatibilities between them.
That and functionality. If my static library is tested against 1.34, I can only support it with 1.34. Changing a library like Boost which is all over the place in some code bases requires a new testing cycle. God help you if your tests miss some edge case that is now relevant after upgrading to 1.35 b/c of a third-party component. -- Sohail Somani http://uint32t.blogspot.com
on Sun Sep 14 2008, Stefan Seefeld <seefeld-AT-sympatico.ca> wrote:
David Abrahams wrote:
on Sat Sep 13 2008, Stefan Seefeld <seefeld-AT-sympatico.ca> wrote:
David Abrahams wrote:
I don't know if anyone at Boost has taken the time to solve the problem in general, so I'm really glad to see the issue raised.
That may not exactly be the solution you have in mind, but I can't resist pointing out that a somewhat different approach to backward compatibility might have helped alleviating the effects that ultimately cause people to be looking for such alternative solutions.
I guess you'd have to be more specific if you want me to understand what you're driving at.
I figure the main reason people are urged to use multiple boost versions side-by-side is because of API incompatibilities between them.
More like ABI incompatibility. Our APIs have been relatively stable, IMO, but if you have precompiled binaries from one vendor that uses 1.34 and from another that uses 1.35, what choice do you have? -- Dave Abrahams BoostPro Computing http://www.boostpro.com
From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of David Abrahams Sent: Monday, September 15, 2008 1:49 PM
on Sun Sep 14 2008, Stefan Seefeld <seefeld-AT-sympatico.ca> wrote:
I figure the main reason people are urged to use multiple boost versions side-by-side is because of API incompatibilities between them.
More like ABI incompatibility. Our APIs have been relatively stable, IMO, but if you have precompiled binaries from one vendor that uses 1.34 and from another that uses 1.35, what choice do you have?
We have a somewhat unique twist on this. We dynamically-link our application against several Boost shared libraries, but we also have a facility for our customers to build plug-ins which can be loaded into our application. The plug-ins take the form of shared libraries. Some customers have run into difficulty using Boost within the plug-ins they build due to version conflicts between the version our application is using and the version the customer is using. This isn't strictly an ABI issue-- Boost classes never cross the boundary of plug-ins. But it has some of the same issues, since loading two different shared libraries which define the same symbols creates a real mess. Putting Boost into a versioned namespace and using namespace aliasing at build time could allow multiple versions to co-exist within a single process. Admittedly our use-case is on the fringe. -Chris
participants (8)
-
Bjørn Roald -
Chris Newbold -
David Abrahams -
Emil Dotchevski -
Rene Rivera -
Sohail Somani -
Stefan Seefeld -
Steven Watanabe