On Saturday 24 September 2005 01:09, David Abrahams wrote:
Bjørn Roald <bjorn@4roald.org> writes:
I have been working for some time to get acceptance for using Boost libraries as elements in a higher level portable platform for library development. It has surprised me how hard it is to make compelling and attractive set of arguments for using Boost (or other libraries) without attempting to ignore some pretty serious concerns. These concerns are:
- commercial support for our use of Boost
I know of a certain company offering commercial support for boost :)
Yes, I know about Boost Consulting. I did not mean to insinuate that there are no comercial support for Boost, just that it for many managers is important to have this kind of arrangement as part of a risk midigation in relation to open source. I expected you to react on the above, but left it anyway for completness, it also highlights that what you are doing is important. Also in face of licency issues it seems like it is hard for many to trust that you really are allowed to do what you are allowed to do with Boost or similar licences, for those none-beleivers it becommes easier to accept it if they acctually buy a product from somebody. I have no first hand experience in dealing with Boost Consulting, nether do I know anybody that have evaluated the company and their services. So I am in no possition to have an opinion about the quallity of what you can offer. It is certainly a positive sign that the 3 of you are so active on the developers list :-)
- quality assurance of Boost itself
Specifically?
Again, and as I mentioned to John, there are allways room for improvement, but I do think the boost developers do a lot. I bring this up because it is a genuine concern whenever you deal with sub-contractors or whatever, and it also apply to use of open source. Boost and or Boost Consulting could possibly make a more prominent reference to QA efforts and procedures on their web sites. At least have a bullet for it an tie together the procedures you have with peer reviews, testing, regression testing, bug tracking, change management, public developer mailing list. Then you can call this web page and its guidelines your Quallity Assuruance Plan.
- stability of Boost API
Yeah, there are no rules about it. However, we generally avoid breaking source compatibility.
If the library is good enough, this is a risk many will choose to live with. But it would be nice if library maintainers made ther _current_ view of how stable the API was public. Maybe on a scale used for all Boost libraries.
- maintainability of Boost source code
I actually find that most of the time it's pretty easy to dig into a Boost library and fix things,
good
although I seldom have to because the authors tend to be responsive to bug reports.
I have noticed that
However, there are no uniform standards for certain things that make maintenance easier for some people, like code formatting. But you probably had something else in mind?
Well I am pragmatic, we (programmers) will never agree on that stuff anyway ;-)
- deployability of our library into environments using different versions of Boost or libraries used in boost
Hmm, that's a tough one, especially if your library traffics in boost types in its public API, or if you use compilers that don't support symbol visibility control.
yes
- configuration management issues in general for us and our customers
Such as?
mostly a concern if we have to maintain other brances of boost than the library maintainers on boost.org.
- platform support in Boost ( or if the platform support Boost if you like ;-)
Sorry; what do you mean?
Just trying to be funny, sorry... the pun was that if boost is standard conformant, then the platforms are to blame. However, there are often some truth in what I try to joke about.
All of these concerns can be declared acceptable, given that you are willing to accept trade offs and risk. The issue I try to raze is if it is possible to reduce the magnitude of these risks and trade offs.
Good question.
One of the key issues is that you have to decide on some strategy of how to maintain your boost version. The most obvious strategy may be to stay with the latest Boost release at any time, this will however add risk as you have to adopt new changes as they are introduced into Boost and not as it fits your own release plans. The obvious alternative is to stay with a Boost release longer, possibly until you are forced to switch. This strategy will quickly force you into maintaining your own branch as Boost developers can not be expected to support old releases.
A company offering commercial Boost support can support old releases by integrating fixes into older release branches. Whether or not those branches can ever become "official Boost releases" is not clear, but even if they don't, at least you can share a common set of fixes with everyone else who is depending on that old release.
agree
Between these there are many shades of gray which may fit more or less well depending on your project and your customers. If your customers themselves use boost you are likely to meet a added configuration management, testing and release related cost and risk.
As library developers have a very different type of customer relationship than what is the case for end application developers there are a number of concerns that make adaptation of Boost, or any other 3rd party library for that sake, more problematic for library developers. The main issue is to what degree a library depends on other libraries and hence introduces indirect or direct dependencies to its users. These dependencies tend to be troublesome for the library developer to support, as well as they may be troublesome for users to accept. In fact users may easily be in situations where such dependencies are completely unacceptable. To help to understand the issue I like to define categories of libraries which affect the degree they are affected by these concerns.
Cat1. open source, source distributions, allowing encapsulation
Cat2. open source, source distributions in general
Cat3. open source, binary distributions
Cat4. closed source
Cat1 and Cat2 are build-able source code distributions.
I believe there are closed source libraries that are also delivered int the form of buildable source... but that's probably beside the point.
Depend of the definition of closed source, I consider it a Cat3 if it technically is buildable, but you have no licence to modify and then build. Often source code is open for debugging purposes only.
This give the user the ability to build the library and ensure that it is built correctly with other library it depends on, on the target system. Cat1 add to this the ability to hide the use of it inside other libraries, i.e. it can live concurrently in an end system with other versions of the same Cat1 library.
E.g. via symbol isolation or namespace renaming?
That is what we have in the toolbox, at least as far as I am aware of, yes.
Boost is a Cat2 library. Even if the extensive use of headers only libraries help some of the issues addressed in this post, it is not a general solution that makes Boost or part of Boost a Cat1 library.
Now an example showing some dependencies:
MyCustomersApplication
|---- Boost 1.33 | |---- MyLib | |---- Boost 1.32
If MyLib is Cat1 or Cat2 the chances are good that we may be ok. Most likely MyLib will soon adopt the migration to new Boost releases by the actions of the maintainer or by patches from end users. If however MyLib is Cat 3 or 4 we have a problem. This problem is real even if the use of Boost in MyLib is strictly internal. In my view this is an especially bad problem. Even if I do not intend to use Boost in any way in the API of MyLib I still have this issue of incompatibility.
Not if you maintain symbol isolation. Of course, that may not be possible for you.
I do not know how on all platforms I have to deal with.
Ok, back to the example. If MyLib have no intention of using Boost in the API, is there a way?
There are several ways to maintain symbol isolation. On Windows, it's fairly easy: simply use a shared library and don't dlexport anything involving boost. On *nix, recent GCCs can give you control over symbol visibility, so you can play the same kind of game. Otherwise -- and this is inconvenient at best -- you can do it by using dlopen to load a shared library containing the boost symbols, and speak to that library through one or more known entry points. That approach can be followed on Windows as well (using LoadLibrary) if symmetry with *nix turns out to be more appropriate than using the platform facilities.
Use of dlopen/dlsym and the win32 equallents is sertainly possible. Good point -- I will think more about that one. This could be appropriate if internal use of boost can be statically linked into a loadable library, and that module's services can be accessed through simple methods, alternatively through some framework - component hook-up. Interfaces to the module have to be Boost less.
Side Note:
I am aware of symbol encapsulation abilities in windows DLLs and similar new features supported by GCC, however these are not generally supported on all platforms, so I have selected to ignore them. They may however be helpful to many.
Okay, then you're stuck with the "plug-in" approach using dlopen.
yes, and renaming namespaces..
The general idea is that there are no references to symbols in the boost namespace between any MyLib libraries, nor in any header files in or referenced from MyLib API. How can this be achieved? I have concluded that accessing boost trough namespace wrapping header files are unpractical at best. This needs to be designed into all of boost for this to work, and I do not know exactly how. The main challenge is #include <boost/xxxx.hpp> directives which does not refer to the wrapping headers.
Other libraries that allow that sort of thing have all of their headers #include both a prefix AND a suffix header. It would be a big change.
Yes, and I do not like the prospect of that. David, you left out my text about use of bcp to rename the boost namespace here, that is what I wrap up.
So to wrap up:
I do not believe anything prevents me from going along this path if I decide to do so. I am however interested in the boost developers attitude toward this approach, and in particular if this is something you will encourage and/or support. I am willing to work on the tool support for this in boost or to maintain a modified bcp tool if that is more appropriate.
I have mixed feelings about this. One of the things some people complain about when they look at the boost code is that the many workarounds get in the way of understanding. In fact, maintainability was one of the issues you listed at the beginning. So if we put in facilities for namespace renaming, our code would get uglier and less transparent. Maybe that's not a big deal, as we've already stepped off the "workaround cliff," but it is a concern for me.
Ok, I agree with everything you write above, except the part about somebody falling off a cliff. But I feel like you missed one important point I tried to make. What I try to propose is something requiering no change in the way boost source code is written or maintained. I propose tool support for a _production_step_ ensuring namespace renaming in a subset of boost selected with the bcp tool. I would strongly advice all users of this tool support _not_ to maintain boost sources in the form where namespaces are renamed. I think of the original boost code as a model for generation of buildable source code. Kind of like CORBA IDL for idl2cpp compilers. Nowbody with their sences intact would modify the generated C++ binding code, at least not manually. -- Bjørn