On 6/18/2017 12:09 PM, Stefan Seefeld via Boost wrote:
One of Boost's bottlenecks is its monolithic build infrastructure. It would be beneficial for many different use cases (testing, packaging, etc.) to decouple the building of Boost libraries (well, those that require building, see the listing in http://www.boost.org/doc/libs/1_64_0/more/getting_started/unix-variants.html...) such that they may optionally be built stand-alone.
At present, building a Boost library requires the entire (super-)repository to be checked out, and the build logic itself involves traversing the entire source tree, looking for a "Jamroot".
What would be beneficial to many would be a workflow like this:
1) Have a development environment with (some version of) Boost pre-installed (or at least the parts that the library being built depends on).
2) Check out a single Boost repository (e.g., https://github.com/boostorg/python)
3) Invoke a command to build it (if there is anything to build)
4) Invoke a command to test it
5) Invoke a command to install it
6) Invoke a command to package it (optional)
While it's of course already possible to do all the above by adding support for another build infrastructure (point in case: Boost.Python right now uses SCons for this very reason), this means duplication of effort, as Boost as a whole is still built, tested (and even packaged for binary packages) using Boost.Build, meaning I need to maintain two sets of build infrastructure.
This proposal thus has two parts:
1) Make the nested build logic independent of the outer bits, so individual libraries can be built stand-alone (for example, using b2 by adding a `--with-boost` set of options to point to the location of the prerequisite parts of Boost).
2) Define a clear interface the outer build logic will use to invoke the nested build commands.
Note that for my own case above (Boost.Python), providing 2) would be enough, i.e. by having a way for me to "plug in" my SCons commands to build and test Boost.Python could obsolete the existing Boost.Build logic as defined in Boost.Python (as well as the prerequisite parts in Boost.Build's "python" module).
However, as my proposal is *not* actually advocating to move away from Boost.Build, but rather to modularize it, I think 1) is essential to let everyone else (who may not be inclined to use anything other than Boost.Build) to also take advantage of modularization.
While I'd rather avoid delving into the technical details of how this could possibly be implemented (if you really must, *please* do so in a new thread !), let me outline a few use-cases that would be made possible by the above:
* individual projects would be free to switch to their preferred infrastructure, including CMake, SCons, etc., if they so wish
* individual projects could be much more easily developed and contributed to
* individual projects could be much more easily tested, notably in CI environments
* individual projects could be much more easily packaged
All of the above advantages are *huge*, and reflect real-world needs, and the technical issues to solve these are all minor. The question really is whether there is enough will to move into that direction.
I'd be very happy to participate in the work needed to implement this. But first we need to agree that this is where we want to go.
A serious problem to consider, whenever anyone speaks of a modularized Boost where a library can be distributed on its own, is library dependencies. This problem exists for all libraries, not just Boost. But most libraries have their own dependency system, usually based on whatever OS that library is being used, whereas Boost libraries almost always intend to be cross-platform. How should Boost solve this problem is for me the bottleneck of distributing a particular Boost library. When I speak of dependencies I am not just speaking of library X depending on other libraries A, B, and C etc. I am also speaking of library X depending on particular versions of library A, B, C etc. But since the only versioning system Boost has is a single version number for a Boost release, and since their is no way a library can check even that single version number of a Boost release either at compile or run-time, Boost libraries have no way to check versioning of other individual Boost libraries on which a library may depend. If you say, I am going to distribute my library X with particular releases of library A, B, and C etc. with which I know my library X will work correctly you then have end-users of your library who may have multiple copies of libraries X, A, B, and C etc. on their systems, running each set within its own environment, not knowing how to identify each set of libraries, and hoping they can run these things and avoid the well-known shared library hell which has plagued end-users for years. Let's be realistic, this is a real problem which only some sort of Boost individual library versioning system for starters can hope to solve.
Thanks, Stefan