The state of modular Boost

Hello fellow developers, As you might remember many months ago I started asking library maintainers to merge the build (and sometimes source) changes needed to make it possible to consume Boost libraries in a modular way. At this point a good number of libraries have merged those PRs. And there have been a pair of Boost releases containing those changes. But there is still work to do. So far, we have: * 119 PRs merged so far! * 4 PRs needing work before they can be merged. * 1 PR that was merged to master but never merged to develop. * 45 PRs that can be merged immediately. What I'm asking now is for "forgiveness" as it were. The 45 PRs above are either PRs where the authors gave permission to the Boost github owners to review and merge. Or are for libraries that are effectively unmaintained (the authors did not respond to the PR or to the various requests for how to manage the changes, and haven't seen changes in them for some years). And the recent changes I made to the PRs was not with the build changes. But instead to fix the unmaintained CI of the libraries as they otherwise just did not work. What I want it for the github org owners to go ahead and merge these 45 PRs: https://github.com/users/grafikrobot/projects/1/views/7 Thank you. -- -- René Ferdinand Rivera Morell -- Don't Assume Anything -- No Supongas Nada -- Robot Dreams - http://robot-dreams.net

On 14/04/2025 14:53, René Ferdinand Rivera Morell via Boost wrote:
Hello fellow developers,
As you might remember many months ago I started asking library maintainers to merge the build (and sometimes source) changes needed to make it possible to consume Boost libraries in a modular way. At this point a good number of libraries have merged those PRs. And there have been a pair of Boost releases containing those changes. But there is still work to do. So far, we have:
* 119 PRs merged so far! * 4 PRs needing work before they can be merged. * 1 PR that was merged to master but never merged to develop. * 45 PRs that can be merged immediately.
First off, thank you for this effort, we really DO all appreciate this, however I do have some questions and concerns: 1) Documentation: recently CI for Boost.Math completely broke and despite 3 or us looking at it, we couldn't figure out why. Commenting out random stuff, I found that these project requirements were causing the issue: project : requirements <library>/boost/math//boost_math <library>/boost/multiprecision//boost_multiprecision <library>/boost/test//included <library>/boost/type_index//boost_type_index <library>/boost/ublas//boost_ublas If I removed them then everything started working again, so assuming that they were ancient cruft that was no longer needed I removed them. Woohoo, CI works again and we can get on with fixing some bugs, except now I realize these were part of the modular boost support code, so *probably* I've broken something else somewhere else, but I have no idea what. Adding some boilerplate comments: project : requirements # modular boost dependency list, please edit with care! <library>/boost/math//boost_math <library>/boost/multiprecision//boost_multiprecision <library>/boost/test//included <library>/boost/type_index//boost_type_index <library>/boost/ublas//boost_ublas # end dependency list would have helped a lot. 2) Some of these dependencies appear to be added somewhat sledgehammer fashion: digging further the cause of the breakage was <library>/boost/test//included which was causing Boost.Exception *the library* to be built for every single target in the Jamfile, irrespective of whether it uses it or not (not one of them does, nor does Boost.Test I think), and for every single build configuration too. It was an <exception-handling>off test target that then caused Boost.exception to built with exceptions off which lead to a hard error. 3) There are other heavyweight dependencies, for example <library>/boost/math//boost_math is a binary target, if a testing target depends on it, then *it's already listed in the targets source* same for all other binaries. Why do these need to be global project requirements? 4) This is terribly fragile, I had last week earmarked for squashing some bugs and pretty much all I've done is track down why CI was failing, first in Math then in TypeTraits and Config. It's not all your fault: deprecated test runners were a bit part of it, but it is awfully frustrating. More to the point, as both the type_traits and math failures showed, random changes to other libraries modular-dependency lists can cause completely inscrutable failures downstream. I can't help feeling that we need some tests for this modular code too - they may well exist already, but not within each libraries own test suite. Have I broken Math's modular support with the changes I made? Genuinely I have no idea, and don't know where to look to find out. I'm sorry if this comes across as overly negative, I don't want to stop you from modularising Boost, quite the reverse, and some of this may well be teething problems, but hopefully by raising this now, we can find a way to make this more robust and less inscrutable. Thanks, John.

John Maddock wrote:
First off, thank you for this effort, we really DO all appreciate this, however I do have some questions and concerns:
1) Documentation: recently CI for Boost.Math completely broke and despite 3 or us looking at it, we couldn't figure out why. Commenting out random stuff, I found that these project requirements were causing the issue:
project : requirements <library>/boost/math//boost_math <library>/boost/multiprecision//boost_multiprecision <library>/boost/test//included <library>/boost/type_index//boost_type_index <library>/boost/ublas//boost_ublas
If I removed them then everything started working again, so assuming that they were ancient cruft that was no longer needed I removed them. Woohoo, CI works again and we can get on with fixing some bugs, except now I realize these were part of the modular boost support code, so *probably* I've broken something else somewhere else, but I have no idea what. Adding some boilerplate comments: ... would have helped a lot.
Listing dependencies in project requirements is a cheat and generally not what one wants in any non-trivial Jamfile (same goes for build.jam). In CMake parlance, which is what more people speak today than b2, this corresponds to using link_libraries instead of target_link_libraries. Unfortunately we have a lot of libraries containing those now. I've been fixing them in the PRs I merged, but I can't be expected to go over everything else.

ср, 16 апр. 2025 г. в 14:14, Peter Dimov via Boost <boost@lists.boost.org>:
Listing dependencies in project requirements is a cheat and generally not what one wants in any non-trivial Jamfile (same goes for build.jam). In CMake parlance, which is what more people speak today than b2, this corresponds to using link_libraries instead of target_link_libraries.
I think that it may be useful to explain what the "modular changes" actually do. First, why build.jam? That file is a project jamfile that can act as both the root of the project tree and not the root. It in principle allows the project to be built both as part of the Boost superproject and without it. In addition, Boost superproject checks if that file exists to determine if that particular library has been modularised or not. Inside that file we can see something like this: constant boost_dependencies : /boost/assert//boost_assert /boost/config//boost_config /boost/core//boost_core /boost/static_assert//boost_static_assert /boost/system//boost_system ; This constant is not actually necessary, but it is very helpful to **third party tools* that attempt to determine interdependencies between Boost libraries. For that reason it is better to keep each dependency on a separate line. These dependencies should be the direct dependencies of the **libraries** from the project, including header-only libraries. If you need extra dependencies for tests, examples, benchmarks, etc. put them somewhere else. Next we have the declaration of the project and "public" targets, which are intended for other projects to depend on. E.g. project /boost/mp11 ; The project declaration should be self-explanatory. Note that it explicitly positions itself as /boost/mp11, because otherwise it would have become /boost/libs/mp11 as that's the path from the Boost superproject root. alias boost_mp11 : : : : <include>include <library>$(boost_dependencies) ; This is a target for a header-only library. We used to not need those, because effectively **every library implicitly depends on every other library's headers via /boost/headers target**. But in a modularised setup there is no superproject, so targets should be explicit about their dependencies, including header-only dependencies. The actual utility that the target provides is having <include>include in its usage requirements and also in forwarding its dependencies to its consumer (target that depends on it). I would actually amend this to be alias boost_mp11 : : : : <include>include <library>$(boost_dependencies)//<warnings-as-errors>off ; This makes sure that warnings emitted when building dependencies do not fail your builds. Some projects had a problem when they had something akin to lib A : $(srcs) : <library>B/<warnings-as-errors>off <library>B ; Such a setup may or may not cause the build to fail depending on whether b2 will first update B/<warnings-as-errors>off or B. I recommend to simply always add /<warnings-as-errors>off to targets from other projects. Let's look now at a target for a compiled library. We usually declare those in a build subproject. Hence in build.jam we put an alias to that target: alias boost_json : build//boost_json ; And in the build subproject we add things similarly to the header-only library. Here's a (simplified) declaration from Boost.JSON: lib boost_json : src.cpp : <include>../include <use>$(boost_dependencies)/<warnings-as-errors>off <link>shared:<define>BOOST_JSON_DYN_LINK=1 <link>static:<define>BOOST_JSON_STATIC_LINK=1 : : <use>$(boost_dependencies)/<warnings-as-errors>off <library>/boost/container//boost_container/<warnings-as-errors>off ; The target puts its dependencies both into requirements and usage requirements. I used <use> feature, because most of the dependencies are actually header-only, and they only add usage requirements. The one link dependency is repeated with <library>. Finally call-if : boost-library json : install boost_json ; calls boost-install boost_json if part of the superproject.
participants (4)
-
John Maddock
-
Peter Dimov
-
René Ferdinand Rivera Morell
-
Дмитрий Архипов