Making Boost modular. IMPORTANT! URGENT! <insert-superlative-here>!
Dear fellow Boost authors, As some of you may have heard.. Some of us have been working on making it possible to fully consume Boost in a modular arrangement. First some FAQs about this: Q: What is a "modular arrangement"? A: It's when the libraries can be used, and hence built, without creating the monolithic headers, without needing the root build files, and without needing the libraries to be arranged in the usual root/libs/<name>. Q: Will a modular Boost remove the Boost release? A: No. The collection of libraries is still a single release. See recent discussions about this. Q: Will this change the testing? A: No, unless you want to. You will still be able to test the same non-modular way. But you could also test the modular way. Q: Will this require that we change the current Boost source structure? A: Yes. Unfortunately there is one restriction that adhering to a modular Boost requires. We would not allow sublibs. That is, that we can't support having libraries in root/libs/thing/<library> style locations. All libraries must be single libraries under the root/libs directory. Thankfully there's only a handful of such libraries. The root/libs/numeric/* group of libraries. Q: Why do we want a modular Boost? A: I'm not going to enumerate the reasons here. But let me just say.. That we've wanted a way to reduce, i.e. modularize, Boost since at least 19 years ago. When we added the "--without-<library>" build option to the root build file <https://github.com/boostorg/boost/blame/master/Jamroot#L107>. Now the work part.. Over the past year or so I've been making patches to the Boost build files, in my own repo, that cleans up the many, many, problems that make libraries depend on the current source structure. I've also made changes to B2 to make it easier/possible/automatic to deal with the new modular freedom. The B2 changes were naturally easy to put in since I control the repo and distribution. And the last bit of them shipped with the 1.83 release last Summer. But now I need to start pushing the changes to the many Boost repos to get this finished. A couple of weeks ago I started posting PRs to incrementally, and unobtrusively, make the changes to the Jamfiles. So far I have these: <https://github.com/boostorg/ublas/pull/189> <https://github.com/boostorg/interval/pull/38> I'm not sure how else to say this so.. PLEASE, PLEASE, merge those and future PRs ASAP. -- -- René Ferdinand Rivera Morell -- Don't Assume Anything -- No Supone Nada -- Robot Dreams - http://robot-dreams.net
On 1/14/24 08:41, René Ferdinand Rivera Morell via Boost wrote:
Dear fellow Boost authors,
As some of you may have heard.. Some of us have been working on making it possible to fully consume Boost in a modular arrangement. First some FAQs about this:
Q: What is a "modular arrangement"? A: It's when the libraries can be used, and hence built, without creating the monolithic headers, without needing the root build files, and without needing the libraries to be arranged in the usual root/libs/<name>.
Q: Will a modular Boost remove the Boost release? A: No. The collection of libraries is still a single release. See recent discussions about this.
Q: Will this change the testing? A: No, unless you want to. You will still be able to test the same non-modular way. But you could also test the modular way.
Q: Will this require that we change the current Boost source structure? A: Yes. Unfortunately there is one restriction that adhering to a modular Boost requires. We would not allow sublibs. That is, that we can't support having libraries in root/libs/thing/<library> style locations. All libraries must be single libraries under the root/libs directory. Thankfully there's only a handful of such libraries. The root/libs/numeric/* group of libraries.
Just out of curiosity. Above you're saying that the "modular" Boost will not require the root/libs/<name> structure, and yet the root/libs/thing/<library> arrangement is a problem. Could you elaborate why? Also, how will the "monolithic" Boost work if there is no root/libs/<name> structure? Or is the structure still required for the "monolithic" Boost?
On Sun, Jan 14, 2024 at 4:54 AM Andrey Semashev via Boost <boost@lists.boost.org> wrote:
On 1/14/24 08:41, René Ferdinand Rivera Morell via Boost wrote:
Dear fellow Boost authors,
As some of you may have heard.. Some of us have been working on making it possible to fully consume Boost in a modular arrangement. First some FAQs about this:
Q: What is a "modular arrangement"? A: It's when the libraries can be used, and hence built, without creating the monolithic headers, without needing the root build files, and without needing the libraries to be arranged in the usual root/libs/<name>.
Q: Will a modular Boost remove the Boost release? A: No. The collection of libraries is still a single release. See recent discussions about this.
Q: Will this change the testing? A: No, unless you want to. You will still be able to test the same non-modular way. But you could also test the modular way.
Q: Will this require that we change the current Boost source structure? A: Yes. Unfortunately there is one restriction that adhering to a modular Boost requires. We would not allow sublibs. That is, that we can't support having libraries in root/libs/thing/<library> style locations. All libraries must be single libraries under the root/libs directory. Thankfully there's only a handful of such libraries. The root/libs/numeric/* group of libraries.
Just out of curiosity. Above you're saying that the "modular" Boost will not require the root/libs/<name> structure, and yet the root/libs/thing/<library> arrangement is a problem. Could you elaborate why?
That requirement, which will become obvious in future PRs, comes from supporting an automatic/easy way to discover references to Boost libs once the modular changes to the Jamroot file are done. Okay, that doesn't really say anything useful. Let me clarify.. Currently the Jamroot "includes" each of the Boost libraries by searching the libs/* and libs/*/* locations (doing a glob and loop). That search will no longer exist. Instead the Jamroot file will contain a single "project-search /boost : libs ;". With that a reference to a boost lib (/boost/predef) will search in the "libs" location for it. Now, it's technically possible to instead do a "project-search /boost : libs libs/numeric ;". But if there is no Jamroot, i.e. there's no super-project, and that search declaration is in a user-config.jam (or equivalent) it means communicating that to users. I.e. it's just easier on everyone if we adopt a flat hierarchy.
Also, how will the "monolithic" Boost work if there is no root/libs/<name> structure? Or is the structure still required for the "monolithic" Boost?
The structure is still required for things like testing and documentation building. I.e. for release management. As those scripts do file globbing to figure out what they are doing. -- -- René Ferdinand Rivera Morell -- Don't Assume Anything -- No Supone Nada -- Robot Dreams - http://robot-dreams.net
Q: Will this require that we change the current Boost source structure? A: Yes. Unfortunately there is one restriction that adhering to a modular Boost requires. We would not allow sublibs. That is, that we can't support having libraries in root/libs/thing/<library> style locations. All libraries must be single libraries under the root/libs directory. Thankfully there's only a handful of such libraries. The root/libs/numeric/* group of libraries.
Does referencing other Boost libs in Jamfiles change? Is /boost/test//boost_test (or whatever other lib) still the way? How can I CI test whether my lib adheres to modular Boost practices? Thanks, Ruben.
On Sun, Jan 14, 2024 at 8:39 AM Ruben Perez <rubenperez038@gmail.com> wrote:
Q: Will this require that we change the current Boost source structure? A: Yes. Unfortunately there is one restriction that adhering to a modular Boost requires. We would not allow sublibs. That is, that we can't support having libraries in root/libs/thing/<library> style locations. All libraries must be single libraries under the root/libs directory. Thankfully there's only a handful of such libraries. The root/libs/numeric/* group of libraries.
Does referencing other Boost libs in Jamfiles change? Is /boost/test//boost_test (or whatever other lib) still the way? How can I CI test whether my lib adheres to modular Boost practices?
No (there might be exceptions that I don't remember at the moment). Your example is invalid though. The current reference for that would be something like /boost/test//boost_unit_test_framework (or other targets based on what you are using). -- -- René Ferdinand Rivera Morell -- Don't Assume Anything -- No Supone Nada -- Robot Dreams - http://robot-dreams.net
Does referencing other Boost libs in Jamfiles change? Is /boost/test//boost_test (or whatever other lib) still the way? How can I CI test whether my lib adheres to modular Boost practices?
No (there might be exceptions that I don't remember at the moment). Your example is invalid though. The current reference for that would be something like /boost/test//boost_unit_test_framework (or other targets based on what you are using).
I wrote that from the top of my head. My point was the target structure. Thanks, Ruben.
On Sun, Jan 14, 2024 at 9:02 AM Ruben Perez <rubenperez038@gmail.com> wrote:
Does referencing other Boost libs in Jamfiles change? Is /boost/test//boost_test (or whatever other lib) still the way? How can I CI test whether my lib adheres to modular Boost practices?
No (there might be exceptions that I don't remember at the moment). Your example is invalid though. The current reference for that would be something like /boost/test//boost_unit_test_framework (or other targets based on what you are using).
I wrote that from the top of my head. My point was the target structure.
I suspected that was the case, but wasn't sure. :-) Interestingly though is that technically the references will change. As with the modular setup *all* libraries will need a target to reference when used. And many, many, header only libraries do not have such targets. Adding those targets will be a big chunk of the PRs I will submit. -- -- René Ferdinand Rivera Morell -- Don't Assume Anything -- No Supone Nada -- Robot Dreams - http://robot-dreams.net
PS. For reference here's a GitHb project that tracks the work <https://github.com/users/grafikrobot/projects/1>. -- -- René Ferdinand Rivera Morell -- Don't Assume Anything -- No Supone Nada -- Robot Dreams - http://robot-dreams.net
On 1/14/24 8:11 AM, René Ferdinand Rivera Morell via Boost wrote:
PS. For reference here's a GitHb project that tracks the work <https://github.com/users/grafikrobot/projects/1>.
Very impressive effort!! I'm sure this has turned into a lot more work than it started out to be. Thanks for addressing all my questions in a constructive way in a post packed with useful information. Robert Ramey
René Ferdinand Rivera Morell via Boost <boost@lists.boost.org> writes:
Q: Will this require that we change the current Boost source structure? A: Yes. Unfortunately there is one restriction that adhering to a modular Boost requires. We would not allow sublibs. That is, that we can't support having libraries in root/libs/thing/<library> style locations. All libraries must be single libraries under the root/libs directory. Thankfully there's only a handful of such libraries. The root/libs/numeric/* group of libraries.
This caused us quite a bit of pain when packaging Boost for build2. FWIW, we split numeric/* into four packages: libboost-numeric-conversion/ libboost-numeric-interval/ libboost-numeric-odeint/ libboost-numeric-ublas/ Just to confirm, the changes that you are proposing will be backwards- compatible from the user's perspective, right? In particular, the public header inclusion paths will still be <boost/numeric/**.hpp> rather than, say, <boost/numeric-conversion/*.hpp>, correct?
On Mon, Jan 15, 2024 at 4:48 AM Boris Kolpackov <boris@codesynthesis.com> wrote:
Just to confirm, the changes that you are proposing will be backwards- compatible from the user's perspective, right? In particular, the public header inclusion paths will still be <boost/numeric/**.hpp> rather than, say, <boost/numeric-conversion/*.hpp>, correct?
Correct. -- -- René Ferdinand Rivera Morell -- Don't Assume Anything -- No Supone Nada -- Robot Dreams - http://robot-dreams.net
@Rene Rivera <rene@cppalliance.org> This seems like good information to go in the Boost FAQ. I will assign an Issue to myself to scan this conversation and add the questions and answers to a "Modular Boost" section of the User Guide FAQ - if that makes sense! On Mon, Jan 15, 2024 at 6:12 AM René Ferdinand Rivera Morell via Boost < boost@lists.boost.org> wrote:
On Mon, Jan 15, 2024 at 4:48 AM Boris Kolpackov <boris@codesynthesis.com> wrote:
Just to confirm, the changes that you are proposing will be backwards- compatible from the user's perspective, right? In particular, the public header inclusion paths will still be <boost/numeric/**.hpp> rather than, say, <boost/numeric-conversion/*.hpp>, correct?
Correct.
-- -- René Ferdinand Rivera Morell -- Don't Assume Anything -- No Supone Nada -- Robot Dreams - http://robot-dreams.net
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
participants (7)
-
Andrey Semashev -
Boris Kolpackov -
Peter Turcan -
René Ferdinand Rivera Morell -
Robert Ramey -
Ruben Perez -
Vinnie Falco