On Boost and modules: second part
Hi all, Following the recent discussion on Boost and C++20 modules, I've been performing further investigations on the possible benefits and costs of offering module support in Boost. I've summarized my findings here: https://anarthal.github.io/cppblog/modules2 TL;DR: * While there isn't too much benefit for clean builds, incremental builds tend to be much faster with modules. * Right now, modularizing requires an effort for each modularized library. * Writing CMake files for module consumption won't be trivial. I'd like to know everyone's opinion on this: do you think these efforts would make sense? In the mid-term, this will require authors to review and merge module-related PRs, at the very least. I'm especially interested in hearing René and Peter's opinions, as maintainers of B2, Boost.CMake and many other core parts (as well as other authors'). If the community decides these efforts may be worthwhile, I volunteer to perform most of the required work. My first step would be to create a working prototype with a modularized Boost.Url (as a maintained, separately compiled library, with non-trivial dependencies), together with CMake tooling, to spot any pitfalls I've missed during my analysis. I'd also like to drag any present MSVC developers' attention on this bug which has nasty consequences on code like Asio: https://developercommunity.visualstudio.com/t/Using-%5f%5ftry-in-an-inline-f... Thanks, Ruben.
[Ruben Perez]
I'd also like to drag any present MSVC developers' attention on this bug which has nasty consequences on code like Asio: https://developercommunity.visualstudio.com/t/Using-__try-in-an-inline-funct...
I've pinged the compiler devs. Thanks, STL
Ruben Perez via Boost
Under MSVC, mixing standard includes and imports is problematic. This is true even if the mix happens in different translation units, as long as they interact via import.
We have tried this with 17.10 preview4 and there is definitely an improvement. Not only this works: #include <iostream> import std; But also something like this (which is trying to test the above- mentioned interaction via import): // a.mxx module; #include <iostream> export module a; export void a (std::ostream& os) {os << "a";} // b.mxx export module b; import std; export void b (std::ostream& os) {os << "b";} // c.mxx export module c; import a; import b; import std; export void c (std::ostream& os) {a (os); b (os);} // main.cxx #include <iostream> import c; int main () {c (std::cout);} This works for all the permutations of the import order in c.mxx. Having said that, when we tried to enable `import std;` in a more realistic project, things did fall apart pretty quickly due to what looks like hard to pin-point interactions between include and import.
* Create a BOOST_XXX_MODULE_EXPORT macro, defined to export if we’re doing modules, empty otherwise.
* Annotate all public entities in our headers with the aforementioned macro.
* Ifdef-out all third-party includes from our headers if we’re doing modules (including standard library and other Boost library includes) - these will be made available with import.
In the early modules days I tried this dual modules/headers approach on a relatively small library. It didn't go well, to put it mildly. The resulting headers/module interfaces got really hairy due to all the macros and ifdef's. One thing I found particularly dizzying (literally) is keeping straight all the imports/includes in the module interface and implementation units. Remember that when you do, for example,`import std;` in the module interface in the module's purview, all the imported names are automatically made visible in the module implementation units without an explicit `import std;`. But that's not the case with headers and you will need to pause and think where you need to include each header. If you are interested to see what it used to look like, here is the commit that ripped all this dual support out: https://github.com/build2/libbutl/commit/df1ef68cd8e85
[John Maddock]
5) Using the regex is an all or nothing affair, with msvc this works: import boost.regex; [...] as does: import std; import boost.regex; [...] But this blows up: #include <iostream> import boost.regex;
[Boris Kolpackov]
We have tried this with 17.10 preview4 and there is definitely an improvement.
Yep, this was recorded in our Changelog. See https://github.com/microsoft/STL/wiki/Changelog#vs-2022-1710-preview-1 (or https://github.com/microsoft/STL/wiki/Changelog#vs-2022-1710 if you're reading this in the near future after 17.10 has released for production): [STL Changelog]
* Wrapped the STL in extern "C++" as a temporary workaround to allow #include <meow> to coexist with import std; in the same translation unit, in that order. #4154 + The other order, import std; before #include <meow>, will still cause compiler errors. We're working on a long-term solution.
[Boris Kolpackov]
Having said that, when we tried to enable `import std;` in a more realistic project, things did fall apart pretty quickly due to what looks like hard to pin-point interactions between include and import.
If you can reduce this to something self-contained that rigorously follows our current implementation restriction of include-before-import, we would be happy to investigate. I know the experience is messy, but it's improving, and the only way it can improve is for eager early adopters (including me; the STL is the compiler's first and closest user) to report the problems they encounter so that "it doesn't work" can be resolved into fine-grained issues that can be fixed - see https://github.com/microsoft/STL/issues/1694 for the laundry list of bugs affecting the STL (which is a small subset of all modules bugs in the compiler that have been fixed) that we've been hammering away at since VS 2019 16.8 when I started looking into getting header units and then named modules working. Thanks, STL
On 18/04/2024 14:48, Ruben Perez via Boost wrote:
Hi all,
Following the recent discussion on Boost and C++20 modules, I've been performing further investigations on the possible benefits and costs of offering module support in Boost. I've summarized my findings here: https://anarthal.github.io/cppblog/modules2
Thank you Ruben for a nice summary. I've also been experimenting with modules in Regex - it's a smaller more confined effort than the one I also started in Math - but even so it's proved to be extremely time consuming and rather frustrating. The current state of play in the draft PR here: https://github.com/boostorg/regex/pull/174 is as follows: 1) It more or less works with latest MSVC and clang. Latest GCC fails hopelessly. 2) Build time improvements are similar to yours, time for a serial build of all the examples is 14s with modules and 38s with #includes. That excludes the module build time. I also had to move the most common template instances into module implementation files (these are hidden behind non-template factory functions) to see the speedup. 3) Tooling support for CI just plain sucks, I ended up writing batch and shell scripts just to test this. 4) It's ever so very flaky. For example I simply could not figure out the correct way to #include third party headers in module implementation units - this is not helped by different tutorials giving different answers to this issue. The only method that clang and msvc seem to agree on is: module; #include "third_Party.hpp" module boost.regex; // note not exported in implementation unit. I'm actually reasonably sure this is not legal code outside of the module definition. 5) Using the regex is an all or nothing affair, with msvc this works: import boost.regex; int main(int argc) { boost::regex e; return 0; } as does: import std; import boost.regex; int main(int argc) { boost::regex e; return 0; } But this blows up: #include <iostream> import boost.regex; int main(int argc) { boost::regex e; return 0; } This makes it literally impossible to port the main test program to test the module because it relies on third party headers which pull in std lib headers (ie other boost headers). I'm probably halting this for now... it's been a somewhat fun experiment, but I feel we need a few more compiler releases and better tooling before this can move forward, oh and a well written tutorial on modules which isn't full of misleading information. John.
participants (4)
-
Boris Kolpackov
-
John Maddock
-
Ruben Perez
-
Stephan T. Lavavej