Using Boost.Test header-only from CMake
Some libraries (I encountered this when porting the tests of Signals2) use Boost.Test in header-only mode, by including <boost/test/included/unit_test.hpp> without linking to anything. This worked in the "legacy" structure where all headers are always available, but doesn't work under CMake where you have to link to header-only libraries in order to get the proper include path. Should we make an attempt to support this use by declaring a special header-only target in Boost.Test (e.g. Boost::included_unit_test), or is that a waste of time? One can as well link to the proper Boost::unit_test_framework (and then even use it header-only, although there's not much point in doing so.) I'm inclined towards "waste of time".
On Sun, Jun 6, 2021 at 9:12 PM Peter Dimov via Boost <boost@lists.boost.org> wrote:
Some libraries (I encountered this when porting the tests of Signals2) use Boost.Test in header-only mode, by including <boost/test/included/unit_test.hpp> without linking to anything. This worked in the "legacy" structure where all headers are always available, but doesn't work under CMake where you have to link to header-only libraries in order to get the proper include path.
Should we make an attempt to support this use by declaring a special header-only target in Boost.Test (e.g. Boost::included_unit_test), or is that a waste of time? One can as well link to the proper Boost::unit_test_framework (and then even use it header-only, although there's not much point in doing so.)
I'm inclined towards "waste of time".
Likewise. What is the list of libraries that do this? I'm curious if most of them are unmaintained or would otherwise be better served migrating to LWT. Glen
Glen Fernandes wrote:
On Sun, Jun 6, 2021 at 9:12 PM Peter Dimov via Boost <boost@lists.boost.org> wrote:
Some libraries (I encountered this when porting the tests of Signals2) use Boost.Test in header-only mode, by including <boost/test/included/unit_test.hpp> without linking to anything. This worked in the "legacy" structure where all headers are always available, but doesn't work under CMake where you have to link to header-only libraries in order to get the proper include path.
Should we make an attempt to support this use by declaring a special header-only target in Boost.Test (e.g. Boost::included_unit_test), or is that a waste of time? One can as well link to the proper Boost::unit_test_framework (and then even use it header-only, although there's not much point in doing so.)
I'm inclined towards "waste of time".
Likewise.
What is the list of libraries that do this? I'm curious if most of them are unmaintained or would otherwise be better served migrating to LWT.
The libraries that use the "included/" versions of the Test libraries are actually the maintained ones, because the unmaintained ones are still on the deprecated <boost/test/minimal.hpp>. :-) The `included` ones are algorithm filesystem geometry heap lockfree math mpi numeric/conversion numeric/odeint numeric/ublas process ptr_container random range signals2 sort type_traits uuid yap so this is widely used, and in fairness, it's a legitimate and documented way to use Boost.Test. I don't think we need to change anything in the libraries, but I also don't think we need to duplicate the Boost.Test CMake targets to make them have header-only versions. Linking to the normal target will also enable the header-only use to compile. For completeness, the libraries using the deprecated boost/test/minimal.hpp are: foreach graph graph_parallel iterator logic numeric/conversion numeric/interval property_map tokenizer units yap These need to migrate to either LWT or the non-deprecated Boost.Test.
On Sun, Jun 6, 2021 at 9:51 PM Peter Dimov <pdimov@gmail.com> wrote:
Glen Fernandes wrote:
What is the list of libraries that do this? I'm curious if most of them are unmaintained or would otherwise be better served migrating to LWT.
The libraries that use the "included/" versions of the Test libraries are actually the maintained ones, because the unmaintained ones are still on the deprecated <boost/test/minimal.hpp>. :-)
For completeness, the libraries using the deprecated boost/test/minimal.hpp are:
foreach graph graph_parallel iterator logic numeric/conversion numeric/interval property_map tokenizer units yap
These need to migrate to either LWT or the non-deprecated Boost.Test.
I've migrated Yap, Tokenizer, Iterator, and Logic to LWT. Almost finished migrating Units to LWT too, and will do the others afterwards. Glen
On 6/6/2021 9:11 PM, Peter Dimov via Boost wrote:
Some libraries (I encountered this when porting the tests of Signals2) use Boost.Test in header-only mode, by including <boost/test/included/unit_test.hpp> without linking to anything. This worked in the "legacy" structure where all headers are always available, but doesn't work under CMake where you have to link to header-only libraries in order to get the proper include path.
Should we make an attempt to support this use by declaring a special header-only target in Boost.Test (e.g. Boost::included_unit_test), or is that a waste of time? One can as well link to the proper Boost::unit_test_framework (and then even use it header-only, although there's not much point in doing so.)
I'm inclined towards "waste of time".
Since it is totally legitimate to use Boost.Test in header only mode, and many libraries currently do this, it is hardly a "waste of time" to support this under CMake.
Edward Diener wrote:
Since it is totally legitimate to use Boost.Test in header only mode, and many libraries currently do this, it is hardly a "waste of time" to support this under CMake.
Under CMake you can't use header-only libraries without linking to their targets. So there's no real benefit from having a separate header-only target to link to; you can just link to the ordinary target Boost::unit_test_framework.
On 07/06/2021 17:26, Peter Dimov via Boost wrote:
Edward Diener wrote:
Since it is totally legitimate to use Boost.Test in header only mode, and many libraries currently do this, it is hardly a "waste of time" to support this under CMake. Under CMake you can't use header-only libraries without linking to their targets. So there's no real benefit from having a separate header-only target to link to; you can just link to the ordinary target Boost::unit_test_framework.
Doesn't that a) cause the unit test lib to be build when it doesn't need to be, and b) add the resulting binary to library link list when it doesn't need to be? But I do appreciate the simplicity of a single target ;) John. -- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus
On 6/7/21 7:48 PM, John Maddock via Boost wrote:
On 07/06/2021 17:26, Peter Dimov via Boost wrote:
Since it is totally legitimate to use Boost.Test in header only mode, and many libraries currently do this, it is hardly a "waste of time" to support this under CMake. Under CMake you can't use header-only libraries without linking to
Edward Diener wrote: their targets. So there's no real benefit from having a separate header-only target to link to; you can just link to the ordinary target Boost::unit_test_framework.
Doesn't that a) cause the unit test lib to be build when it doesn't need to be, and b) add the resulting binary to library link list when it doesn't need to be?
But I do appreciate the simplicity of a single target ;)
Another question is whether unnecessarily linking would cause duplicate symbols or ODR issues. I believe, you're not supposed to use header-only Boost.Test and then link against it, too, as the same symbols would be defined both in the test and the library. Boost.Test could define two sets of targets - one for header-only and the other for separately built library. One benefit of that is that you could use different macros for these two targets, if needed (e.g. if the library sources need to discriminate between the two configs).
Andrey Semashev wrote:
On 6/7/21 7:48 PM, John Maddock via Boost wrote:
On 07/06/2021 17:26, Peter Dimov via Boost wrote:
Edward Diener wrote:
Since it is totally legitimate to use Boost.Test in header only mode, and many libraries currently do this, it is hardly a "waste of time" to support this under CMake.
Under CMake you can't use header-only libraries without linking to their targets. So there's no real benefit from having a separate header-only target to link to; you can just link to the ordinary target Boost::unit_test_framework.
Doesn't that a) cause the unit test lib to be build when it doesn't need to be, and b) add the resulting binary to library link list when it doesn't need to be?
But I do appreciate the simplicity of a single target ;)
Another question is whether unnecessarily linking would cause duplicate symbols or ODR issues. I believe, you're not supposed to use header-only Boost.Test and then link against it, too, as the same symbols would be defined both in the test and the library.
It works because the symbols in the executable override those in libraries, but it's indeed not quite right in principle.
Boost.Test could define two sets of targets - one for header-only and the other for separately built library. One benefit of that is that you could use different macros for these two targets, if needed (e.g. if the library sources need to discriminate between the two configs).
That's probably what we'll have to do, yes. Any input from the Boost.Test maintainer? Raffi?
On 07.06.21 19:27, Peter Dimov via Boost wrote:
Andrey Semashev wrote:
On 6/7/21 7:48 PM, John Maddock via Boost wrote:
On 07/06/2021 17:26, Peter Dimov via Boost wrote:
Edward Diener wrote:
Since it is totally legitimate to use Boost.Test in header only mode, and many libraries currently do this, it is hardly a "waste of time" to support this under CMake.
Under CMake you can't use header-only libraries without linking to their targets. So there's no real benefit from having a separate header-only target to link to; you can just link to the ordinary target Boost::unit_test_framework.
Doesn't that a) cause the unit test lib to be build when it doesn't need to be, and b) add the resulting binary to library link list when it doesn't need to be?
But I do appreciate the simplicity of a single target ;)
Another question is whether unnecessarily linking would cause duplicate symbols or ODR issues. I believe, you're not supposed to use header-only Boost.Test and then link against it, too, as the same symbols would be defined both in the test and the library.
It works because the symbols in the executable override those in libraries, but it's indeed not quite right in principle.
Hi all, I am skeptical about this: * the overriding behaviour sounds to me system dependent, * we may duplicate some of the singletons, * some compilation information may be different between the build time and the consumption time of the library. The 2 steps build (library generation with one set of options, header only consumption with potentially another set of options) may create difficult to debug situations. I would avoid this.
Boost.Test could define two sets of targets - one for header-only and the other for separately built library. One benefit of that is that you could use different macros for these two targets, if needed (e.g. if the library sources need to discriminate between the two configs).
That's probably what we'll have to do, yes.
Any input from the Boost.Test maintainer? Raffi?
I believe the correct way to do that in the CMake world is to use an INTERFACE library, as it carries the include folder, compiler options, defines etc together with the target cmake object, without actually building anything, and that is what we need for header only: https://cmake.org/cmake/help/latest/manual/cmake-buildsystem.7.html#interfac... I have not looked at the recent developments for CMake in Boost. If the CMakeLists.txt I developed (under build/) is still there, it contained 2 targets: one for static and one for shared. Those were having different and explicit names. Having a 3rd target for header only sounds good to me. I can give a go if you want. Raffi PS.: for those hitting the deprecated API... it's been deprecated since almost a decade now :)
Raffi Enficiaud wrote:
I have not looked at the recent developments for CMake in Boost. If the CMakeLists.txt I developed (under build/) is still there, it contained 2 targets: one for static and one for shared. Those were having different and explicit names.
I decided to not do this and instead use the "normal" CMake way of relying on BUILD_SHARED_LIBS. It causes problems in the more complicated cases where a compiled Boost::X links to header-only Boost::Y which links to a compiled Boost::Z. Client code can choose to link explicitly to Boost::static_X, and if X used Z directly, it could link to Boost::static_Z, but it doesn't. And requiring all header-only libraries to also provide triplicate targets so that static/shared-ness can be propagated correctly is (I think) infeasible. So X links to Boost::Y, and Y has to choose whether to link to the static or shared Z, but it doesn't know how. (b2 does this "right" by having static/shared as a build feature, and propagating it automatically. No need to split your targets by hand.) Swimming upstream is sometimes justified when it results in something objectively better than the accepted CMake way, but I don't believe that's the case here, so BUILD_SHARED_LIBS will have to do. (Similarly, I tried to do something b2-like with the tests, but at the end reverting to the way CMake wants to do things proved better.) Getting back to Boost.Test, I've duplicated the targets with an included_ prefix: https://github.com/boostorg/test/commit/bce2d24c8b32f47f0403766fe4fee3e2e93a...
On 6/7/2021 7:58 PM, Peter Dimov via Boost wrote:
Raffi Enficiaud wrote:
I have not looked at the recent developments for CMake in Boost. If the CMakeLists.txt I developed (under build/) is still there, it contained 2 targets: one for static and one for shared. Those were having different and explicit names.
I decided to not do this and instead use the "normal" CMake way of relying on BUILD_SHARED_LIBS.
It causes problems in the more complicated cases where a compiled Boost::X links to header-only Boost::Y which links to a compiled Boost::Z. Client code can choose to link explicitly to Boost::static_X, and if X used Z directly, it could link to Boost::static_Z, but it doesn't. And requiring all header-only libraries to also provide triplicate targets so that static/shared-ness can be propagated correctly is (I think) infeasible. So X links to Boost::Y, and Y has to choose whether to link to the static or shared Z, but it doesn't know how.
I do not know CMake, so maybe my comment is irrelevant, but it seems natural to me that a header-only library would always choose to use some other dependent library as header-only, even when that other library had static or shared variants. In other words I applaud the decision of Boost.Test to provide a header-only variant and can not even begin to process the fact that CMake can not deal with Boost.Test as a header-only library, if that is indeed the case. If you are saying that the use of Boost.Test within Boost's creation of CMakeLists.txt for its libraries can never be as a header-only only library, please reconsider. If you are not saying that I apologize for my noise.
On 8/06/2021 4:53 pm, Edward Diener wrote:
I do not know CMake, so maybe my comment is irrelevant, but it seems natural to me that a header-only library would always choose to use some other dependent library as header-only, even when that other library had static or shared variants. In other words I applaud the decision of Boost.Test to provide a header-only variant and can not even begin to process the fact that CMake can not deal with Boost.Test as a header-only library, if that is indeed the case.
A problem occurs when program P uses library A as static or shared and then wants to use header-only library B which also uses library A. For this to work, one of the following must be true: 1. library B can figure out how program P chose to link to it and links to it the exact same way. 2. library A is written such that in header-only mode it uses entirely separate namespaces from its compiled modes; then B can link to it in header-only mode without conflicting with other uses. #2 is rarely true, because it's more work for both users and maintainers (inline namespaces make it actually *feasible*, but still require people to actually have considered the problem in the first place). #1 is putting the cart before the horse, which can be problematic (but nevertheless, is the "solution" most often taken, by "bubbling down" settings from the parent projects) -- but it only works as long as everyone respects them and everything gets compiled with matching settings. (#1 also affect dependencies on the CRT itself, which is why most have big global hammers to specify whether linking to the CRT statically or dynamically, and why everything explodes when these settings don't match between compilation units.) When these conditions are not met, you end up with an ODR violation. Which often works anyway by coincidence, especially in libraries without singletons, but you're tap-dancing over a minefield. (And even doing #2 will land you with disjoint singleton "islands", but at least that way it's presumably deliberate.) (The safest way out of this minefield is to ensure that header-only libraries either have no separate dependencies, or at least only ever reference dependencies that do not have a compiled variant. This is of course a scaling problem.) TLDR: C++ is extremely ODR-prone and libraries are very fragile things.
Mere moments ago, quoth I:
TLDR: C++ is extremely ODR-prone and libraries are very fragile things.
Another way of putting all this is that libraries that can be configured between static/shared are mostly fine these days, because the tools have some blunt hammers that mostly do the right things with them. (But still bite people on occasion.) But libraries that can be configured between compiled and header-only are inherently a "here be dragons" and unless done *really* carefully (or sufficiently small that they can exist without dependencies of their own), are best avoided. Pick one of header-only or compiled, and commit to it. Don't try to hedge it to configuration, as this will only lead to pain and suffering. And don't try to be header-only unless all of your dependencies are also header-only, or you're *really darn sure* that the build tools can unify the static/shared configuration across the whole program.
On 8/06/2021 7:42 pm, I wrote:
Another way of putting all this is that libraries that can be configured between static/shared are mostly fine these days, because the tools have some blunt hammers that mostly do the right things with them. (But still bite people on occasion.)
FWIW, the most common biting issue is when mixing static/shared. All-static and all-shared are usually pretty safe. But when libraries A and B both use dependency C, and either of A and B is shared, then C must be shared as well -- things will often break horribly if C is static. (Having said that, what actually happens is highly dependent on whether using Windows or something UNIXy, and for the latter what symbol visibility level is being used. And for some libraries and some platforms/settings then there will be no apparent problems.) But this sort of thing is usually why tools like CMake prefer to offer global settings to configure static/shared rather than letting it be configured on a target-by-target basis. You *can* override that, but it is at your own peril. (Or that of the downstream app.)
FWIW, the most common biting issue is when mixing static/shared.
All-static and all-shared are usually pretty safe.
But when libraries A and B both use dependency C, and either of A and B is shared, then C must be shared as well -- things will often break horribly if C is static.
(Having said that, what actually happens is highly dependent on whether using Windows or something UNIXy, and for the latter what symbol visibility level is being used. And for some libraries and some platforms/settings then there will be no apparent problems.)
But this sort of thing is usually why tools like CMake prefer to offer global settings to configure static/shared rather than letting it be configured on a target-by-target basis. You *can* override that, but it is at your own peril. (Or that of the downstream app.)
A header-only library cannot *link* to anything, or it is not
Yes, that is why I think providing foo::shared and foo::static is usually(!) an anti-pattern. You should just have foo (i.e. Boost::foo) and let CMake decide what that should be based on BUILD_SHARED (the one variable whose naming is bad) Of course this still allows to build your Boost::foo explicitely as static or shared in which case the problem is also avoided as then only one version of it exists anyway. header-only. Depends on what you mean by "link". In CMake-terms this includes "depends on", which IMO is correct. So e.g. a header-only threading library will depend on pthreads, so it will link to pthreads. Of course you could omit linking to pthread at the cost of the user having to find out on what this "header-only" library depends on and having to link all those manually.
On 11/06/2021 6:55 pm, Alexander Grund wrote:
Of course this still allows to build your Boost::foo explicitely as static or shared in which case the problem is also avoided as then only one version of it exists anyway.
No, if you choose to build Boost::foo as static and it happens to be used by at least two different consumers (at least one of which is shared) that end up as part of a single final application, that's exactly the problem case that I was talking about. This is because that can end up with multiple copies of the library in the same application (but in separate binaries), subject to the whims of the platform and linker.
Am 11.06.21 um 09:33 schrieb Gavin Lambert via Boost:
On 11/06/2021 6:55 pm, Alexander Grund wrote:
Of course this still allows to build your Boost::foo explicitely as static or shared in which case the problem is also avoided as then only one version of it exists anyway.
No, if you choose to build Boost::foo as static and it happens to be used by at least two different consumers (at least one of which is shared) that end up as part of a single final application, that's exactly the problem case that I was talking about.
This is because that can end up with multiple copies of the library in the same application (but in separate binaries), subject to the whims of the platform and linker.
I meant the case where Boost::foo is ONLY ever build as static. I.e. `add_library(Boost::Foo STATIC source.cpp)` as opposed to `add_library(Boost::Foo source.cpp)` in CMake terms. So there is only static Boost::Foo and no shared version EVER. This would work, wouldn't it?
On 11/06/2021 7:52 pm, Alexander Grund wrote:
I meant the case where Boost::foo is ONLY ever build as static. I.e. `add_library(Boost::Foo STATIC source.cpp)` as opposed to `add_library(Boost::Foo source.cpp)` in CMake terms.
So there is only static Boost::Foo and no shared version EVER. This would work, wouldn't it?
Only if the library doesn't have any singletons, or otherwise can cope with multiple independent "islands".
On 08.06.21 09:33, Gavin Lambert via Boost wrote:
On 8/06/2021 4:53 pm, Edward Diener wrote:
I do not know CMake, so maybe my comment is irrelevant, but it seems natural to me that a header-only library would always choose to use some other dependent library as header-only, even when that other library had static or shared variants. In other words I applaud the decision of Boost.Test to provide a header-only variant and can not even begin to process the fact that CMake can not deal with Boost.Test as a header-only library, if that is indeed the case.
A problem occurs when program P uses library A as static or shared and then wants to use header-only library B which also uses library A.
For this to work, one of the following must be true: 1. library B can figure out how program P chose to link to it and links to it the exact same way.
A header-only library cannot *link* to anything, or it is not header-only. A header-only library B can depend on symbols being provided by the calling program P, which can delegate the task to library A, or to another library C which provides the same interface as A. There should be no direct connection from B to A. Depending on the design of B, this may involve #including a header from A before #including a header from B, or it can involve making the headers from A available to B so that B can #include them directly. Where these symbols and/or headers come from is the business of P, not of B. -- Rainer Deyke (rainerd@eldwood.com)
On 6/9/21 2:05 PM, Rainer Deyke via Boost wrote:
On 08.06.21 09:33, Gavin Lambert via Boost wrote:
On 8/06/2021 4:53 pm, Edward Diener wrote:
I do not know CMake, so maybe my comment is irrelevant, but it seems natural to me that a header-only library would always choose to use some other dependent library as header-only, even when that other library had static or shared variants. In other words I applaud the decision of Boost.Test to provide a header-only variant and can not even begin to process the fact that CMake can not deal with Boost.Test as a header-only library, if that is indeed the case.
A problem occurs when program P uses library A as static or shared and then wants to use header-only library B which also uses library A.
For this to work, one of the following must be true: 1. library B can figure out how program P chose to link to it and links to it the exact same way.
A header-only library cannot *link* to anything, or it is not header-only.
CMake defines targets for header-only libraries, which can have dependencies on other targets, including on static or shared libraries. So the program that "links" to the header-only library picks up its dependencies recursively. As a result, the linker is invoked with all (binary) libraries in the dependency tree. Whether you can consider such library header-only in the first place is a philosophical question. I do, because otherwise any library that calls the standard library cannot be called header-only.
A header-only library B can depend on symbols being provided by the calling program P, which can delegate the task to library A, or to another library C which provides the same interface as A. There should be no direct connection from B to A. Depending on the design of B, this may involve #including a header from A before #including a header from B, or it can involve making the headers from A available to B so that B can #include them directly. Where these symbols and/or headers come from is the business of P, not of B.
Reverse dependencies are rather uncommon, and it's not what is being discussed here. The question is how the header-only library selects the targets to depend on. I think, it should not select any specific target unless it absolutely must (i.e. it won't work otherwise). In terms of the earlier example, the library B should depend on the generic target A. Now, that target A can be defined as a header-only, shared or static variant, depending on the user's choice or some default.
On 10.06.21 23:36, Andrey Semashev via Boost wrote:
On 6/9/21 2:05 PM, Rainer Deyke via Boost wrote:
On 08.06.21 09:33, Gavin Lambert via Boost wrote:
For this to work, one of the following must be true: 1. library B can figure out how program P chose to link to it and links to it the exact same way.
A header-only library cannot *link* to anything, or it is not header-only.
CMake defines targets for header-only libraries, which can have dependencies on other targets, including on static or shared libraries. So the program that "links" to the header-only library picks up its dependencies recursively. As a result, the linker is invoked with all (binary) libraries in the dependency tree.
If this functionality works the way I think does, then I think it should not be used.
Whether you can consider such library header-only in the first place is a philosophical question. I do, because otherwise any library that calls the standard library cannot be called header-only.
Actually, the standard library is a great example of what I am talking about. A header-only library (B) should not care about the specific implementation of the standard library (A) being used by the program (P). There is no link dependency from B to A. There is a requirement from B to P that P must provide /a/ implementation of the C++ standard library, but B doesn't care if it is libstdc++ or libc++ or even a custom standard library implementation that is part of P and not a separate library at all. Again, to make my point absolutely clear: header-only library B "using" library A does not imply a dependency from B to A. Instead, it implies a requirement on a program P that uses B to also link to A. Header-only libraries use interfaces. Programs link to implementations. -- Rainer Deyke (rainerd@eldwood.com)
Am 11.06.21 um 11:17 schrieb Rainer Deyke via Boost:
On 10.06.21 23:36, Andrey Semashev via Boost wrote:
On 6/9/21 2:05 PM, Rainer Deyke via Boost wrote:
On 08.06.21 09:33, Gavin Lambert via Boost wrote:
For this to work, one of the following must be true: 1. library B can figure out how program P chose to link to it and links to it the exact same way.
A header-only library cannot *link* to anything, or it is not header-only.
CMake defines targets for header-only libraries, which can have dependencies on other targets, including on static or shared libraries. So the program that "links" to the header-only library picks up its dependencies recursively. As a result, the linker is invoked with all (binary) libraries in the dependency tree.
If this functionality works the way I think does, then I think it should not be used.
And in what way do you think it works? (Honest question.)
Whether you can consider such library header-only in the first place is a philosophical question. I do, because otherwise any library that calls the standard library cannot be called header-only.
Actually, the standard library is a great example of what I am talking about. A header-only library (B) should not care about the specific implementation of the standard library (A) being used by the program (P). There is no link dependency from B to A. There is a requirement from B to P that P must provide /a/ implementation of the C++ standard library, but B doesn't care if it is libstdc++ or libc++ or even a custom standard library implementation that is part of P and not a separate library at all.
That works for the standard-library, because you can be sure that one is available. (Otherwise your C++ compiler would be broken.) However, for other libraries A there is no way to guarantee that. Of course, you need to make sure that (any) dependency A is available on your build machine, otherwise building could never succeed. But the extra burden, to prepare the build-files (aka CMakeLists.txt) of your program P to find and explicitly configure even indirect dependencies (using `find_package`, `target_link_libraries`, `target_include_directories` etc.) is not to be underestimated. For a complex dependency-hierarchy this can become a nightmare, in particular if you need to find out in what order to process the individual build-files (`add_subdirectories`, `find_package` etc.). Instead, program P should only be expected to declare its direct dependencies (using `target_link_libraries`) and automatically get all the usage-requirements (aka dependencies) of these. This only requires, that every dependency (CMake target) carries its own usage-requirements. And, IMO, this should be true for header-only CMake targets, too.
Again, to make my point absolutely clear: header-only library B "using" library A does not imply a dependency from B to A. Instead, it implies a requirement on a program P that uses B to also link to A. Header-only libraries use interfaces. Programs link to implementations.
I just want to emphasize that even such interfaces are part of the usage-requirements I was talking about. Therefore B has a **usage-dependency** on A. And the CMake target for B should communicate this (through its usage-requirements) to its user, the program P (or more specific its CMake target). The usage-requirements of the CMake target for B will become the build-requirements of the CMake target for P (and could possibly even become its usage-requirements as well). And whether A is header-only or not is irrelevant. This (indirect) dependency from P to A can either mean linking against A or just having the correct include-search-path for the headers of A. The CMakeLists.txt file for building P needs to have this information either way. And this can be provided explicitly (as you prefer) which can become quite complex and tedious, or automatically and indirectly through B. Deniz PS: Maybe watching the first 11 minutes of my CMake talk [1] from Meeting C++ 2018 might help to better understand this target-centric approach CMake took. [1] https://youtu.be/y7ndUhdQuU8 -- BENOCS GmbH Dipl.-Inform. Deniz Bahadir Reuchlinstr. 10 D 10553 Berlin Germany Phone: +49 - 30 / 577 0004-22 Email: deniz.bahadir@benocs.com www.benocs.com Board of Management: Stephan Schroeder, Dr.-Ing. Ingmar Poese Commercial Register: Amtsgericht Bonn HRB 19378
On 11.06.21 16:20, Deniz Bahadir via Boost wrote:
Am 11.06.21 um 11:17 schrieb Rainer Deyke via Boost:
If this functionality works the way I think does, then I think it should not be used.
And in what way do you think it works? (Honest question.)
I think it forces the users of library B to (indirectly) link to library A, as opposed any other library that implements the interface of A. In the worst case, it might even unnecessarily force a specific version of A.
Actually, the standard library is a great example of what I am talking about. A header-only library (B) should not care about the specific implementation of the standard library (A) being used by the program (P). There is no link dependency from B to A. There is a requirement from B to P that P must provide /a/ implementation of the C++ standard library, but B doesn't care if it is libstdc++ or libc++ or even a custom standard library implementation that is part of P and not a separate library at all.
That works for the standard-library, because you can be sure that one is available. (Otherwise your C++ compiler would be broken.) However, for other libraries A there is no way to guarantee that.
You guarantee it by documenting it as a requirement for the users of B. The human users, not CMake.
Of course, you need to make sure that (any) dependency A is available on your build machine, otherwise building could never succeed. But the extra burden, to prepare the build-files (aka CMakeLists.txt) of your program P to find and explicitly configure even indirect dependencies (using `find_package`, `target_link_libraries`, `target_include_directories` etc.) is not to be underestimated. For a complex dependency-hierarchy this can become a nightmare, in particular if you need to find out in what order to process the individual build-files (`add_subdirectories`, `find_package` etc.).
In order to build P, you need to build the specific versions of all of the libraries that P requires, with the specific build settings that P requires. The build process should never pick up random library versions found in system directories. Program P must be able to say "use this modified version of libpng instead of the upstream version", "use LibreSSL instead of OpenSSL", and "don't use zlib at all, I'll provide my own version of the zlib functions". -- Rainer Deyke (rainerd@eldwood.com)
Am 11.06.21 um 22:17 schrieb Rainer Deyke via Boost:
On 11.06.21 16:20, Deniz Bahadir via Boost wrote:
Am 11.06.21 um 11:17 schrieb Rainer Deyke via Boost:
If this functionality works the way I think does, then I think it should not be used.
And in what way do you think it works? (Honest question.)
I think it forces the users of library B to (indirectly) link to library A, as opposed any other library that implements the interface of A. In the worst case, it might even unnecessarily force a specific version of A.
Actually, the standard library is a great example of what I am talking about. A header-only library (B) should not care about the specific implementation of the standard library (A) being used by the program (P). There is no link dependency from B to A. There is a requirement from B to P that P must provide /a/ implementation of the C++ standard library, but B doesn't care if it is libstdc++ or libc++ or even a custom standard library implementation that is part of P and not a separate library at all.
That works for the standard-library, because you can be sure that one is available. (Otherwise your C++ compiler would be broken.) However, for other libraries A there is no way to guarantee that.
You guarantee it by documenting it as a requirement for the users of B. The human users, not CMake.
Of course, you need to make sure that (any) dependency A is available on your build machine, otherwise building could never succeed. But the extra burden, to prepare the build-files (aka CMakeLists.txt) of your program P to find and explicitly configure even indirect dependencies (using `find_package`, `target_link_libraries`, `target_include_directories` etc.) is not to be underestimated. For a complex dependency-hierarchy this can become a nightmare, in particular if you need to find out in what order to process the individual build-files (`add_subdirectories`, `find_package` etc.).
In order to build P, you need to build the specific versions of all of the libraries that P requires, with the specific build settings that P requires. The build process should never pick up random library versions found in system directories. Program P must be able to say "use this modified version of libpng instead of the upstream version", "use LibreSSL instead of OpenSSL", and "don't use zlib at all, I'll provide my own version of the zlib functions".
You can even achieve that with CMake and its target-centric approach: If library B is a good citizen it uses `find_package` to look for its dependencies (more specifically, for the CMake target of its dependencies) and provides customization points (CMake options or simple variables). So if Program P wants to force a specific version of the indirect dependency A upon B it can set these customization points beforehand or it can just call `find_package` for that indirect dependency explicitly beforehand. The later call to `find_package` from B's CMakeLists.txt should then just return the same target (without even bothering searching). Of course, this requires careful crafting of B's CMakeLists.txt. (But it really can be worth it.) In case the indirect dependency A is header-only itself, it becomes even simpler because you do not have to cope with a library that might have been compiled using different/incompatible compiler-flags. IMO, CMake is quite powerful and one can achieve some really "funky stuff". But, as always, one should remember to not make it too complex or one will be unable to find anyone else who is willing to maintain it later. Deniz -- BENOCS GmbH Dipl.-Inform. Deniz Bahadir Reuchlinstr. 10 D 10553 Berlin Germany Phone: +49 - 30 / 577 0004-22 Email: deniz.bahadir@benocs.com www.benocs.com Board of Management: Stephan Schroeder, Dr.-Ing. Ingmar Poese Commercial Register: Amtsgericht Bonn HRB 19378
Rainer Deyke wrote:
On 10.06.21 23:36, Andrey Semashev via Boost wrote:
CMake defines targets for header-only libraries, which can have dependencies on other targets, including on static or shared libraries. So the program that "links" to the header-only library picks up its dependencies recursively. As a result, the linker is invoked with all (binary) libraries in the dependency tree.
If this functionality works the way I think does, then I think it should not be used.
Not "linking" to header-only libraries properly only works when all the headers are dumped into a central location, e.g. /usr/include or, in our case, $BOOST_ROOT/boost. That it works with our b2 setup is an artifact of this physical organization (and requires our `b2 headers` step). If the libraries are properly separated, which should really be required by any sane package management scheme, you have to "link" to header-only libraries in order to get the proper directory into your include path. As a side effect, this also makes the compiled -> header-only -> compiled case work without any effort from the end user (who would otherwise be forced to change his link line if an update to 'header-only' changes the list of compiled libraries that need to be linked).
On 11/06/2021 9:17 pm, Rainer Deyke wrote:
Actually, the standard library is a great example of what I am talking about. A header-only library (B) should not care about the specific implementation of the standard library (A) being used by the program (P). There is no link dependency from B to A. There is a requirement from B to P that P must provide /a/ implementation of the C++ standard library, but B doesn't care if it is libstdc++ or libc++ or even a custom standard library implementation that is part of P and not a separate library at all.
It does, however, need to be sure that it is using the same standard library as P (and with the same settings). Mixing multiple standard libraries into the same P will only end poorly -- which is usually also true (but less immediately obvious) when mixing multiple variants of other single libraries into P. You're assuming that by not specifying an explicit link from B to the standard library that it will inherently have the same settings as P. You're even mostly correct, because the standard library has a bit of special handling in the build tools. But this issue is not unique to the standard library, but applies to any dependency, including ones that you do have to explicitly tell the build tools about.
On 14.06.21 01:43, Gavin Lambert via Boost wrote:
On 11/06/2021 9:17 pm, Rainer Deyke wrote:
Actually, the standard library is a great example of what I am talking about. A header-only library (B) should not care about the specific implementation of the standard library (A) being used by the program (P). There is no link dependency from B to A. There is a requirement from B to P that P must provide /a/ implementation of the C++ standard library, but B doesn't care if it is libstdc++ or libc++ or even a custom standard library implementation that is part of P and not a separate library at all.
It does, however, need to be sure that it is using the same standard library as P (and with the same settings). Mixing multiple standard libraries into the same P will only end poorly -- which is usually also true (but less immediately obvious) when mixing multiple variants of other single libraries into P.
You're assuming that by not specifying an explicit link from B to the standard library that it will inherently have the same settings as P. You're even mostly correct, because the standard library has a bit of special handling in the build tools.
No, I'm saying that by not having a separate compilation step, there is literally no way for B to have different settings than P, because there is nothing that could be compiled with different settings. This is true regardless of the build system, because it is true at the compile level. There is no set of arguments that the build system can pass to the compiler that would cause B to be built with different settings than P. B is a header-only library. To use B, the C++ preprocessor literally pastes the code from B into P. Therefore, B /has/ no build settings. There are only the build settings from P, which are applied indiscriminately to P's own code and the code from B that P #includes. -- Rainer Deyke (rainerd@eldwood.com)
On 14/06/2021 5:34 pm, Rainer Deyke wrote:
You're assuming that by not specifying an explicit link from B to the standard library that it will inherently have the same settings as P. You're even mostly correct, because the standard library has a bit of special handling in the build tools.
No, I'm saying that by not having a separate compilation step, there is literally no way for B to have different settings than P, because there is nothing that could be compiled with different settings. This is true regardless of the build system, because it is true at the compile level. There is no set of arguments that the build system can pass to the compiler that would cause B to be built with different settings than P.
B is a header-only library. To use B, the C++ preprocessor literally pastes the code from B into P. Therefore, B /has/ no build settings. There are only the build settings from P, which are applied indiscriminately to P's own code and the code from B that P #includes.
That's actually completely false. The preprocessor itself *is* build settings -- defines can be defined differently (along with other compiler options) between the two builds. And yes, there may still be two builds. There is the build of P itself (which includes A) and the build of another library C that is linked into P but also includes A. Both of these can be via B or separately, and both can cause different code for B to be generated if the settings differ, which is an ODR violation. (Or cause two distinct copies of B to be generated, where C is a shared library with private visibility, for example.) For some examples, there's usually a compiler option to choose whether the standard library is linked as static or as shared, which in turn causes different #defines to be defined that will affect compilation. Another example is things that can vary due to the C++ language selected, or whether or not you've defined various "posix extension" flags, or library-specific configuration defines. Or something as simple as compiling one with exception support and one without. Having a header-only library is not a silver bullet to avoid these kinds of compilation incompatibilities. In fact it makes some of them worse.
On 15.06.21 01:32, Gavin Lambert via Boost wrote:
On 14/06/2021 5:34 pm, Rainer Deyke wrote:
B is a header-only library. To use B, the C++ preprocessor literally pastes the code from B into P. Therefore, B /has/ no build settings. There are only the build settings from P, which are applied indiscriminately to P's own code and the code from B that P #includes.
That's actually completely false. The preprocessor itself *is* build settings -- defines can be defined differently (along with other compiler options) between the two builds.
There are no two builds in my example. There is only P. Yes, using B modifies the build settings of P, but B has no build settings of its own. By the same token, linking P to separately compiled library A also modifies the build settings of P, but this is separate from the build settings that are actually used to /build/ A.
And yes, there may still be two builds. There is the build of P itself (which includes A) and the build of another library C that is linked into P but also includes A. Both of these can be via B or separately, and both can cause different code for B to be generated if the settings differ, which is an ODR violation.
That's a different situation. If you add a separately compiled library C to the equation, then C does have its own build settings, which need to be compatible with P's build settings. This is true whether C uses B or not, and whether B is header-only or not. Or you could, if C allows it, use C as a header-only library, which completely solves the problem. -- Rainer Deyke (rainerd@eldwood.com)
On 08.06.21 01:58, Peter Dimov via Boost wrote:
Raffi Enficiaud wrote:
I have not looked at the recent developments for CMake in Boost. If the CMakeLists.txt I developed (under build/) is still there, it contained 2 targets: one for static and one for shared. Those were having different and explicit names.
I decided to not do this and instead use the "normal" CMake way of relying on BUILD_SHARED_LIBS.
It causes problems in the more complicated cases where a compiled Boost::X links to header-only Boost::Y which links to a compiled Boost::Z. Client code can choose to link explicitly to Boost::static_X, and if X used Z directly, it could link to Boost::static_Z, but it doesn't. And requiring all header-only libraries to also provide triplicate targets so that static/shared-ness can be propagated correctly is (I think) infeasible. So X links to Boost::Y, and Y has to choose whether to link to the static or shared Z, but it doesn't know how.
(b2 does this "right" by having static/shared as a build feature, and propagating it automatically. No need to split your targets by hand.)
Swimming upstream is sometimes justified when it results in something objectively better than the accepted CMake way, but I don't believe that's the case here, so BUILD_SHARED_LIBS will have to do. (Similarly, I tried to do something b2-like with the tests, but at the end reverting to the way CMake wants to do things proved better.)
Getting back to Boost.Test, I've duplicated the targets with an included_ prefix:
https://github.com/boostorg/test/commit/bce2d24c8b32f47f0403766fe4fee3e2e93a...
Thanks, it looks good to me and if it works for Edward, then all good :) BUILD_SHARED_LIBS is a global property inherited by all targets, with the drawback of not being able to explain all desirable configurations (# exponential in the number of targets). One example where it is not working: I need to run the unit tests of Boost.Test with shared, static and header only, so those 3 targets must exist. But this is not an isolated usage: as a project grow, it is sometimes needed that a specific lib exists in both shared and static variant. In any case, it is always possible (later, if needed) to define the shared/static explicitly and make an alias on the target that mimics BUILD_SHARED_LIBS. I have to thank Peter and all other boosters that participated to this effort. On a side note: I believe the "democratic" way of doing it does not work inside the current boost setup, and yield endless discussions not producing any artifacts. OTOH, the person doing that should have enough established authority & being respected within Boost for doing such a job. Peter has this, I do not. Coming back to Boost.Test, Edward + Peter: your call. Raffi
On 6/8/2021 3:33 AM, Raffi Enficiaud via Boost wrote:
On 08.06.21 01:58, Peter Dimov via Boost wrote:
Raffi Enficiaud wrote:
I have not looked at the recent developments for CMake in Boost. If the CMakeLists.txt I developed (under build/) is still there, it contained 2 targets: one for static and one for shared. Those were having different and explicit names.
I decided to not do this and instead use the "normal" CMake way of relying on BUILD_SHARED_LIBS.
It causes problems in the more complicated cases where a compiled Boost::X links to header-only Boost::Y which links to a compiled Boost::Z. Client code can choose to link explicitly to Boost::static_X, and if X used Z directly, it could link to Boost::static_Z, but it doesn't. And requiring all header-only libraries to also provide triplicate targets so that static/shared-ness can be propagated correctly is (I think) infeasible. So X links to Boost::Y, and Y has to choose whether to link to the static or shared Z, but it doesn't know how.
(b2 does this "right" by having static/shared as a build feature, and propagating it automatically. No need to split your targets by hand.)
Swimming upstream is sometimes justified when it results in something objectively better than the accepted CMake way, but I don't believe that's the case here, so BUILD_SHARED_LIBS will have to do. (Similarly, I tried to do something b2-like with the tests, but at the end reverting to the way CMake wants to do things proved better.)
Getting back to Boost.Test, I've duplicated the targets with an included_ prefix:
https://github.com/boostorg/test/commit/bce2d24c8b32f47f0403766fe4fee3e2e93a...
Thanks, it looks good to me and if it works for Edward, then all good :)
Again I do not know CMake, and I am totally appreciative of the work Peter has done to bring CMake to Boost, but I was simply arguing that if Boost's implementation of CMake can not deal with the header-only variation of Boost.Test then conceptually this appears to be a failure. However if whatever Peter has done allows other code using CMake to treat Boost.Test as header-only, as well as compiled as either a static or shared library when chosen, I think that is optimal. I know that in a project on which I am working I use Boost.Test in header-only mode and it works fine when using b2 to run tests for the project. I would like to think that if I try to use CMake to run tests for the project using the header-only variant of Boost.Test it will continue to work fine. That's all really.
I know that in a project on which I am working I use Boost.Test in header-only mode and it works fine when using b2 to run tests for the project. I would like to think that if I try to use CMake to run tests for the project using the header-only variant of Boost.Test it will continue to work fine. That's all really.
Peter wrote that he implemented that in https://github.com/boostorg/test/commit/bce2d24c8b32f47f0403766fe4fee3e2e93a... So all you got to do is "link" to `Boost::included_unit_test_framework` and you can use it header-only. "Link" in CMake terms, no actual linking done, only declaring a dependency, and getting the headers into the include path. Although I would have used `header_only` as a suffix instead, but well, naming ;)
On 6/8/21 2:58 AM, Peter Dimov via Boost wrote:
Raffi Enficiaud wrote:
I have not looked at the recent developments for CMake in Boost. If the CMakeLists.txt I developed (under build/) is still there, it contained 2 targets: one for static and one for shared. Those were having different and explicit names.
I decided to not do this and instead use the "normal" CMake way of relying on BUILD_SHARED_LIBS.
It causes problems in the more complicated cases where a compiled Boost::X links to header-only Boost::Y which links to a compiled Boost::Z. Client code can choose to link explicitly to Boost::static_X, and if X used Z directly, it could link to Boost::static_Z, but it doesn't. And requiring all header-only libraries to also provide triplicate targets so that static/shared-ness can be propagated correctly is (I think) infeasible. So X links to Boost::Y, and Y has to choose whether to link to the static or shared Z, but it doesn't know how.
(b2 does this "right" by having static/shared as a build feature, and propagating it automatically. No need to split your targets by hand.)
Swimming upstream is sometimes justified when it results in something objectively better than the accepted CMake way, but I don't believe that's the case here, so BUILD_SHARED_LIBS will have to do. (Similarly, I tried to do something b2-like with the tests, but at the end reverting to the way CMake wants to do things proved better.)
Getting back to Boost.Test, I've duplicated the targets with an included_ prefix:
https://github.com/boostorg/test/commit/bce2d24c8b32f47f0403766fe4fee3e2e93a...
Not to start a bikeshedding argument, but maybe a more generic naming scheme would be preferable? I'm thinking something like Boost::x names the default config, Boost::x::header_only means header-only variant, Boost::x::static means static library, and so on. This makes it possible to add more configurations in the future, including the library-specific ones, if needed.
Andrey Semashev wrote:
Getting back to Boost.Test, I've duplicated the targets with an included_ prefix:
https://github.com/boostorg/test/commit/bce2d24c8b32f47f0403766fe4fee3e2e93a...
Not to start a bikeshedding argument, but maybe a more generic naming scheme would be preferable?
Since the normal unit_test_framework is used with #include <boost/test/unit_test.hpp> and the "included" one is used with #include <boost/test/included/unit_test.hpp> it seemed natural to call the target Boost::included_unit_test_framework in the second case. (Similarly with the execution monitors.)
participants (10)
-
Alexander Grund -
Andrey Semashev -
Deniz Bahadir -
Edward Diener -
Gavin Lambert -
Glen Fernandes -
John Maddock -
Peter Dimov -
Raffi Enficiaud -
Rainer Deyke