Boost CMake support - Request for Comment
Background ========== My personal knowledge of build systems is somewhat limited. a) I don't really understand CMake all that well. I have made CMakeLists.txt files for both the serialization library and the safe numerics libraries as an aid to build and maintain IDE and make files used in running tests and posting results to CDash. b) I don't really understand our current system, Boost Build, very well either. I've made jam files for building and running tests for the serialization and safe numerics libraries. c) I recently reviewed C++Now 2017: Daniel Pfeifer's excellent video from C++Now 2017 “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk . I recommend it highly. Despite my lack of knowledge in this area, or perhaps because of it, I've volunteered to direct the effort to evolve Boost so that users of Boost can avail themselves of the benefits of CMake. This effort will be organized as follows: a) A discussion on the boost developers mailing list with the goal of reaching a consensus as to what benefits CMake can and should provide to users and developers of Boost. This effort will commence with the posting of this notice. I hope that this discussion can be more or less resolved within 30 days. b) I will then synthesize from the above discussion a call for proposals which lists the requirements and requirements. c) Those proposing CMake for boost will have approximately 90 days to prepare their proposals. Their proposal are expected to look similar to a boost library proposal. That is they are expected to have some (CMake) code, along with tests, documentation including tutorials and reference. This is only an expectation. It's conceivable that such a proposal might only contain conventions and and samples of what boost library should contain to fulfill the requirements. As with boost libraries, submissions are not required to totally complete. But they have to be sufficiently complete to convince reviewers the the submission fulfills the stated requirements and can be finished in good time. d) After the 90 day period, submissions will be reviewed simultaneously. Note that this is at a variance from normal boost procedure of reviewing submissions one at a time. Since we expect to receive a number of submissions and only one can be accepted, the normal boost protocol can't really function. Target date for this review is 1 February 2019. e) After the review period the review manager will prepare a report which includes the decision of which submission will be accepted into the official boost distribution. Currently, I, Robert Ramey, expect to be review manager. However, it's possible that this by the time the review is undertaken, this coveted position could be assigned to some more worthy candidate. Scope, Requirements and Features ================================ We presume that submissions will fulfill the applicable requirements of any boost submission. In addition we would like to see the following addressed. Library Build ------------- “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk Recommends that all libraries - including header only libraries have a CMakeList.txt file. What should this include for Boost Libraries? Library Test ------------ Should facilities for "testing" be only done by developers? or should users who acquire library packages also be able to test libraries. Should CMake testing results posting be used - CDASH? Should we just skip the issue of Library Testing and continue to depend on Boost Build. Library Packaging ----------------- Is the library packaging facility provide by CMake - CPACK - useful to boost. Should boost libraries be updated to support it? Dependencies ------------ If the above is implemented, can we depend upon CMake to handle library dependencies? Or does some special functionality Do circular dependencies constitute a problem? Modularity ---------- Currently boost is organized as a tightly integrated group of libraries. This organization manifests itself in a number of ways. We have a Boost "super project" in github. Boost libraries are "sub projects" of this super project. This known structure is exploited by and depended upon by boost tools. Should CMake support continue this policy/design. We distribute boost as "super project". Should we continue to do this or distribute libraries on a library by library basis. Does CMake/CPACK etc. make this "simple"? Documentation/testing --------------------- What support should be require for uses of Boost CMake implementation. Documentation, examples, templates for helping library developers. How should the CMake design/implementation be tested? Other considerations -------------------- add your requirement/features here. Useful Resources =============== “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk https://gist.github.com/mbinna/c61dbb39bca0e4fb7d1f73b0d66a4fd1 https://github.com/purpleKarrot/Boost.CMake https://www.youtube.com/watch?v=eC9-iRN2b04 Please try to keep this discussion on this list. I know that there has been a lot going on on slack. But having it in two places makes it harder for me to review and summarize. Also slack doesn't conveniently maintain comment hierarchy. So please try to keep this discussion on this list. Robert Ramey
On 10/1/2018 1:07 PM, Robert Ramey via Boost wrote:
Background ==========
My personal knowledge of build systems is somewhat limited.
a) I don't really understand CMake all that well. I have made CMakeLists.txt files for both the serialization library and the safe numerics libraries as an aid to build and maintain IDE and make files used in running tests and posting results to CDash.
b) I don't really understand our current system, Boost Build, very well either. I've made jam files for building and running tests for the serialization and safe numerics libraries.
c) I recently reviewed C++Now 2017: Daniel Pfeifer's excellent video from C++Now 2017 “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk . I recommend it highly.
Despite my lack of knowledge in this area, or perhaps because of it, I've volunteered to direct the effort to evolve Boost so that users of Boost can avail themselves of the benefits of CMake. This effort will be organized as follows:
a) A discussion on the boost developers mailing list with the goal of reaching a consensus as to what benefits CMake can and should provide to users and developers of Boost. This effort will commence with the posting of this notice. I hope that this discussion can be more or less resolved within 30 days.
b) I will then synthesize from the above discussion a call for proposals which lists the requirements and requirements.
c) Those proposing CMake for boost will have approximately 90 days to prepare their proposals. Their proposal are expected to look similar to a boost library proposal. That is they are expected to have some (CMake) code, along with tests, documentation including tutorials and reference. This is only an expectation. It's conceivable that such a proposal might only contain conventions and and samples of what boost library should contain to fulfill the requirements. As with boost libraries, submissions are not required to totally complete. But they have to be sufficiently complete to convince reviewers the the submission fulfills the stated requirements and can be finished in good time.
d) After the 90 day period, submissions will be reviewed simultaneously. Note that this is at a variance from normal boost procedure of reviewing submissions one at a time. Since we expect to receive a number of submissions and only one can be accepted, the normal boost protocol can't really function. Target date for this review is 1 February 2019.
e) After the review period the review manager will prepare a report which includes the decision of which submission will be accepted into the official boost distribution. Currently, I, Robert Ramey, expect to be review manager. However, it's possible that this by the time the review is undertaken, this coveted position could be assigned to some more worthy candidate.
Scope, Requirements and Features ================================
We presume that submissions will fulfill the applicable requirements of any boost submission. In addition we would like to see the following addressed.
Library Build ------------- “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk Recommends that all libraries - including header only libraries have a CMakeList.txt file. What should this include for Boost Libraries?
Library Test ------------ Should facilities for "testing" be only done by developers? or should users who acquire library packages also be able to test libraries. Should CMake testing results posting be used - CDASH? Should we just skip the issue of Library Testing and continue to depend on Boost Build.
Library Packaging ----------------- Is the library packaging facility provide by CMake - CPACK - useful to boost. Should boost libraries be updated to support it?
Dependencies ------------ If the above is implemented, can we depend upon CMake to handle library dependencies? Or does some special functionality Do circular dependencies constitute a problem?
Modularity ---------- Currently boost is organized as a tightly integrated group of libraries. This organization manifests itself in a number of ways. We have a Boost "super project" in github. Boost libraries are "sub projects" of this super project. This known structure is exploited by and depended upon by boost tools. Should CMake support continue this policy/design. We distribute boost as "super project". Should we continue to do this or distribute libraries on a library by library basis. Does CMake/CPACK etc. make this "simple"?
Documentation/testing --------------------- What support should be require for uses of Boost CMake implementation. Documentation, examples, templates for helping library developers. How should the CMake design/implementation be tested?
Other considerations -------------------- add your requirement/features here.
Useful Resources =============== “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk https://gist.github.com/mbinna/c61dbb39bca0e4fb7d1f73b0d66a4fd1 https://github.com/purpleKarrot/Boost.CMake https://www.youtube.com/watch?v=eC9-iRN2b04
Please try to keep this discussion on this list.
I know that there has been a lot going on on slack. But having it in two places makes it harder for me to review and summarize. Also slack doesn't conveniently maintain comment hierarchy. So please try to keep this discussion on this list.
I strongly suggest that the goal of using CMake with Boost begin with making Boost libraries, whether header-only or built, available to CMake projects. This also means that we have a way of testing whether or not such a solution actually works for any given Boost library. My own preference for testing whether or not such a solution would work or not would be to use our current b2/Boost Build test jamfiles, but if this is too difficult to do we still need some way to test the solution for each library so each library maintainer can see whether or not the solution we choose works for his library. This should be our first goal and all other goals should be tabled until and when we can have a solution where this works flawlessly and there is a general agreement that whatever proposal is chosen to implement this is best. Only when the above is fully accomplished should Boost then look into the goal of using CMake internally, perhaps for test, docs, building libraries, or what-not. We need to work this way because every solution is bound to end with comments that it does X but does not do Y or it does X and Y but does not do Z, and we all know where these endless arguments end: that we end up doing nothing because we start by trying to do too much and we can never reach agreement.
Robert Ramey
I strongly suggest that the goal of using CMake with Boost begin with making Boost libraries, whether header-only or built, available to CMake projects. This also means that we have a way of testing whether or not such a solution actually works for any given Boost library. My own preference for testing whether or not such a solution would work or not would be to use our current b2/Boost Build test jamfiles, but if this is too difficult to do we still need some way to test the solution for each library so each library maintainer can see whether or not the solution we choose works for his library.
Are you talking about writing a CMake Config for installed boost? If so I'm strongly in favor of doing so! You'd need some BoostConfig.cmake files with appropriate contents. They are similar to pkgconfig files, so small text files. Those can be (mostly) generated by CMake when it is used to build boost and in the end they replace the FindBoost.cmake currently shipped with CMake (and which the CMake maintainers dislike) Starting from the "other side" would allow some experimentation on how Boost can be consumed by CMake projects and will show many pitfalls early. For example one would need to think about how to deal with the variants (static/dynamic, debug/release[actually simple], static/dynamic runtime, versions installed to same folder, ...) If this is solved, one can go further and implement the actual build system. Note that at this point naming conventions are already established and the current build system can be used to generate these files requiring less effort.
-----Original Message----- From: Boost <boost-bounces@lists.boost.org> On Behalf Of Edward Diener via Boost Sent: Monday, October 1, 2018 8:00 PM
On 10/1/2018 1:07 PM, Robert Ramey via Boost wrote:
Background ==========
My personal knowledge of build systems is somewhat limited.
a) I don't really understand CMake all that well. I have made CMakeLists.txt files for both the serialization library and the safe numerics libraries as an aid to build and maintain IDE and make files used in running tests and posting results to CDash.
b) I don't really understand our current system, Boost Build, very well either. I've made jam files for building and running tests for the serialization and safe numerics libraries.
c) I recently reviewed C++Now 2017: Daniel Pfeifer's excellent video from C++Now 2017 “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk . I recommend it highly.
Despite my lack of knowledge in this area, or perhaps because of it, I've volunteered to direct the effort to evolve Boost so that users of Boost can avail themselves of the benefits of CMake. This effort will be organized as follows:
a) A discussion on the boost developers mailing list with the goal of reaching a consensus as to what benefits CMake can and should provide to users and developers of Boost. This effort will commence with the posting of this notice. I hope that this discussion can be more or less resolved within 30 days.
b) I will then synthesize from the above discussion a call for proposals which lists the requirements and requirements.
c) Those proposing CMake for boost will have approximately 90 days to prepare their proposals. Their proposal are expected to look similar to a boost library proposal. That is they are expected to have some (CMake) code, along with tests, documentation including tutorials and reference. This is only an expectation. It's conceivable that such a proposal might only contain conventions and and samples of what boost library should contain to fulfill the requirements. As with boost libraries, submissions are not required to totally complete. But they have to be sufficiently complete to convince reviewers the the submission fulfills the stated requirements and can be finished in good time.
d) After the 90 day period, submissions will be reviewed simultaneously. Note that this is at a variance from normal boost procedure of reviewing submissions one at a time. Since we expect to receive a number of submissions and only one can be accepted, the normal boost protocol can't really function. Target date for this review is 1 February 2019.
e) After the review period the review manager will prepare a report which includes the decision of which submission will be accepted into the official boost distribution. Currently, I, Robert Ramey, expect to be review manager. However, it's possible that this by the time the review is undertaken, this coveted position could be assigned to some more worthy candidate.
Scope, Requirements and Features ================================
We presume that submissions will fulfill the applicable requirements of any boost submission. In addition we would like to see the following addressed.
Library Build ------------- “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk Recommends that all libraries - including header only libraries have a CMakeList.txt file. What should this include for Boost Libraries?
Library Test ------------ Should facilities for "testing" be only done by developers? or should users who acquire library packages also be able to test libraries. Should CMake testing results posting be used - CDASH? Should we just skip the issue of Library Testing and continue to depend on Boost Build.
Library Packaging ----------------- Is the library packaging facility provide by CMake - CPACK - useful to boost. Should boost libraries be updated to support it?
Dependencies ------------ If the above is implemented, can we depend upon CMake to handle library dependencies? Or does some special functionality Do circular dependencies constitute a problem?
Modularity ---------- Currently boost is organized as a tightly integrated group of libraries. This organization manifests itself in a number of ways. We have a Boost "super project" in github. Boost libraries are "sub projects" of this super project. This known structure is exploited by and depended upon by boost tools. Should CMake support continue this policy/design. We distribute boost as "super project". Should we continue to do this or distribute libraries on a library by library basis. Does CMake/CPACK etc. make this "simple"?
Documentation/testing --------------------- What support should be require for uses of Boost CMake implementation. Documentation, examples, templates for helping library developers. How should the CMake design/implementation be tested?
Other considerations -------------------- add your requirement/features here.
Useful Resources =============== “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk https://gist.github.com/mbinna/c61dbb39bca0e4fb7d1f73b0d66a4fd1 https://github.com/purpleKarrot/Boost.CMake https://www.youtube.com/watch?v=eC9-iRN2b04
Please try to keep this discussion on this list.
I know that there has been a lot going on on slack. But having it in two places makes it harder for me to review and summarize. Also slack doesn't conveniently maintain comment hierarchy. So please try to keep this discussion on this list.
I strongly suggest that the goal of using CMake with Boost begin with making Boost libraries, whether header-only or built, available to CMake projects. This also means that we have a way of testing whether or not such a solution actually works for any given Boost library. My own preference for testing whether or not such a solution would work or not would be to use our current b2/Boost Build test jamfiles, but if this is too difficult to do we still need some way to test the solution for each library so each library maintainer can see whether or not the solution we choose works for his library.
This should be our first goal and all other goals should be tabled until and when we can have a solution where this works flawlessly and there is a general agreement that whatever proposal is chosen to implement this is best.
Only when the above is fully accomplished should Boost then look into the goal of using CMake internally, perhaps for test, docs, building libraries, or what-not. We need to work this way because every solution is bound to end with comments that it does X but does not do Y or it does X and Y but does not do Z, and we all know where these endless arguments end: that we end up doing nothing because we start by trying to do too much and we can never reach agreement.
I agree with you in so far, that the focus should be on the interface (how to consume boost libraries) and less on the implementation details, but I see two practical reasons, why it is beneficial to immediately use cmake (at least) for building too: 1) Creating a CMake target for a separately compiled binary that can be consumed by the user is about the same effort as using cmake to build the library (if necessary) in which case you get the interface information for free (switching to cmake for unit-tests and docs is a different topic). 2) "making Boost libraries [...] available to CMake projects" for me also means I have an easy way to compile those libraries with specific compiler flags that match my project settings - otherwise, what is the advantage compared to the current mechanism that is already build into cmake? Ensuring consistent build flags is relatively easy if all libraries use the same build system (in this case cmake) but can become quite complex if you want to implement an automatic translation from cmake to b2 or vice versa. So yes, while a library (build-) interface and consummation should be the focus of the review, I'd expect that at least building the libraries with cmake will be the simplest way to implement that interface. Just to be completely clear: Just because the "reference implementation" of a cmake interface is probably a cmake-based build, that doesn't mean everyone has to actually adopt that reference implementation as long as the interface is consistent. Mike
On 10/8/2018 6:15 AM, Mike Dev via Boost wrote:
-----Original Message----- From: Boost <boost-bounces@lists.boost.org> On Behalf Of Edward Diener via Boost Sent: Monday, October 1, 2018 8:00 PM
On 10/1/2018 1:07 PM, Robert Ramey via Boost wrote:
Background ==========
My personal knowledge of build systems is somewhat limited.
a) I don't really understand CMake all that well. I have made CMakeLists.txt files for both the serialization library and the safe numerics libraries as an aid to build and maintain IDE and make files used in running tests and posting results to CDash.
b) I don't really understand our current system, Boost Build, very well either. I've made jam files for building and running tests for the serialization and safe numerics libraries.
c) I recently reviewed C++Now 2017: Daniel Pfeifer's excellent video from C++Now 2017 “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk . I recommend it highly.
Despite my lack of knowledge in this area, or perhaps because of it, I've volunteered to direct the effort to evolve Boost so that users of Boost can avail themselves of the benefits of CMake. This effort will be organized as follows:
a) A discussion on the boost developers mailing list with the goal of reaching a consensus as to what benefits CMake can and should provide to users and developers of Boost. This effort will commence with the posting of this notice. I hope that this discussion can be more or less resolved within 30 days.
b) I will then synthesize from the above discussion a call for proposals which lists the requirements and requirements.
c) Those proposing CMake for boost will have approximately 90 days to prepare their proposals. Their proposal are expected to look similar to a boost library proposal. That is they are expected to have some (CMake) code, along with tests, documentation including tutorials and reference. This is only an expectation. It's conceivable that such a proposal might only contain conventions and and samples of what boost library should contain to fulfill the requirements. As with boost libraries, submissions are not required to totally complete. But they have to be sufficiently complete to convince reviewers the the submission fulfills the stated requirements and can be finished in good time.
d) After the 90 day period, submissions will be reviewed simultaneously. Note that this is at a variance from normal boost procedure of reviewing submissions one at a time. Since we expect to receive a number of submissions and only one can be accepted, the normal boost protocol can't really function. Target date for this review is 1 February 2019.
e) After the review period the review manager will prepare a report which includes the decision of which submission will be accepted into the official boost distribution. Currently, I, Robert Ramey, expect to be review manager. However, it's possible that this by the time the review is undertaken, this coveted position could be assigned to some more worthy candidate.
Scope, Requirements and Features ================================
We presume that submissions will fulfill the applicable requirements of any boost submission. In addition we would like to see the following addressed.
Library Build ------------- “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk Recommends that all libraries - including header only libraries have a CMakeList.txt file. What should this include for Boost Libraries?
Library Test ------------ Should facilities for "testing" be only done by developers? or should users who acquire library packages also be able to test libraries. Should CMake testing results posting be used - CDASH? Should we just skip the issue of Library Testing and continue to depend on Boost Build.
Library Packaging ----------------- Is the library packaging facility provide by CMake - CPACK - useful to boost. Should boost libraries be updated to support it?
Dependencies ------------ If the above is implemented, can we depend upon CMake to handle library dependencies? Or does some special functionality Do circular dependencies constitute a problem?
Modularity ---------- Currently boost is organized as a tightly integrated group of libraries. This organization manifests itself in a number of ways. We have a Boost "super project" in github. Boost libraries are "sub projects" of this super project. This known structure is exploited by and depended upon by boost tools. Should CMake support continue this policy/design. We distribute boost as "super project". Should we continue to do this or distribute libraries on a library by library basis. Does CMake/CPACK etc. make this "simple"?
Documentation/testing --------------------- What support should be require for uses of Boost CMake implementation. Documentation, examples, templates for helping library developers. How should the CMake design/implementation be tested?
Other considerations -------------------- add your requirement/features here.
Useful Resources =============== “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk https://gist.github.com/mbinna/c61dbb39bca0e4fb7d1f73b0d66a4fd1 https://github.com/purpleKarrot/Boost.CMake https://www.youtube.com/watch?v=eC9-iRN2b04
Please try to keep this discussion on this list.
I know that there has been a lot going on on slack. But having it in two places makes it harder for me to review and summarize. Also slack doesn't conveniently maintain comment hierarchy. So please try to keep this discussion on this list.
I strongly suggest that the goal of using CMake with Boost begin with making Boost libraries, whether header-only or built, available to CMake projects. This also means that we have a way of testing whether or not such a solution actually works for any given Boost library. My own preference for testing whether or not such a solution would work or not would be to use our current b2/Boost Build test jamfiles, but if this is too difficult to do we still need some way to test the solution for each library so each library maintainer can see whether or not the solution we choose works for his library.
This should be our first goal and all other goals should be tabled until and when we can have a solution where this works flawlessly and there is a general agreement that whatever proposal is chosen to implement this is best.
Only when the above is fully accomplished should Boost then look into the goal of using CMake internally, perhaps for test, docs, building libraries, or what-not. We need to work this way because every solution is bound to end with comments that it does X but does not do Y or it does X and Y but does not do Z, and we all know where these endless arguments end: that we end up doing nothing because we start by trying to do too much and we can never reach agreement.
I agree with you in so far, that the focus should be on the interface (how to consume boost libraries) and less on the implementation details, but I see two practical reasons, why it is beneficial to immediately use cmake (at least) for building too:
1) Creating a CMake target for a separately compiled binary that can be consumed by the user is about the same effort as using cmake to build the library (if necessary) in which case you get the interface information for free (switching to cmake for unit-tests and docs is a different topic).
2) "making Boost libraries [...] available to CMake projects" for me also means I have an easy way to compile those libraries with specific compiler flags that match my project settings - otherwise, what is the advantage compared to the current mechanism that is already build into cmake? Ensuring consistent build flags is relatively easy if all libraries use the same build system (in this case cmake) but can become quite complex if you want to implement an automatic translation from cmake to b2 or vice versa.
So yes, while a library (build-) interface and consummation should be the focus of the review, I'd expect that at least building the libraries with cmake will be the simplest way to implement that interface.
Just to be completely clear: Just because the "reference implementation" of a cmake interface is probably a cmake-based build, that doesn't mean everyone has to actually adopt that reference implementation as long as the interface is consistent.
Boost Build has a feature which allows the programmer to test at build time what C++ features are supported for a particular compiler implementation and compile parameters, and change the build internally on-the-fly accordingly. I do not know if any Boost non-header only library uses this feature but if one does it will be difficult translating this to CMake terms, if such a similar feature exists in CMake. It is for such a reason that as soon as you start talking about using CMake for anything else but identifying Boost libraries for the purposes of end-user use of Boost libraries in their own CMake scripts, that you may run into much more work than you can imagine. You just can't tell individual library developers to translate all their own use of Boost Build into CMake, as if this will be simple for them to do, and they will just do it with little time spent and as if it will be no work for them. Boost Build, for better or worse, has some extensive features which have no easy correspondence to CMake. Unless you or someone else is willing to develop a bridge between Boost Build and CMake for features of Boost Build which libraries rely on, then individual Boost libraries which use such Boost Build features are going to find it very difficult to transition to CMake, possibly even for something as seemingly innocuous as building the library itself using a CMake script. This is why I have urged the move to CMake by Boost to proceed slowly by baby steps, rather than to try doing things that may take a huge initial effort to get done, and therefore may end up never accomplishing anything.
Mike
In article <ppge4m$2d5$1@blaine.gmane.org>, Edward Diener via Boost <boost@lists.boost.org> writes:
Boost Build has a feature which allows the programmer to test at build time what C++ features are supported for a particular compiler implementation and compile parameters, and change the build internally on-the-fly accordingly. I do not know if any Boost non-header only library uses this feature but if one does it will be difficult translating this to CMake terms, if such a similar feature exists in CMake.
CMake has supported this for years, it is not a blocking issue. -- "The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline> The Terminals Wiki <http://terminals-wiki.org> The Computer Graphics Museum <http://ComputerGraphicsMuseum.org> Legalize Adulthood! (my blog) <http://LegalizeAdulthood.wordpress.com>
On 10/8/2018 6:46 PM, Richard via Boost wrote:
In article <ppge4m$2d5$1@blaine.gmane.org>, Edward Diener via Boost <boost@lists.boost.org> writes:
Boost Build has a feature which allows the programmer to test at build time what C++ features are supported for a particular compiler implementation and compile parameters, and change the build internally on-the-fly accordingly. I do not know if any Boost non-header only library uses this feature but if one does it will be difficult translating this to CMake terms, if such a similar feature exists in CMake.
CMake has supported this for years, it is not a blocking issue.
Good ! Then I am sure you will be glad to be nominated as the person who converts all instances of the Boost Build use of this feature, in any Boost library which uses it, to the exact equivalent in CMake. It is great to know that you have volunteered to do this for Boost !
In article <ppgt9u$6a1$1@blaine.gmane.org>, Edward Diener via Boost <boost@lists.boost.org> writes:
Good ! Then I am sure you will be glad to be nominated as the person who converts all instances of the Boost Build use of this feature,
You can nominate about whatever you like, but I am not volunteering for anything in boost. It takes forever to get anything done. -- "The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline> The Terminals Wiki <http://terminals-wiki.org> The Computer Graphics Museum <http://ComputerGraphicsMuseum.org> Legalize Adulthood! (my blog) <http://LegalizeAdulthood.wordpress.com>
On 2018-10-09 05:13 PM, Richard via Boost wrote:
In article <ppgt9u$6a1$1@blaine.gmane.org>, Edward Diener via Boost <boost@lists.boost.org> writes:
Good ! Then I am sure you will be glad to be nominated as the person who converts all instances of the Boost Build use of this feature, You can nominate about whatever you like, but I am not volunteering for anything in boost. It takes forever to get anything done.
I think the point wasn't so much to nominate you, but rather to point out that a statement like the one you made is rater gratuitous to make if you don't have to back it up by actually doing the implied work. Stefan -- ...ich hab' noch einen Koffer in Berlin...
In article <c7576691-195e-7ed6-eb4a-4b3a76fa8a08@seefeld.name>, Stefan Seefeld via Boost <boost@lists.boost.org> writes:
On 2018-10-09 05:13 PM, Richard via Boost wrote:
In article <ppgt9u$6a1$1@blaine.gmane.org>, Edward Diener via Boost <boost@lists.boost.org> writes:
Good ! Then I am sure you will be glad to be nominated as the person who converts all instances of the Boost Build use of this feature, You can nominate about whatever you like, but I am not volunteering for anything in boost. It takes forever to get anything done.
I think the point wasn't so much to nominate you,
Au contraire, he quite specifically nominates me in the first line I quoted and further implies that I will be happy about it. The question he raised was whether or not CMake supports this kind of feature testing and I simply pointed out that it has been supported for many years. Why he goes further and tries to volunteer me for work is beyond my understanding.
but rather to point out that a statement like the one you made is rater gratuitous to make if you don't have to back it up by actually doing the implied work.
Oh nonsense. The question was: "Does CMake do X?" to which I answered in the affirmative. Providing information about capabilities in no way obligates me to do work. This is just more data supporting my assertion above that it takes forever to get anything done. This sort of needless snark is just wasting everyone's time and further alienates me to contribute anything to boost. -- "The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline> The Terminals Wiki <http://terminals-wiki.org> The Computer Graphics Museum <http://ComputerGraphicsMuseum.org> Legalize Adulthood! (my blog) <http://LegalizeAdulthood.wordpress.com>
AMDG On 10/10/2018 10:32 AM, Richard via Boost wrote:
In article <c7576691-195e-7ed6-eb4a-4b3a76fa8a08@seefeld.name>, Stefan Seefeld via Boost <boost@lists.boost.org> writes:
On 2018-10-09 05:13 PM, Richard via Boost wrote:
In article <ppgt9u$6a1$1@blaine.gmane.org>, Edward Diener via Boost <boost@lists.boost.org> writes:
Good ! Then I am sure you will be glad to be nominated as the person who converts all instances of the Boost Build use of this feature, You can nominate about whatever you like, but I am not volunteering for anything in boost. It takes forever to get anything done.
I think the point wasn't so much to nominate you,
Au contraire, he quite specifically nominates me in the first line I quoted and further implies that I will be happy about it.
The question he raised was whether or not CMake supports this kind of feature testing and I simply pointed out that it has been supported for many years.
Why he goes further and tries to volunteer me for work is beyond my understanding.
It's because you misread the post that you responded to:
...it will be difficult translating this to CMake terms, if such a similar feature exists in CMake.
CMake has supported this for years, it is not a blocking issue.
From Edward's later responses, it seems clear that he really did mean exactly what he said rather than "...if a similar feature *does not* exist...", which is what you responded to.
In Christ, Steven Watanabe
On 2018-10-10 12:32 PM, Richard via Boost wrote:
This is just more data supporting my assertion above that it takes forever to get anything done. This sort of needless snark is just wasting everyone's time and further alienates me to contribute anything to boost.
What is wasting everyone's time is if people with no intent to contribute get involved in discussions without even putting any effort into listening (a highly under-rated activity !) and understanding what their peers are saying. Stefan -- ...ich hab' noch einen Koffer in Berlin...
On 10/8/18 5:40 PM, Edward Diener via Boost wrote:
On 10/8/2018 6:46 PM, Richard via Boost wrote:
In article <ppge4m$2d5$1@blaine.gmane.org>, Edward Diener via Boost <boost@lists.boost.org> writes:
Boost Build has a feature which allows the programmer to test at build time what C++ features are supported for a particular compiler implementation and compile parameters, and change the build internally on-the-fly accordingly. I do not know if any Boost non-header only library uses this feature but if one does it will be difficult translating this to CMake terms, if such a similar feature exists in CMake.
CMake has supported this for years, it is not a blocking issue.
Good ! Then I am sure you will be glad to be nominated as the person who converts all instances of the Boost Build use of this feature, in any Boost library which uses it, to the exact equivalent in CMake. It is great to know that you have volunteered to do this for Boost !
I don't think that at this point you can presume that such a person would be necessary. But then again I don't presume it wouldn't either. I'm going restate this as a suggested requirement: "Any new CMake implemenation for Boost shouldn't require non-trivial efforts on existing libraries by library developers." I'll take this under advisement. Robert Ramey
-----Original Message----- From: Boost <boost-bounces@lists.boost.org> On Behalf Of Edward Diener via Boost Sent: Monday, October 8, 2018 10:22 PM
On 10/8/2018 6:15 AM, Mike Dev via Boost wrote:
-----Original Message----- From: Boost <boost-bounces@lists.boost.org> On Behalf Of Edward Diener via Boost Sent: Monday, October 1, 2018 8:00 PM
On 10/1/2018 1:07 PM, Robert Ramey via Boost wrote:
Background ==========
My personal knowledge of build systems is somewhat limited.
a) I don't really understand CMake all that well. I have made CMakeLists.txt files for both the serialization library and the safe numerics libraries as an aid to build and maintain IDE and make files used in running tests and posting results to CDash.
b) I don't really understand our current system, Boost Build, very well either. I've made jam files for building and running tests for the serialization and safe numerics libraries.
c) I recently reviewed C++Now 2017: Daniel Pfeifer's excellent video from C++Now 2017 “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk . I recommend it highly.
Despite my lack of knowledge in this area, or perhaps because of it, I've volunteered to direct the effort to evolve Boost so that users of Boost can avail themselves of the benefits of CMake. This effort will be organized as follows:
a) A discussion on the boost developers mailing list with the goal of reaching a consensus as to what benefits CMake can and should provide to users and developers of Boost. This effort will commence with the posting of this notice. I hope that this discussion can be more or less resolved within 30 days.
b) I will then synthesize from the above discussion a call for proposals which lists the requirements and requirements.
c) Those proposing CMake for boost will have approximately 90 days to prepare their proposals. Their proposal are expected to look similar to a boost library proposal. That is they are expected to have some (CMake) code, along with tests, documentation including tutorials and reference. This is only an expectation. It's conceivable that such a proposal might only contain conventions and and samples of what boost library should contain to fulfill the requirements. As with boost libraries, submissions are not required to totally complete. But they have to be sufficiently complete to convince reviewers the the submission fulfills the stated requirements and can be finished in good time.
d) After the 90 day period, submissions will be reviewed simultaneously. Note that this is at a variance from normal boost procedure of reviewing submissions one at a time. Since we expect to receive a number of submissions and only one can be accepted, the normal boost protocol can't really function. Target date for this review is 1 February 2019.
e) After the review period the review manager will prepare a report which includes the decision of which submission will be accepted into the official boost distribution. Currently, I, Robert Ramey, expect to be review manager. However, it's possible that this by the time the review is undertaken, this coveted position could be assigned to some more worthy candidate.
Scope, Requirements and Features ================================
We presume that submissions will fulfill the applicable requirements of any boost submission. In addition we would like to see the following addressed.
Library Build ------------- “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk Recommends that all libraries - including header only libraries have a CMakeList.txt file. What should this include for Boost Libraries?
Library Test ------------ Should facilities for "testing" be only done by developers? or should users who acquire library packages also be able to test libraries. Should CMake testing results posting be used - CDASH? Should we just skip the issue of Library Testing and continue to depend on Boost Build.
Library Packaging ----------------- Is the library packaging facility provide by CMake - CPACK - useful to boost. Should boost libraries be updated to support it?
Dependencies ------------ If the above is implemented, can we depend upon CMake to handle library dependencies? Or does some special functionality Do circular dependencies constitute a problem?
Modularity ---------- Currently boost is organized as a tightly integrated group of libraries. This organization manifests itself in a number of ways. We have a Boost "super project" in github. Boost libraries are "sub projects" of this super project. This known structure is exploited by and depended upon by boost tools. Should CMake support continue this policy/design. We distribute boost as "super project". Should we continue to do this or distribute libraries on a library by library basis. Does CMake/CPACK etc. make this "simple"?
Documentation/testing --------------------- What support should be require for uses of Boost CMake implementation. Documentation, examples, templates for helping library developers. How should the CMake design/implementation be tested?
Other considerations -------------------- add your requirement/features here.
Useful Resources =============== “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk https://gist.github.com/mbinna/c61dbb39bca0e4fb7d1f73b0d66a4fd1 https://github.com/purpleKarrot/Boost.CMake https://www.youtube.com/watch?v=eC9-iRN2b04
Please try to keep this discussion on this list.
I know that there has been a lot going on on slack. But having it in two places makes it harder for me to review and summarize. Also slack doesn't conveniently maintain comment hierarchy. So please try to keep this discussion on this list.
I strongly suggest that the goal of using CMake with Boost begin with making Boost libraries, whether header-only or built, available to CMake projects. This also means that we have a way of testing whether or not such a solution actually works for any given Boost library. My own preference for testing whether or not such a solution would work or not would be to use our current b2/Boost Build test jamfiles, but if this is too difficult to do we still need some way to test the solution for each library so each library maintainer can see whether or not the solution we choose works for his library.
This should be our first goal and all other goals should be tabled until and when we can have a solution where this works flawlessly and there is a general agreement that whatever proposal is chosen to implement this is best.
Only when the above is fully accomplished should Boost then look into the goal of using CMake internally, perhaps for test, docs, building libraries, or what-not. We need to work this way because every solution is bound to end with comments that it does X but does not do Y or it does X and Y but does not do Z, and we all know where these endless arguments end: that we end up doing nothing because we start by trying to do too much and we can never reach agreement.
I agree with you in so far, that the focus should be on the interface (how to consume boost libraries) and less on the implementation details, but I see two practical reasons, why it is beneficial to immediately use cmake (at least) for building too:
1) Creating a CMake target for a separately compiled binary that can be consumed by the user is about the same effort as using cmake to build the library (if necessary) in which case you get the interface information for free (switching to cmake for unit-tests and docs is a different topic).
2) "making Boost libraries [...] available to CMake projects" for me also means I have an easy way to compile those libraries with specific compiler flags that match my project settings - otherwise, what is the advantage compared to the current mechanism that is already build into cmake? Ensuring consistent build flags is relatively easy if all libraries use the same build system (in this case cmake) but can become quite complex if you want to implement an automatic translation from cmake to b2 or vice versa.
So yes, while a library (build-) interface and consummation should be the focus of the review, I'd expect that at least building the libraries with cmake will be the simplest way to implement that interface.
Just to be completely clear: Just because the "reference implementation" of a cmake interface is probably a cmake-based build, that doesn't mean everyone has to actually adopt that reference implementation as long as the interface is consistent.
Boost Build has a feature which allows the programmer to test at build time what C++ features are supported for a particular compiler implementation and compile parameters, and change the build internally on-the-fly accordingly. I do not know if any Boost non-header only library uses this feature but if one does it will be difficult translating this to CMake terms, if such a similar feature exists in CMake.
As Richard pointed out, such features exist in cmake too and have been in common use for years. E.g. It has a built-in mechanism that lets you directly query for most of the post c++03 compiler features: (https://cmake.org/cmake/help/v3.13/prop_gbl/CMAKE_CXX_KNOWN_FEATURES.html) and as a more general feature, you can always check if a particular source file using some feature or type compiles or not: (https://cmake.org/cmake/help/v3.13/module/CheckCXXSourceCompiles.html) Could you maybe point me to some examples where this feature is used so one could estimate how difficult a conversion is?
It is for such a reason that as soon as you start talking about using CMake for anything else but identifying Boost libraries for the purposes of end-user use of Boost libraries in their own CMake scripts,
Just consuming precompiled/pre-installed boost libraries in a cmake script is already possible thanks to the work of cmake contributors and the people providing boost packages for vcpkg, conan and other package management systems (some of those contributors are probably also boost library maintainers?). Those solutions have to keep playing catch up with the latest boost release if something significant changes (like name-mangling or a new library), but by and large they work quite well. Imho an official boost solution should provide a bit more functionality to justify the effort.
that you may run into much more work than you can imagine. You just can't tell individual library developers to translate all their own use of Boost Build into CMake, as if this will be simple for them to do, and they will just do it with little time spent and as if it will be no work for them.
Let's get one thing straight: Switching to cmake will require work, no one is denying that and it will not happen during the course of a single release. And if library developers are not willing to invest some time to do this, we can forget about it all together. However, it is my impression that many developers are willing to make that effort. All that is lacking is a clear decision of what the (semi-) final solution should look like to make sure the effort isn't wasted. Now as I said. I'm not convinced, that only providing interface information is actually less work than building boost with cmake and letting it generate the cmake information automatically. That probably depends to a large degree if those cmake files can be auto generated by b2. However, just as there seem to be more complex build scenarios, there are probably scenarios where the auto-generated cmake-config files are not enough.
Boost Build, for better or worse, has some extensive features which have no easy correspondence to CMake.
Such as? If plain cmake is lacking any important features that are needed by boost, then a replacement is exactly what a boost cmake library should provide.
Unless you or someone else is willing to develop a bridge between Boost Build and CMake for features of Boost Build which libraries rely on, then individual Boost libraries which use such Boost Build features are going to find it very difficult to transition to CMake, possibly even for something as seemingly innocuous as building the library itself using a CMake script.
Let's not forget, that there are already solutions out there that build (almost?) all of boost with cmake. Here is one: https://github.com/boost-cmake/boost Now instead of adopting and tweaking one of those existing solutions during review, you are imho asking for the development of yet another solution. I'm not sure if that is actually an easier path forward. That doesn't mean that an individual library author can't decide he wants to stick to b2 internally and just provide a cmake interface file, but we for the time being and
This is why I have urged the move to CMake by Boost to proceed slowly by baby steps, rather than to try doing things that may take a huge initial effort to get done, and therefore may end up never accomplishing anything.
That is kind of what I'm trying to achieve with my individual mini PRs. They are in no way a replacement for the existing boost build system, as they only cover a very narrow use case, but even those also build the boost libraries. Again, I don't think building is the issue (as I said, it has been done already) and if someone really encounters a problem, I'm sure people on the ML and elsewhere are willing to help. IMHO it will be far more difficult to agree all the bike shedding topics related to how exactly boost libraries should be made available to the consumer. Also, proceeding by Baby steps means that boost will stay in limbo between cmake and boost build for quite some time, where library maintainers will have to maintain two systems and more or less constantly upgrading their CMake files. It doesn't have to happen all in one big step (I agree that that is not realistic), but considering that consumers can already use boost libraries in their cmake projects (albeit with a few pain points here and there) and that there are already proof of concepts for more fully fledged solutions out there, I think the initial goal should be a tad more ambitious. Mike
On 10/9/2018 7:09 PM, mike via Boost wrote:
-----Original Message----- From: Boost <boost-bounces@lists.boost.org> On Behalf Of Edward Diener via Boost Sent: Monday, October 8, 2018 10:22 PM
On 10/8/2018 6:15 AM, Mike Dev via Boost wrote:
-----Original Message----- From: Boost <boost-bounces@lists.boost.org> On Behalf Of Edward Diener via Boost Sent: Monday, October 1, 2018 8:00 PM
On 10/1/2018 1:07 PM, Robert Ramey via Boost wrote:
Background ==========
My personal knowledge of build systems is somewhat limited.
a) I don't really understand CMake all that well. I have made CMakeLists.txt files for both the serialization library and the safe numerics libraries as an aid to build and maintain IDE and make files used in running tests and posting results to CDash.
b) I don't really understand our current system, Boost Build, very well either. I've made jam files for building and running tests for the serialization and safe numerics libraries.
c) I recently reviewed C++Now 2017: Daniel Pfeifer's excellent video from C++Now 2017 “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk . I recommend it highly.
Despite my lack of knowledge in this area, or perhaps because of it, I've volunteered to direct the effort to evolve Boost so that users of Boost can avail themselves of the benefits of CMake. This effort will be organized as follows:
a) A discussion on the boost developers mailing list with the goal of reaching a consensus as to what benefits CMake can and should provide to users and developers of Boost. This effort will commence with the posting of this notice. I hope that this discussion can be more or less resolved within 30 days.
b) I will then synthesize from the above discussion a call for proposals which lists the requirements and requirements.
c) Those proposing CMake for boost will have approximately 90 days to prepare their proposals. Their proposal are expected to look similar to a boost library proposal. That is they are expected to have some (CMake) code, along with tests, documentation including tutorials and reference. This is only an expectation. It's conceivable that such a proposal might only contain conventions and and samples of what boost library should contain to fulfill the requirements. As with boost libraries, submissions are not required to totally complete. But they have to be sufficiently complete to convince reviewers the the submission fulfills the stated requirements and can be finished in good time.
d) After the 90 day period, submissions will be reviewed simultaneously. Note that this is at a variance from normal boost procedure of reviewing submissions one at a time. Since we expect to receive a number of submissions and only one can be accepted, the normal boost protocol can't really function. Target date for this review is 1 February 2019.
e) After the review period the review manager will prepare a report which includes the decision of which submission will be accepted into the official boost distribution. Currently, I, Robert Ramey, expect to be review manager. However, it's possible that this by the time the review is undertaken, this coveted position could be assigned to some more worthy candidate.
Scope, Requirements and Features ================================
We presume that submissions will fulfill the applicable requirements of any boost submission. In addition we would like to see the following addressed.
Library Build ------------- “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk Recommends that all libraries - including header only libraries have a CMakeList.txt file. What should this include for Boost Libraries?
Library Test ------------ Should facilities for "testing" be only done by developers? or should users who acquire library packages also be able to test libraries. Should CMake testing results posting be used - CDASH? Should we just skip the issue of Library Testing and continue to depend on Boost Build.
Library Packaging ----------------- Is the library packaging facility provide by CMake - CPACK - useful to boost. Should boost libraries be updated to support it?
Dependencies ------------ If the above is implemented, can we depend upon CMake to handle library dependencies? Or does some special functionality Do circular dependencies constitute a problem?
Modularity ---------- Currently boost is organized as a tightly integrated group of libraries. This organization manifests itself in a number of ways. We have a Boost "super project" in github. Boost libraries are "sub projects" of this super project. This known structure is exploited by and depended upon by boost tools. Should CMake support continue this policy/design. We distribute boost as "super project". Should we continue to do this or distribute libraries on a library by library basis. Does CMake/CPACK etc. make this "simple"?
Documentation/testing --------------------- What support should be require for uses of Boost CMake implementation. Documentation, examples, templates for helping library developers. How should the CMake design/implementation be tested?
Other considerations -------------------- add your requirement/features here.
Useful Resources =============== “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk https://gist.github.com/mbinna/c61dbb39bca0e4fb7d1f73b0d66a4fd1 https://github.com/purpleKarrot/Boost.CMake https://www.youtube.com/watch?v=eC9-iRN2b04
Please try to keep this discussion on this list.
I know that there has been a lot going on on slack. But having it in two places makes it harder for me to review and summarize. Also slack doesn't conveniently maintain comment hierarchy. So please try to keep this discussion on this list.
I strongly suggest that the goal of using CMake with Boost begin with making Boost libraries, whether header-only or built, available to CMake projects. This also means that we have a way of testing whether or not such a solution actually works for any given Boost library. My own preference for testing whether or not such a solution would work or not would be to use our current b2/Boost Build test jamfiles, but if this is too difficult to do we still need some way to test the solution for each library so each library maintainer can see whether or not the solution we choose works for his library.
This should be our first goal and all other goals should be tabled until and when we can have a solution where this works flawlessly and there is a general agreement that whatever proposal is chosen to implement this is best.
Only when the above is fully accomplished should Boost then look into the goal of using CMake internally, perhaps for test, docs, building libraries, or what-not. We need to work this way because every solution is bound to end with comments that it does X but does not do Y or it does X and Y but does not do Z, and we all know where these endless arguments end: that we end up doing nothing because we start by trying to do too much and we can never reach agreement.
I agree with you in so far, that the focus should be on the interface (how to consume boost libraries) and less on the implementation details, but I see two practical reasons, why it is beneficial to immediately use cmake (at least) for building too:
1) Creating a CMake target for a separately compiled binary that can be consumed by the user is about the same effort as using cmake to build the library (if necessary) in which case you get the interface information for free (switching to cmake for unit-tests and docs is a different topic).
2) "making Boost libraries [...] available to CMake projects" for me also means I have an easy way to compile those libraries with specific compiler flags that match my project settings - otherwise, what is the advantage compared to the current mechanism that is already build into cmake? Ensuring consistent build flags is relatively easy if all libraries use the same build system (in this case cmake) but can become quite complex if you want to implement an automatic translation from cmake to b2 or vice versa.
So yes, while a library (build-) interface and consummation should be the focus of the review, I'd expect that at least building the libraries with cmake will be the simplest way to implement that interface.
Just to be completely clear: Just because the "reference implementation" of a cmake interface is probably a cmake-based build, that doesn't mean everyone has to actually adopt that reference implementation as long as the interface is consistent.
Boost Build has a feature which allows the programmer to test at build time what C++ features are supported for a particular compiler implementation and compile parameters, and change the build internally on-the-fly accordingly. I do not know if any Boost non-header only library uses this feature but if one does it will be difficult translating this to CMake terms, if such a similar feature exists in CMake.
As Richard pointed out, such features exist in cmake too and have been in common use for years. E.g. It has a built-in mechanism that lets you directly query for most of the post c++03 compiler features: (https://cmake.org/cmake/help/v3.13/prop_gbl/CMAKE_CXX_KNOWN_FEATURES.html) and as a more general feature, you can always check if a particular source file using some feature or type compiles or not: (https://cmake.org/cmake/help/v3.13/module/CheckCXXSourceCompiles.html)
Could you maybe point me to some examples where this feature is used so one could estimate how difficult a conversion is?
The feature is essentially programmed by config and predef to provide build-time testing of compiler support and testing of such things like the OS versions and compiler versions etc. See https://www.boost.org/doc/libs/1_68_0/libs/config/doc/html/boost_config/buil... for the explanation in config and https://www.boost.org/doc/libs/1_68_0/doc/html/predef/check_utilities.html for checking predef values at build time. The basics of the feature are offered by Boost Build as explained at https://boostorg.github.io/build/manual/develop/index.html#bbv2.reference.ru... in reference to the check-target-builds rule. Plenty of libraries use the feature during testing and I imagine that some of them may use the feature during building. Asking individual libraries to convert their tests or build to use some CMake equivalent, for every possible setting in config and predef which they use, is not realistic. I understand that CMake may have an equivalent technique, but who is going to program that technique to offer all the equivalents which config and predef currently offer ?
It is for such a reason that as soon as you start talking about using CMake for anything else but identifying Boost libraries for the purposes of end-user use of Boost libraries in their own CMake scripts,
Just consuming precompiled/pre-installed boost libraries in a cmake script is already possible thanks to the work of cmake contributors and the people providing boost packages for vcpkg, conan and other package management systems (some of those contributors are probably also boost library maintainers?). Those solutions have to keep playing catch up with the latest boost release if something significant changes (like name-mangling or a new library), but by and large they work quite well. Imho an official boost solution should provide a bit more functionality to justify the effort.
that you may run into much more work than you can imagine. You just can't tell individual library developers to translate all their own use of Boost Build into CMake, as if this will be simple for them to do, and they will just do it with little time spent and as if it will be no work for them.
Let's get one thing straight: Switching to cmake will require work, no one is denying that and it will not happen during the course of a single release. And if library developers are not willing to invest some time to do this, we can forget about it all together. However, it is my impression that many developers are willing to make that effort. All that is lacking is a clear decision of what the (semi-) final solution should look like to make sure the effort isn't wasted.
Now as I said. I'm not convinced, that only providing interface information is actually less work than building boost with cmake and letting it generate the cmake information automatically. That probably depends to a large degree if those cmake files can be auto generated by b2. However, just as there seem to be more complex build scenarios, there are probably scenarios where the auto-generated cmake-config files are not enough.
Boost Build, for better or worse, has some extensive features which have no easy correspondence to CMake.
Such as? If plain cmake is lacking any important features that are needed by boost, then a replacement is exactly what a boost cmake library should provide.
Unless you or someone else is willing to develop a bridge between Boost Build and CMake for features of Boost Build which libraries rely on, then individual Boost libraries which use such Boost Build features are going to find it very difficult to transition to CMake, possibly even for something as seemingly innocuous as building the library itself using a CMake script.
Let's not forget, that there are already solutions out there that build (almost?) all of boost with cmake. Here is one: https://github.com/boost-cmake/boost
Now instead of adopting and tweaking one of those existing solutions during review, you are imho asking for the development of yet another solution. I'm not sure if that is actually an easier path forward. That doesn't mean that an individual library author can't decide he wants to stick to b2 internally and just provide a cmake interface file, but we for the time being and
This is why I have urged the move to CMake by Boost to proceed slowly by baby steps, rather than to try doing things that may take a huge initial effort to get done, and therefore may end up never accomplishing anything.
That is kind of what I'm trying to achieve with my individual mini PRs. They are in no way a replacement for the existing boost build system, as they only cover a very narrow use case, but even those also build the boost libraries. Again, I don't think building is the issue (as I said, it has been done already) and if someone really encounters a problem, I'm sure people on the ML and elsewhere are willing to help. IMHO it will be far more difficult to agree all the bike shedding topics related to how exactly boost libraries should be made available to the consumer.
Also, proceeding by Baby steps means that boost will stay in limbo between cmake and boost build for quite some time, where library maintainers will have to maintain two systems and more or less constantly upgrading their CMake files. It doesn't have to happen all in one big step (I agree that that is not realistic), but considering that consumers can already use boost libraries in their cmake projects (albeit with a few pain points here and there) and that there are already proof of concepts for more fully fledged solutions out there, I think the initial goal should be a tad more ambitious.
Mike
On Wed, Oct 10, 2018 at 2:43 AM Edward Diener via Boost <boost@lists.boost.org> wrote:
The feature is essentially programmed by config and predef to provide build-time testing of compiler support and testing of such things like the OS versions and compiler versions etc. See https://www.boost.org/doc/libs/1_68_0/libs/config/doc/html/boost_config/buil... for the explanation in config and https://www.boost.org/doc/libs/1_68_0/doc/html/predef/check_utilities.html for checking predef values at build time. The basics of the feature are offered by Boost Build as explained at https://boostorg.github.io/build/manual/develop/index.html#bbv2.reference.ru... in reference to the check-target-builds rule. Plenty of libraries use the feature during testing and I imagine that some of them may use the feature during building. Asking individual libraries to convert their tests or build to use some CMake equivalent, for every possible setting in config and predef which they use, is not realistic. I understand that CMake may have an equivalent technique, but who is going to program that technique to offer all the equivalents which config and predef currently offer ?
Why do you think a potential Boost.CMake proposal cannot offer an equivalent of check-target-builds that internally uses the checks based on Boost.Config or Boost.Predef? Because, AFAIU, check-target-builds is exactly that - a compile test that involves a preprocessor check of one of the macros + a bit of caching on Boost.Build side. Doesn't sound like something that can't be done in CMake. As for the conversion process, I don't think it is reasonable to expect a single person, a Boost.CMake candidate author, to convert all Boost Jamfiles to CMake. Library authors *will* have to learn the new build system and do the conversion of their libraries. Hopefully, a proposed Boost.CMake should make it easier, but it won't be a free ride. PS: Also, let's not over-quote, please.
On 10/9/18 4:58 PM, Andrey Semashev via Boost wrote:
in reference to the check-target-builds rule. Plenty of libraries use the feature during testing and I imagine that some of them may use the feature during building. Asking individual libraries to convert their tests or build to use some CMake equivalent, for every possible setting in config and predef which they use, is not realistic. I understand that CMake may have an equivalent technique, but who is going to program that technique to offer all the equivalents which config and predef currently offer ?
At this point I have no idea what it might take for a CMake fulfill such a requirement. But of course that's not relevant at this point. So this would be another suggested requirement: "Build and Test should be able to adjust target requirements according to the test results of boost/config - similar to how boost build does today." Again, This is noted. I'll take it under advismment when I craft the first cut of the "scope and requirements" proposal. Robert Ramey
On 10/9/2018 7:58 PM, Andrey Semashev via Boost wrote:
On Wed, Oct 10, 2018 at 2:43 AM Edward Diener via Boost <boost@lists.boost.org> wrote:
The feature is essentially programmed by config and predef to provide build-time testing of compiler support and testing of such things like the OS versions and compiler versions etc. See https://www.boost.org/doc/libs/1_68_0/libs/config/doc/html/boost_config/buil... for the explanation in config and https://www.boost.org/doc/libs/1_68_0/doc/html/predef/check_utilities.html for checking predef values at build time. The basics of the feature are offered by Boost Build as explained at https://boostorg.github.io/build/manual/develop/index.html#bbv2.reference.ru... in reference to the check-target-builds rule. Plenty of libraries use the feature during testing and I imagine that some of them may use the feature during building. Asking individual libraries to convert their tests or build to use some CMake equivalent, for every possible setting in config and predef which they use, is not realistic. I understand that CMake may have an equivalent technique, but who is going to program that technique to offer all the equivalents which config and predef currently offer ?
Why do you think a potential Boost.CMake proposal cannot offer an equivalent of check-target-builds that internally uses the checks based on Boost.Config or Boost.Predef?
I do not think what you propose. I am just pointing out that this feature, used extensively from config and/or predef for a number of Boost libraries, needs to have an equally easy CMake equivalent so that converting the library's build or test jamfile to CMake is fairly trivial.
Because, AFAIU, check-target-builds is exactly that - a compile test that involves a preprocessor check of one of the macros + a bit of caching on Boost.Build side. Doesn't sound like something that can't be done in CMake.
Why do you think a potential Boost.CMake proposal cannot offer an equivalent of check-target-builds that internally uses the checks based on Boost.Config or Boost.Predef?
I do not think what you propose. I am just pointing out that this feature, used extensively from config and/or predef for a number of Boost libraries, needs to have an equally easy CMake equivalent so that converting the library's build or test jamfile to CMake is fairly trivial.
Sorry, but WHY exactly do you need to have the checks from Boost.Config/Predef in CMake? Do they exist in B2? In which form? Isn't Boost.Config/Predef a set of headers defining macros for consumption in source code?
On 10/15/2018 3:10 AM, Alexander Grund via Boost wrote:
Why do you think a potential Boost.CMake proposal cannot offer an equivalent of check-target-builds that internally uses the checks based on Boost.Config or Boost.Predef?
I do not think what you propose. I am just pointing out that this feature, used extensively from config and/or predef for a number of Boost libraries, needs to have an equally easy CMake equivalent so that converting the library's build or test jamfile to CMake is fairly trivial.
Sorry, but WHY exactly do you need to have the checks from Boost.Config/Predef in CMake? Do they exist in B2? In which form? Isn't Boost.Config/Predef a set of headers defining macros for consumption in source code?
A great many Boost libraries use the checks from Boost.Config/Predef in their jamfiles. What do you propose to do when these jamfiles need to be converted for the purposes of using CMake for these libraries ? Asking each library maintainer to provide some CMake equivalent for the checks that they do is not realistic. Providing a CMake equivalent in the Boost.Config and Boost.Predef libraries for all the current checks which are offered by those libraries, and then using these equivalents by other libraries which currently use them, is realistic. But someone has to do the work of converting all the current checks in Boost.Config and Boost.Predef to a CMake equivalent, and which can be used by other libraries in their CMake implementation.
Sorry, but WHY exactly do you need to have the checks from Boost.Config/Predef in CMake? Do they exist in B2? In which form? Isn't Boost.Config/Predef a set of headers defining macros for consumption in source code?
A great many Boost libraries use the checks from Boost.Config/Predef in their jamfiles. What do you propose to do when these jamfiles need to be converted for the purposes of using CMake for these libraries ?
Providing a CMake equivalent in the Boost.Config and Boost.Predef
What I meant was: I was only aware of a C++ part (header files) of Boost.Config/Predef. Can you point me to where those header defines are transported into bjam so they can be used by those other libs in their jamfiles? libraries for all the current checks which are offered by those libraries, and then using these equivalents by other libraries which currently use them Exactly this would be the proposal. I can comment on possible effort when I have seen how it is done in bjam.
On Mon, Oct 15, 2018 at 8:21 AM Alexander Grund via Boost < boost@lists.boost.org> wrote:
Sorry, but WHY exactly do you need to have the checks from Boost.Config/Predef in CMake? Do they exist in B2? In which form? Isn't Boost.Config/Predef a set of headers defining macros for consumption in source code?
A great many Boost libraries use the checks from Boost.Config/Predef in their jamfiles. What do you propose to do when these jamfiles need to be converted for the purposes of using CMake for these libraries ?
What I meant was: I was only aware of a C++ part (header files) of Boost.Config/Predef. Can you point me to where those header defines are transported into bjam so they can be used by those other libs in their jamfiles?
Providing a CMake equivalent in the Boost.Config and Boost.Predef libraries for all the current checks which are offered by those libraries, and then using these equivalents by other libraries which currently use them
Exactly this would be the proposal. I can comment on possible effort when I have seen how it is done in bjam.
For Predef <https://github.com/boostorg/predef/tree/develop/tools/check> with documentation for usage here < https://www.boost.org/doc/libs/1_68_0/doc/html/predef/check_utilities.html>. For Config <https://github.com/boostorg/config/tree/develop/checks> and documentation for that here < https://www.boost.org/doc/libs/1_68_0/libs/config/doc/html/boost_config/buil...
.
-- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net
For Predef <https://github.com/boostorg/predef/tree/develop/tools/check> with documentation for usage here <https://www.boost.org/doc/libs/1_68_0/doc/html/predef/check_utilities.html>.
For Config <https://github.com/boostorg/config/tree/develop/checks> and documentation for that here <https://www.boost.org/doc/libs/1_68_0/libs/config/doc/html/boost_config/build_config.html>. Thanks. Seeing this I approximate the work required to convert the checks of Config to ~1h, maybe 2h. Usage would be: include(BoostConfigFeatures) if(BOOST_FEATURE_cxx11_local_class_template_parameters) add_executable(...) endif()
Some bike-shedding required: - Name for the CMake file that contains this - Naming scheme for the results of the checks (the variable in the "if") - Eagerly test all or on demand: boost_config_has(RESULT cxx11_local_class_template_parameters) if(RESULT) add_executable(...) endif() Similar for predef: include(BoostPredefCheck) boost_predef_check(RESULT "BOOST_OS_WINDOWS != 0" "BOOST_OS_VMS != 0") if(RESULT) add_executable(...) endif() Work is about the same and yes some checks can be omitted (e.g. boost_predef_check(RESULT "BOOST_OS_WINDOWS != 0") --> RESULT==WIN32 standard CMake variable)
On 10/16/2018 3:34 AM, Alexander Grund via Boost wrote:
For Predef <https://github.com/boostorg/predef/tree/develop/tools/check> with documentation for usage here <https://www.boost.org/doc/libs/1_68_0/doc/html/predef/check_utilities.html>.
For Config <https://github.com/boostorg/config/tree/develop/checks> and documentation for that here <https://www.boost.org/doc/libs/1_68_0/libs/config/doc/html/boost_config/build_config.html>.
Thanks. Seeing this I approximate the work required to convert the checks of Config to ~1h, maybe 2h. Usage would be: include(BoostConfigFeatures) if(BOOST_FEATURE_cxx11_local_class_template_parameters) add_executable(...) endif()
Programmers are notorious for estimating the work to do any programming as about an order of magnitude less than it really takes <g>.
Some bike-shedding required: - Name for the CMake file that contains this
Anything reasonable with 'config' in the name for config and 'predef' in the name for predef.
- Naming scheme for the results of the checks (the variable in the "if")
As close as possible, using CMake naming conventions, to the name in config or predef.
- Eagerly test all or on demand:
What is the difference in CMake terms ? In Boost Build I am fairly sure the checks are done on demand, and usually involves some sort of compile/build/run/test-result cycle which would make pre-testing all checks in a library, and holding the result for further inquiry, pretty expensive.
boost_config_has(RESULT cxx11_local_class_template_parameters) if(RESULT) add_executable(...) endif()
Similar for predef: include(BoostPredefCheck) boost_predef_check(RESULT "BOOST_OS_WINDOWS != 0" "BOOST_OS_VMS != 0") if(RESULT) add_executable(...) endif()
Work is about the same and yes some checks can be omitted (e.g. boost_predef_check(RESULT "BOOST_OS_WINDOWS != 0") --> RESULT==WIN32 standard CMake variable)
Even if there is a standard CMake variable it would be worthwhile to have the check so that libraries which use the check can use the same syntax throughout.
Programmers are notorious for estimating the work to do any programming as about an order of magnitude less than it really takes <g>. True ;) So essence: Very low effort to port this to CMake due to existing bjam code What is the difference in CMake terms ? In Boost Build I am fairly sure the checks are done on demand, and usually involves some sort of compile/build/run/test-result cycle which would make pre-testing all checks in a library, and holding the result for further inquiry, pretty expensive. On demand: User calls a function and uses the result (see example) Eager: All checks executed on including the CMake file. (Note: Results are cached, hence this overhead is only once for all libraries for the mono-build), yes high overhead for a single lib, but this is what CMake also does to e.g. get compiler type, version, OS, ... and usage is easier (no additional calls, see examples)
On 10/16/2018 10:22 AM, Alexander Grund via Boost wrote:
Programmers are notorious for estimating the work to do any programming as about an order of magnitude less than it really takes <g>. True ;) So essence: Very low effort to port this to CMake due to existing bjam code
That is great !
What is the difference in CMake terms ? In Boost Build I am fairly sure the checks are done on demand, and usually involves some sort of compile/build/run/test-result cycle which would make pre-testing all checks in a library, and holding the result for further inquiry, pretty expensive. On demand: User calls a function and uses the result (see example) Eager: All checks executed on including the CMake file. (Note: Results are cached, hence this overhead is only once for all libraries for the mono-build), yes high overhead for a single lib, but this is what CMake also does to e.g. get compiler type, version, OS, ... and usage is easier (no additional calls, see examples)
Thanks for the explanation.
On Tue, Oct 16, 2018 at 9:13 AM Edward Diener via Boost < boost@lists.boost.org> wrote:
On 10/16/2018 3:34 AM, Alexander Grund via Boost wrote:
Similar for predef: include(BoostPredefCheck) boost_predef_check(RESULT "BOOST_OS_WINDOWS != 0" "BOOST_OS_VMS != 0") if(RESULT) add_executable(...) endif()
Work is about the same and yes some checks can be omitted (e.g. boost_predef_check(RESULT "BOOST_OS_WINDOWS != 0") --> RESULT==WIN32 standard CMake variable)
Even if there is a standard CMake variable it would be worthwhile to have the check so that libraries which use the check can use the same syntax throughout.
Actually the CMake built-in is not equivalent to the Predef version AFAICT. For instance the Predef version can compare against version numbers, and not just zero and non-zero unlike the CMake WIN32 boolean var. The next closest CMake var would be CMAKE_SYSTEM_VERSION.. Although I do love how the format of the version number is not documented for that :-) Hence the closer equivalent would be "WIN32 && CMAKE_SYSTEM_VERSION != 0" (where 0 could be 8.1 maybe?). Also in B2 you wouldn't use Predef for checking the OS like above. You would just use the built in "<target-os>windows" feature. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net
On 09/10/2018 23:42, Edward Diener via Boost wrote:
On 10/9/2018 7:09 PM, mike via Boost wrote:
As Richard pointed out, such features exist in cmake too and have been in common use for years. E.g. It has a built-in mechanism that lets you directly query for most of the post c++03 compiler features: (https://cmake.org/cmake/help/v3.13/prop_gbl/CMAKE_CXX_KNOWN_FEATURES.html)
and as a more general feature, you can always check if a particular source file using some feature or type compiles or not: (https://cmake.org/cmake/help/v3.13/module/CheckCXXSourceCompiles.html)
Could you maybe point me to some examples where this feature is used so one could estimate how difficult a conversion is?
The feature is essentially programmed by config and predef to provide build-time testing of compiler support and testing of such things like the OS versions and compiler versions etc. See https://www.boost.org/doc/libs/1_68_0/libs/config/doc/html/boost_config/buil... for the explanation in config and https://www.boost.org/doc/libs/1_68_0/doc/html/predef/check_utilities.html for checking predef values at build time. The basics of the feature are offered by Boost Build as explained at https://boostorg.github.io/build/manual/develop/index.html#bbv2.reference.ru... in reference to the check-target-builds rule. Plenty of libraries use the feature during testing and I imagine that some of them may use the feature during building. Asking individual libraries to convert their tests or build to use some CMake equivalent, for every possible setting in config and predef which they use, is not realistic. I understand that CMake may have an equivalent technique, but who is going to program that technique to offer all the equivalents which config and predef currently offer ?
This is all supported by CMake for years as mike said, and it's utterly trivial to write such feature tests. If you would like some examples, here's some I wrote for Xerces-C++: https://github.com/apache/xerces-c/tree/trunk/cmake For example, here's a simple test for const [it's a port of Autoconf logic] with CheckCXXSourceCompiles: https://github.com/apache/xerces-c/blob/trunk/cmake/XercesConst.cmake this could be adapted for any feature you care to check, from constexpr to user-defined literals, or any other feature which can be checked by running the compiler. This is a set of checks for integer type sizes: https://github.com/apache/xerces-c/blob/trunk/cmake/XercesIntTypes.cmake This is a more complex set of checks for SSE intrinsics: https://github.com/apache/xerces-c/blob/trunk/cmake/XercesSSE2.cmake Here's some checks to verify Boost is working properly: https://gitlab.com/codelibre/ome-common-cpp/blob/master/cmake/BoostChecks.cm... Testing of linker version scripts: https://gitlab.com/libtiff/libtiff/blob/master/CMakeLists.txt#L198 Checking include presence: https://gitlab.com/libtiff/libtiff/blob/master/CMakeLists.txt#L212 Checking compiler keyword presence and fallbacks: https://gitlab.com/libtiff/libtiff/blob/master/CMakeLists.txt#L226 Checking structure presence: https://gitlab.com/libtiff/libtiff/blob/master/CMakeLists.txt#L226 Checking library symbol exports: https://gitlab.com/libtiff/libtiff/blob/master/CMakeLists.txt#L226 I don't think any of this is difficult. If anything, you can likely reuse the existing source code used by the Boost.Build checks, maybe even sourced directly without any changes--just wrap it with check_cxx_source_compiles or other feature test macros. Regards, Roger
On 10/10/2018 4:44 AM, Roger Leigh via Boost wrote:
On 09/10/2018 23:42, Edward Diener via Boost wrote:
On 10/9/2018 7:09 PM, mike via Boost wrote:
As Richard pointed out, such features exist in cmake too and have been in common use for years. E.g. It has a built-in mechanism that lets you directly query for most of the post c++03 compiler features: (https://cmake.org/cmake/help/v3.13/prop_gbl/CMAKE_CXX_KNOWN_FEATURES.html)
and as a more general feature, you can always check if a particular source file using some feature or type compiles or not: (https://cmake.org/cmake/help/v3.13/module/CheckCXXSourceCompiles.html)
Could you maybe point me to some examples where this feature is used so one could estimate how difficult a conversion is?
The feature is essentially programmed by config and predef to provide build-time testing of compiler support and testing of such things like the OS versions and compiler versions etc. See https://www.boost.org/doc/libs/1_68_0/libs/config/doc/html/boost_config/buil... for the explanation in config and https://www.boost.org/doc/libs/1_68_0/doc/html/predef/check_utilities.html for checking predef values at build time. The basics of the feature are offered by Boost Build as explained at https://boostorg.github.io/build/manual/develop/index.html#bbv2.reference.ru... in reference to the check-target-builds rule. Plenty of libraries use the feature during testing and I imagine that some of them may use the feature during building. Asking individual libraries to convert their tests or build to use some CMake equivalent, for every possible setting in config and predef which they use, is not realistic. I understand that CMake may have an equivalent technique, but who is going to program that technique to offer all the equivalents which config and predef currently offer ?
This is all supported by CMake for years as mike said, and it's utterly trivial to write such feature tests. If you would like some examples, here's some I wrote for Xerces-C++:
https://github.com/apache/xerces-c/tree/trunk/cmake
For example, here's a simple test for const [it's a port of Autoconf logic] with CheckCXXSourceCompiles: https://github.com/apache/xerces-c/blob/trunk/cmake/XercesConst.cmake this could be adapted for any feature you care to check, from constexpr to user-defined literals, or any other feature which can be checked by running the compiler.
This is a set of checks for integer type sizes: https://github.com/apache/xerces-c/blob/trunk/cmake/XercesIntTypes.cmake
This is a more complex set of checks for SSE intrinsics: https://github.com/apache/xerces-c/blob/trunk/cmake/XercesSSE2.cmake
Here's some checks to verify Boost is working properly: https://gitlab.com/codelibre/ome-common-cpp/blob/master/cmake/BoostChecks.cm...
Testing of linker version scripts: https://gitlab.com/libtiff/libtiff/blob/master/CMakeLists.txt#L198
Checking include presence: https://gitlab.com/libtiff/libtiff/blob/master/CMakeLists.txt#L212
Checking compiler keyword presence and fallbacks: https://gitlab.com/libtiff/libtiff/blob/master/CMakeLists.txt#L226
Checking structure presence: https://gitlab.com/libtiff/libtiff/blob/master/CMakeLists.txt#L226
Checking library symbol exports: https://gitlab.com/libtiff/libtiff/blob/master/CMakeLists.txt#L226
I don't think any of this is difficult. If anything, you can likely reuse the existing source code used by the Boost.Build checks, maybe even sourced directly without any changes--just wrap it with check_cxx_source_compiles or other feature test macros.
My point is that you can not ask each library maintainer to re-invent what he already uses from Boost config and/or Boost predef. You need to reprogram this in CMake for each case in config and predef, and then provide a CMake-like interface to it for other libraries. I do not care how easy you find it in CMake, individual library maintainers are not going to spend time re-duplicating everything they need from config and predef just so they can build their library and/or run their library tests. We are talking about some 130+ potential Boost libraries we have to deal with, and any conversion to CMake that is non-trivial is going to be very unpopular.
On 10/10/18 15:23, Edward Diener via Boost wrote:
On 10/10/2018 4:44 AM, Roger Leigh via Boost wrote:
This is all supported by CMake for years as mike said, and it's utterly trivial to write such feature tests. If you would like some examples, here's some I wrote for Xerces-C++:
[examples]
I don't think any of this is difficult. If anything, you can likely reuse the existing source code used by the Boost.Build checks, maybe even sourced directly without any changes--just wrap it with check_cxx_source_compiles or other feature test macros.
My point is that you can not ask each library maintainer to re-invent what he already uses from Boost config and/or Boost predef. You need to reprogram this in CMake for each case in config and predef, and then provide a CMake-like interface to it for other libraries. I do not care how easy you find it in CMake, individual library maintainers are not going to spend time re-duplicating everything they need from config and predef just so they can build their library and/or run their library tests. We are talking about some 130+ potential Boost libraries we have to deal with, and any conversion to CMake that is non-trivial is going to be very unpopular.
The logic could be written to be either in a single place (which can provide CMake logic for other components to use), or duplicated in the different components. Which is better depends upon how decoupled you wish the components to be. If you always require a monolithic build, then one component could provide shared functionality. Otherwise, duplication is better to avoid tight coupling. As an example: https://gitlab.com/rleigh/ome-common-cpp/tree/master/cmake are modules I routinely share between related components of a larger project. Not the size of Boost though. I looked at the Boost config and predef links Rene provided in another email in this thread. The Boost.Config checks all look pretty straightforward. Should be possible to iterate over them and store each one in a variable, then we could provide them in the exported configuration for dependent components to consume. (While I see this component does have CMake support merged in, I don't see any configuration export. Where is that handled? Has anyone worked on it yet?) For Boost predef this isn't much different; run the program and store the results, export the variables. In both cases, CMake users can then simply run conditional logic such as: if (boost_feature) list(APPEND sources foo.cpp) endif() or if (boost_feature2 AND boost_feature3 AND NOT boost_feature 4) add_executable(foo …) … endif() and so on. Note that a number of the predef checks are already supported directly by CMake, such as if(MSVC) if(WIN32) etc. While the predef checks can be carried over for completeness, they are not necessary. I would be happy to work on some of this. I'm not sure exactly how to do so however. Where would I submit it? Is CMake development discussion happening on any other lists or other places? I don't want to tread on any toes or work at cross-purposes. Regards, Roger
On 10/15/2018 4:36 PM, Roger Leigh via Boost wrote:
On 10/10/18 15:23, Edward Diener via Boost wrote:
On 10/10/2018 4:44 AM, Roger Leigh via Boost wrote:
This is all supported by CMake for years as mike said, and it's utterly trivial to write such feature tests. If you would like some examples, here's some I wrote for Xerces-C++:
[examples]
I don't think any of this is difficult. If anything, you can likely reuse the existing source code used by the Boost.Build checks, maybe even sourced directly without any changes--just wrap it with check_cxx_source_compiles or other feature test macros.
My point is that you can not ask each library maintainer to re-invent what he already uses from Boost config and/or Boost predef. You need to reprogram this in CMake for each case in config and predef, and then provide a CMake-like interface to it for other libraries. I do not care how easy you find it in CMake, individual library maintainers are not going to spend time re-duplicating everything they need from config and predef just so they can build their library and/or run their library tests. We are talking about some 130+ potential Boost libraries we have to deal with, and any conversion to CMake that is non-trivial is going to be very unpopular.
The logic could be written to be either in a single place (which can provide CMake logic for other components to use), or duplicated in the different components. Which is better depends upon how decoupled you wish the components to be. If you always require a monolithic build, then one component could provide shared functionality. Otherwise, duplication is better to avoid tight coupling. As an example:
https://gitlab.com/rleigh/ome-common-cpp/tree/master/cmake
are modules I routinely share between related components of a larger project. Not the size of Boost though.
Nearly every library depends on Boost.Config. Boost.Predef was created much, much later than Boost.Config so I do not know how many libraries use its functionality. Creating support for Boost.Config check builds should be done once in Boost.Config and creating support Boost.Predef check builds should be done once in Boost.Predef. Expecting individual libraries to duplicate what is current in those libraries, for CMake, is not something the maintainers of those other libraries will want to do.
I looked at the Boost config and predef links Rene provided in another email in this thread.
The Boost.Config checks all look pretty straightforward. Should be possible to iterate over them and store each one in a variable, then we could provide them in the exported configuration for dependent components to consume.
(While I see this component does have CMake support merged in, I don't see any configuration export. Where is that handled? Has anyone worked on it yet?)
For Boost predef this isn't much different; run the program and store the results, export the variables.
In both cases, CMake users can then simply run conditional logic such as:
if (boost_feature) list(APPEND sources foo.cpp) endif()
or
if (boost_feature2 AND boost_feature3 AND NOT boost_feature 4) add_executable(foo …) … endif()
and so on.
Note that a number of the predef checks are already supported directly by CMake, such as if(MSVC) if(WIN32) etc. While the predef checks can be carried over for completeness, they are not necessary.
I would be happy to work on some of this. I'm not sure exactly how to do so however. Where would I submit it? Is CMake development discussion happening on any other lists or other places? I don't want to tread on any toes or work at cross-purposes.
It would be best to mention that you would like to help convert the check builds functionality in those two libraries to CMake on the appropriate library's Github page. You could bring this up as Issues for each of the two libraries and eventually create PRs.
Regards, Roger
Edward Diener wrote: ...
My point is that you can not ask each library maintainer to re-invent what he already uses from Boost config and/or Boost predef.
It should be noted here that CMake's philosophy with respect to C++ feature checks is different from Boost.Build's one. While in b2 we [ require cxx11_constexpr ] and do not build if this is not satisfied, in CMake one does target_compile_features(... cxx_constexpr) and then this requirement is propagated upwards to the targets that depend on the target requiring constexpr, the effect being that the top target is automatically compiled with -std=c++11. That's how CMake works and we probably should respect it and not try to swim against the current and try to force it into what we do now. Either way, this is basically the least of our worries. CMake can easily do all manners of configure-style checks.
On 10/10/2018 12:09, mike wrote:
It is for such a reason that as soon as you start talking about using CMake for anything else but identifying Boost libraries for the purposes of end-user use of Boost libraries in their own CMake scripts,
Just consuming precompiled/pre-installed boost libraries in a cmake script is already possible thanks to the work of cmake contributors and the people providing boost packages for vcpkg, conan and other package management systems (some of those contributors are probably also boost library maintainers?). Those solutions have to keep playing catch up with the latest boost release if something significant changes (like name-mangling or a new library), but by and large they work quite well. Imho an official boost solution should provide a bit more functionality to justify the effort.
I'm mostly watching this from the sidelines (since other than building Boost itself I use neither b2 nor cmake), so take this with a grain of salt, but: I think that "onboarding" the consumption of Boost libraries into Boost could be valuable in itself with no need to justify additional functionality, simply because it eliminates that catch-up process and helps ensure things are correct before release instead of after. And Boost is probably in a better position to track library dependencies. Having said that, I'm not sure how feasible it is in practice simply due to historically package deployment being left entirely to outside parties (especially on Linux), thus there being significant variation in the wild.
-----Original Message----- From: Boost <boost-bounces@lists.boost.org> On Behalf Of Robert Ramey via Boost Sent: Monday, October 1, 2018 7:07 PM
[...]
a) A discussion on the boost developers mailing list with the goal of reaching a consensus as to what benefits CMake can and should provide to users and developers of Boost. This effort will commence with the posting of this notice. I hope that this discussion can be more or less resolved within 30 days.
Personally, I hope to gain two things by this effort: 1) I would like to be able to clone the boost super project and/or individual boost libraries into a subfolder of my cmake based app and use those libraries along the lines of - add_subdirectory( libs/boost ) // assuming the super projects resides in \ (<app_root>/libs/boost) - target_link_libraries( my_app PRIVATE Boost::static_assert Boost::filesystem Boost::circular_buffer ) Which should make sure that a) All used boost libraries are built with the compiler flags specified in my cmake toolchain file, or the top level CMakeLists.txt file from my app project. (In particular c++ version and anything that influences ABI/API) b) Headers are found at their original location without any need to copy them to some installation/staging directory c) (Which is probably the same as b)) My IDE (that understands CMake files) should be able to find and parse the source and header files (e.g. for "go to definition" operations) 2) It should be possible to compile and install a boost release via the customary - cmake <path to boost_root> \ -DCMAKE_INSTALL_PREFIX=<path> \ -DCMAKE_TOOLCHAIN_FILE=<path> \ [other options] - cmake --build . - cmake --build . --target install Again respecting all compilation flags specified in the toolchain file and installing a) headers, b) compiled libraries c) the respective library cmake config files. The goal is to provide a cmake based installation procedure that can be easily used by package managers like conan, vcpkg, apt, cget and so on. In addition to those points, it would probably be prudent to also allow tests (for a individual library as well as the whole boost release) to be run via cmake / ctest and cdash, but this is less important for me as a user and I have no opinions on the details of that. Best Mike
Let me start by thanking you, Robert, for getting this process started. CMake has been a hot topic for a long time with little action and getting a review going is important. On Mon, Oct 1, 2018 at 8:07 PM Robert Ramey via Boost <boost@lists.boost.org> wrote:
Scope, Requirements and Features ================================
Library Build ------------- “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk Recommends that all libraries - including header only libraries have a CMakeList.txt file. What should this include for Boost Libraries?
I suppose, every library, header-only or not, will have to include CMakeLists.txt. For separately compiled libraries, CMakeLists.txt should support building the library, similarly to Boost.Build. These CMakeLists.txt files, along with the one of the superproject, will have to be included in the source code packages we distribute. And also in the Windows binary packages we distribute. Note that here I assume that at this point we are continuing to distribute monolithic Boost releases for the time being. This may change in the future as we modularize Boost but for now having CMake alone is a big enough task by itself. Of course, if a certain CMake proposal already presents a potential route for further modularization, it can only be a plus.
Library Test ------------ Should facilities for "testing" be only done by developers? or should users who acquire library packages also be able to test libraries.
I think, the ability to test libraries is essential for both Boost developers and users (primarily, developers, though). I think, potential solutions should include this functionality.
Should CMake testing results posting be used - CDASH?
I don't really know what CDash is, I've never used it.
Should we just skip the issue of Library Testing and continue to depend on Boost Build.
Building library docs is missing in this section. Having CMake able to build docs is less important than being able to build and test libraries, so we may omit this functionality at first. Docs are available online and pretty readable in source form most of the time. However, it would seem that this task should be the easiest one from the build system standpoint, so it shouldn't be too difficult to add.
Library Packaging ----------------- Is the library packaging facility provide by CMake - CPACK - useful to boost. Should boost libraries be updated to support it?
I would really like us to not get into the packaging business beyond preparing source packages of Boost releases. Packaging is a difficult topic, very target system dependent. Let the people who has the domain knowledge do it. As someone who has built Debian packages of Boost for my project I can tell that it is unlikely that I would use CPack. Not because something is wrong with it (in fact, I've never had to use it, so I really don't know), but because the current building pipeline doesn't involve it and works backwards. I.e., as far as I know, CPack is designed to be part of the build system, to which it adds packaging as one of the steps, while the common workflow is that you invoke packaging tools (dpkg, debhelper in case of Debian packages), which invokes the build process and then collects and packages the artifacts from it. I don't really see the benefit from changing the common workflow. I suppose, CPack could be useful if we did packaging and deploy as part of our CI process, but we don't.
Dependencies ------------ If the above is implemented, can we depend upon CMake to handle library dependencies? Or does some special functionality Do circular dependencies constitute a problem?
I think, dependency management will be limited to enumerating immediate dependencies for the targets in CMakeLists.txt, much like what we have now with Boost.Build. Since we won't do packaging, we won't have to deal with package dependencies, including dependencies from system libraries, like zlib, for example. I'm not sure how CMake manages dependencies between targets and whether circular dependencies constitute a problem. CMake proposals will have to clarify that, I guess.
Modularity ---------- Currently boost is organized as a tightly integrated group of libraries. This organization manifests itself in a number of ways. We have a Boost "super project" in github. Boost libraries are "sub projects" of this super project. This known structure is exploited by and depended upon by boost tools. Should CMake support continue this policy/design. We distribute boost as "super project". Should we continue to do this or distribute libraries on a library by library basis. Does CMake/CPACK etc. make this "simple"?
As I said earlier, I don't think we should specifically target modilarization at this point as just adding support for CMake alone is a big enough task. Let us leave further modularization for future and not require immediate support for it. Therefore, I think, we will have a CMakeLists.txt in the superproject, as well as in the libs directory.
Documentation/testing --------------------- What support should be require for uses of Boost CMake implementation. Documentation, examples, templates for helping library developers. How should the CMake design/implementation be tested?
I think, documentation and examples are required. Given that, presumably, library maintainers will be the ones who will have to write CMakeLists.txt for their libraries, the docs quality is very important. Besides the obvious stuff, like how to create a library or a test, the docs should describe more intricate stuff, like how to add compiler switches depending on compiler or how to test if the compiler supports a certain feature and do something depending on that, or how to set a custom visibility for a project, etc. From experience with Boost.Build, I can say that it is these little details that are difficult to find out. It would help if, as a proof of concept, demonstrated one or two converted libraries which cover most common and more inticate use cases we have. As for tests, these are, of course, welcome, but ultimately it is the tool that helps the maintainer most, so I'd leave this to the authors' discretion. I'm not even sure how one would test CMake scripts.
On 09/10/2018 23:08, Andrey Semashev via Boost wrote:
Library Test ------------ Should facilities for "testing" be only done by developers? or should users who acquire library packages also be able to test libraries.
I think, the ability to test libraries is essential for both Boost developers and users (primarily, developers, though). I think, potential solutions should include this functionality.
Should CMake testing results posting be used - CDASH?
I don't really know what CDash is, I've never used it.
It's a "dashboard" which displays submitted test results. It can be used to aggregate testing on multiple platforms. https://open.cdash.org/index.php?project=CMake is the dashboard for CMake itself. Boost could potentially use it, either the open dashboard or a self-hosted one, but it's strictly optional. Maybe Travis is sufficient, or you could integrate something else entirely.
Library Packaging ----------------- Is the library packaging facility provide by CMake - CPACK - useful to boost. Should boost libraries be updated to support it?
I would really like us to not get into the packaging business beyond preparing source packages of Boost releases. Packaging is a difficult topic, very target system dependent. Let the people who has the domain knowledge do it.
As someone who has built Debian packages of Boost for my project I can tell that it is unlikely that I would use CPack. Not because something is wrong with it (in fact, I've never had to use it, so I really don't know), but because the current building pipeline doesn't involve it and works backwards.
Last time I tried it, it was inferior to the native packaging tools. Particularly when packaging libraries, it didn't do as good a job with library version dependencies--if you build Debian packages on a non-Debian platform it can't compute the minimum library version from shlibs. I think it's primarily useful for packaging standalone applications where there are no complex library dependencies. Certainly for Windows and MacOS X it makes sense. For packaging a set of libraries like Boost, I don't think it's as useful; we already have good Boost packaging anyway, so I'd suggest ignoring it at least for now. Regards, Roger
On 10/10/18 1:24 AM, Roger Leigh via Boost wrote:
I think it's primarily useful for packaging standalone applications where there are no complex library dependencies. Certainly for Windows and MacOS X it makes sense. For packaging a set of libraries like Boost, I don't think it's as useful;
we already have good Boost packaging anyway, so I'd suggest ignoring it at least for now.
Hmmm - what is "Boost packaging" refering to? Robert Ramey
Regards, Roger
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On 10/10/2018 16:02, Robert Ramey via Boost wrote:
On 10/10/18 1:24 AM, Roger Leigh via Boost wrote:
I think it's primarily useful for packaging standalone applications where there are no complex library dependencies. Certainly for Windows and MacOS X it makes sense. For packaging a set of libraries like Boost, I don't think it's as useful;
we already have good Boost packaging anyway, so I'd suggest ignoring it at least for now.
Hmmm - what is "Boost packaging" refering to?
I was referring to the existing packaging for the various Linux distributions, BSD ports, MacOS MacPorts/homebrew, vcpkg etc. I simply meant that the problem is already taken care of by others for the most part, and that CPack doesn't bring much (if any) added value on top of that. Regards, Roger
On 10.10.18 22:25, Roger Leigh via Boost wrote:
On 10/10/2018 16:02, Robert Ramey via Boost wrote:
On 10/10/18 1:24 AM, Roger Leigh via Boost wrote:
I think it's primarily useful for packaging standalone applications where there are no complex library dependencies. Certainly for Windows and MacOS X it makes sense. For packaging a set of libraries like Boost, I don't think it's as useful;
we already have good Boost packaging anyway, so I'd suggest ignoring it at least for now.
Hmmm - what is "Boost packaging" refering to?
I was referring to the existing packaging for the various Linux distributions, BSD ports, MacOS MacPorts/homebrew, vcpkg etc. I simply meant that the problem is already taken care of by others for the most part, and that CPack doesn't bring much (if any) added value on top of that.
I believe it would be interesting to distribute boost via Launchpad/PPA for Ubuntu. CPack can generate debian packages for that. An example that (used to) work with multiple packages: https://bitbucket.org/renficiaud/yayi-debian-ppa/src/master/debian/rules Raffi
On Mon, Oct 1, 2018 at 10:07 AM Robert Ramey via Boost <boost@lists.boost.org> wrote:
We distribute boost as "super project". Should we continue to do this or distribute libraries on a library by library basis. Does CMake/CPACK etc. make this "simple"?
Here's an idea, we leave our distribution as-is, and let the community step forward and decide if and how they want to package individual libraries. We can make it easier for them by making adjustments as needed. This way the stakeholders have a direct say in how it goes.
Here's an idea, we leave our distribution as-is, and let the community step forward and decide if and how they want to package individual
Vinnie Falco wrote: libraries. This is not possible at the moment because all headers are copied into boost/ and removed from libs/$lib/include. The 'community' can't package an individual library from the contents of a release.
On 10/11/18 11:59 AM, Peter Dimov via Boost wrote:
Vinnie Falco wrote:
Here's an idea, we leave our distribution as-is, and let the community step forward and decide if and how they want to package individual libraries.
This is not possible at the moment because all headers are copied into boost/ and removed from libs/$lib/include.
off topic - which is a very bad idea.
The 'community' can't package an individual library from the contents of a release.
Why not? What is a "release"? I don't see why and "deployment" system can't walk through the collection of boost library and pick out anything it wants. Robert Ramey
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On 10/11/18 11:38 AM, Vinnie Falco via Boost wrote:
On Mon, Oct 1, 2018 at 10:07 AM Robert Ramey via Boost <boost@lists.boost.org> wrote:
We distribute boost as "super project". Should we continue to do this or distribute libraries on a library by library basis. Does CMake/CPACK etc. make this "simple"?
Here's an idea, we leave our distribution as-is, and let the community step forward and decide if and how they want to package individual libraries. We can make it easier for them by making adjustments as needed. This way the stakeholders have a direct say in how it goes.
Right. This is really a question about deployment of modular Boost. The purpose of this thread is to solicit suggestions as to what a CMake system for Boost should include. I'm aware that CMake includes CPACK which I believe touches upon this subject. I hope that someone who knows more about this than I can supply more information as to whether we should include support for CPACK in the coming list of requirements contained in the forthcoming call for submissions. Robert Ramey
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
First, thank you Robert for starting this discussion. Second, I think the scope and goals of supporting CMake, even though well intentioned, fall short of what Boost needs to do to resolve the varied afflictions that plague us. In what follows when I refer to: "users" I mean those programmers that obtain Boost and use the libraries in their projects, and "authors" as the programmers that develop and support the Boost Libraries and tools. With that in mind, what I would like us to do is what I've mentioned many times in the past.. Move towards a truly modular Boost distribution and development that: a) supports users no matter what environment they choose to use Boost Libraries in, b) supports developers to flexibly develop and test their libraries, c) allows Boost to grow in a way that doesn't cripple development resources. I think that without a "fully normalized modular Boost" a CMake, or any other build system, effort is a waste of time. As it doesn't address the long term viability of Boost. And hence if we are going to spend considerable effort it should be towards modularity. And everything else can flow from that. In addition to the standard requirements that Robert mentioned here's what I think is required for a modular Boost: 1) Fully normalized library file structure that can be introspected. 2) Clear inter-library, and non-cyclic, dependency arrangement and information. 3) Abandon the super-project structure as a development and distribution vehicle. To telegraph my intent for when it's time for proposals.. I think that for #1 above I think we should be forward looking and follow the Pitchfork structure proposal <http://bit.ly/2Edy4pC>. As for build system.. I think I only have one additional requirement given the above: 1) We should support generating multiple build system files for distribution and development from the introspection of the modular Boost libraries to support the varied user and developer needs. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net
On 10/11/18 9:45 PM, Rene Rivera via Boost wrote:
Second, I think the scope and goals of supporting CMake, even though well intentioned, fall short of what Boost needs to do to resolve the varied afflictions that plague us.
Well, the scope, goals and requirements of CMake as it relates to Boost aren't decided yet. That is the current discussion. I don't believe that CMake will address all the afflictions that plague us. I believe it will make boost in someway better. That is my aspiration.
In what follows when I refer to: "users" I mean those programmers that obtain Boost and use the libraries in their projects, and "authors" as the programmers that develop and support the Boost Libraries and tools.
With that in mind, what I would like us to do is what I've mentioned many times in the past.. Move towards a truly modular Boost distribution and development that: a) supports users no matter what environment they choose to use Boost Libraries in, b) supports developers to flexibly develop and test their libraries, c) allows Boost to grow in a way that doesn't cripple development resources.
This is my goal as well. I believe that a CMake support of boost will take use closer to that goal.
I think that without a "fully normalized modular Boost" a CMake, or any other build system, effort is a waste of time.
Of course we differ. I don't believe it a grand (intelligent?) design. I believe in evolution. It is my intention that CMake support of boost will bring us close to the goal of a modular boost - which I've referred to in the past as Boost 2.0.
As it doesn't address the long term viability of Boost.
Not explicitly. But I believe it will contribute to the long term visibility of Boost.
And hence if we are going to spend considerable effort it should be towards modularity.
By keeping the scope relatively narrow, I hope to be able to make actual progress. This is a key feature of my proposal and strategy. Other more ambitious and grandiose initiatives have failed in the past because they were too ambitious and grandiose. This makes it impossible to reach consensus which is the bedrock of Boost. We've tried the marxist approach to move to Boost 2.0 and it's failed us. Now we're going to try capitalism - which accepts and embraces failure is an inevitable cost of moving forward.
In addition to the standard requirements that Robert mentioned here's what I think is required for a modular Boost:
1) Fully normalized library file structure that can be introspected.
This would be nice - but I don't think it's essential to making a CMake Boost integration.
2) Clear inter-library, and non-cyclic, dependency arrangement and information.
LOL - the hole grail! It's never going to happen. The advocates of this view have totally misunderstood the meaning and role of dependency in this context. Like the knights of old, they will continue to pursue this holy grail - always getting closer - but never capturing the prize. I've recently described my views on this in detail (for the Nth time!) on slack in a thread with Peter Dimov. And for the Nth time I've failed to convince anyone! LOL If he's sir galahad, maybe I'm don quijote - both pure of heart but totally misguided in different ways. (I'm just being entertaining here - I know I'm right and have proven it many times.)
3) Abandon the super-project structure as a development and distribution vehicle.
I agree with this. It's my current intention that the "scope and requirements" include the statement like: "Any CMake support of Boost should not presume any super project or similar directory structure outside of any particular library"
To telegraph my intent for when it's time for proposals.. I think that for #1 above I think we should be forward looking and follow the Pitchfork structure proposal <http://bit.ly/2Edy4pC>.
Interesting - but as I said before - out of scope.
As for build system.. I think I only have one additional requirement given the above:
1) We should support generating multiple build system files for distribution and development from the introspection of the modular Boost libraries to support the varied user and developer needs.
LOL - that could mean anything - maybe you should run for office! I think I get the idea but would phrase it differently: "A CMake integration should not conflict with the current build system or other boost tools" Robert Ramey
On Fri, Oct 12, 2018 at 12:06 PM Robert Ramey via Boost < boost@lists.boost.org> wrote:
On 10/11/18 9:45 PM, Rene Rivera via Boost wrote:
I believe it [CMake] will make boost in someway better.
And that's the aspect that I've never seen.
In what follows when I refer to: "users" I mean those programmers that obtain Boost and use the libraries in their projects, and "authors" as the programmers that develop and support the Boost Libraries and tools.
With that in mind, what I would like us to do is what I've mentioned many times in the past.. Move towards a truly modular Boost distribution and development that: a) supports users no matter what environment they choose to use Boost Libraries in, b) supports developers to flexibly develop and test their libraries, c) allows Boost to grow in a way that doesn't cripple development resources.
This is my goal as well. I believe that a CMake support of boost will take use closer to that goal.
Again, I disagree, no build system will take us closer to that goal because it's not the build system at fault.
I think that without a "fully normalized modular Boost" a CMake, or any other build system, effort is a waste of time.
Of course we differ. I don't believe it a grand (intelligent?) design. I believe in evolution. It is my intention that CMake support of boost will bring us close to the goal of a modular boost - which I've referred to in the past as Boost 2.0.
I've tried evolution. I spent the past 10 years slowly moving Boost to be modular. No one cared enough to make the needed changes. People are adverse to changes that they have to implement (this itself being a prime example -- in multiple meanings).
As it doesn't address the long term viability of Boost.
Not explicitly. But I believe it will contribute to the long term visibility of Boost.
I don't thin Boost has a visibility problem.
And hence if we are going to spend considerable effort it should be towards modularity.
By keeping the scope relatively narrow, I hope to be able to make actual progress.
Been there, done that, failed.
This is a key feature of my proposal and strategy. Other more ambitious and grandiose initiatives have failed in the past because they were too ambitious and grandiose.
I disagree. They failed because they attempted to make *multiple* ambitious changes at once. Single, focused, ambitious changes are achievable.
This makes it impossible to reach consensus which is the bedrock of Boost.
So far in this discussion multiple people have mentioned modularity as a goal. How is that not consensus?
We've tried the marxist approach to move to Boost 2.0 and it's failed us. Now we're going to try capitalism - which accepts and embraces failure is an inevitable cost of moving forward.
In addition to the standard requirements that Robert mentioned here's what I think is required for a modular Boost:
1) Fully normalized library file structure that can be introspected.
This would be nice - but I don't think it's essential to making a CMake Boost integration.
True it's not essential. But it would make any build system integration a magnitude easier (and I'm including b2 reintegration here).
2) Clear inter-library, and non-cyclic, dependency arrangement and information.
LOL - the hole grail! It's never going to happen. The advocates of this view have totally misunderstood the meaning and role of dependency in this context. Like the knights of old, they will continue to pursue this holy grail - always getting closer - but never capturing the prize. I've recently described my views on this in detail (for the Nth time!) on slack in a thread with Peter Dimov. And for the Nth time I've failed to convince anyone! LOL If he's sir galahad, maybe I'm don quijote - both pure of heart but totally misguided in different ways. (I'm just being entertaining here - I know I'm right and have proven it many times.)
Yep, saw that, and was in that argument. And been in that argument multiple times now :-) I've lived in multiple audiences in this argument as an author, user, and packager. I hate to break it to you.. But you're going to keep not convincing anyone ;-)
3) Abandon the super-project structure as a development and distribution vehicle.
I agree with this. It's my current intention that the "scope and requirements" include the statement like:
"Any CMake support of Boost should not presume any super project or similar directory structure outside of any particular library"
Sounds good.. Just replace "CMake" with "build system".
To telegraph my intent for when it's time for proposals.. I think that for #1 above I think we should be forward looking and follow the Pitchfork structure proposal <http://bit.ly/2Edy4pC>.
Interesting - but as I said before - out of scope.
Not entirely out of scope. Colby and others are working to make CMake directly use such a structure. It would make the CMake effort easier to accomplish with less work from authors.
As for build system.. I think I only have one additional requirement given
the above:
1) We should support generating multiple build system files for distribution and development from the introspection of the modular Boost libraries to support the varied user and developer needs.
LOL - that could mean anything - maybe you should run for office!
I've thought about running for office :-) But gave up changing the world through politics at my last stint with political party involvement.
I think I get the idea but would phrase it differently:
"A CMake integration should not conflict with the current build system or other boost tools"
Perhaps. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net
Hi Rene, Robert, all, On 10/12/18 3:04 PM, Rene Rivera via Boost wrote:
With that in mind, what I would like us to do is what I've mentioned many times in the past.. Move towards a truly modular Boost distribution and development that: a) supports users no matter what environment they choose to use Boost Libraries in, b) supports developers to flexibly develop and test their libraries, c) allows Boost to grow in a way that doesn't cripple development resources. This is my goal as well. I believe that a CMake support of boost will take use closer to that goal.
Again, I disagree, no build system will take us closer to that goal because it's not the build system at fault.
+10
I think that without a "fully normalized modular Boost" a CMake, or any other build system, effort is a waste of time. Of course we differ. I don't believe it a grand (intelligent?) design. I believe in evolution. It is my intention that CMake support of boost will bring us close to the goal of a modular boost - which I've referred to in the past as Boost 2.0.
I'm a big fan of incremental progress (which is my attempt at paraphrasing your use of the term "evolution"), and it is precisely for that reason that I think "modularity" must include organizational / administrative aspects (e.g. the ability to take certain decisions per project rather than monolithically for the whole organization) rather than only technical ones (e.g. the ability to build individual projects). Not only would a solution that involves some projects using CMake and some using Boost.Build be a much more convincing demonstration of modularity, it would also address a range of concerns voiced by others (including Edward Diener in https://lists.boost.org/Archives/boost/2018/10/243620.php and myself in https://lists.boost.org/Archives/boost/2018/09/243293.php et al.)
This is a key feature of my proposal and strategy. Other more ambitious and grandiose initiatives have failed in the past because they were too ambitious and grandiose.
This seems to be a question of perspective. To me a proposal to "switch from Boost.Build to CMake for the entirety of Boost" is a way more ambitious goal than to "change the Boost infrastructure to support individual projects to switch to CMake".
This makes it impossible to reach consensus which is the bedrock of Boost.
Consensus would be great, but we (the community) have demonstrated that it has its limits, i.e. it doesn't scale well. And of course, to state the obvious: This isn't a democracy. It's the developers and maintainers who put in the effort to develop and maintain individual projects who should have the ultimate power to decide.
1) Fully normalized library file structure that can be introspected. This would be nice - but I don't think it's essential to making a CMake Boost integration.
True it's not essential. But it would make any build system integration a magnitude easier (and I'm including b2 reintegration here).
"Provide the following integration points: ..." would be a much simpler set of requirements. But I agree of course with the general idea: establish fewer points to be standardized, to improve the ability to encapsulate and insulate "implementation details".
2) Clear inter-library, and non-cyclic, dependency arrangement and information.
+10
3) Abandon the super-project structure as a development and distribution vehicle.
+10
I agree with this. It's my current intention that the "scope and requirements" include the statement like:
"Any CMake support of Boost should not presume any super project or similar directory structure outside of any particular library"
Sounds good.. Just replace "CMake" with "build system".
Yes !! Stefan -- ...ich hab' noch einen Koffer in Berlin...
-----Original Message----- From: Boost <boost-bounces@lists.boost.org> On Behalf Of Robert Ramey via Boost Sent: Monday, October 1, 2018 7:07 PM
[...]
Scope, Requirements and Features ================================
We presume that submissions will fulfill the applicable requirements of any boost submission. In addition we would like to see the following addressed.
I'll start with the points I can answer (relatively) short and follow up with another email for the rest.
Library Build ------------- “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk Recommends that all libraries - including header only libraries have a CMakeList.txt file. What should this include for Boost Libraries?
[See next email]
Library Test ------------ Should facilities for "testing" be only done by developers? or should users who acquire library packages also be able to test libraries. Should CMake testing results posting be used - CDASH? Should we just skip the issue of Library Testing and continue to depend on Boost Build.
I (a user) would very much like to be able to run boost tests locally, to make sure it works as expected with the specific set of compiler and linker flags chosen for my project. Personally, I don't need to be able to run all boost library unit tests with a single" ctest . <some_parameter>" command, but for release management that might be desirable. I guess CDASH would be nice, but that is mainly a question for whoever is in charge of the current testing infrastructure.
Library Packaging ----------------- Is the library packaging facility provide by CMake - CPACK - useful to boost. Should boost libraries be updated to support it?
If I want to use a boost package, I'd use a package manager (vcpkg, conan, apt, brew ...) for that. Unless using CPACK significantly simplifies packaging boost for those package managers, I don't think that would be an advantage
Dependencies ------------ If the above is implemented, can we depend upon CMake to handle library dependencies? Or does some special functionality Do circular dependencies constitute a problem?
CMake can handle circular dependencies. I think the main questions is how a cmake library should find its dependencies. I.e. target_link_libraries(my_lib PUBLIC Boost::other_library1 Boost::other_library2 Boost::other_library2 ) Should be the universal default (of course you can pick another naming convention), but there are different possible ways how those targets could be found if they are not generated by the same (super-) project: - should each project do its own find_package ...? - Should the user do this centrally? - Should boost somehow stick to find_package(BOOST COMPONETS x,y,z) or rather find_package(boost_x), find_package(boost_y) ..? - Should there be a single find_package that pulls in all available header only targets? - Also, should a library first check if a target is already available before calling find_package? Not saying I find all of these choices reasonable. Just that there are different possibilities and if boost absolutely wants or doesn't want to support one of those it should probably be part of the requirements.
Modularity ---------- Currently boost is organized as a tightly integrated group of libraries. This organization manifests itself in a number of ways. We have a Boost "super project" in github. Boost libraries are "sub projects" of this super project. This known structure is exploited by and depended upon by boost tools. Should CMake support continue this policy/design. We distribute boost as "super project". Should we continue to do this or distribute libraries on a library by library basis. Does CMake/CPACK etc. make this "simple"?
For backwards compatibility there should still be a super project (and e.g. the release should probably still provide the headers (or rather a link to them) in one place. However, I'd very much prefer if a common include directory was no longer required by default.
Documentation/testing --------------------- What support should be require for uses of Boost CMake implementation. Documentation, examples, templates for helping library developers. How should the CMake design/implementation be tested?
Other considerations -------------------- add your requirement/features here.
Useful Resources =============== “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk https://gist.github.com/mbinna/c61dbb39bca0e4fb7d1f73b0d66a4fd1 https://github.com/purpleKarrot/Boost.CMake https://www.youtube.com/watch?v=eC9-iRN2b04
Also relevant even if not CMake specific: https://www.youtube.com/watch?v=sBP17HQAQjk Mike
participants (15)
-
Alexander Grund -
Andrey Semashev -
Edward Diener -
Gavin Lambert -
Mike Dev -
mike.dev＠gmx.de
-
Peter Dimov -
Raffi Enficiaud -
Rene Rivera -
Richard -
Robert Ramey -
Roger Leigh -
Stefan Seefeld -
Steven Watanabe -
Vinnie Falco