CMake and Boost Build tests

Following John Maddock's appeal for practical solutions related to the move to CMake, I would like to know what the CMake equivalent is to the Boost Build unit test functionality. In other words what do I write for CMake in order to do a Boost Build compile, compile-fail, link, link-fail, run, and run-fail unit tests ? Does CMake have an equivalent to the Boost Build alias rule ? or the Boost Build project rule ? The great majority of the Boost libraries are header-only libraries where Boost Build is used to run the unit tests and, optionally, Boost Build is used to build the documentation using a quickbook, boostbook, doxygen toolset to generate html and pdf files. While I realize that building a Boost library, and making that built library available to end-users via CMake, has been the focus of most of the discussion so far regarding CMake and Boost, I am assuming the move to CMake also means that running unit tests and building docs where applicable must also be done through CMake. If I am wrong about this then I personally will be only too happy to continue using Boost Build to do these things.

On 7/26/17 11:49 AM, Edward Diener via Boost wrote:
Following John Maddock's appeal for practical solutions related to the move to CMake, I would like to know what the CMake equivalent is to the Boost Build unit test functionality.
I can tell you what I did for the serialization library. This predates the current discussions so will likely be considered out of date by the current contributors to the discussion. On the other hand, it does actually work and you can test it yourself.
In other words what do I write for CMake in order to do a Boost Build compile, compile-fail, link, link-fail, run, and run-fail unit tests ?
One adds targets (which for me is just the name of the source/excutable without the cpp, exe, etc. to a list to get compile and run functionality. Its pretty simple. I couldn't make the compile-fail and run-fail targets to work so I just removed them from the cmake test suite. So if your thinking that CMake has a ways to go to be able to offer what we expect, you're right.
Does CMake have an equivalent to the Boost Build alias rule ? or the Boost Build project rule ?
LOL - I honestly don't remember what these rules do in boost build. But then I've only been using boost build for 15 years.
The great majority of the Boost libraries are header-only libraries where Boost Build is used to run the unit tests and,
true and this case is pretty simple in both cmake and boost build.
optionally, Boost Build is used to build the documentation using a quickbook, boostbook, doxygen toolset to generate html and pdf files.
Personally, I could never make this tool chain work despite a large investment of effort. I gave up on doxygen and quickbook after concluding them to be essentially non-functional. I did manage to use the xml tools loaded with boost tools along with some downloaded FOP, a version of XMLMind to create toolchain which is about 100 times better and 1/100 as complex as the boost one for creating documents. So you're life will be better when you're forced to leave these behind. While I realize that
building a Boost library, and making that built library available to end-users via CMake, has been the focus of most of the discussion so far regarding CMake and Boost,
You are correct. There are two independent issues related to this. a) Replacing boost build with cmake b) providing CMake files to be helpful to users who use CMake to build their applications. Boost should first address b) first while leaving a) untouched for now. If it can't manage to do that, then there is not point in trying to address b) I am assuming the move to CMake also means that running unit tests I think that is the goal and building docs where applicable must also be done through CMake. I don't see this as a requirement nor do I see it as necessary or desirable.
If I am wrong about this then I personally will be only too happy to continue using Boost Build to do these things.
LOL - it certainly looks that way. I wouldn't panic yet. Nothing is actually going to happen until someone actually does something (besides offer advice). It doesn't seem that that is going to happen very soon, so you can just relax. But you might want to experiment a little with the following useful orthogonal ideas. a) investigate how CMake Find<your library> should work and consider adding the files necessary to support it to your library. This wouldn't mess up any of your other work. b) Consider making a simple CMake project to build library and run tests such as I did for the serialization library. CMake gurus it seems dismiss this effort. But it has worked for me. The main benefit for me is that permits to to build a working IDE which I can used to build and debug. For me, this has turned out to be a great time saver. Also I use boost build for running unit tests on my local system and it works very well for that. It's what I really depend on. It doesn't consume any effort to maintain (it was a MAJOR effort to setup). And I created my own program to display test results (library_test) so I'm very happy with this. One thing that CMake has that Boost Build doesn't have is CDash does have is the ability to upload test results to a special site where all test results from all users can be browsed. This is desperately needed for Boost but no one has made the effort to build it. So all in all, I'm hoping that we can evolve to something better and more widely useful. But I don't see anyone actually undertaking efforts to make that happen. So I don't think you and I really have to worry about this all that much. We'll Boost build will likely out live you and me. Robert Ramey

On 7/26/2017 3:33 PM, Robert Ramey via Boost wrote:
On 7/26/17 11:49 AM, Edward Diener via Boost wrote:
Following John Maddock's appeal for practical solutions related to the move to CMake, I would like to know what the CMake equivalent is to the Boost Build unit test functionality.
I can tell you what I did for the serialization library. This predates the current discussions so will likely be considered out of date by the current contributors to the discussion. On the other hand, it does actually work and you can test it yourself.
Robert I appreciate your answer. Is this the CMakeLists.txt in the CMake subdirectory of the serialization library ? Remember I am talking about header-only libraries where nothing is "built" and I am only trying to run the unit tests in the library's 'test' subdirectory using CMake rather than Boost Build using CMake's equivalent of a script file rather than a Boost Build jamfile.
In other words what do I write for CMake in order to do a Boost Build compile, compile-fail, link, link-fail, run, and run-fail unit tests ?
One adds targets (which for me is just the name of the source/excutable without the cpp, exe, etc. to a list to get compile and run functionality. Its pretty simple.
I want to see actual cmake code. Saying "Its pretty simple" only means it is pretty simple for those who already know CMake well. <RANT> I find the CMake documentation at https://cmake.org/cmake/help/v3.9/ nearly impossible to understand due to the fact that there is no overriding explanation of how CMake is supposed to work as practical examples of setting up CMake scripts. While I realize that all the information is supposedly there I have just about no idea how it all comes together. I am guessing that the only way I can really learn CMake is buying the "Mastering CMake" book, which I don't mind doing so if I have to do it. If I am wrong and someone would like to point me to an actual overview which explains CMake in a regular and meaningful way, rather than a bunch of loosely related topics which is the actual CMake documentation in the above URL, it would be appreciated. </RANT>
I couldn't make the compile-fail and run-fail targets to work so I just removed them from the cmake test suite. So if your thinking that CMake has a ways to go to be able to offer what we expect, you're right.
So you are saying that you cannot run a unit test with CMake that is meant to show a failure. Wow, what a brilliant system !
Does CMake have an equivalent to the Boost Build alias rule ? or the Boost Build project rule ?
LOL - I honestly don't remember what these rules do in boost build. But then I've only been using boost build for 15 years.
The project rule for unit tests is normally used to set usage requirements for all tests being run. The alias rule for unit tests is normally used as a means of grouping some unit tests together so that the end-user of the tests can test individual groups rather than all the tests.
The great majority of the Boost libraries are header-only libraries where Boost Build is used to run the unit tests and,
true and this case is pretty simple in both cmake and boost build.
Once again saying it is pretty simple in cmake isn't showing me anything.
optionally, Boost Build is used to build the documentation using a quickbook, boostbook, doxygen toolset to generate html and pdf files.
Personally, I could never make this tool chain work despite a large investment of effort. I gave up on doxygen and quickbook after concluding them to be essentially non-functional. I did manage to use the xml tools loaded with boost tools along with some downloaded FOP, a version of XMLMind to create toolchain which is about 100 times better and 1/100 as complex as the boost one for creating documents. So you're life will be better when you're forced to leave these behind.
I heartily disagree. Using quickbook and doxygen is a no-brainer for me, but there is no point of debating this. Are both perfect ? No. But using them makes creating documentation supremely easy. I did say that there is no Boost requirement for creating documentation using Boost Build. Some documentation is straight html. But for those libraries that do go the quickbook and doxygen way using Boost Build we need a CMake way of doing this if that is what the Boost Steering Committee is mandating.
While I realize that
building a Boost library, and making that built library available to end-users via CMake, has been the focus of most of the discussion so far regarding CMake and Boost,
You are correct. There are two independent issues related to this.
a) Replacing boost build with cmake b) providing CMake files to be helpful to users who use CMake to build their applications.
Boost should first address b) first while leaving a) untouched for now. If it can't manage to do that, then there is not point in trying to address b)
I am assuming the move to CMake also means that running unit tests
I think that is the goal
and building docs where applicable must also be done through CMake.
I don't see this as a requirement nor do I see it as necessary or desirable.
If I am wrong about this then I personally will be only too happy to continue using Boost Build to do these things.
LOL - it certainly looks that way.
I wouldn't panic yet. Nothing is actually going to happen until someone actually does something (besides offer advice). It doesn't seem that that is going to happen very soon, so you can just relax.
OK, I will relax and await the outcome of those, whoever they are, who are evidently taking on the work of providing CMake scripts for what is currently Boost Build tasks.
But you might want to experiment a little with the following useful orthogonal ideas.
a) investigate how CMake Find<your library> should work and consider adding the files necessary to support it to your library. This wouldn't mess up any of your other work.
b) Consider making a simple CMake project to build library and run tests such as I did for the serialization library. CMake gurus it seems dismiss this effort. But it has worked for me. The main benefit for me is that permits to to build a working IDE which I can used to build and debug. For me, this has turned out to be a great time saver.
I do not know how you learned enough of CMake to do what you want but, apropos my RANT, I am finding my ability to do this severely hampered by the piecemeal documentation which CMake deems its own docs.
Also I use boost build for running unit tests on my local system and it works very well for that. It's what I really depend on. It doesn't consume any effort to maintain (it was a MAJOR effort to setup). And I created my own program to display test results (library_test) so I'm very happy with this. One thing that CMake has that Boost Build doesn't have is CDash does have is the ability to upload test results to a special site where all test results from all users can be browsed. This is desperately needed for Boost but no one has made the effort to build it.
So all in all, I'm hoping that we can evolve to something better and more widely useful. But I don't see anyone actually undertaking efforts to make that happen. So I don't think you and I really have to worry about this all that much. We'll Boost build will likely out live you and me.
Robert Ramey

On 26/07/17 21:28, Edward Diener via Boost wrote:
On 7/26/2017 3:33 PM, Robert Ramey via Boost wrote:
On 7/26/17 11:49 AM, Edward Diener via Boost wrote:
optionally, Boost Build is used to build the documentation using a quickbook, boostbook, doxygen toolset to generate html and pdf files.
Personally, I could never make this tool chain work despite a large investment of effort. I gave up on doxygen and quickbook after concluding them to be essentially non-functional. I did manage to use the xml tools loaded with boost tools along with some downloaded FOP, a version of XMLMind to create toolchain which is about 100 times better and 1/100 as complex as the boost one for creating documents. So you're life will be better when you're forced to leave these behind.
I heartily disagree. Using quickbook and doxygen is a no-brainer for me, but there is no point of debating this. Are both perfect ? No. But using them makes creating documentation supremely easy.
I did say that there is no Boost requirement for creating documentation using Boost Build. Some documentation is straight html. But for those libraries that do go the quickbook and doxygen way using Boost Build we need a CMake way of doing this if that is what the Boost Steering Committee is mandating.
https://github.com/ome/ome-files-cpp/blob/master/docs/doxygen/CMakeLists.txt https://github.com/ome/ome-files-cpp/blob/master/cmake/DoxygenCheck.cmake are one way to generate doxygen docs. I'm sure something similar to this could be made to meet Boost's needs. Regards, Roger

On 7/26/17 1:36 PM, Roger Leigh via Boost wrote:
are one way to generate doxygen docs. I'm sure something similar to this could be made to meet Boost's needs.
I've avoided addressing doxygen ... until you brought it up. This tool is probably the single biggest reason why software development documentation is soo very bad. Things would be much, much better it had never been invented. On the bright side, it does generate crappy documentation in the shortest time. Since authors are going to create crappy documentation regardless, I guess it's a win. Robert Ramey

On Wed, Jul 26, 2017 at 2:06 PM, Robert Ramey via Boost <boost@lists.boost.org> wrote:
I've avoided addressing doxygen ... until you brought it up. This tool is probably the single biggest reason why software development documentation is soo very bad. Things would be much, much better it had never been invented. On the bright side, it does generate crappy documentation in the shortest time. Since authors are going to create crappy documentation regardless, I guess it's a win.
I disagree. This page is generated from Doxygen and it looks great: <http://vinniefalco.github.io/beast/beast/ref/beast__http__async_read/overload1.html> Thanks

On 27/07/2017 09:08, Vinnie Falco wrote:
On Wed, Jul 26, 2017 at 2:06 PM, Robert Ramey wrote:
I've avoided addressing doxygen ... until you brought it up. This tool is probably the single biggest reason why software development documentation is soo very bad. Things would be much, much better it had never been invented. On the bright side, it does generate crappy documentation in the shortest time. Since authors are going to create crappy documentation regardless, I guess it's a win.
I disagree. This page is generated from Doxygen and it looks great: <http://vinniefalco.github.io/beast/beast/ref/beast__http__async_read/overload1.html>
One of the problems I've found with Doxygen is that many people don't actually get to that page, but rather only get as far as http://vinniefalco.github.io/beast/beast/ref/beast__http__async_read.html. Once you've encountered enough Doxygen docs you might eventually notice the little tiny "more" links and achieve enlightenment.

On Wed, Jul 26, 2017 at 5:49 PM, Gavin Lambert via Boost <boost@lists.boost.org> wrote:
<http://vinniefalco.github.io/beast/beast/ref/beast__http__async_read/overload1.html> ... One of the problems I've found with Doxygen is that many people don't actually get to that page, but rather only get as far as http://vinniefalco.github.io/beast/beast/ref/beast__http__async_read.html.
That's not a problem with Doxygen (which I have configured to output only the XML description of the source code) but rather the XSLT style sheet I have written. I copied the xsl from Asio and made some improvements but the format is the same. Clicking on a function with multiple overloads takes you to the "overload resolution" page which you are complaining about (and to be fair you're not the only person). My xsl skills aren't that great since I only just learned it after copying Asio's style sheet. I'm waiting for a consultant to free up some time to help me make the style sheets great. In the meanwhile I have opened an issue: <https://github.com/vinniefalco/docca/issues/28> If anyone who is an XSL expert on the list might want to help, I would be grateful. I spun off the xsl into its own reusable project called "docca": <https://github.com/vinniefalco/docca> Here is the specific template which emits the overload selection page: <https://github.com/vinniefalco/docca/blob/324a50da1b3dd0efa5416ff81dc6e7bf89181230/include/docca/doxygen.xsl#L1379> Thanks

On 7/26/2017 4:36 PM, Roger Leigh via Boost wrote:
On 26/07/17 21:28, Edward Diener via Boost wrote:
On 7/26/2017 3:33 PM, Robert Ramey via Boost wrote:
On 7/26/17 11:49 AM, Edward Diener via Boost wrote:
optionally, Boost Build is used to build the documentation using a quickbook, boostbook, doxygen toolset to generate html and pdf files.
Personally, I could never make this tool chain work despite a large investment of effort. I gave up on doxygen and quickbook after concluding them to be essentially non-functional. I did manage to use the xml tools loaded with boost tools along with some downloaded FOP, a version of XMLMind to create toolchain which is about 100 times better and 1/100 as complex as the boost one for creating documents. So you're life will be better when you're forced to leave these behind.
I heartily disagree. Using quickbook and doxygen is a no-brainer for me, but there is no point of debating this. Are both perfect ? No. But using them makes creating documentation supremely easy.
I did say that there is no Boost requirement for creating documentation using Boost Build. Some documentation is straight html. But for those libraries that do go the quickbook and doxygen way using Boost Build we need a CMake way of doing this if that is what the Boost Steering Committee is mandating.
https://github.com/ome/ome-files-cpp/blob/master/docs/doxygen/CMakeLists.txt
https://github.com/ome/ome-files-cpp/blob/master/cmake/DoxygenCheck.cmake
are one way to generate doxygen docs. I'm sure something similar to this could be made to meet Boost's needs.
I certainly hope so since the creation of docs using the quickbook/boostbook/doxygen toolsets via a Boost Build jam file is pretty flexible and easy, once you understand the jam file structure for it.
Regards, Roger

On 7/26/17 7:51 PM, Edward Diener via Boost wrote:
I certainly hope so since the creation of docs using the quickbook/boostbook/doxygen toolsets via a Boost Build jam file is pretty flexible and easy, once you understand the jam file structure for it.
Right - I guess 15 years with it isn't enough time. Robert Ramey

I couldn't make the compile-fail and run-fail targets to work so I just removed them from the cmake test suite. So if your thinking that CMake has a ways to go to be able to offer what we expect, you're right.
So you are saying that you cannot run a unit test with CMake that is meant to show a failure. Wow, what a brilliant system !
Actually cmake's support for fail tests is richer than Boost.Build's. Here is cmake to implement simple compile fail testing where a regex on the error messages from the compiler must be matched for the test to pass: https://github.com/ned14/quickcpplib/blob/master/cmakelib/QuickCppLibMakeSta... This cmake reads the regex from the first line of the fail test's source code. There is lots of flexibility in cmake's fail test design. You can make tests which must compile but must fail to link. You can make tests which must complete within a certain time period. Lots of flexibility. ctest generates XML of all the test results. That can be merged with JUnit XML from the individual tests easily enough. And the whole lot sent up to CDash e.g. http://my.cdash.org/index.php?project=Boost.Outcome&date=2017-07-23 where fails and successes can be drilled into, and the cause figured out. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On 7/26/2017 4:39 PM, Niall Douglas via Boost wrote:
I couldn't make the compile-fail and run-fail targets to work so I just removed them from the cmake test suite. So if your thinking that CMake has a ways to go to be able to offer what we expect, you're right.
So you are saying that you cannot run a unit test with CMake that is meant to show a failure. Wow, what a brilliant system !
Actually cmake's support for fail tests is richer than Boost.Build's.
Here is cmake to implement simple compile fail testing where a regex on the error messages from the compiler must be matched for the test to pass:
https://github.com/ned14/quickcpplib/blob/master/cmakelib/QuickCppLibMakeSta...
Thanks ! I will look at it. It does seem like a great deal of code just to run a bunch of tests.
This cmake reads the regex from the first line of the fail test's source code.
There is lots of flexibility in cmake's fail test design. You can make tests which must compile but must fail to link. You can make tests which must complete within a certain time period. Lots of flexibility.
ctest generates XML of all the test results. That can be merged with JUnit XML from the individual tests easily enough. And the whole lot sent up to CDash e.g. http://my.cdash.org/index.php?project=Boost.Outcome&date=2017-07-23 where fails and successes can be drilled into, and the cause figured out.
I have a feeling that CTest is not usable for the current Boost testing system, but maybe I am wrong.
Niall

Actually cmake's support for fail tests is richer than Boost.Build's.
Here is cmake to implement simple compile fail testing where a regex on the error messages from the compiler must be matched for the test to pass:
https://github.com/ned14/quickcpplib/blob/master/cmakelib/QuickCppLibMakeSta...
Thanks ! I will look at it. It does seem like a great deal of code just to run a bunch of tests.
Only four lines or so are needed to make a compile-fail test in cmake. You can extract those out from my code easily enough. The reason for all the code is that QuickCppLib automates everything. You simply point it at a directory tree laid out in boost format, fill in a few parameters like <library name>, and whoosh it all just works with a complete package of support for docs, sanitiser tests, C++ Modules, precompiled headers, versioning, mangling, the whole kaboosh. But that's cmake innovation. And I wouldn't recommend that for Boost. You guys need something as simple and as minimal as possible which matches, as close as possible, Boost.Build. Otherwise you'll never finish converting over the build system, you'll get 60% there and that long tail will become insurmountable given it's volunteer time. We saw exactly the same happen with the git migration, it's that long tail which kills you by causing people to quit the whole project rather than put up with the never ending criticism. I personally think that a build directory per configuration design is too dissimilar to Boost.Build that such a conversion could ever be finished without injecting paid death march hours. But I won't be doing the work, so it's up to those who are to choose their design poison.
This cmake reads the regex from the first line of the fail test's source code.
There is lots of flexibility in cmake's fail test design. You can make tests which must compile but must fail to link. You can make tests which must complete within a certain time period. Lots of flexibility.
ctest generates XML of all the test results. That can be merged with JUnit XML from the individual tests easily enough. And the whole lot sent up to CDash e.g. http://my.cdash.org/index.php?project=Boost.Outcome&date=2017-07-23 where fails and successes can be drilled into, and the cause figured out.
I have a feeling that CTest is not usable for the current Boost testing system, but maybe I am wrong.
CTest I think is more than able. The sole thing I've ever found it falls badly down on is running tests on real remote devices e.g. Android ones. It really wants to run on-device, not remotely. Still, it can be beaten into submission using evil trickery. CDash probably isn't up to Boost's requirements. But the XML CTest outputs is standardised, it's not even Kitware's XML design. So the tooling currently producing the Boost regression matrix etc ought to be very easy to teach to understand CTest XML output, and thus all becomes well. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On 26/07/17 21:28, Edward Diener via Boost wrote:
On 7/26/2017 3:33 PM, Robert Ramey via Boost wrote:
On 7/26/17 11:49 AM, Edward Diener via Boost wrote:
I couldn't make the compile-fail and run-fail targets to work so I just removed them from the cmake test suite. So if your thinking that CMake has a ways to go to be able to offer what we expect, you're right.
So you are saying that you cannot run a unit test with CMake that is meant to show a failure. Wow, what a brilliant system !
ctest itself treats a nonzero return as failure. It's pretty trivial to run a wrapper to invert the status for expected failures, and customise the test running in any way you see fit. See the EXPECT_FAIL lines in https://github.com/apache/xerces-c/blob/f7654ee16b1dc2fb6e2b68777895c5384cd5... and the wrapper logic in https://github.com/apache/xerces-c/blob/f7654ee16b1dc2fb6e2b68777895c5384cd5... as one way to do this. Regards, Roger

On Wed, 2017-07-26 at 21:48 +0100, Roger Leigh via Boost wrote:
On 26/07/17 21:28, Edward Diener via Boost wrote:
On 7/26/2017 3:33 PM, Robert Ramey via Boost wrote:
On 7/26/17 11:49 AM, Edward Diener via Boost wrote:
I couldn't make the compile-fail and run-fail targets to work so I just removed them from the cmake test suite. So if your thinking that CMake has a ways to go to be able to offer what we expect, you're right.
So you are saying that you cannot run a unit test with CMake that is meant to show a failure. Wow, what a brilliant system !
ctest itself treats a nonzero return as failure. It's pretty trivial to run a wrapper to invert the status for expected failures, and customise the test running in any way you see fit. See the EXPECT_FAIL lines in
https://github.com/apache/xerces-c/blob/f7654ee16b1dc2fb6e2b68777895c5384cd5 1c28/tests/CMakeLists.txt#L176
and the wrapper logic in
https://github.com/apache/xerces-c/blob/f7654ee16b1dc2fb6e2b68777895c5384cd5 1c28/cmake/RunTest.cmake#L45
as one way to do this.
There's really no need for that because you can just set the test property WILL_FAIL to true, like I showed in my previous email.

On 26/07/17 22:13, paul wrote:
On Wed, 2017-07-26 at 21:48 +0100, Roger Leigh via Boost wrote:
https://github.com/apache/xerces-c/blob/f7654ee16b1dc2fb6e2b68777895c5384cd5 1c28/tests/CMakeLists.txt#L176
and the wrapper logic in
https://github.com/apache/xerces-c/blob/f7654ee16b1dc2fb6e2b68777895c5384cd5 1c28/cmake/RunTest.cmake#L45
as one way to do this.
There's really no need for that because you can just set the test property WILL_FAIL to true, like I showed in my previous email.
For the simple case that's definitely possible. It depends entirely upon what you're testing and how you determine success or failure for those tests. Regards, Roger

On 7/26/2017 4:48 PM, Roger Leigh via Boost wrote:
On 26/07/17 21:28, Edward Diener via Boost wrote:
On 7/26/2017 3:33 PM, Robert Ramey via Boost wrote:
On 7/26/17 11:49 AM, Edward Diener via Boost wrote:
I couldn't make the compile-fail and run-fail targets to work so I just removed them from the cmake test suite. So if your thinking that CMake has a ways to go to be able to offer what we expect, you're right.
So you are saying that you cannot run a unit test with CMake that is meant to show a failure. Wow, what a brilliant system !
ctest itself treats a nonzero return as failure. It's pretty trivial to run a wrapper to invert the status for expected failures, and customise the test running in any way you see fit. See the EXPECT_FAIL lines in
https://github.com/apache/xerces-c/blob/f7654ee16b1dc2fb6e2b68777895c5384cd5...
and the wrapper logic in
https://github.com/apache/xerces-c/blob/f7654ee16b1dc2fb6e2b68777895c5384cd5...
as one way to do this.
That is just applicable to a run test. Current Boost Build also has compile and link tests, although I will admit I have never found uses for the link test. But in the compile test, only whether a compile is successful or not is measured, while the run test does follow CTest treating a nonzero return as failure. But is CTest really applicable as a CMake alternative to Boost testing as it now exists under Boost Build ? The documentation for CTest does not explain what it actually does amidst it enumeration of a trillion or so command line options, but why should it do that, right ?
Regards, Roger

On 7/26/17 1:28 PM, Edward Diener via Boost wrote:
On 7/26/2017 3:33 PM, Robert Ramey via Boost wrote:
On 7/26/17 11:49 AM, Edward Diener via Boost wrote:
Following John Maddock's appeal for practical solutions related to the move to CMake, I would like to know what the CMake equivalent is to the Boost Build unit test functionality.
I can tell you what I did for the serialization library. This predates the current discussions so will likely be considered out of date by the current contributors to the discussion. On the other hand, it does actually work and you can test it yourself.
Robert I appreciate your answer. Is this the CMakeLists.txt in the CMake subdirectory of the serialization library
right ? Remember I am talking about
header-only libraries where nothing is "built" and I am only trying to run the unit tests in the library's 'test' subdirectory using CMake rather than Boost Build using CMake's equivalent of a script file rather than a Boost Build jamfile.
right that's what it does. It builds the library and executes the tests
In other words what do I write for CMake in order to do a Boost Build compile, compile-fail, link, link-fail, run, and run-fail unit tests ?
One adds targets (which for me is just the name of the source/excutable without the cpp, exe, etc. to a list to get compile and run functionality. Its pretty simple.
I want to see actual cmake code. Saying "Its pretty simple" only means it is pretty simple for those who already know CMake well.
aaa - the CMakeLists.txt IS the "cmake code". You invoke CMake - I use the GUI version and it produces and makefile, or an IDE file (I generate the one for XCode). They you build the library (if it has one) and run the tests from the project it created for you. It's a two step process.
<RANT>
I find the CMake documentation at https://cmake.org/cmake/help/v3.9/ nearly impossible to understand due to the fact that there is no overriding explanation of how CMake is supposed to work as practical examples of setting up CMake scripts. While I realize that all the information is supposedly there I have just about no idea how it all comes together. I am guessing that the only way I can really learn CMake is buying the "Mastering CMake" book, which I don't mind doing so if I have to do it. If I am wrong and someone would like to point me to an actual overview which explains CMake in a regular and meaningful way, rather than a bunch of loosely related topics which is the actual CMake documentation in the above URL, it would be appreciated.
This is a serious short coming of CMake. Given this it's a mystery that CMake has been successful. But then then look at all the competition it's ever worse!!! To addressed this, I wrote a walk through in the boost library inclubator to help library developers get started with CMake. It uses on the safe numerics library CMake as an example. This example/narrative addresses only header-only files.
</RANT>
I couldn't make the compile-fail and run-fail targets to work so I just removed them from the cmake test suite. So if your thinking that CMake has a ways to go to be able to offer what we expect, you're right.
So you are saying that you cannot run a unit test with CMake that is meant to show a failure. Wow, what a brilliant system !
Right - I presume that there's a way around it if one has nothing else to do. I didn't find a simple one. This fact, combined with the fact that the system is well maintained, continually enhanced, and responsive to it's users for over 10 years says a lot the demands and state of library development. It seems almost no one ever asked kitware for this.
Does CMake have an equivalent to the Boost Build alias rule ? or the Boost Build project rule ?
LOL - I honestly don't remember what these rules do in boost build. But then I've only been using boost build for 15 years.
The project rule for unit tests is normally used to set usage requirements for all tests being run. The alias rule for unit tests is normally used as a means of grouping some unit tests together so that the end-user of the tests can test individual groups rather than all the tests.
My real point is that even one with a lot of experience with boost build (I would guess that the serialization library carpet bombing test suite is the most complex one in boost) still can't sit down and use it without spending huge amounts of time re-figuring it out. This is a serious indictment of boost build which has now brought us to this unhappy juncture.
The great majority of the Boost libraries are header-only libraries where Boost Build is used to run the unit tests and,
true and this case is pretty simple in both cmake and boost build.
Once again saying it is pretty simple in cmake isn't showing me anything.
I've provide two working examples for two very different libraries and a narrative with the information I gleaned from making them. I would hope they would be helpful. It's the best anyone can/has done.
optionally, Boost Build is used to build the documentation using a quickbook, boostbook, doxygen toolset to generate html and pdf files.
Personally, I could never make this tool chain work despite a large investment of effort. I gave up on doxygen and quickbook after concluding them to be essentially non-functional. I did manage to use the xml tools loaded with boost tools along with some downloaded FOP, a version of XMLMind to create toolchain which is about 100 times better and 1/100 as complex as the boost one for creating documents. So you're life will be better when you're forced to leave these behind.
I heartily disagree. Using quickbook and doxygen is a no-brainer for me, but there is no point of debating this. Are both perfect ? No. But using them makes creating documentation supremely easy.
Right - we just have to disagree.
I did say that there is no Boost requirement for creating documentation using Boost Build. Some documentation is straight html. But for those libraries that do go the quickbook and doxygen way using Boost Build we need a CMake way of doing this if that is what the Boost Steering Committee is mandating.
Actually we don't. We can just use a small shell script or batch file. It's only about 5 lines long and it's about 20 times simpler than going into boost build trying to train an ant to train a flee. This is another think we'll just have to disagree on.
snip
But you might want to experiment a little with the following useful orthogonal ideas.
a) investigate how CMake Find<your library> should work and consider adding the files necessary to support it to your library. This wouldn't mess up any of your other work.
b) Consider making a simple CMake project to build library and run tests such as I did for the serialization library. CMake gurus it seems dismiss this effort. But it has worked for me. The main benefit for me is that permits to to build a working IDE which I can used to build and debug. For me, this has turned out to be a great time saver.
I do not know how you learned enough of CMake to do what you want but, apropos my RANT, I am finding my ability to do this severely hampered by the piecemeal documentation which CMake deems its own docs.
Ahh docs ... This is not a CMake issue. It's a software development issue. Documentation for software libraries and tools is generally atrocious. The thing that bugs me the most though is that developers don't agree with this statement. they think their stuff is pretty good an that the users just need to be smarter. They're dead wrong. They design crappy stuff then package then (under duress) write some docs which only serve to illustrate how lacking in coherence their original designs are in the first place. But then it's too late and everyone else is left with the job of shoveling a path through it. I've got a lot to say about this and I will be giving a presentation on the subject at CPPCon 2017 for anyone who might be interested. That is the both of you. Robert Ramey

On 7/26/2017 5:03 PM, Robert Ramey via Boost wrote:
On 7/26/17 1:28 PM, Edward Diener via Boost wrote:
On 7/26/2017 3:33 PM, Robert Ramey via Boost wrote:
On 7/26/17 11:49 AM, Edward Diener via Boost wrote:
Following John Maddock's appeal for practical solutions related to the move to CMake, I would like to know what the CMake equivalent is to the Boost Build unit test functionality.
I can tell you what I did for the serialization library. This predates the current discussions so will likely be considered out of date by the current contributors to the discussion. On the other hand, it does actually work and you can test it yourself.
Robert I appreciate your answer. Is this the CMakeLists.txt in the CMake subdirectory of the serialization library
right
? Remember I am talking about
header-only libraries where nothing is "built" and I am only trying to run the unit tests in the library's 'test' subdirectory using CMake rather than Boost Build using CMake's equivalent of a script file rather than a Boost Build jamfile.
right that's what it does. It builds the library and executes the tests
But the case I brought up is that there is no "build' of the library, since it is header only, but the library needs to execute its tests using CMake. That appears to me to be different from your case.
In other words what do I write for CMake in order to do a Boost Build compile, compile-fail, link, link-fail, run, and run-fail unit tests ?
One adds targets (which for me is just the name of the source/excutable without the cpp, exe, etc. to a list to get compile and run functionality. Its pretty simple.
I want to see actual cmake code. Saying "Its pretty simple" only means it is pretty simple for those who already know CMake well.
aaa - the CMakeLists.txt IS the "cmake code". You invoke CMake - I use the GUI version and it produces and makefile, or an IDE file (I generate the one for XCode). They you build the library (if it has one) and run the tests from the project it created for you. It's a two step process.
So you are saying that even when their is no actual library to be built, as is the case with a header only library, you still specify a CMakeLists.txt file as if you were building a library ?
<RANT>
I find the CMake documentation at https://cmake.org/cmake/help/v3.9/ nearly impossible to understand due to the fact that there is no overriding explanation of how CMake is supposed to work as practical examples of setting up CMake scripts. While I realize that all the information is supposedly there I have just about no idea how it all comes together. I am guessing that the only way I can really learn CMake is buying the "Mastering CMake" book, which I don't mind doing so if I have to do it. If I am wrong and someone would like to point me to an actual overview which explains CMake in a regular and meaningful way, rather than a bunch of loosely related topics which is the actual CMake documentation in the above URL, it would be appreciated.
This is a serious short coming of CMake. Given this it's a mystery that CMake has been successful. But then then look at all the competition it's ever worse!!! To addressed this, I wrote a walk through in the boost library inclubator to help library developers get started with CMake. It uses on the safe numerics library CMake as an example. This example/narrative addresses only header-only files.
Would you please give a URL ?
</RANT>
I couldn't make the compile-fail and run-fail targets to work so I just removed them from the cmake test suite. So if your thinking that CMake has a ways to go to be able to offer what we expect, you're right.
So you are saying that you cannot run a unit test with CMake that is meant to show a failure. Wow, what a brilliant system !
Right - I presume that there's a way around it if one has nothing else to do. I didn't find a simple one. This fact, combined with the fact that the system is well maintained, continually enhanced, and responsive to it's users for over 10 years says a lot the demands and state of library development. It seems almost no one ever asked kitware for this.
Does CMake have an equivalent to the Boost Build alias rule ? or the Boost Build project rule ?
LOL - I honestly don't remember what these rules do in boost build. But then I've only been using boost build for 15 years.
The project rule for unit tests is normally used to set usage requirements for all tests being run. The alias rule for unit tests is normally used as a means of grouping some unit tests together so that the end-user of the tests can test individual groups rather than all the tests.
My real point is that even one with a lot of experience with boost build (I would guess that the serialization library carpet bombing test suite is the most complex one in boost) still can't sit down and use it without spending huge amounts of time re-figuring it out. This is a serious indictment of boost build which has now brought us to this unhappy juncture.
Do you know if CMake has a way of setting usage requirements for all builds in a directory ? That is the effect of the project rule in Boost Build when running tests. It is very useful and means that you do not have to repeat a bunch of usage requirements for each and every test.
The great majority of the Boost libraries are header-only libraries where Boost Build is used to run the unit tests and,
true and this case is pretty simple in both cmake and boost build.
Once again saying it is pretty simple in cmake isn't showing me anything.
I've provide two working examples for two very different libraries and a narrative with the information I gleaned from making them. I would hope they would be helpful. It's the best anyone can/has done.
I will look at your serialization CMake code and I assume the other working example is for your Safe Numerics library, so I will look at that also. Thanks !
optionally, Boost Build is used to build the documentation using a quickbook, boostbook, doxygen toolset to generate html and pdf files.
Personally, I could never make this tool chain work despite a large investment of effort. I gave up on doxygen and quickbook after concluding them to be essentially non-functional. I did manage to use the xml tools loaded with boost tools along with some downloaded FOP, a version of XMLMind to create toolchain which is about 100 times better and 1/100 as complex as the boost one for creating documents. So you're life will be better when you're forced to leave these behind.
I heartily disagree. Using quickbook and doxygen is a no-brainer for me, but there is no point of debating this. Are both perfect ? No. But using them makes creating documentation supremely easy.
Right - we just have to disagree.
I did say that there is no Boost requirement for creating documentation using Boost Build. Some documentation is straight html. But for those libraries that do go the quickbook and doxygen way using Boost Build we need a CMake way of doing this if that is what the Boost Steering Committee is mandating.
Actually we don't. We can just use a small shell script or batch file. It's only about 5 lines long and it's about 20 times simpler than going into boost build trying to train an ant to train a flee. This is another think we'll just have to disagree on.
snip
But you might want to experiment a little with the following useful orthogonal ideas.
a) investigate how CMake Find<your library> should work and consider adding the files necessary to support it to your library. This wouldn't mess up any of your other work.
b) Consider making a simple CMake project to build library and run tests such as I did for the serialization library. CMake gurus it seems dismiss this effort. But it has worked for me. The main benefit for me is that permits to to build a working IDE which I can used to build and debug. For me, this has turned out to be a great time saver.
I do not know how you learned enough of CMake to do what you want but, apropos my RANT, I am finding my ability to do this severely hampered by the piecemeal documentation which CMake deems its own docs.
Ahh docs ... This is not a CMake issue. It's a software development issue. Documentation for software libraries and tools is generally atrocious. The thing that bugs me the most though is that developers don't agree with this statement. they think their stuff is pretty good an that the users just need to be smarter. They're dead wrong. They design crappy stuff then package then (under duress) write some docs which only serve to illustrate how lacking in coherence their original designs are in the first place. But then it's too late and everyone else is left with the job of shoveling a path through it.
I've got a lot to say about this and I will be giving a presentation on the subject at CPPCon 2017 for anyone who might be interested. That is the both of you.
I did not realize I was a split personality <g>.
Robert Ramey

On 7/26/17 7:28 PM, Edward Diener via Boost wrote:
But the case I brought up is that there is no "build' of the library, since it is header only, but the library needs to execute its tests using CMake. That appears to me to be different from your case.
Well, there is the safe numerics library which has CMake tests and is header only. It's much simpler than the serialization library. https://github.com/robertramey/safe_numerics You'll find a few CMakeLists.txt files in different places like include/example/test etc. Not that much unlike bjam where one has jam files in test, example, etc.
So you are saying that even when their is no actual library to be built, as is the case with a header only library, you still specify a CMakeLists.txt file as if you were building a library ?
yep - just like for bjam you specify a jam file in the test directory, for CMake you specifiy a CMakeLists.txt file in the test directory.
This is a serious short coming of CMake. Given this it's a mystery that CMake has been successful. But then then look at all the competition it's ever worse!!! To addressed this, I wrote a walk through in the boost library inclubator to help library developers get started with CMake. It uses on the safe numerics library CMake as an example. This example/narrative addresses only header-only files.
Would you please give a URL ?
https://github.com/robertramey/safe_numerics I'm really disappointed you couldn't find this. It recently was reviewed for inclusion in boost and accepted!
Do you know if CMake has a way of setting usage requirements for all builds in a directory ? That is the effect of the project rule in Boost Build when running tests. It is very useful and means that you do not have to repeat a bunch of usage requirements for each and every test.
Right. Every build system has to address this problem. In bjam I can make a "rule" which passes "usage requirements". In CMake I make a "function" which passes arguments. Similar but different. For some reason every build system designer seems to fancy himself a language designer and what we get is some oddball/irregular "language" for describing the "build". When it barely works the release and then start patching it up to so it can be made to work. The result is what we have to today. An assortment of build languages none of which are very good. This is addressed by inventing a NEW build language which isn't really any better than the one it's meant to replace. This happens again and again - this is how we came to be where we are today.
I will look at your serialization CMake code and I assume the other working example is for your Safe Numerics library, so I will look at that also. Thanks !
There are other examples already in boost - made by people who know more about CMake than I do. (I hate wasting time on this kind of stuff). Unfortunately, many of these examples seem to be more elaborate than necessary and end up totally obscuring that which starts out simple. Its a chronic programmer disease to start with something understandable which works and then try to make it perfect there by mucking it up. Hollywood plastic surgeries have the exact same problem.
I've got a lot to say about this and I will be giving a presentation on the subject at CPPCon 2017 for anyone who might be interested. That is the both of you.
I did not realize I was a split personality <g>.
Hmmmm - It didn't occur to me that you might be going to CPPCon. So I guess I'll plan for three then. Robert

On 7/27/2017 1:12 AM, Robert Ramey via Boost wrote:
On 7/26/17 7:28 PM, Edward Diener via Boost wrote:
But the case I brought up is that there is no "build' of the library, since it is header only, but the library needs to execute its tests using CMake. That appears to me to be different from your case.
Well, there is the safe numerics library which has CMake tests and is header only. It's much simpler than the serialization library.
https://github.com/robertramey/safe_numerics
You'll find a few CMakeLists.txt files in different places like include/example/test etc. Not that much unlike bjam where one has jam files in test, example, etc.
So you are saying that even when their is no actual library to be built, as is the case with a header only library, you still specify a CMakeLists.txt file as if you were building a library ?
yep - just like for bjam you specify a jam file in the test directory, for CMake you specifiy a CMakeLists.txt file in the test directory.
Great, thanks !
This is a serious short coming of CMake. Given this it's a mystery that CMake has been successful. But then then look at all the competition it's ever worse!!! To addressed this, I wrote a walk through in the boost library inclubator to help library developers get started with CMake. It uses on the safe numerics library CMake as an example. This example/narrative addresses only header-only files.
Would you please give a URL ?
https://github.com/robertramey/safe_numerics
I'm really disappointed you couldn't find this. It recently was reviewed for inclusion in boost and accepted!
You said you wrote a walk through in the boost library incubator and I asked for the URL of the walk through. You gave me a link to your safe numerics library on github. I have found http://blincubator.com/tools_cmak/. I will read this.
Do you know if CMake has a way of setting usage requirements for all builds in a directory ? That is the effect of the project rule in Boost Build when running tests. It is very useful and means that you do not have to repeat a bunch of usage requirements for each and every test.
Right. Every build system has to address this problem.
In bjam I can make a "rule" which passes "usage requirements".
You do not have to make a rule since the builtin "project" rule already does this for you.
In CMake I make a "function" which passes arguments.
So CMake has no way built-in command to say "here are the usage requirements for every target in this directory" other than to have to write a "function" each time ? That seems a bit poor.
Similar but different. For some reason every build system designer seems to fancy himself a language designer and what we get is some oddball/irregular "language" for describing the "build". When it barely works the release and then start patching it up to so it can be made to work. The result is what we have to today. An assortment of build languages none of which are very good. This is addressed by inventing a NEW build language which isn't really any better than the one it's meant to replace. This happens again and again - this is how we came to be where we are today.
I will look at your serialization CMake code and I assume the other working example is for your Safe Numerics library, so I will look at that also. Thanks !
There are other examples already in boost - made by people who know more about CMake than I do. (I hate wasting time on this kind of stuff). Unfortunately, many of these examples seem to be more elaborate than necessary and end up totally obscuring that which starts out simple. Its a chronic programmer disease to start with something understandable which works and then try to make it perfect there by mucking it up. Hollywood plastic surgeries have the exact same problem.
I've got a lot to say about this and I will be giving a presentation on the subject at CPPCon 2017 for anyone who might be interested. That is the both of you.
I did not realize I was a split personality <g>.
Hmmmm - It didn't occur to me that you might be going to CPPCon. So I guess I'll plan for three then.
No I don't go to CPPCon but if you can get programmers to create better documentation by your talk there I am all for it.
Robert

On 27/07/2017 10:24, Edward Diener via Boost wrote:
On 7/27/2017 1:12 AM, Robert Ramey via Boost wrote:
On 7/26/17 7:28 PM, Edward Diener via Boost wrote:
But the case I brought up is that there is no "build' of the library, since it is header only, but the library needs to execute its tests using CMake. That appears to me to be different from your case.
Well, there is the safe numerics library which has CMake tests and is header only. It's much simpler than the serialization library.
https://github.com/robertramey/safe_numerics
You'll find a few CMakeLists.txt files in different places like include/example/test etc. Not that much unlike bjam where one has jam files in test, example, etc.
So you are saying that even when their is no actual library to be built, as is the case with a header only library, you still specify a CMakeLists.txt file as if you were building a library ?
yep - just like for bjam you specify a jam file in the test directory, for CMake you specifiy a CMakeLists.txt file in the test directory.
Great, thanks !
This is a serious short coming of CMake. Given this it's a mystery that CMake has been successful. But then then look at all the competition it's ever worse!!! To addressed this, I wrote a walk through in the boost library inclubator to help library developers get started with CMake. It uses on the safe numerics library CMake as an example. This example/narrative addresses only header-only files.
Would you please give a URL ?
https://github.com/robertramey/safe_numerics
I'm really disappointed you couldn't find this. It recently was reviewed for inclusion in boost and accepted!
You said you wrote a walk through in the boost library incubator and I asked for the URL of the walk through. You gave me a link to your safe numerics library on github. I have found http://blincubator.com/tools_cmak/. I will read this.
Do you know if CMake has a way of setting usage requirements for all builds in a directory ? That is the effect of the project rule in Boost Build when running tests. It is very useful and means that you do not have to repeat a bunch of usage requirements for each and every test.
Right. Every build system has to address this problem.
In bjam I can make a "rule" which passes "usage requirements".
You do not have to make a rule since the builtin "project" rule already does this for you.
In CMake I make a "function" which passes arguments.
So CMake has no way built-in command to say "here are the usage requirements for every target in this directory" other than to have to write a "function" each time ? That seems a bit poor.
It does has some support for things like that, but it scales poorly. What you can do instead is have a target with all the requirements you want and link against it and they will bubble up. Or you just use a function, which is not a bad thing, they can remove a lot of boilerplate and make the intent clearer. Would you write ASM all the time? No, you write high level C++. A build system is no different.
Similar but different. For some reason every build system designer seems to fancy himself a language designer and what we get is some oddball/irregular "language" for describing the "build". When it barely works the release and then start patching it up to so it can be made to work. The result is what we have to today. An assortment of build languages none of which are very good. This is addressed by inventing a NEW build language which isn't really any better than the one it's meant to replace. This happens again and again - this is how we came to be where we are today.
I will look at your serialization CMake code and I assume the other working example is for your Safe Numerics library, so I will look at that also. Thanks !
There are other examples already in boost - made by people who know more about CMake than I do. (I hate wasting time on this kind of stuff). Unfortunately, many of these examples seem to be more elaborate than necessary and end up totally obscuring that which starts out simple. Its a chronic programmer disease to start with something understandable which works and then try to make it perfect there by mucking it up. Hollywood plastic surgeries have the exact same problem.
I've got a lot to say about this and I will be giving a presentation on the subject at CPPCon 2017 for anyone who might be interested. That is the both of you.
I did not realize I was a split personality <g>.
Hmmmm - It didn't occur to me that you might be going to CPPCon. So I guess I'll plan for three then.
No I don't go to CPPCon but if you can get programmers to create better documentation by your talk there I am all for it.
Robert
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On 7/27/2017 8:12 AM, Florent Castelli via Boost wrote:
On 27/07/2017 10:24, Edward Diener via Boost wrote:
On 7/27/2017 1:12 AM, Robert Ramey via Boost wrote:
On 7/26/17 7:28 PM, Edward Diener via Boost wrote:
But the case I brought up is that there is no "build' of the library, since it is header only, but the library needs to execute its tests using CMake. That appears to me to be different from your case.
Well, there is the safe numerics library which has CMake tests and is header only. It's much simpler than the serialization library.
https://github.com/robertramey/safe_numerics
You'll find a few CMakeLists.txt files in different places like include/example/test etc. Not that much unlike bjam where one has jam files in test, example, etc.
So you are saying that even when their is no actual library to be built, as is the case with a header only library, you still specify a CMakeLists.txt file as if you were building a library ?
yep - just like for bjam you specify a jam file in the test directory, for CMake you specifiy a CMakeLists.txt file in the test directory.
Great, thanks !
This is a serious short coming of CMake. Given this it's a mystery that CMake has been successful. But then then look at all the competition it's ever worse!!! To addressed this, I wrote a walk through in the boost library inclubator to help library developers get started with CMake. It uses on the safe numerics library CMake as an example. This example/narrative addresses only header-only files.
Would you please give a URL ?
https://github.com/robertramey/safe_numerics
I'm really disappointed you couldn't find this. It recently was reviewed for inclusion in boost and accepted!
You said you wrote a walk through in the boost library incubator and I asked for the URL of the walk through. You gave me a link to your safe numerics library on github. I have found http://blincubator.com/tools_cmak/. I will read this.
Do you know if CMake has a way of setting usage requirements for all builds in a directory ? That is the effect of the project rule in Boost Build when running tests. It is very useful and means that you do not have to repeat a bunch of usage requirements for each and every test.
Right. Every build system has to address this problem.
In bjam I can make a "rule" which passes "usage requirements".
You do not have to make a rule since the builtin "project" rule already does this for you.
In CMake I make a "function" which passes arguments.
So CMake has no way built-in command to say "here are the usage requirements for every target in this directory" other than to have to write a "function" each time ? That seems a bit poor.
It does has some support for things like that, but it scales poorly. What you can do instead is have a target with all the requirements you want and link against it and they will bubble up. Or you just use a function, which is not a bad thing, they can remove a lot of boilerplate and make the intent clearer. Would you write ASM all the time? No, you write high level C++. A build system is no different.
What would be the CMake syntax for such a target ? In Boost Build the 'project' rule lets you set requirements. How would I do that with your target. Some CMake code please ?
Similar but different. For some reason every build system designer seems to fancy himself a language designer and what we get is some oddball/irregular "language" for describing the "build". When it barely works the release and then start patching it up to so it can be made to work. The result is what we have to today. An assortment of build languages none of which are very good. This is addressed by inventing a NEW build language which isn't really any better than the one it's meant to replace. This happens again and again - this is how we came to be where we are today.
I will look at your serialization CMake code and I assume the other working example is for your Safe Numerics library, so I will look at that also. Thanks !
There are other examples already in boost - made by people who know more about CMake than I do. (I hate wasting time on this kind of stuff). Unfortunately, many of these examples seem to be more elaborate than necessary and end up totally obscuring that which starts out simple. Its a chronic programmer disease to start with something understandable which works and then try to make it perfect there by mucking it up. Hollywood plastic surgeries have the exact same problem.
I've got a lot to say about this and I will be giving a presentation on the subject at CPPCon 2017 for anyone who might be interested. That is the both of you.
I did not realize I was a split personality <g>.
Hmmmm - It didn't occur to me that you might be going to CPPCon. So I guess I'll plan for three then.
No I don't go to CPPCon but if you can get programmers to create better documentation by your talk there I am all for it.
Robert

On 7/27/17 1:24 AM, Edward Diener via Boost wrote:
In bjam I can make a "rule" which passes "usage requirements".
You do not have to make a rule since the builtin "project" rule already does this for you.
Right - but in anything more than a trivial case, I always had to make my own rule - then I have to start learning a whole bunch of stuff I never wanted to know. Then it sucks gobs of time. Then ... This is not so much a critcism of bjam - all systems - CMake included, have similar problems.
So CMake has no way built-in command to say "here are the usage requirements for every target in this directory" other than to have to write a "function" each time ? That seems a bit poor.
Well there is a long list of built in functions just as in bjam there is a long list of built in "rules". As I've said, the systems are actually more alike than different. This fact is obscured by the very irregular syntax and unorthogonal design used by all such systems. This is forced in turn by the fact that the "build" problem is itself ad-hoc with designers adding features and facilities and requirements willy-nilly and trying to patch all this up with a "build" system. IDEs sort of "fix" this problem - for one release then they change it for the next one. The whole thing is continuing agony. I can't fix this, and tool providers have failed to do so. I have my own theory for why this is so, but it's too off topic. Bottom line: a) All of these systems work well in trivial toy problems b) None of these systems "work" well in real world cases c) All of these systems can be made to "work" for each case d) All of the require investment of effort to realize c) above One thing about CMake which is unique and actually a genius idea. Rather than try to actually build the project, build the make, or ide files that build the project. This extra level of indirection has made CMake more useful to me overall than bjam - in spite of having it's own short comings. Among the things I've never understood about bjam/boost build is why no one ever was tempted to add the options to create ide/make projects. This would have added to bjam the one thing that CMake has but boost build doesn't. I'm guessing that doing this would be a lot of unrewarded work. Doesn't matter now.
No I don't go to CPPCon but if you can get programmers to create better documentation by your talk there I am all for it.
I'm sort of pessimistic: a) I don't think that most developers think there is a problem. That is that the documentation they create is just fine. b) If there is a problem, it's that the users aren't smart enough c) Hence, I think that most developers will read the abstract and conclude that the subject is trivial and doesn't hold any value for them. The funny thing is that this conflicts with users experience - even when the users are library developers themselves!!! But I'm a sucker for lost causes. Robert Ramey

On 2017-07-27 17:08, Robert Ramey via Boost wrote:
No I don't go to CPPCon but if you can get programmers to create better documentation by your talk there I am all for it.
I'm sort of pessimistic:
a) I don't think that most developers think there is a problem. That is that the documentation they create is just fine. b) If there is a problem, it's that the users aren't smart enough c) Hence, I think that most developers will read the abstract and conclude that the subject is trivial and doesn't hold any value for them.
For what it's worth: - I know that my documentation is incomplete - I am amazed by what users figure out - I just read your abstract. I won't promise to be in the room (there is just too much going on at CppCon), but it sounds valuable to me. I sincerely hope that you touch on expression templates, too. Seems to me I need concepts and metaclasses to describe them. Cheers, Roland

On 26/07/2017 20:33, Robert Ramey via Boost wrote:
On 7/26/17 11:49 AM, Edward Diener via Boost wrote:
Following John Maddock's appeal for practical solutions related to the move to CMake, I would like to know what the CMake equivalent is to the Boost Build unit test functionality.
I can tell you what I did for the serialization library. This predates the current discussions so will likely be considered out of date by the current contributors to the discussion. On the other hand, it does actually work and you can test it yourself.
In other words what do I write for CMake in order to do a Boost Build compile, compile-fail, link, link-fail, run, and run-fail unit tests ?
One adds targets (which for me is just the name of the source/excutable without the cpp, exe, etc. to a list to get compile and run functionality. Its pretty simple.
I couldn't make the compile-fail and run-fail targets to work so I just removed them from the cmake test suite. So if your thinking that CMake has a ways to go to be able to offer what we expect, you're right.
compile-fail tests a re often an essential part of testing certain classes of library.
optionally, Boost Build is used to build the documentation using a quickbook, boostbook, doxygen toolset to generate html and pdf files.
Personally, I could never make this tool chain work despite a large investment of effort. I gave up on doxygen and quickbook after concluding them to be essentially non-functional. I did manage to use the xml tools loaded with boost tools along with some downloaded FOP, a version of XMLMind to create toolchain which is about 100 times better and 1/100 as complex as the boost one for creating documents. So you're life will be better when you're forced to leave these behind.
I'm sorry you found them that way, Doxygen I hate (to be fair it's mostly the way folks use it rather than the tool itself), but quickbook works extremely well for me. As for the toolchain.... I find it amusing that this is the one thing folks most often complain about... even citing it as an example of why we shouldn't invent stuff ourselves... when it's actually the one thing that was *not* invented here! Best, John. --- This email has been checked for viruses by AVG. http://www.avg.com

-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of John Maddock via Boost Sent: 27 July 2017 10:10 To: boost@lists.boost.org Cc: John Maddock Subject: Re: [boost] CMake and Boost Build tests
<snip>
optionally, Boost Build is used to build the documentation using a quickbook, boostbook, doxygen toolset to generate html and pdf files.
Personally, I could never make this tool chain work despite a large investment of effort. I gave up on doxygen and quickbook after concluding them to be essentially non-functional. I did manage to use the xml tools loaded with boost tools along with some downloaded FOP, a version of XMLMind to create toolchain which is about 100 times better and 1/100 as complex as the boost one for creating documents. So you're life will be better when you're forced to leave these behind.
Works for me.
I'm sorry you found them that way, Doxygen I hate
I also don't like the *look* of its readable output. But *Doxygen comment syntax for annotating classes, functions, macros etc works OK, and is the nearest thing to a Standard for annotating code, and not just C++.* These comments are readable in the code without cluttering and obscuring too much. Some examples of the *syntax*: File information: \brief This file is a partial reference implementation for the proposed "C++ binary fixed-point arithmetic" as specified in N3352.\n \details There is optional support for certain variations of fixed_point using preprocessor definitions. Not all of these are supported at the moment. The potential options include: Is supported now : \#define BOOST_FIXED_POINT_DISABLE_IOSTREAM\n Is supported now : \#define BOOST_FIXED_POINT_DISABLE_MULTIPRECISION\n Information above a class, here the constructor /*! Constructor for a data series to plot \tparam T an STL container: for example: @c multimap. \param begin Starting iterator into container of data series begin() to start at the beginning. \param end Ending iterator into container of data series, end() to finish with the last item. \param title Title for the plot. */ function svg_2d_plot_series& svg_2d_plot_series::line_on(bool on_) { //! Set @c true to draw line linking data points. line_style_.line_on_ = on_; return *this; //! \return Reference to svg_2d_plot_series to make chainable. Doxygen can now use the Clang compiler to extract all this into a form that other programs can then use to lay out in a human-readable (or other machine readable form). (Doxygen makes a good stab at template-heavy C++, but isn't a compiler). I believe that this is the way forward to producing documentation in the style we prefer (or in the style that others that others prefer).
(to be fair it's mostly the way folks use it rather than the tool itself),
+1 but Quickbook works extremely well for me + more than 1
As for the toolchain.... I find it amusing that this is the one thing folks most often complain about... even citing it as an example of why we shouldn't invent stuff ourselves... when it's actually the one thing that was *not* invented here!
It works for me - most of the trouble is with bjam syntax and dealing with MS file naming. Paul --- Paul A. Bristow Prizet Farmhouse Kendal UK LA8 8AB +44 (0) 1539 561830

On 7/27/17 3:24 AM, Paul A. Bristow via Boost wrote:
Works for me. LOL - of course that makes me feel much better
I'm sorry you found them that way, Doxygen I hate
I also don't like the *look* of its readable output.
Actually, I don't have much complaint about the look. This is subjective after all. And much / most of the look is in the xslt transforms which create the html.
But
*Doxygen comment syntax for annotating classes, functions, macros etc works OK, and is the nearest thing to a Standard for annotating code, and not just C++.*
Right - and that's a problem. In a recent survey, over 90% of users stated their biggest problem with open source is the documentation or lack thereof. Composing the two facts above yields the conclusion: "Standard user documentation of software is found to be unsatisfactory by over 90% of users" How can this be anything but bad. <snip> Doxygen excerpt.
Doxygen can now use the Clang compiler to extract all this into a form that other programs can then use to lay out in a human-readable (or other machine readable form). (Doxygen makes a good stab at template-heavy C++, but isn't a compiler).
I believe that this is the way forward to producing documentation in the style we prefer (or in the style that others that others prefer).
(to be fair it's mostly the way folks use it rather than the tool itself),
+1
This can be said for many tools. And DOxygen is founded on a nobel idea - literate programming. But it fails to keep up when the problem is more complex - e.g. Templated programming libraries. And in fact leads a programmer to think he's explained something when in fact all he has done has places a good looking facade on a header file.
but Quickbook works extremely well for me
+ more than 1
In defense of quickbook, it's a reasonable approach to editing XML for which very few, if any, reasonable alternatives have been proposed. I've had limited exposure to quickbook. But what I have had does not encourage me. It does work but when I wanted to add something which seemed simple, had to go back to the mailing list. Little languages is an attractive concept. But in practice it turns in to "unfinished" language. In any case I think a big source of the problem is DOxygen which gives the programmer a false sense of security that he's solve an annoyingly tedious problem and can now move on - when in fact he hasn't.
As for the toolchain.... I find it amusing that this is the one thing folks most often complain about... even citing it as an example of why we shouldn't invent stuff ourselves... when it's actually the one thing that was *not* invented here!
It works for me - most of the trouble is with bjam syntax and dealing with MS file naming.
The tool chain is a problem - so I use a 4/5 line shell script. This is bad enough. But then we have to compound the problem by trying embed the substance of this script into bjam - a whole other system - just to save a 5 line script. Now when you need to customization - you have to dive back into bjam. Sorry this is not an effective usage of one's time. Robert Ramey

On Wed, 2017-07-26 at 14:49 -0400, Edward Diener via Boost wrote:
Following John Maddock's appeal for practical solutions related to the move to CMake, I would like to know what the CMake equivalent is to the Boost Build unit test functionality.
In other words what do I write for CMake in order to do a Boost Build compile, compile-fail, link, link-fail, run, and run-fail unit tests ?
For a run test, you would do, something like: add_executable(footest footest.cpp) add_test(NAME footest COMMAND $<TARGET_FILE:footest>) For a compile/link test: add_library(footest STATIC EXCLUDE_FROM_ALL footest.cpp) add_test(NAME footest COMMAND ${CMAKE_COMMAND} --build . --target footest -- config $<CONFIGURATION> WORKING_DIRECTORY ${CMAKE_BINARY_DIR}) To make the tests with expected failures, just set a property: set_tests_properties(footest PROPERTIES WILL_FAIL TRUE) You also can check for certain output, which nice for checking that the compile fail tests has triggered the static assert. Ideally, the BCMTest module can make this simpler: http://bcm.readthedocs.io/en/latest/src/BCMTest.html Which you can write the equivalent tests like this: Run test: bcm_test(NAME footest SOURCES footest.cpp) Run fail test: bcm_test(NAME footest SOURCES footest.cpp WILL_FAIL) Compile test: bcm_test(NAME footest SOURCES footest.cpp COMPILE_ONLY) Compile fail test: bcm_test(NAME footest SOURCES footest.cpp COMPILE_ONLY WILL_FAIL)
Does CMake have an equivalent to the Boost Build alias rule ?
You can create interface target that includes the set of targets. add_library(core INTERFACE) target_link_libraries(core INTERFACE im reader writer)
or the Boost Build project rule ?
Hmm, I am not sure how this is relevant to cmake.
The great majority of the Boost libraries are header-only libraries where Boost Build is used to run the unit tests and, optionally, Boost Build is used to build the documentation using a quickbook, boostbook, doxygen toolset to generate html and pdf files. While I realize that building a Boost library, and making that built library available to end-users via CMake, has been the focus of most of the discussion so far regarding CMake and Boost, I am assuming the move to CMake also means that running unit tests and building docs where applicable must also be done through CMake. If I am wrong about this then I personally will be only too happy to continue using Boost Build to do these things.
There has been cmake written in the past that has done the documentation, I don't recall where it was at. Either way, why do you think that generating documentation is not possible with cmake?

On 7/26/2017 3:42 PM, paul via Boost wrote:
On Wed, 2017-07-26 at 14:49 -0400, Edward Diener via Boost wrote:
Following John Maddock's appeal for practical solutions related to the move to CMake, I would like to know what the CMake equivalent is to the Boost Build unit test functionality.
In other words what do I write for CMake in order to do a Boost Build compile, compile-fail, link, link-fail, run, and run-fail unit tests ?
For a run test, you would do, something like:
add_executable(footest footest.cpp) add_test(NAME footest COMMAND $<TARGET_FILE:footest>)
OK, that makes sense.
For a compile/link test:
add_library(footest STATIC EXCLUDE_FROM_ALL footest.cpp) add_test(NAME footest COMMAND ${CMAKE_COMMAND} --build . --target footest -- config $<CONFIGURATION> WORKING_DIRECTORY ${CMAKE_BINARY_DIR})
Why am I adding a library when all I am trying to do is see whether source file(s) can be compiled to their object files successfully ( the compile test ) or whether source file(s) can be compiled to their object files successfully then linked as an executable without linking errors ( the link test ) ?
To make the tests with expected failures, just set a property:
set_tests_properties(footest PROPERTIES WILL_FAIL TRUE)
OK, thanks !
You also can check for certain output, which nice for checking that the compile fail tests has triggered the static assert.
Ideally, the BCMTest module can make this simpler:
http://bcm.readthedocs.io/en/latest/src/BCMTest.html
Which you can write the equivalent tests like this:
Run test: bcm_test(NAME footest SOURCES footest.cpp)
Run fail test: bcm_test(NAME footest SOURCES footest.cpp WILL_FAIL)
Compile test: bcm_test(NAME footest SOURCES footest.cpp COMPILE_ONLY)
Compile fail test: bcm_test(NAME footest SOURCES footest.cpp COMPILE_ONLY WILL_FAIL)
Thanks !
Does CMake have an equivalent to the Boost Build alias rule ?
You can create interface target that includes the set of targets.
add_library(core INTERFACE) target_link_libraries(core INTERFACE im reader writer)
Thanks I will look at the use of INTERFACE.
or the Boost Build project rule ?
Hmm, I am not sure how this is relevant to cmake.
For the purposes of unit tests the project rule lets the developer set usage requirements for all tests in the jamfile, rather than having to repeat the usage requirements for each test. Since usage requirements in Boost Build can be pretty specific depending on the compiler being used the entire usage requirements for all tests can be pretty extensive. Having to repeat that extensive list of usage requirements for every test would be a PITA.
The great majority of the Boost libraries are header-only libraries where Boost Build is used to run the unit tests and, optionally, Boost Build is used to build the documentation using a quickbook, boostbook, doxygen toolset to generate html and pdf files. While I realize that building a Boost library, and making that built library available to end-users via CMake, has been the focus of most of the discussion so far regarding CMake and Boost, I am assuming the move to CMake also means that running unit tests and building docs where applicable must also be done through CMake. If I am wrong about this then I personally will be only too happy to continue using Boost Build to do these things.
There has been cmake written in the past that has done the documentation, I don't recall where it was at. Either way, why do you think that generating documentation is not possible with cmake?
I never said that generating documentation from the quickbook/boostbook/doxygen tools was impossible with CMake. I would love to be shown how to do it, since it is pretty flexible with Boost Build jam files.

On Wed, 2017-07-26 at 22:48 -0400, Edward Diener via Boost wrote:
On 7/26/2017 3:42 PM, paul via Boost wrote:
On Wed, 2017-07-26 at 14:49 -0400, Edward Diener via Boost wrote:
Following John Maddock's appeal for practical solutions related to the move to CMake, I would like to know what the CMake equivalent is to the Boost Build unit test functionality.
In other words what do I write for CMake in order to do a Boost Build compile, compile-fail, link, link-fail, run, and run-fail unit tests ?
For a run test, you would do, something like:
add_executable(footest footest.cpp) add_test(NAME footest COMMAND $<TARGET_FILE:footest>)
OK, that makes sense.
For a compile/link test:
add_library(footest STATIC EXCLUDE_FROM_ALL footest.cpp) add_test(NAME footest COMMAND ${CMAKE_COMMAND} --build . --target footest -- config $<CONFIGURATION> WORKING_DIRECTORY ${CMAKE_BINARY_DIR})
Why am I adding a library when all I am trying to do is see whether source file(s) can be compiled to their object files successfully ( the compile test ) or whether source file(s) can be compiled to their object files successfully then linked as an executable without linking errors ( the link test ) ?
You have to add a target to be built, and then the test will invoke this target. Tests in cmake are just commands that can do anything, and in this case we instruct cmake to build a target. The reason its a library and not an executable is so that a `main` function is not needed. Of course, this will test both compiling and linking together, as there is no separation for this in cmake.
To make the tests with expected failures, just set a property:
set_tests_properties(footest PROPERTIES WILL_FAIL TRUE)
OK, thanks !
You also can check for certain output, which nice for checking that the compile fail tests has triggered the static assert.
Ideally, the BCMTest module can make this simpler:
http://bcm.readthedocs.io/en/latest/src/BCMTest.html
Which you can write the equivalent tests like this:
Run test: bcm_test(NAME footest SOURCES footest.cpp)
Run fail test: bcm_test(NAME footest SOURCES footest.cpp WILL_FAIL)
Compile test: bcm_test(NAME footest SOURCES footest.cpp COMPILE_ONLY)
Compile fail test: bcm_test(NAME footest SOURCES footest.cpp COMPILE_ONLY WILL_FAIL)
Thanks !
Does CMake have an equivalent to the Boost Build alias rule ?
You can create interface target that includes the set of targets.
add_library(core INTERFACE) target_link_libraries(core INTERFACE im reader writer)
Thanks I will look at the use of INTERFACE.
or the Boost Build project rule ?
Hmm, I am not sure how this is relevant to cmake.
For the purposes of unit tests the project rule lets the developer set usage requirements for all tests in the jamfile, rather than having to repeat the usage requirements for each test. Since usage requirements in Boost Build can be pretty specific depending on the compiler being used the entire usage requirements for all tests can be pretty extensive. Having to repeat that extensive list of usage requirements for every test would be a PITA.
Well you can use the global function such as `add_compile_options` or `link_libraries` but these are discouraged as this will apply to every target created in the directory(and the usage requirements are not propagated). So in general using per-target is preferred. However, in BCM, I have `bcm_link_libraries` that will link in those libraries when calling `bcm_test`, and can be excluded with the `NO_TEST_LIBS` option. So this provides the same convenience as the project rule. Although, a lot of cmake projects just write their own custom function to add the tests.

On 7/27/2017 10:45 AM, paul via Boost wrote:
On Wed, 2017-07-26 at 22:48 -0400, Edward Diener via Boost wrote:
On 7/26/2017 3:42 PM, paul via Boost wrote:
On Wed, 2017-07-26 at 14:49 -0400, Edward Diener via Boost wrote:
Following John Maddock's appeal for practical solutions related to the move to CMake, I would like to know what the CMake equivalent is to the Boost Build unit test functionality.
In other words what do I write for CMake in order to do a Boost Build compile, compile-fail, link, link-fail, run, and run-fail unit tests ?
For a run test, you would do, something like:
add_executable(footest footest.cpp) add_test(NAME footest COMMAND $<TARGET_FILE:footest>)
OK, that makes sense.
For a compile/link test:
add_library(footest STATIC EXCLUDE_FROM_ALL footest.cpp) add_test(NAME footest COMMAND ${CMAKE_COMMAND} --build . --target footest -- config $<CONFIGURATION> WORKING_DIRECTORY ${CMAKE_BINARY_DIR})
Why am I adding a library when all I am trying to do is see whether source file(s) can be compiled to their object files successfully ( the compile test ) or whether source file(s) can be compiled to their object files successfully then linked as an executable without linking errors ( the link test ) ?
You have to add a target to be built, and then the test will invoke this target. Tests in cmake are just commands that can do anything, and in this case we instruct cmake to build a target. The reason its a library and not an executable is so that a `main` function is not needed.
But a 'compile' or 'link' rule always attempts at compiling or linking an executable anyway, which has a 'main' function.
Of course, this will test both compiling and linking together, as there is no separation for this in cmake.
To make the tests with expected failures, just set a property:
set_tests_properties(footest PROPERTIES WILL_FAIL TRUE)
OK, thanks !
You also can check for certain output, which nice for checking that the compile fail tests has triggered the static assert.
Ideally, the BCMTest module can make this simpler:
http://bcm.readthedocs.io/en/latest/src/BCMTest.html
Which you can write the equivalent tests like this:
Run test: bcm_test(NAME footest SOURCES footest.cpp)
Run fail test: bcm_test(NAME footest SOURCES footest.cpp WILL_FAIL)
Compile test: bcm_test(NAME footest SOURCES footest.cpp COMPILE_ONLY)
Compile fail test: bcm_test(NAME footest SOURCES footest.cpp COMPILE_ONLY WILL_FAIL)
Thanks !
Does CMake have an equivalent to the Boost Build alias rule ?
You can create interface target that includes the set of targets.
add_library(core INTERFACE) target_link_libraries(core INTERFACE im reader writer)
Thanks I will look at the use of INTERFACE.
or the Boost Build project rule ?
Hmm, I am not sure how this is relevant to cmake.
For the purposes of unit tests the project rule lets the developer set usage requirements for all tests in the jamfile, rather than having to repeat the usage requirements for each test. Since usage requirements in Boost Build can be pretty specific depending on the compiler being used the entire usage requirements for all tests can be pretty extensive. Having to repeat that extensive list of usage requirements for every test would be a PITA.
Well you can use the global function such as `add_compile_options` or `link_libraries` but these are discouraged as this will apply to every target created in the directory(and the usage requirements are not propagated).
That is actually what I want and is the equivalent of the Boost Build 'project' rule in a test jamfile with the 'requirements' options. Why this is discouraged I would not know. Surely if you run a large number of tests in a Boost Build test jamfile, it is much easier to specify requirements for all test in one place rather than repeat them for each and every test. This is especially true when requirements are based on particular compilers possibly with particular versions of those compilers.
So in general using per-target is preferred.
However, in BCM, I have `bcm_link_libraries` that will link in those libraries when calling `bcm_test`, and can be excluded with the `NO_TEST_LIBS` option. So this provides the same convenience as the project rule. Although, a lot of cmake projects just write their own custom function to add the tests.
Where is your BCM ? Is this going to be used by the CMake implementation in Boost as mandated by the steering committee ? I am all in favor of CMake code which makes the transition between Boost Build and CMake as transparent as possible. If I, as a library developer, am expected to provide CMake equivalents to my Boost Build unit tests and CMake equivalents to my Boost Build doc generation I want as high a boilerplate level as possible.

On Thu, 2017-07-27 at 11:19 -0400, Edward Diener via Boost wrote:
On 7/27/2017 10:45 AM, paul via Boost wrote:
On Wed, 2017-07-26 at 22:48 -0400, Edward Diener via Boost wrote:
On 7/26/2017 3:42 PM, paul via Boost wrote:
On Wed, 2017-07-26 at 14:49 -0400, Edward Diener via Boost wrote:
Following John Maddock's appeal for practical solutions related to the move to CMake, I would like to know what the CMake equivalent is to the Boost Build unit test functionality.
In other words what do I write for CMake in order to do a Boost Build compile, compile-fail, link, link-fail, run, and run-fail unit tests ?
For a run test, you would do, something like:
add_executable(footest footest.cpp) add_test(NAME footest COMMAND $<TARGET_FILE:footest>)
OK, that makes sense.
For a compile/link test:
add_library(footest STATIC EXCLUDE_FROM_ALL footest.cpp) add_test(NAME footest COMMAND ${CMAKE_COMMAND} --build . --target footest -- config $<CONFIGURATION> WORKING_DIRECTORY ${CMAKE_BINARY_DIR})
Why am I adding a library when all I am trying to do is see whether source file(s) can be compiled to their object files successfully ( the compile test ) or whether source file(s) can be compiled to their object files successfully then linked as an executable without linking errors ( the link test ) ?
You have to add a target to be built, and then the test will invoke this target. Tests in cmake are just commands that can do anything, and in this case we instruct cmake to build a target. The reason its a library and not an executable is so that a `main` function is not needed.
But a 'compile' or 'link' rule always attempts at compiling or linking an executable anyway, which has a 'main' function.
Yes, and it will still work with a main function, but I dont see the need for the main function. The compile-fail tests in the Fit library don't use a main function either.
Of course, this will test both compiling and linking together, as there is no separation for this in cmake.
To make the tests with expected failures, just set a property:
set_tests_properties(footest PROPERTIES WILL_FAIL TRUE)
OK, thanks !
You also can check for certain output, which nice for checking that the compile fail tests has triggered the static assert.
Ideally, the BCMTest module can make this simpler:
http://bcm.readthedocs.io/en/latest/src/BCMTest.html
Which you can write the equivalent tests like this:
Run test: bcm_test(NAME footest SOURCES footest.cpp)
Run fail test: bcm_test(NAME footest SOURCES footest.cpp WILL_FAIL)
Compile test: bcm_test(NAME footest SOURCES footest.cpp COMPILE_ONLY)
Compile fail test: bcm_test(NAME footest SOURCES footest.cpp COMPILE_ONLY WILL_FAIL)
Thanks !
Does CMake have an equivalent to the Boost Build alias rule ?
You can create interface target that includes the set of targets.
add_library(core INTERFACE) target_link_libraries(core INTERFACE im reader writer)
Thanks I will look at the use of INTERFACE.
or the Boost Build project rule ?
Hmm, I am not sure how this is relevant to cmake.
For the purposes of unit tests the project rule lets the developer set usage requirements for all tests in the jamfile, rather than having to repeat the usage requirements for each test. Since usage requirements in Boost Build can be pretty specific depending on the compiler being used the entire usage requirements for all tests can be pretty extensive. Having to repeat that extensive list of usage requirements for every test would be a PITA.
Well you can use the global function such as `add_compile_options` or `link_libraries` but these are discouraged as this will apply to every target created in the directory(and the usage requirements are not propagated).
That is actually what I want and is the equivalent of the Boost Build 'project' rule in a test jamfile with the 'requirements' options. Why this is discouraged I would not know. Surely if you run a large number of tests in a Boost Build test jamfile, it is much easier to specify requirements for all test in one place rather than repeat them for each and every test. This is especially true when requirements are based on particular compilers possibly with particular versions of those compilers.
Yes, but a custom function is usually what is done to avoid repeating the requirements everywhere.
So in general using per-target is preferred.
However, in BCM, I have `bcm_link_libraries` that will link in those libraries when calling `bcm_test`, and can be excluded with the `NO_TEST_LIBS` option. So this provides the same convenience as the project rule. Although, a lot of cmake projects just write their own custom function to add the tests.
Where is your BCM ?
https://github.com/boost-cmake/bcm http://bcm.readthedocs.io
Is this going to be used by the CMake implementation in Boost as mandated by the steering committee ?
I don't know. I think having a set cmake modules with common functionality will be necessary, and I am working on setting boost building and testing with these modules.
I am all in favor of CMake code which makes the transition between Boost Build and CMake as transparent as possible. If I, as a library developer, am expected to provide CMake equivalents to my Boost Build unit tests and CMake equivalents to my Boost Build doc generation I want as high a boilerplate level as possible.
Yea, I think copying and pasting these solutions for every library is not maintainable, but we also need a balance. Originally, I had the BCM modules use a high-level `bcm_boost_package` to setup a library like this: bcm_boost_package(core VERSION 1.61.0 DEPENDS assert config ) Which it would take care of a lot of cmake setup under the hood. The problem with this is it is unfamiliar to cmake developers and to non-cmake developers as well. So its kind of the worst of both worlds. So instead I am focusing on building functions that can reduce the boilerplate for boost-like tasks while still making the cmake as transparent as possible. So instead the above is written as: bcm_setup_version(VERSION 1.61.0) find_package(boost_assert) find_package(boost_config) add_library(boost_core INTERFACE) set_property(TARGET boost_core PROPERTY EXPORT_NAME core) target_link_libraries(boost_core INTERFACE boost::assert boost::config) bcm_deploy(TARGETS boost_core INCLUDE include NAMESPACE boost::) There could be more work to improve this, but I think this is a good balance.

Am 27.07.2017 um 16:45 schrieb paul via Boost:
On Wed, 2017-07-26 at 22:48 -0400, Edward Diener via Boost wrote:
On 7/26/2017 3:42 PM, paul via Boost wrote:
[...]
For a compile/link test:
add_library(footest STATIC EXCLUDE_FROM_ALL footest.cpp) add_test(NAME footest COMMAND ${CMAKE_COMMAND} --build . --target footest -- config $<CONFIGURATION> WORKING_DIRECTORY ${CMAKE_BINARY_DIR})
Why am I adding a library when all I am trying to do is see whether source file(s) can be compiled to their object files successfully ( the compile test ) or whether source file(s) can be compiled to their object files successfully then linked as an executable without linking errors ( the link test ) ?
You have to add a target to be built, and then the test will invoke this target. Tests in cmake are just commands that can do anything, and in this case we instruct cmake to build a target. The reason its a library and not an executable is so that a `main` function is not needed.
Of course, this will test both compiling and linking together, as there is no separation for this in cmake.
CMake has support for "object-libraries", which are targets that just compile source-files into object-files I have not tried it but I would think the following would be possible to only test compilation without linking: ```cmake add_library(OBJECT footest EXCLUDE_FROM_ALL footest.cpp # ... further source-files can go here ... ) add_test(NAME footest COMMAND ${CMAKE_COMMAND} --build . --target footest --config $<CONFIGURATION> WORKING_DIRECTORY ${CMAKE_BINARY_DIR} ) ``` HTH, Deniz

On 7/27/2017 11:41 AM, Deniz Bahadir via Boost wrote:
Am 27.07.2017 um 16:45 schrieb paul via Boost:
On Wed, 2017-07-26 at 22:48 -0400, Edward Diener via Boost wrote:
On 7/26/2017 3:42 PM, paul via Boost wrote:
[...]
For a compile/link test:
add_library(footest STATIC EXCLUDE_FROM_ALL footest.cpp) add_test(NAME footest COMMAND ${CMAKE_COMMAND} --build . --target footest -- config $<CONFIGURATION> WORKING_DIRECTORY ${CMAKE_BINARY_DIR})
Why am I adding a library when all I am trying to do is see whether source file(s) can be compiled to their object files successfully ( the compile test ) or whether source file(s) can be compiled to their object files successfully then linked as an executable without linking errors ( the link test ) ?
You have to add a target to be built, and then the test will invoke this target. Tests in cmake are just commands that can do anything, and in this case we instruct cmake to build a target. The reason its a library and not an executable is so that a `main` function is not needed.
Of course, this will test both compiling and linking together, as there is no separation for this in cmake.
CMake has support for "object-libraries", which are targets that just compile source-files into object-files I have not tried it but I would think the following would be possible to only test compilation without linking:
```cmake
add_library(OBJECT footest EXCLUDE_FROM_ALL footest.cpp # ... further source-files can go here ... ) add_test(NAME footest COMMAND ${CMAKE_COMMAND} --build . --target footest --config $<CONFIGURATION> WORKING_DIRECTORY ${CMAKE_BINARY_DIR} )
Thanks, that looks reasonable for a compile test.
```
HTH, Deniz

On 26/07/2017 20:42, paul via Boost wrote:
On Wed, 2017-07-26 at 14:49 -0400, Edward Diener via Boost wrote:
Following John Maddock's appeal for practical solutions related to the move to CMake, I would like to know what the CMake equivalent is to the Boost Build unit test functionality.
In other words what do I write for CMake in order to do a Boost Build compile, compile-fail, link, link-fail, run, and run-fail unit tests ? For a run test, you would do, something like:
add_executable(footest footest.cpp) add_test(NAME footest COMMAND $<TARGET_FILE:footest>)
For a compile/link test:
add_library(footest STATIC EXCLUDE_FROM_ALL footest.cpp) add_test(NAME footest COMMAND ${CMAKE_COMMAND} --build . --target footest -- config $<CONFIGURATION> WORKING_DIRECTORY ${CMAKE_BINARY_DIR})
I think I'm losing the will to live looking at that :(
To make the tests with expected failures, just set a property:
set_tests_properties(footest PROPERTIES WILL_FAIL TRUE)
You also can check for certain output, which nice for checking that the compile fail tests has triggered the static assert.
Ideally, the BCMTest module can make this simpler:
http://bcm.readthedocs.io/en/latest/src/BCMTest.html
Which you can write the equivalent tests like this:
Run test: bcm_test(NAME footest SOURCES footest.cpp)
Run fail test: bcm_test(NAME footest SOURCES footest.cpp WILL_FAIL)
Compile test: bcm_test(NAME footest SOURCES footest.cpp COMPILE_ONLY)
Compile fail test: bcm_test(NAME footest SOURCES footest.cpp COMPILE_ONLY WILL_FAIL)
That looks way better! I assume we can also add annotation for things like: * Defines. * Configuration if-then-else logic. * Compiler specific flags (probably in conjunction with some configuration test). Thanks, John. --- This email has been checked for viruses by AVG. http://www.avg.com

For a run test, you would do, something like:
add_executable(footest footest.cpp) add_test(NAME footest COMMAND $<TARGET_FILE:footest>)
"If <command> specifies an executable target (created by add_executable()) it will automatically be replaced by the location of the executable created at build time." So, add_test(NAME footest COMMAND footest)

On Thu, 2017-07-27 at 10:16 +0100, John Maddock via Boost wrote:
On 26/07/2017 20:42, paul via Boost wrote:
On Wed, 2017-07-26 at 14:49 -0400, Edward Diener via Boost wrote:
Following John Maddock's appeal for practical solutions related to the move to CMake, I would like to know what the CMake equivalent is to the Boost Build unit test functionality.
In other words what do I write for CMake in order to do a Boost Build compile, compile-fail, link, link-fail, run, and run-fail unit tests ?
For a run test, you would do, something like:
add_executable(footest footest.cpp) add_test(NAME footest COMMAND $<TARGET_FILE:footest>)
For a compile/link test:
add_library(footest STATIC EXCLUDE_FROM_ALL footest.cpp) add_test(NAME footest COMMAND ${CMAKE_COMMAND} --build . --target footest -- config $<CONFIGURATION> WORKING_DIRECTORY ${CMAKE_BINARY_DIR})
I think I'm losing the will to live looking at that :(
It is low-level, which is why people just write a custom function for it.
To make the tests with expected failures, just set a property:
set_tests_properties(footest PROPERTIES WILL_FAIL TRUE)
You also can check for certain output, which nice for checking that the compile fail tests has triggered the static assert.
Ideally, the BCMTest module can make this simpler:
http://bcm.readthedocs.io/en/latest/src/BCMTest.html
Which you can write the equivalent tests like this:
Run test: bcm_test(NAME footest SOURCES footest.cpp)
Run fail test: bcm_test(NAME footest SOURCES footest.cpp WILL_FAIL)
Compile test: bcm_test(NAME footest SOURCES footest.cpp COMPILE_ONLY)
Compile fail test: bcm_test(NAME footest SOURCES footest.cpp COMPILE_ONLY WILL_FAIL)
That looks way better!
I assume we can also add annotation for things like:
* Defines. * Configuration if-then-else logic. * Compiler specific flags (probably in conjunction with some configuration test).
You mean add annotations to the function? You can do all those things directly with cmake as the name is the name of the target and test, so you can just write: bcm_test(NAME footest SOURCES footest.cpp) target_compile_options(footest -Wall) You can see an example of how the tests would be written in cmake for Boost.Config on PR#129 I have, here: https://github.com/boostorg/config/pull/129/files#diff-15547c54d3d4898a882b4... 7b3cee381 In addition to what is done in bjam, this also generates the tests from the .ipp files so there is no need to commit generated files to the repo.

On 26/07/2017 20:49, Edward Diener via Boost wrote:
Following John Maddock's appeal for practical solutions related to the move to CMake, I would like to know what the CMake equivalent is to the Boost Build unit test functionality.
In other words what do I write for CMake in order to do a Boost Build compile, compile-fail, link, link-fail, run, and run-fail unit tests ?
In my own Boost-CMake project, I have implemented regular "RUN" in a way that looks similar to the original Boost Build using functions: https://github.com/Orphis/boost-cmake/blob/master/libs/system.cmake Other commands should be easy enough to implement as well, but I haven't had the interest in doing that in my project yet (which predates the SC decision). It is possible to do "compile" by creating a regular static library target with that file and then invoking "cmake --build . --target <compile test name>". Compile failures can be done similarly and check they fail. Link and Link-fail can be done the same way using actual binaries that are built, but not run. The difference is making sure you have a clear separation of building Boost itself and running the tests. I found out that failures are pretty common with those tests and many of them will fail to run without errors on all supported compiler and platform, so isolation is quite important. The problem is that some build tools will not support parallel invocations (Ninja doesn't for example, see https://groups.google.com/forum/#!topic/ninja-build/4VP7whvWSH8 ) and thus running tests would be need to be linearized or moved to make, which should support that scenario better.
Does CMake have an equivalent to the Boost Build alias rule ? or the Boost Build project rule ?
The great majority of the Boost libraries are header-only libraries where Boost Build is used to run the unit tests and, optionally, Boost Build is used to build the documentation using a quickbook, boostbook, doxygen toolset to generate html and pdf files. While I realize that building a Boost library, and making that built library available to end-users via CMake, has been the focus of most of the discussion so far regarding CMake and Boost, I am assuming the move to CMake also means that running unit tests and building docs where applicable must also be done through CMake. If I am wrong about this then I personally will be only too happy to continue using Boost Build to do these things.
CMake support header-only libraries using the INTERFACE keyword of "add_library()". Those will propagate usage requirements like any other library to their users. You can have you tests depend on those easily. You can always have custom targets in each library to generate their documentation in any way you like. Building the documentation for the whole Boost project can be done by building all of those targets.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On 7/27/2017 8:43 AM, Florent Castelli via Boost wrote:
On 26/07/2017 20:49, Edward Diener via Boost wrote:
Following John Maddock's appeal for practical solutions related to the move to CMake, I would like to know what the CMake equivalent is to the Boost Build unit test functionality.
In other words what do I write for CMake in order to do a Boost Build compile, compile-fail, link, link-fail, run, and run-fail unit tests ?
In my own Boost-CMake project, I have implemented regular "RUN" in a way that looks similar to the original Boost Build using functions: https://github.com/Orphis/boost-cmake/blob/master/libs/system.cmake
What is RUN supposed to be in your link above ?
Other commands should be easy enough to implement as well, but I haven't had the interest in doing that in my project yet (which predates the SC decision).
It is possible to do "compile" by creating a regular static library target with that file and then invoking "cmake --build . --target <compile test name>". Compile failures can be done similarly and check they fail. Link and Link-fail can be done the same way using actual binaries that are built, but not run.
This is a poor solution.
The difference is making sure you have a clear separation of building Boost itself and running the tests. I found out that failures are pretty common with those tests and many of them will fail to run without errors on all supported compiler and platform, so isolation is quite important.
Again I need to mention that the tests are run for header only libraries which are not "built" at all in most cases.
The problem is that some build tools will not support parallel invocations (Ninja doesn't for example, see https://groups.google.com/forum/#!topic/ninja-build/4VP7whvWSH8 ) and thus running tests would be need to be linearized or moved to make, which should support that scenario better.
That is the end-users problem. I am just interested in Boost solving its own problem of moving from the Boost Build testing rules to something with CMake which does the same thing.
Does CMake have an equivalent to the Boost Build alias rule ? or the Boost Build project rule ?
The great majority of the Boost libraries are header-only libraries where Boost Build is used to run the unit tests and, optionally, Boost Build is used to build the documentation using a quickbook, boostbook, doxygen toolset to generate html and pdf files. While I realize that building a Boost library, and making that built library available to end-users via CMake, has been the focus of most of the discussion so far regarding CMake and Boost, I am assuming the move to CMake also means that running unit tests and building docs where applicable must also be done through CMake. If I am wrong about this then I personally will be only too happy to continue using Boost Build to do these things.
CMake support header-only libraries using the INTERFACE keyword of "add_library()". Those will propagate usage requirements like any other library to their users. You can have you tests depend on those easily.
You can always have custom targets in each library to generate their documentation in any way you like. Building the documentation for the whole Boost project can be done by building all of those targets.

On Jul 27, 2017 16:41, "Edward Diener via Boost" <boost@lists.boost.org> wrote: On 7/27/2017 8:43 AM, Florent Castelli via Boost wrote:
On 26/07/2017 20:49, Edward Diener via Boost wrote:
Following John Maddock's appeal for practical solutions related to the move to CMake, I would like to know what the CMake equivalent is to the Boost Build unit test functionality.
In other words what do I write for CMake in order to do a Boost Build compile, compile-fail, link, link-fail, run, and run-fail unit tests ?
In my own Boost-CMake project, I have implemented regular "RUN" in a way that looks similar to the original Boost Build using functions: https://github.com/Orphis/boost-cmake/blob/master/libs/system.cmake
What is RUN supposed to be in your link above ? An equivalent of run from Boost Build. See https://github.com/boostorg/system/blob/develop/test/Jamfile.v2 Other commands should be easy enough to implement as well, but I haven't
had the interest in doing that in my project yet (which predates the SC decision).
It is possible to do "compile" by creating a regular static library target with that file and then invoking "cmake --build . --target <compile test name>". Compile failures can be done similarly and check they fail. Link and Link-fail can be done the same way using actual binaries that are built, but not run.
This is a poor solution. Could you please explain why? Using Cmake directly to build the sources with the same flags as set in the toolchain is not controversial I'd say. It's actually working better than other solutions I've seen trying to reproduce the features.
The difference is making sure you have a clear separation of building Boost itself and running the tests. I found out that failures are pretty common with those tests and many of them will fail to run without errors on all supported compiler and platform, so isolation is quite important.
Again I need to mention that the tests are run for header only libraries which are not "built" at all in most cases. That's irrelevant for CMake. You end up building the tests still, they will depend on the header only library and its dependencies.
The problem is that some build tools will not support parallel invocations (Ninja doesn't for example, see https://groups.google.com/foru m/#!topic/ninja-build/4VP7whvWSH8 ) and thus running tests would be need to be linearized or moved to make, which should support that scenario better.
That is the end-users problem. I am just interested in Boost solving its own problem of moving from the Boost Build testing rules to something with CMake which does the same thing. Not really, because if a solution that works correctly and doing perfect forwarding of build flags can't be parallelized in some case, it would need fixing. If a solution doesn't have perfect forwarding of compiler settings and requirements, it's not really acceptable either.
Does CMake have an equivalent to the Boost Build alias rule ? or the Boost Build project rule ?
The great majority of the Boost libraries are header-only libraries where Boost Build is used to run the unit tests and, optionally, Boost Build is used to build the documentation using a quickbook, boostbook, doxygen toolset to generate html and pdf files. While I realize that building a Boost library, and making that built library available to end-users via CMake, has been the focus of most of the discussion so far regarding CMake and Boost, I am assuming the move to CMake also means that running unit tests and building docs where applicable must also be done through CMake. If I am wrong about this then I personally will be only too happy to continue using Boost Build to do these things.
CMake support header-only libraries using the INTERFACE keyword of "add_library()". Those will propagate usage requirements like any other library to their users. You can have you tests depend on those easily.
You can always have custom targets in each library to generate their documentation in any way you like. Building the documentation for the whole Boost project can be done by building all of those targets.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman /listinfo.cgi/boost

On 7/27/2017 12:58 PM, Florent Castelli via Boost wrote:
On Jul 27, 2017 16:41, "Edward Diener via Boost" <boost@lists.boost.org> wrote:
On 7/27/2017 8:43 AM, Florent Castelli via Boost wrote:
On 26/07/2017 20:49, Edward Diener via Boost wrote:
Following John Maddock's appeal for practical solutions related to the move to CMake, I would like to know what the CMake equivalent is to the Boost Build unit test functionality.
In other words what do I write for CMake in order to do a Boost Build compile, compile-fail, link, link-fail, run, and run-fail unit tests ?
In my own Boost-CMake project, I have implemented regular "RUN" in a way that looks similar to the original Boost Build using functions: https://github.com/Orphis/boost-cmake/blob/master/libs/system.cmake
What is RUN supposed to be in your link above ?
An equivalent of run from Boost Build. See https://github.com/boostorg/system/blob/develop/test/Jamfile.v2
Please show the code for RUN in CMake. To ease the transition from Boost Build to CMake we really need CMake equivalents to the unit testing rules of Boost Build at the very least.
Other commands should be easy enough to implement as well, but I haven't
had the interest in doing that in my project yet (which predates the SC decision).
It is possible to do "compile" by creating a regular static library target with that file and then invoking "cmake --build . --target <compile test name>". Compile failures can be done similarly and check they fail. Link and Link-fail can be done the same way using actual binaries that are built, but not run.
This is a poor solution.
Could you please explain why? Using Cmake directly to build the sources with the same flags as set in the toolchain is not controversial I'd say. It's actually working better than other solutions I've seen trying to reproduce the features.
I said it was a poor solution because all the Boost Build 'compile' rule does is to try to compile one or more source files. Success is if there are no compilations errors, while failure is if there is at least one compilation error. Why should one have to create an actual library to do this, especially as the source(s) may contain a main() function, and in fact usually does ? I think the right solution, as suggested to me by someone else, is to create a CMake OBJECT library, which as I understand it is not really a library but just OBJECT files, and then run a test based on that. But of course I do not know if CMake supports that.
The difference is making sure you have a clear separation of building Boost itself and running the tests. I found out that failures are pretty common with those tests and many of them will fail to run without errors on all supported compiler and platform, so isolation is quite important.
Again I need to mention that the tests are run for header only libraries which are not "built" at all in most cases.
That's irrelevant for CMake. You end up building the tests still, they will depend on the header only library and its dependencies.
The problem is that some build tools will not support parallel invocations (Ninja doesn't for example, see https://groups.google.com/foru m/#!topic/ninja-build/4VP7whvWSH8 ) and thus running tests would be need to be linearized or moved to make, which should support that scenario better.
That is the end-users problem. I am just interested in Boost solving its own problem of moving from the Boost Build testing rules to something with CMake which does the same thing.
Not really, because if a solution that works correctly and doing perfect forwarding of build flags can't be parallelized in some case, it would need fixing. If a solution doesn't have perfect forwarding of compiler settings and requirements, it's not really acceptable either.
Why would not each test have the correct flags whether the tests run in parallel or not ? Why would you want tests to run in parallel and not be linearized ?

On 27 Jul 2017, at 22:29, Edward Diener via Boost <boost@lists.boost.org> wrote:
On 7/27/2017 12:58 PM, Florent Castelli via Boost wrote:
On Jul 27, 2017 16:41, "Edward Diener via Boost" <boost@lists.boost.org> wrote: On 7/27/2017 8:43 AM, Florent Castelli via Boost wrote:
On 26/07/2017 20:49, Edward Diener via Boost wrote:
Following John Maddock's appeal for practical solutions related to the move to CMake, I would like to know what the CMake equivalent is to the Boost Build unit test functionality.
In other words what do I write for CMake in order to do a Boost Build compile, compile-fail, link, link-fail, run, and run-fail unit tests ?
In my own Boost-CMake project, I have implemented regular "RUN" in a way that looks similar to the original Boost Build using functions: https://github.com/Orphis/boost-cmake/blob/master/libs/system.cmake
What is RUN supposed to be in your link above ? An equivalent of run from Boost Build. See https://github.com/boostorg/system/blob/develop/test/Jamfile.v2
Please show the code for RUN in CMake. To ease the transition from Boost Build to CMake we really need CMake equivalents to the unit testing rules of Boost Build at the very least.
Let’s start over. Basically, you end up with those equivalents. Run is compiling, linking an executable and running it, checking the return code is success. It could translate to: add_executable(test_name test_file.cpp) target_link_libraries(test_name PRIVATE boost_library) add_test(NAME test_name COMMAND test_name) But, if you do that, you need 2 steps to run the test. First you compile the program (using regular "make test_name" or even “cmake —build . —target test_name”) and then you run your test through ctest “ctest . -R test_name” (or you run all the tests without the filter). The problem there is you have 2 commands, and ctest won’t capture the compilation or linking errors. To work around that, you can use this instead: add_test(NAME test_name COMMAND cmake —build . —target test_name && $<TARGET_FILE:test_name>) This will for each test build it with the proper build tool (make, ninja, msbuild, xcodebuild…) and then run the program. The whole output will be captured by ctest and can be transfered to any test dashboard, you won’t have to look for errors in 2 places. The big issue I talk about later if that if you asked ctest to run several tests in parallel, it will have several “cmake —build . —target …” in parallel for those tests, which is not supported by some tools (mainly Ninja).
Other commands should be easy enough to implement as well, but I haven't
had the interest in doing that in my project yet (which predates the SC decision).
It is possible to do "compile" by creating a regular static library target with that file and then invoking "cmake --build . --target <compile test name>". Compile failures can be done similarly and check they fail. Link and Link-fail can be done the same way using actual binaries that are built, but not run.
This is a poor solution. Could you please explain why? Using Cmake directly to build the sources with the same flags as set in the toolchain is not controversial I'd say. It's actually working better than other solutions I've seen trying to reproduce the features.
I said it was a poor solution because all the Boost Build 'compile' rule does is to try to compile one or more source files. Success is if there are no compilations errors, while failure is if there is at least one compilation error. Why should one have to create an actual library to do this, especially as the source(s) may contain a main() function, and in fact usually does ?
I think the right solution, as suggested to me by someone else, is to create a CMake OBJECT library, which as I understand it is not really a library but just OBJECT files, and then run a test based on that. But of course I do not know if CMake supports that.
The thing is, static libraries are guaranteed to work and fairly standard, object libraries are a bit finicky and may not work depending on the generator. Using one or the other is merely an optimisation though, static libraries are just an archive of object files after all. Similar to the code above, you could implement them with this code: add_library(test_name STATIC (or OBJECT) test_file.cpp) target_link_libraries(test_name PRIVATE boost_library) add_test(NAME test_name COMMAND cmake —build . —target test_name) In this code, ctest will just build it and capture any compilation error. Easy enough. It doesn’t matter if there are functions called main or not since it’s not linked, the compiler won’t care for the most part about the name of the functions (well, main has some special semantics, but I doubt any test would rely on that). For completeness, here’s a possible implementation of “LINK”. I understand this one links the source files into a binary, but doesn’t run it. This is simple CMake code as well: add_executable(test_name test_file.cpp) add_test(NAME test_name COMMAND cmake —build . —target test_name) In this case, CMake will build and link the files from the ctest invocation. To support the failing variants for all the above, you need this set_test_properties(test_name PROPERTIES WILL_FAIL TRUE). Note that if you expect a test to fail running, but still succeed linking, that’s not directly supported by the code above and some changes need to be made. Overall, that’s a lot of boilerplate for each test, which is why you need some convenience functions to make the build files clean and simple, with a clear intent. In complex test cases, it will certainly be possible to have people using CMake commands directly, but that should be rare.
The difference is making sure you have a clear separation of building Boost itself and running the tests. I found out that failures are pretty common with those tests and many of them will fail to run without errors on all supported compiler and platform, so isolation is quite important.
Again I need to mention that the tests are run for header only libraries which are not "built" at all in most cases. That's irrelevant for CMake. You end up building the tests still, they will depend on the header only library and its dependencies.
The problem is that some build tools will not support parallel invocations (Ninja doesn't for example, see https://groups.google.com/foru m/#!topic/ninja-build/4VP7whvWSH8 ) and thus running tests would be need to be linearized or moved to make, which should support that scenario better.
That is the end-users problem. I am just interested in Boost solving its own problem of moving from the Boost Build testing rules to something with CMake which does the same thing. Not really, because if a solution that works correctly and doing perfect forwarding of build flags can't be parallelized in some case, it would need fixing. If a solution doesn't have perfect forwarding of compiler settings and requirements, it's not really acceptable either.
Why would not each test have the correct flags whether the tests run in parallel or not ? Why would you want tests to run in parallel and not be linearized ?
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost <http://lists.boost.org/mailman/listinfo.cgi/boost>

On 7/27/2017 5:15 PM, Florent Castelli via Boost wrote:
On 27 Jul 2017, at 22:29, Edward Diener via Boost <boost@lists.boost.org> wrote:
On 7/27/2017 12:58 PM, Florent Castelli via Boost wrote:
On Jul 27, 2017 16:41, "Edward Diener via Boost" <boost@lists.boost.org> wrote: On 7/27/2017 8:43 AM, Florent Castelli via Boost wrote:
On 26/07/2017 20:49, Edward Diener via Boost wrote:
Following John Maddock's appeal for practical solutions related to the move to CMake, I would like to know what the CMake equivalent is to the Boost Build unit test functionality.
In other words what do I write for CMake in order to do a Boost Build compile, compile-fail, link, link-fail, run, and run-fail unit tests ?
In my own Boost-CMake project, I have implemented regular "RUN" in a way that looks similar to the original Boost Build using functions: https://github.com/Orphis/boost-cmake/blob/master/libs/system.cmake
What is RUN supposed to be in your link above ? An equivalent of run from Boost Build. See https://github.com/boostorg/system/blob/develop/test/Jamfile.v2
Please show the code for RUN in CMake. To ease the transition from Boost Build to CMake we really need CMake equivalents to the unit testing rules of Boost Build at the very least.
Let’s start over. Basically, you end up with those equivalents. Run is compiling, linking an executable and running it, checking the return code is success. It could translate to:
add_executable(test_name test_file.cpp) target_link_libraries(test_name PRIVATE boost_library) add_test(NAME test_name COMMAND test_name)
But, if you do that, you need 2 steps to run the test. First you compile the program (using regular "make test_name" or even “cmake —build . —target test_name”) and then you run your test through ctest “ctest . -R test_name” (or you run all the tests without the filter). The problem there is you have 2 commands, and ctest won’t capture the compilation or linking errors. To work around that, you can use this instead:
add_test(NAME test_name COMMAND cmake —build . —target test_name && $<TARGET_FILE:test_name>)
This will for each test build it with the proper build tool (make, ninja, msbuild, xcodebuild…) and then run the program. The whole output will be captured by ctest and can be transfered to any test dashboard, you won’t have to look for errors in 2 places. The big issue I talk about later if that if you asked ctest to run several tests in parallel, it will have several “cmake —build . —target …” in parallel for those tests, which is not supported by some tools (mainly Ninja).
Understood.
Other commands should be easy enough to implement as well, but I haven't
had the interest in doing that in my project yet (which predates the SC decision).
It is possible to do "compile" by creating a regular static library target with that file and then invoking "cmake --build . --target <compile test name>". Compile failures can be done similarly and check they fail. Link and Link-fail can be done the same way using actual binaries that are built, but not run.
This is a poor solution. Could you please explain why? Using Cmake directly to build the sources with the same flags as set in the toolchain is not controversial I'd say. It's actually working better than other solutions I've seen trying to reproduce the features.
I said it was a poor solution because all the Boost Build 'compile' rule does is to try to compile one or more source files. Success is if there are no compilations errors, while failure is if there is at least one compilation error. Why should one have to create an actual library to do this, especially as the source(s) may contain a main() function, and in fact usually does ?
I think the right solution, as suggested to me by someone else, is to create a CMake OBJECT library, which as I understand it is not really a library but just OBJECT files, and then run a test based on that. But of course I do not know if CMake supports that.
The thing is, static libraries are guaranteed to work and fairly standard, object libraries are a bit finicky and may not work depending on the generator. Using one or the other is merely an optimisation though, static libraries are just an archive of object files after all. Similar to the code above, you could implement them with this code:
add_library(test_name STATIC (or OBJECT) test_file.cpp) target_link_libraries(test_name PRIVATE boost_library) add_test(NAME test_name COMMAND cmake —build . —target test_name)
In this code, ctest will just build it and capture any compilation error. Easy enough. It doesn’t matter if there are functions called main or not since it’s not linked, the compiler won’t care for the most part about the name of the functions (well, main has some special semantics, but I doubt any test would rely on that).
But now you are filling up an end-user's build output with a static library for each 'compile' tested. Ughhh ! That seems so silly to me, where just adding object files on success seems normal.
For completeness, here’s a possible implementation of “LINK”. I understand this one links the source files into a binary, but doesn’t run it. This is simple CMake code as well:
add_executable(test_name test_file.cpp) add_test(NAME test_name COMMAND cmake —build . —target test_name)
In this case, CMake will build and link the files from the ctest invocation.
To support the failing variants for all the above, you need this set_test_properties(test_name PROPERTIES WILL_FAIL TRUE). Note that if you expect a test to fail running, but still succeed linking, that’s not directly supported by the code above and some changes need to be made.
Overall, that’s a lot of boilerplate for each test, which is why you need some convenience functions to make the build files clean and simple, with a clear intent. In complex test cases, it will certainly be possible to have people using CMake commands directly, but that should be rare.
Understood. Thanks !

On Thu, 2017-07-27 at 17:53 -0400, Edward Diener via Boost wrote:
On 7/27/2017 5:15 PM, Florent Castelli via Boost wrote:
On 27 Jul 2017, at 22:29, Edward Diener via Boost <boost@lists.boost.org
wrote:
On 7/27/2017 12:58 PM, Florent Castelli via Boost wrote:
On Jul 27, 2017 16:41, "Edward Diener via Boost" <boost@lists.boost.or g> wrote: On 7/27/2017 8:43 AM, Florent Castelli via Boost wrote:
On 26/07/2017 20:49, Edward Diener via Boost wrote:
Following John Maddock's appeal for practical solutions related to the move to CMake, I would like to know what the CMake equivalent is to the Boost Build unit test functionality.
In other words what do I write for CMake in order to do a Boost Build compile, compile-fail, link, link-fail, run, and run-fail unit tests ?
In my own Boost-CMake project, I have implemented regular "RUN" in a way that looks similar to the original Boost Build using functions: https://github.com/Orphis/boost-cmake/blob/master/libs/system.cmake
What is RUN supposed to be in your link above ? An equivalent of run from Boost Build. See https://github.com/boostorg/system/blob/develop/test/Jamfile.v2
Please show the code for RUN in CMake. To ease the transition from Boost Build to CMake we really need CMake equivalents to the unit testing rules of Boost Build at the very least.
Let’s start over. Basically, you end up with those equivalents. Run is compiling, linking an executable and running it, checking the return code is success. It could translate to:
add_executable(test_name test_file.cpp) target_link_libraries(test_name PRIVATE boost_library) add_test(NAME test_name COMMAND test_name)
But, if you do that, you need 2 steps to run the test. First you compile the program (using regular "make test_name" or even “cmake —build . —target test_name”) and then you run your test through ctest “ctest . -R test_name” (or you run all the tests without the filter). The problem there is you have 2 commands, and ctest won’t capture the compilation or linking errors. To work around that, you can use this instead:
add_test(NAME test_name COMMAND cmake —build . —target test_name && $<TARGET_FILE:test_name>)
This will for each test build it with the proper build tool (make, ninja, msbuild, xcodebuild…) and then run the program. The whole output will be captured by ctest and can be transfered to any test dashboard, you won’t have to look for errors in 2 places. The big issue I talk about later if that if you asked ctest to run several tests in parallel, it will have several “cmake —build . —target …” in parallel for those tests, which is not supported by some tools (mainly Ninja).
Understood.
Other commands should be easy enough to implement as well, but I haven't
had the interest in doing that in my project yet (which predates the SC decision).
It is possible to do "compile" by creating a regular static library target with that file and then invoking "cmake --build . --target <compile test name>". Compile failures can be done similarly and check they fail. Link and Link-fail can be done the same way using actual binaries that are built, but not run.
This is a poor solution. Could you please explain why? Using Cmake directly to build the sources with the same flags as set in the toolchain is not controversial I'd say. It's actually working better than other solutions I've seen trying to reproduce the features.
I said it was a poor solution because all the Boost Build 'compile' rule does is to try to compile one or more source files. Success is if there are no compilations errors, while failure is if there is at least one compilation error. Why should one have to create an actual library to do this, especially as the source(s) may contain a main() function, and in fact usually does ?
I think the right solution, as suggested to me by someone else, is to create a CMake OBJECT library, which as I understand it is not really a library but just OBJECT files, and then run a test based on that. But of course I do not know if CMake supports that.
The thing is, static libraries are guaranteed to work and fairly standard, object libraries are a bit finicky and may not work depending on the generator. Using one or the other is merely an optimisation though, static libraries are just an archive of object files after all. Similar to the code above, you could implement them with this code:
add_library(test_name STATIC (or OBJECT) test_file.cpp) target_link_libraries(test_name PRIVATE boost_library) add_test(NAME test_name COMMAND cmake —build . —target test_name)
In this code, ctest will just build it and capture any compilation error. Easy enough. It doesn’t matter if there are functions called main or not since it’s not linked, the compiler won’t care for the most part about the name of the functions (well, main has some special semantics, but I doubt any test would rely on that).
But now you are filling up an end-user's build output with a static library for each 'compile' tested. Ughhh ! That seems so silly to me, where just adding object files on success seems normal.
Well actually it should be added with `EXCLUDE_FROM_ALL`: add_library(test_name STATIC test_file.cpp EXCLUDE_FROM_ALL) This way it only builds when it is invoked directly(which is when the test is ran), and not in the general build.

On 28/07/2017 00:07, paul via Boost wrote:
On Thu, 2017-07-27 at 17:53 -0400, Edward Diener via Boost wrote:
On 7/27/2017 5:15 PM, Florent Castelli via Boost wrote:
On 27 Jul 2017, at 22:29, Edward Diener via Boost <boost@lists.boost.org
wrote: On 7/27/2017 12:58 PM, Florent Castelli via Boost wrote: On Jul 27, 2017 16:41, "Edward Diener via Boost" <boost@lists.boost.or g> wrote: On 7/27/2017 8:43 AM, Florent Castelli via Boost wrote:
On 26/07/2017 20:49, Edward Diener via Boost wrote:
> Following John Maddock's appeal for practical solutions related to > the > move to CMake, I would like to know what the CMake equivalent is > to the > Boost Build unit test functionality. > > In other words what do I write for CMake in order to do a Boost > Build > compile, compile-fail, link, link-fail, run, and run-fail unit > tests ? > In my own Boost-CMake project, I have implemented regular "RUN" in a way that looks similar to the original Boost Build using functions: https://github.com/Orphis/boost-cmake/blob/master/libs/system.cmake
What is RUN supposed to be in your link above ? An equivalent of run from Boost Build. See https://github.com/boostorg/system/blob/develop/test/Jamfile.v2 Please show the code for RUN in CMake. To ease the transition from Boost Build to CMake we really need CMake equivalents to the unit testing rules of Boost Build at the very least. Let’s start over. Basically, you end up with those equivalents. Run is compiling, linking an executable and running it, checking the return code is success. It could translate to:
add_executable(test_name test_file.cpp) target_link_libraries(test_name PRIVATE boost_library) add_test(NAME test_name COMMAND test_name)
But, if you do that, you need 2 steps to run the test. First you compile the program (using regular "make test_name" or even “cmake —build . —target test_name”) and then you run your test through ctest “ctest . -R test_name” (or you run all the tests without the filter). The problem there is you have 2 commands, and ctest won’t capture the compilation or linking errors. To work around that, you can use this instead:
add_test(NAME test_name COMMAND cmake —build . —target test_name && $<TARGET_FILE:test_name>)
This will for each test build it with the proper build tool (make, ninja, msbuild, xcodebuild…) and then run the program. The whole output will be captured by ctest and can be transfered to any test dashboard, you won’t have to look for errors in 2 places. The big issue I talk about later if that if you asked ctest to run several tests in parallel, it will have several “cmake —build . —target …” in parallel for those tests, which is not supported by some tools (mainly Ninja).
Understood.
Other commands should be easy enough to implement as well, but I haven't
had the interest in doing that in my project yet (which predates the SC decision).
It is possible to do "compile" by creating a regular static library target with that file and then invoking "cmake --build . --target <compile test name>". Compile failures can be done similarly and check they fail. Link and Link-fail can be done the same way using actual binaries that are built, but not run.
This is a poor solution. Could you please explain why? Using Cmake directly to build the sources with the same flags as set in the toolchain is not controversial I'd say. It's actually working better than other solutions I've seen trying to reproduce the features. I said it was a poor solution because all the Boost Build 'compile' rule does is to try to compile one or more source files. Success is if there are no compilations errors, while failure is if there is at least one compilation error. Why should one have to create an actual library to do this, especially as the source(s) may contain a main() function, and in fact usually does ?
I think the right solution, as suggested to me by someone else, is to create a CMake OBJECT library, which as I understand it is not really a library but just OBJECT files, and then run a test based on that. But of course I do not know if CMake supports that.
The thing is, static libraries are guaranteed to work and fairly standard, object libraries are a bit finicky and may not work depending on the generator. Using one or the other is merely an optimisation though, static libraries are just an archive of object files after all. Similar to the code above, you could implement them with this code:
add_library(test_name STATIC (or OBJECT) test_file.cpp) target_link_libraries(test_name PRIVATE boost_library) add_test(NAME test_name COMMAND cmake —build . —target test_name)
In this code, ctest will just build it and capture any compilation error. Easy enough. It doesn’t matter if there are functions called main or not since it’s not linked, the compiler won’t care for the most part about the name of the functions (well, main has some special semantics, but I doubt any test would rely on that). But now you are filling up an end-user's build output with a static library for each 'compile' tested. Ughhh ! That seems so silly to me, where just adding object files on success seems normal. Well actually it should be added with `EXCLUDE_FROM_ALL`:
add_library(test_name STATIC test_file.cpp EXCLUDE_FROM_ALL)
This way it only builds when it is invoked directly(which is when the test is ran), and not in the general build.
Indeed. Another way of doing it would also to use "add_subdirectory(test EXCLUDE_FROM_ALL)" to make sure everything is properly excluded too. It's a possibility some library owners forget them in some special hand crafted tests.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On 27/07/2017 23:53, Edward Diener via Boost wrote:
On 7/27/2017 5:15 PM, Florent Castelli via Boost wrote:
On 27 Jul 2017, at 22:29, Edward Diener via Boost <boost@lists.boost.org> wrote:
On 7/27/2017 12:58 PM, Florent Castelli via Boost wrote:
On Jul 27, 2017 16:41, "Edward Diener via Boost" <boost@lists.boost.org> wrote: On 7/27/2017 8:43 AM, Florent Castelli via Boost wrote:
On 26/07/2017 20:49, Edward Diener via Boost wrote:
Following John Maddock's appeal for practical solutions related to the move to CMake, I would like to know what the CMake equivalent is to the Boost Build unit test functionality.
In other words what do I write for CMake in order to do a Boost Build compile, compile-fail, link, link-fail, run, and run-fail unit tests ?
In my own Boost-CMake project, I have implemented regular "RUN" in a way that looks similar to the original Boost Build using functions: https://github.com/Orphis/boost-cmake/blob/master/libs/system.cmake
What is RUN supposed to be in your link above ? An equivalent of run from Boost Build. See https://github.com/boostorg/system/blob/develop/test/Jamfile.v2
Please show the code for RUN in CMake. To ease the transition from Boost Build to CMake we really need CMake equivalents to the unit testing rules of Boost Build at the very least.
Let’s start over. Basically, you end up with those equivalents. Run is compiling, linking an executable and running it, checking the return code is success. It could translate to:
add_executable(test_name test_file.cpp) target_link_libraries(test_name PRIVATE boost_library) add_test(NAME test_name COMMAND test_name)
But, if you do that, you need 2 steps to run the test. First you compile the program (using regular "make test_name" or even “cmake —build . —target test_name”) and then you run your test through ctest “ctest . -R test_name” (or you run all the tests without the filter). The problem there is you have 2 commands, and ctest won’t capture the compilation or linking errors. To work around that, you can use this instead:
add_test(NAME test_name COMMAND cmake —build . —target test_name && $<TARGET_FILE:test_name>)
This will for each test build it with the proper build tool (make, ninja, msbuild, xcodebuild…) and then run the program. The whole output will be captured by ctest and can be transfered to any test dashboard, you won’t have to look for errors in 2 places. The big issue I talk about later if that if you asked ctest to run several tests in parallel, it will have several “cmake —build . —target …” in parallel for those tests, which is not supported by some tools (mainly Ninja).
Understood.
Other commands should be easy enough to implement as well, but I haven't
had the interest in doing that in my project yet (which predates the SC decision).
It is possible to do "compile" by creating a regular static library target with that file and then invoking "cmake --build . --target <compile test name>". Compile failures can be done similarly and check they fail. Link and Link-fail can be done the same way using actual binaries that are built, but not run.
This is a poor solution. Could you please explain why? Using Cmake directly to build the sources with the same flags as set in the toolchain is not controversial I'd say. It's actually working better than other solutions I've seen trying to reproduce the features.
I said it was a poor solution because all the Boost Build 'compile' rule does is to try to compile one or more source files. Success is if there are no compilations errors, while failure is if there is at least one compilation error. Why should one have to create an actual library to do this, especially as the source(s) may contain a main() function, and in fact usually does ?
I think the right solution, as suggested to me by someone else, is to create a CMake OBJECT library, which as I understand it is not really a library but just OBJECT files, and then run a test based on that. But of course I do not know if CMake supports that.
The thing is, static libraries are guaranteed to work and fairly standard, object libraries are a bit finicky and may not work depending on the generator. Using one or the other is merely an optimisation though, static libraries are just an archive of object files after all. Similar to the code above, you could implement them with this code:
add_library(test_name STATIC (or OBJECT) test_file.cpp) target_link_libraries(test_name PRIVATE boost_library) add_test(NAME test_name COMMAND cmake —build . —target test_name)
In this code, ctest will just build it and capture any compilation error. Easy enough. It doesn’t matter if there are functions called main or not since it’s not linked, the compiler won’t care for the most part about the name of the functions (well, main has some special semantics, but I doubt any test would rely on that).
But now you are filling up an end-user's build output with a static library for each 'compile' tested. Ughhh ! That seems so silly to me, where just adding object files on success seems normal.
I understand that, but some IDEs may have issues with those. I remember having issues with Xcode a while ago. Requires more testing I would say.
For completeness, here’s a possible implementation of “LINK”. I understand this one links the source files into a binary, but doesn’t run it. This is simple CMake code as well:
add_executable(test_name test_file.cpp) add_test(NAME test_name COMMAND cmake —build . —target test_name)
In this case, CMake will build and link the files from the ctest invocation.
To support the failing variants for all the above, you need this set_test_properties(test_name PROPERTIES WILL_FAIL TRUE). Note that if you expect a test to fail running, but still succeed linking, that’s not directly supported by the code above and some changes need to be made.
Overall, that’s a lot of boilerplate for each test, which is why you need some convenience functions to make the build files clean and simple, with a clear intent. In complex test cases, it will certainly be possible to have people using CMake commands directly, but that should be rare.
Understood. Thanks !
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

AMDG On 07/27/2017 06:43 AM, Florent Castelli via Boost wrote:
It is possible to do "compile" by creating a regular static library target with that file and then invoking "cmake --build . --target <compile test name>". Compile failures can be done similarly and check they fail. Link and Link-fail can be done the same way using actual binaries that are built, but not run.
<snip>
The problem is that some build tools will not support parallel invocations (Ninja doesn't for example, see https://groups.google.com/forum/#!topic/ninja-build/4VP7whvWSH8 ) and thus running tests would be need to be linearized or moved to make, which should support that scenario better.
Whoa. I always thought that this recursive CMake invocation was a bit odd, but you're saying that it's basically a hack that isn't really supported. Is there any solution to this? In Christ, Steven Watanabe
participants (14)
-
Deniz Bahadir
-
Edward Diener
-
Florent Castelli
-
Gavin Lambert
-
John Maddock
-
Niall Douglas
-
paul
-
Paul A. Bristow
-
Peter Dimov
-
Robert Ramey
-
Roger Leigh
-
Roland Bock
-
Steven Watanabe
-
Vinnie Falco