[Hana] Integration with the regression Matrix
Hi, With the goal of shipping Hana with the next release of Boost, I am currently trying to satisfy the requirement that Hana's testing results show up in the Boost regression matrix. One way to achieve this would be to have Travis CI upload the results on each push to master. To do so, I would need to somehow create an XML file compatible with what the regression matrix expects. That brings up a few questions: (1) Is there a way to upload the result of CMake tests to the Boost regression matrix? I suspect something like that might have already been done as part of the (unfortunately defunct) CMake effort? (2) If not, would it be difficult to implement? Any input on how to best integrate Hana's testing results with the rest of Boost's regression testing will be greatly appreciated. Also, before someone suggests using Boost.Build: I have been looking at the possibility of maintaining a separate Jamfile. However, the need to maintain two separate build systems is annoying, so I'd like to avoid it if possible. I also think that Travis-based testing could provide a better coverage for Hana than the voluntary testers given the need for cutting edge compilers, but I could be wrong. Regards, Louis -- View this message in context: http://boost.2283326.n4.nabble.com/Hana-Integration-with-the-regression-Matr... Sent from the Boost - Dev mailing list archive at Nabble.com.
On 10/9/2015 6:05 PM, Louis Dionne wrote:
Hi,
With the goal of shipping Hana with the next release of Boost, I am currently trying to satisfy the requirement that Hana's testing results show up in the Boost regression matrix. One way to achieve this would be to have Travis CI upload the results on each push to master.
To do so, I would need to somehow create an XML file compatible with what the regression matrix expects. That brings up a few questions:
(1) Is there a way to upload the result of CMake tests to the Boost regression matrix? I suspect something like that might have already been done as part of the (unfortunately defunct) CMake effort?
(2) If not, would it be difficult to implement?
Any input on how to best integrate Hana's testing results with the rest of Boost's regression testing will be greatly appreciated.
Also, before someone suggests using Boost.Build: I have been looking at the possibility of maintaining a separate Jamfile. However, the need to maintain two separate build systems is annoying, so I'd like to avoid it if possible. I also think that Travis-based testing could provide a better coverage for Hana than the voluntary testers given the need for cutting edge compilers, but I could be wrong.
Boost end-users wishing to use Hana, which may also mean other Boost library authors, will be expecting a jamfile and use of Boost Build to run tests.
On 10/9/15 3:05 PM, Louis Dionne wrote:
Hi,
With the goal of shipping Hana with the next release of Boost, I am currently trying to satisfy the requirement that Hana's testing results show up in the Boost regression matrix.
I have been looking at the possibility of maintaining a separate Jamfile. However, the need to maintain two separate build systems is annoying, so I'd like to avoid it if possible.
As far as I know, including a Jamfile so that boost test can run has always been a boost requirement.
One way to achieve this would be to have Travis CI upload the results on each push to master.
To do so, I would need to somehow create an XML file compatible with what the regression matrix expects. That brings up a few questions:
(1) Is there a way to upload the result of CMake tests to the Boost regression matrix? I suspect something like that might have already been done as part of the (unfortunately defunct) CMake effort?
(2) If not, would it be difficult to implement?
Any input on how to best integrate Hana's testing results with the rest of Boost's regression testing will be greatly appreciated.
I would also like information about that above. But its a totally separate project from Hana or any specific library. No way should you spend time on this until you've got testing of Boost Hana working the current system - this means making a Jamfile
I also think that Travis-based testing could provide a better coverage for Hana than the voluntary testers given the need for cutting edge compilers, but I could be wrong.
LOL - it doesn't matter whether you're right or wrong. Boost gives a lot of leeway to individual library authors. But there are some things we have to agree upon. Right now that means a unitified testing/build structure. This might change in the future, but it's one of the few things we have to agree on. So put your head down, hold your nose and head for daylight and get that Jamfile done. Robert Ramey
-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Robert Ramey Sent: 10 October 2015 01:07 To: boost@lists.boost.org Subject: Re: [boost] [Hana] Integration with the regression Matrix
On 10/9/15 3:05 PM, Louis Dionne wrote:
Hi,
With the goal of shipping Hana with the next release of Boost, I am currently trying to satisfy the requirement that Hana's testing results show up in the Boost regression matrix.
I have been looking at the possibility of maintaining a separate > Jamfile. However, the need to maintain two separate build systems is > annoying, so I'd like to avoid it if possible.
As far as I know, including a Jamfile so that boost test can run has always been a boost requirement.
+1 I hate bjam, but it's the devil we know( and love to hate). Just do it! (Or ask a friend). Paul --- Paul A. Bristow Prizet Farmhouse Kendal UK LA8 8AB +44 (0) 1539 561830
[Puts Testing Manager hat on..] On Fri, Oct 9, 2015 at 5:05 PM, Louis Dionne <ldionne.2@gmail.com> wrote:
Hi,
With the goal of shipping Hana with the next release of Boost, I am currently trying to satisfy the requirement that Hana's testing results show up in the Boost regression matrix. One way to achieve this would be to have Travis CI upload the results on each push to master.
To do so, I would need to somehow create an XML file compatible with what the regression matrix expects. That brings up a few questions:
(1) Is there a way to upload the result of CMake tests to the Boost regression matrix? I suspect something like that might have already been done as part of the (unfortunately defunct) CMake effort?
There is no way, currently. (2) If not, would it be difficult to implement?
It would be difficult. As it would require dealing with it at almost every level of the testing process. Any input on how to best integrate Hana's testing results with the rest of
Boost's regression testing will be greatly appreciated.
As others have mentioned. You need to support Boost Build for integration into the rest of Boost. And that's not just testing. But it could also mean documentation and release/distribution. Also, before someone suggests using Boost.Build: I have been looking at the
possibility of maintaining a separate Jamfile. However, the need to maintain two separate build systems is annoying, so I'd like to avoid it if possible.
Can't be avoided.. But, if it's any consolation, I'm fairly certain that the Jamfile is going to be considerably smaller than the cmakelists.txt you have now.
I also think that Travis-based testing could provide a better coverage for Hana than the voluntary testers given the need for cutting edge compilers, but I could be wrong.
As a user of Travis myself, I can say you are wrong. Travis support for C++ testing is minimal. And you have to go through extra lengths to get any cutting edge compilers. And Travis will never have what Boost testing some times has.. Pre-release proprietary compilers from vendors on non-Unix platforms. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail
On 10.10.2015 01:05, Louis Dionne wrote:
Also, before someone suggests using Boost.Build: I have been looking at the possibility of maintaining a separate Jamfile. However, the need to maintain two separate build systems is annoying, so I'd like to avoid it if possible.
Others have addressed the need for supporting Boost.Build. I can add that it is possible to create a Jamfile that will automatically iterate over the test sources and need not updating at all. You can find an example in Boost.Log: https://github.com/boostorg/log/blob/develop/test/Jamfile.v2
On 10/10/2015 09:55, Andrey Semashev wrote:
On 10.10.2015 01:05, Louis Dionne wrote:
Also, before someone suggests using Boost.Build: I have been looking at the possibility of maintaining a separate Jamfile. However, the need to maintain two separate build systems is annoying, so I'd like to avoid it if possible.
Others have addressed the need for supporting Boost.Build. I can add that it is possible to create a Jamfile that will automatically iterate over the test sources and need not updating at all. You can find an example in Boost.Log:
https://github.com/boostorg/log/blob/develop/test/Jamfile.v2
I was just about to say, that if the tests are uniformly named, then self-updating Jamfiles are easy enough. An even simpler example is in the type_traits lib: https://github.com/boostorg/type_traits/blob/master/test/Jamfile.v2 HTH, John.
John Maddock-2 wrote
On 10/10/2015 09:55, Andrey Semashev wrote:
[...]
Others have addressed the need for supporting Boost.Build. I can add that it is possible to create a Jamfile that will automatically iterate over the test sources and need not updating at all. You can find an example in Boost.Log:
https://github.com/boostorg/log/blob/develop/test/Jamfile.v2
I was just about to say, that if the tests are uniformly named, then self-updating Jamfiles are easy enough. An even simpler example is in the type_traits lib: https://github.com/boostorg/type_traits/blob/master/test/Jamfile.v2
HTH, John.
Thanks a lot guys, I'm gonna do just this! Regards, Louis -- View this message in context: http://boost.2283326.n4.nabble.com/Hana-Integration-with-the-regression-Matr... Sent from the Boost - Dev mailing list archive at Nabble.com.
On 10/10/15 1:55 AM, Andrey Semashev wrote:
On 10.10.2015 01:05, Louis Dionne wrote:
Also, before someone suggests using Boost.Build: I have been looking at the possibility of maintaining a separate Jamfile. However, the need to maintain two separate build systems is annoying, so I'd like to avoid it if possible.
Others have addressed the need for supporting Boost.Build. I can add that it is possible to create a Jamfile that will automatically iterate over the test sources and need not updating at all. You can find an example in Boost.Log:
https://github.com/boostorg/log/blob/develop/test/Jamfile.v2
Great, another cool idea to make jamfile even easier to figure out and debug! Robert Ramey
-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Robert Ramey Sent: 10 October 2015 15:16 To: boost@lists.boost.org Subject: Re: [boost] [Hana] Integration with the regression Matrix
On 10/10/15 1:55 AM, Andrey Semashev wrote:
On 10.10.2015 01:05, Louis Dionne wrote:
Also, before someone suggests using Boost.Build: I have been looking at the possibility of maintaining a separate Jamfile. However, the need to maintain two separate build systems is annoying, so I'd like to avoid it if possible.
Others have addressed the need for supporting Boost.Build. I can add that it is possible to create a Jamfile that will automatically iterate over the test sources and need not updating at all. You can find an example in Boost.Log:
https://github.com/boostorg/log/blob/develop/test/Jamfile.v2
Great, another cool idea to make jamfile even easier to figure out and debug!
Also sets a good example by documenting *why* warnings are disabled etc. Too many jamfiles are write-only ;-) Paul --- Paul A. Bristow Prizet Farmhouse Kendal UK LA8 8AB +44 (0) 1539 561830
participants (7)
-
Andrey Semashev -
Edward Diener -
John Maddock -
Louis Dionne -
Paul A. Bristow -
Rene Rivera -
Robert Ramey