Using static code checkers against the Boost code base
I don't know if there is a policy yet on using static code checkers on the Boost code base as part of the release cycle. Given that the Visual Studio 2012 Analyzer tool I'm using just picked up 3 issues in the 1.57 release (I've posted TRAC items on them already), I suspect not. I would like to encourage such a policy. Boost is, among other things, about quality. This is a way to enhance quality. For people like me who work in safety critical fields, it is vital. I cannot use Boost libraries if they can't be certified. Static analyzers can help insure quality, which makes it easier to qualify these tools. There are many tools available. Some, like CppCheck, are open source. Others are built into development environments (aforementioned VS Analyzer, Clang tools, etc.). Further, I suspect that tool vendors could be convinced it would be good PR to have their tools used by Boost, so I suspect even those with paid licenses can be made available for free. Steve Hickman System Architect, Flight Deck of the Future 480-236-8367
Or maybe help to get Coverity to scan Boost as part of the open source static analysis efforts (if they don't already). http://www.coverity.com/press-releases/coverity-scan-report-finds-open-sourc... SGL From: Boost-users [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Hickman, Steve (AdvTech) Sent: Thursday, December 04, 2014 10:31 AM To: boost-users@lists.boost.org Subject: [Boost-users] Using static code checkers against the Boost code base I don't know if there is a policy yet on using static code checkers on the Boost code base as part of the release cycle. Given that the Visual Studio 2012 Analyzer tool I'm using just picked up 3 issues in the 1.57 release (I've posted TRAC items on them already), I suspect not. I would like to encourage such a policy. Boost is, among other things, about quality. This is a way to enhance quality. For people like me who work in safety critical fields, it is vital. I cannot use Boost libraries if they can't be certified. Static analyzers can help insure quality, which makes it easier to qualify these tools. There are many tools available. Some, like CppCheck, are open source. Others are built into development environments (aforementioned VS Analyzer, Clang tools, etc.). Further, I suspect that tool vendors could be convinced it would be good PR to have their tools used by Boost, so I suspect even those with paid licenses can be made available for free. Steve Hickman System Architect, Flight Deck of the Future 480-236-8367
It appears that Coverity already provides the service free to Open Source projects. All that is required is to sign up for this at: https://scan.coverity.com/ --- Steve H. From: LeMay.Steve [mailto:Steve.Lemay@IGT.com] Sent: Thursday, December 04, 2014 10:56 AM To: boost-users@lists.boost.org Subject: Re: [Boost-users] Using static code checkers against the Boost code base Or maybe help to get Coverity to scan Boost as part of the open source static analysis efforts (if they don't already). http://www.coverity.com/press-releases/coverity-scan-report-finds-open-sourc... SGL From: Boost-users [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Hickman, Steve (AdvTech) Sent: Thursday, December 04, 2014 10:31 AM To: boost-users@lists.boost.org<mailto:boost-users@lists.boost.org> Subject: [Boost-users] Using static code checkers against the Boost code base I don't know if there is a policy yet on using static code checkers on the Boost code base as part of the release cycle. Given that the Visual Studio 2012 Analyzer tool I'm using just picked up 3 issues in the 1.57 release (I've posted TRAC items on them already), I suspect not. I would like to encourage such a policy. Boost is, among other things, about quality. This is a way to enhance quality. For people like me who work in safety critical fields, it is vital. I cannot use Boost libraries if they can't be certified. Static analyzers can help insure quality, which makes it easier to qualify these tools. There are many tools available. Some, like CppCheck, are open source. Others are built into development environments (aforementioned VS Analyzer, Clang tools, etc.). Further, I suspect that tool vendors could be convinced it would be good PR to have their tools used by Boost, so I suspect even those with paid licenses can be made available for free. Steve Hickman System Architect, Flight Deck of the Future 480-236-8367
[Please do not mail me a copy of your followup] "Hickman, Steve (AdvTech)" <Steve.Hickman@honeywell.com> spake the secret code <EA5676973A55CA4BA5D3F921E9FB1C1727837269@de08ex3003.global.ds.honeywell.com> thusly:
It appears that Coverity already provides the service free to Open Source projects. All that is required is to sign up for this at: https://scan.coverity.com/
Nice! I wrote up a blog post describing how I added cppcheck and clang static analyzer checking to my github repo: <http://legalizeadulthood.wordpress.com/2014/12/07/adding-static-analysis-to-your-c-github-repository/> I'll have to try out the coverity analysis, I'll write up a separate blog post for that. Unlike cppcheck/clang there is a limit to the number of builds a free project can do, so the setup is going to be a little more involved. -- "The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline> The Computer Graphics Museum <http://computergraphicsmuseum.org> The Terminals Wiki <http://terminals.classiccmp.org> Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>
On 9 Dec 2014 at 4:10, Richard wrote:
It appears that Coverity already provides the service free to Open Source projects. All that is required is to sign up for this at: https://scan.coverity.com/
I was given some time to implement automated nightly testing for Boost.Thread which is now integrated and appears as a dashboard on https://github.com/boostorg/thread/tree/develop. I'll mux in the clang and MSVC static analysers later today. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
It also appears that CppDepend is available for free to open source projects. See: http://www.cppdepend.com/CppDependForOSS.aspx --- Steve H. From: Hickman, Steve (AdvTech) [mailto:Steve.Hickman@honeywell.com] Sent: Sunday, December 07, 2014 5:02 AM To: boost-users@lists.boost.org Subject: Re: [Boost-users] Using static code checkers against the Boost code base It appears that Coverity already provides the service free to Open Source projects. All that is required is to sign up for this at: https://scan.coverity.com/ --- Steve H. From: LeMay.Steve [mailto:Steve.Lemay@IGT.com] Sent: Thursday, December 04, 2014 10:56 AM To: boost-users@lists.boost.org<mailto:boost-users@lists.boost.org> Subject: Re: [Boost-users] Using static code checkers against the Boost code base Or maybe help to get Coverity to scan Boost as part of the open source static analysis efforts (if they don't already). http://www.coverity.com/press-releases/coverity-scan-report-finds-open-sourc... SGL From: Boost-users [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Hickman, Steve (AdvTech) Sent: Thursday, December 04, 2014 10:31 AM To: boost-users@lists.boost.org<mailto:boost-users@lists.boost.org> Subject: [Boost-users] Using static code checkers against the Boost code base I don't know if there is a policy yet on using static code checkers on the Boost code base as part of the release cycle. Given that the Visual Studio 2012 Analyzer tool I'm using just picked up 3 issues in the 1.57 release (I've posted TRAC items on them already), I suspect not. I would like to encourage such a policy. Boost is, among other things, about quality. This is a way to enhance quality. For people like me who work in safety critical fields, it is vital. I cannot use Boost libraries if they can't be certified. Static analyzers can help insure quality, which makes it easier to qualify these tools. There are many tools available. Some, like CppCheck, are open source. Others are built into development environments (aforementioned VS Analyzer, Clang tools, etc.). Further, I suspect that tool vendors could be convinced it would be good PR to have their tools used by Boost, so I suspect even those with paid licenses can be made available for free. Steve Hickman System Architect, Flight Deck of the Future 480-236-8367
On 4 Dec 2014 at 18:30, Hickman, Steve (AdvTech) wrote:
I don't know if there is a policy yet on using static code checkers on the Boost code base as part of the release cycle. Given that the Visual Studio 2012 Analyzer tool I'm using just picked up 3 issues in the 1.57 release (I've posted TRAC items on them already), I suspect not.
The policy is that this is up to each library maintainer. Some do, some don't. Of those that do, coverage is usually fairly restricted to one or two analysers.
I would like to encourage such a policy. Boost is, among other things, about quality. This is a way to enhance quality. For people like me who work in safety critical fields, it is vital. I cannot use Boost libraries if they can't be certified. Static analyzers can help insure quality, which makes it easier to qualify these tools.
There are many tools available. Some, like CppCheck, are open source. Others are built into development environments (aforementioned VS Analyzer, Clang tools, etc.). Further, I suspect that tool vendors could be convinced it would be good PR to have their tools used by Boost, so I suspect even those with paid licenses can be made available for free.
I think you would be surprised at how unfree licences are for free software. Setting this stuff up is not free, including renting the CI testing resources. As I'm on the Boost.Thread maintainence team, I hereby solicit any funding you or anyone else can provide to improve the static testing of Boost.Thread, and to be specific: 1. The renting of a dedicated server for a Jenkins installation on an ongoing basis. 2. The licencing of the installations of Microsoft Windows required and any static testing tools required. 3. The hourly rate, at approximately $150/hour, of someone qualified in CI config to set all this up for Boost.Thread. I should estimate 160 - 200 hours might do it. And then their hourly rate on an ongoing basis to maintain it e.g. security patches and updates. If you or anyone else can supply any or all of this, we on the Boost.Thread team are very interested to hear from you. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
[Please do not mail me a copy of your followup] "Niall Douglas" <s_sourceforge@nedprod.com> spake the secret code <5481B226.18060.2F9F2BC2@s_sourceforge.nedprod.com> thusly:
Setting this stuff up is not free, including renting the CI testing resources.
I've setup cppcheck to run on travis-ci.org from my github repo. That approach is completely free. -- "The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline> The Computer Graphics Museum <http://computergraphicsmuseum.org> The Terminals Wiki <http://terminals.classiccmp.org> Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>
[Please do not mail me a copy of your followup] legalize+jeeves@mail.xmission.com (Richard) spake the secret code <m5st0d$2mj$1@ger.gmane.org> thusly:
[Please do not mail me a copy of your followup]
"Niall Douglas" <s_sourceforge@nedprod.com> spake the secret code <5481B226.18060.2F9F2BC2@s_sourceforge.nedprod.com> thusly:
Setting this stuff up is not free, including renting the CI testing resources.
I've setup cppcheck to run on travis-ci.org from my github repo. That approach is completely free.
travis-ci.org also supports the clang toolchain, but unfortunately the static analyzer (scan-build) isn't installed. You'd have to build it from source and provide your own package (PITA). I filed a bug on clang saying they should provide this in some binary package. Then it would be easy to get clang static analysis and cppcheck from travis-ci.org If anyone knows of an apt-get repository that has a clang static analyzer package, please let me know. -- "The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline> The Computer Graphics Museum <http://computergraphicsmuseum.org> The Terminals Wiki <http://terminals.classiccmp.org> Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>
[Please do not mail me a copy of your followup] legalize+jeeves@mail.xmission.com (Richard) spake the secret code <m5t4rh$av4$1@ger.gmane.org> thusly:
If anyone knows of an apt-get repository that has a clang static analyzer package, please let me know.
Looks like scan-build is installed on the linux images on travis-ci.org, but for some reason /usr/bin wasn't in the path. So there's two free static analysis tools you can easily hook into continuous integration builds on your boost libraries. -- "The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline> The Computer Graphics Museum <http://computergraphicsmuseum.org> The Terminals Wiki <http://terminals.classiccmp.org> Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>
On 5 Dec 2014 at 18:19, Richard wrote:
Setting this stuff up is not free, including renting the CI testing resources.
I've setup cppcheck to run on travis-ci.org from my github repo. That approach is completely free.
Travis is okay for Linux only toy CI. It is better than nothing, but it's very toy - no results recording, no soak testing, no scheduled testing, no automated fork reconciliation, test and pushing etc etc. It doesn't even approach remotely a proper CI setup. Even my Jenkins CI dashboard for AFIO (https://boostgsoc13.github.io/boost.afio/) isn't a patch on a professional job, and I have four separate analyser passes in there. What I really need in there is historical performance regression analysis and bisection soak testing, but I just don't have the time. Also, with Boost.Thread at least, a large chunk of the unit test suite would need upgrading to output results which can be consumed by a CI. Right now a large chunk doesn't even use Boost.Test, and the unit test suite is woefully incomplete from what it should be. And that's just one small Boost library. Good test engineers get paid more than good developers in recent years. I agree with that market assessment. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
participants (4)
-
Hickman, Steve (AdvTech) -
legalize+jeeves＠mail.xmission.com -
LeMay.Steve -
Niall Douglas