On Wed, Jul 11, 2018 at 7:34 AM James E. King III <jking@apache.org> wrote:
General Code Coverage Tip (for everyone):
Measure your coverage using branches, not lines. Branch coverage tells you how well error cases are tested and therefore a far better measure of quality than line-based. I have seen over 90% line coverage but the branch coverage is more like 60% to 70% on average over the boost CMT-managed repositories.
Below is some proof of what I mentioned about line coverage being (quite) deceptive - Boost.Rational has 98.8% "line" coverage but only 59.7% branch coverage. Line coverage reported by lcov in this case means the line was hit in some way. It doesn't mean every branch on that line was exercised for true and false cases to ensure proper behavior. Good unit tests will exercise all the branch cases and this typically proves that error handling code works as intended. In short, don't rely on line coverage for your quality measurements. Always rely on branch coverage. Extracting /home/travis/build/boostorg/boost-root/boost/rational.hpp Extracted 1 files Writing data to coverage.info Summary coverage rate: lines......: 98.9% (281 of 284 lines) functions..: 97.5% (424 of 435 functions) branches...: 59.7% (964 of 1615 branches) +lcov --gcov-tool=gcov-7 --rc lcov_branch_coverage=1 --list coverage.info Reading tracefile coverage.info |Lines |Functions |Branches Filename |Rate Num|Rate Num|Rate Num ======================================================== [/home/travis/build/boostorg/boost-root/boost/] rational.hpp |98.9% 284|97.5% 435|59.7% 1615 ======================================================== Total:|98.9% 284|97.5% 435|59.7% 1615 - Jim