Test Coverage
Test coverage will usually be higher when robust developer testing is being practiced.
Main Description

Overview

Test Coverage metrics measure the degree to which source code has been tested. There are several coverage metrics to analyze test coverage. For example, statement coverage measures the percentage of source line of code (SLOC) that have been exercised by a test suite. Function coverage measures the percentage of functions in the source code that have been tested. Condition coverage measures the percentage of the evaluation points that have been executed.

Note: High percentage of test coverage is only an indicator of robust developer testing. The team could use a variety of different developer testing techniques (test-first, traditional unit testing, white-box testing, etc) to achieve high test coverage.

Measurement Method

We recommend statement coverage and condition coverage, as function coverage is too coarse a metric.

Statement Coverage (SC)

SC = S-RUN / S-TOTAL* 100

Count: S-RUN = Source lines of code executed by the test suite for a unit of code
           S-TOTAL = Total source lines of code in the unit

Condition Coverage (CC)

CC = C-RUN / C-TOTAL * 100

Count: C-RUN = Number of evaluation points executed by the test suite for a unit of code
           C-TOTAL = Total evaluation points in the unit

The following example has possible 4 evaluation points. If a test that only has x >= 100 as true, this will result in 25% condition coverage.

if (x >= 100) or (y >=100) then 
     print("Do something")
else
     print("Do something else");

This kind of coverage helps identify:

  • Defects in function calls that occur as part of a conditional expression
  • Errors in the conditional expressions themselves.

Measurement Analysis

Strategies such as test-first or rigorous white-box testing yields very high test coverage percentages because the developers write the tests immediately before or after the code is written. If you discover a low percentage of statement coverage, the team is not doing rigorous developer testing.

Condition Coverage is an additional metric to ensure that the tests are sufficiently covering the code.

Metrics Pitfalls

  • Test coverage is not a guarantee of adequate testing. The objective of these metrics is only to assure that code has been executed. The person developing and running the tests also needs to ensure that the results are correct and that the code meets the requirements.

Additional Reading