Overview
Test Coverage metrics measure the degree to which source code has been tested. There are several coverage metrics to
analyze test coverage. For example, statement coverage measures the percentage of source line of code (SLOC) that have
been exercised by a test suite. Function coverage measures the percentage of functions in the source code that
have been tested. Condition coverage measures the percentage of the evaluation points that have been executed.
Note: High percentage of test coverage is only an indicator of robust developer testing. The team
could use a variety of different developer testing techniques (test-first, traditional unit testing, white-box testing,
etc) to achieve high test coverage.
Measurement Method
We recommend statement coverage and condition coverage, as function coverage is too coarse a metric.
Statement Coverage (SC)
SC = S-RUN / S-TOTAL* 100
Count: S-RUN = Source lines of code executed by the test suite for a unit of code
S-TOTAL = Total source lines of code in the unit
Condition Coverage (CC)
CC = C-RUN / C-TOTAL * 100
Count: C-RUN = Number of evaluation points executed by the test suite for a unit of code
C-TOTAL = Total evaluation points in the unit
The following example has possible 4 evaluation points. If a test that only has x >= 100 as true, this will result
in 25% condition coverage.
if (x >= 100) or (y >=100) then
print("Do something")
else
print("Do something else");
This kind of coverage helps identify:
-
Defects in function calls that occur as part of a conditional expression
-
Errors in the conditional expressions themselves.
Measurement Analysis
Strategies such as test-first or rigorous white-box testing yields very high test coverage percentages
because the developers write the tests immediately before or after the code is written. If you discover a low
percentage of statement coverage, the team is not doing rigorous developer testing.
Condition Coverage is an additional metric to ensure that the tests are sufficiently covering the code.
Metrics Pitfalls
-
Test coverage is not a guarantee of adequate testing. The objective of these metrics is only to assure that code
has been executed. The person developing and running the tests also needs to ensure that the results are correct
and that the code meets the requirements.
Additional Reading
|