Test Coverage of Requirements
This metric tracks the percentage of requirements that are covered by tests, and the percentage that have no associated tests.
Main Description

Overview

This metric exposes requirements that do not have test cases. The ultimate goal of this metric is to ensure that all requirements can be validated and that they work as intended.

Measurement Method

  • Number of test cases per requirement
  • Number of requirements with no associated test cases

Measurement Analysis

Software that implements requirements with inadequate test coverage may not work as intended or may have defects. The team should expect 100% coverage of requirements. Verification of coverage requires traceability between requirements and test artifacts.

Tracing requirements to test cases can yield a 10% to 30% defect reduction over several years. Not all requirements will have the same number of test cases or amount of test effort applied.

This pie chart shows the percentage of requirements that have and do not have associated test cases.

Test Coverage of Requirements

Metric Pitfalls

  • The metric shows 100% and the team believes that test coverage is good.
    • The quality of the tests themselves may be bad.
    • Most requirements need more than one test case.
    • The requirements may be wrong, so 100% coverage of them is a waste of time.
    • 100% test coverage is the minimum set of tests.
  • The team is trying to get 100% coverage, despite knowing they have bad requirements.

Countermeasures for this Metric

The following items are indicators of measurement pitfalls and should be used to corroborate this metric:

  • Defects rejected by the development team with a resolution of “works as intended”
  • Defects found after the iteration in which the requirement was tested
  • Change requests vs. requirements
  • Acceptance test failure rate, despite 100% coverage

More Information