Requirement Defect Count
This guideline describes how to measure and use the Requirement Defect Count metric.
Main Description

Overview

The Requirement Defect Count shows the number of defects that have been attributed to the quality of the requirements. This may be due to missing, incorrect, ambiguous, or extraneous requirements. These defects may be found at any phase of the project lifecycle, and in the performance of any discipline, but you will most likely find them during testing.

This measure reflects a form of root cause analysis of defects, attributing defects with one or more attributes to perform analysis and modeling.

There is a range of possible methods for identifying and analyzing this measure. Really simply, this can be measured with a single attribute on defects, which attribute identifies the root cause. With this approach, the possible values could be as simple as: requirements, design, code, or test.

For a more complex and thorough approach, consider Orthogonal Defect Classification, or ODC. In this approach, defects have eight orthogonal attributes that characterize the defects, from which a comprehensive analysis and modeling can be performed. The Impact and Target attributes can reflect upon the quality of the requirements. For more information, see Details of Orthogonal Defect Classification and Adopting ODC to improve software quality: A case study

Within the project lifecycle, it is most effective to find the defects attributed to a requirement as early as possible. The later defects are found, the more that has been invested in the system based upon missing, incorrect, ambiguous, or extraneous requirements.

Measurement Method

Count:

The Number of product defects attributed to requirements management.

Within a project lifecycle, a good way to monitor requirement defects is using a line chart that maps the following through the project lifecycle :

  • The cumulative number of requirement defects over time
  • The number of unresolved defects attributed to requirements over time
  • The number of new requirement defects identified over time   

Measurement Analysis

The cumulative measure will identify the magnitude of defects that are rooted in the requirements management. If a high percentage of defects are attributed to the requirements, then more skills development or increased formality may be necessary.

The unresolved defect chart will identify the responsiveness to making the necessary requirements additions or changes, and any subsequent design and coding changes (for instance, to resolve the defect). A highly responsive team will have a reduced number of unresolved requirement defects. However, the impact of defects due to requirements is usually greater as the lifecycle progresses.

Watch for requirement defects identified late in the project lifecycle. This is an indication that insufficient validation of the requirements is taking place early in the project lifecycle (before significant investment has been made in implementation and testing). It is important to include stakeholders and the project team in the validation activity. Again, more skills development or increased formality may be necessary.

In the latter half of the project lifecycle, it is more difficult to expediently resolve requirement defects, because the impact of change is greater because of the greater investment already made in design, code, and testing.

If you compare these measures across projects, it gives you an indication and feedback on which projects and project teams are more effective, and will identify where additional skills development and formality in their requirements management may be necessary.

If you use the ODC method described previously, the Activity and Trigger attributes will indicate in which activity the defect was found, and thus, how much investment was made in the requirement before it was discovered.