Defect Density at Customer Ship
This guideline describes how to measure and use defect density at the time of customer shipment.
Main Description

Overview

Indicates the code quality attained in the deliverable, based on the number of defects found, fixed, and remaining (not yet fixed). You can see how many defects remain (customers may, of course, encounter these defects). When comparing data over multiple releases, you should be able to draw conclusions about how well you have managed to identify defects.

Measurement Method

Defect Density = Number of total defects found, closed (fixed), and remaining. It can be useful to view defects grouped by:

  • Component, to determine whether any particular component has quality issues (this can be called defect density by component or defect distribution by component)
  • Requirement, to determine whether any particular requirement has quality issues (this can be called defect density by requirement or defect distribution by requirement)

Average Defect Density = Number of total defects found, closed (fixed), and remaining / unit -

In this measurement method, unit can be source line of code or SLOC (total, and new or changed), FunctionPoint, use case, or whatever else you measure. It can also be useful to group by component or requirement, and use in conjunction with defect density (mentioned previously).

Count

  • Total defects found, closed, and remaining.
  • Unit: SLOC, FunctionPoint, use case, or whatever else you measure.

Measurement Analysis

You can use a bar, stacked bar, or grouped (stacked) bar chart to show the number of defects in each category, depending on whether you are viewing by component or requirement, by severity, or simply totals. To show trends over time, you can use a series of bars, or a line chart.

During development, you expect to see the total defect count increase in relation to your testing activities. While you want to see the number closed increasing also, there may still be a large number of defects remaining (a defect is not closed until the fix has been verified).

When you ready the deliverable for shipment, you typically want to see a low number of remaining defects, and few or no remaining high-severity defects. Set targets for an acceptable level of remaining defects (usually a percentage of the total, with perhaps zero outstanding high-severity defects).

A high number of total defects is not necessarily bad, as long as your total fixed defects is acceptable. You can set targets for these numbers based on historical data. For example, this could be your historical defect rate per SLOC (or function point, or whatever you are measuring) on previous similar projects, or within the organization.

Defect density should decline over time: 4-6% annually over the long term. Defect density and trending should be used to estimate maintenance effort on a project. As new code is implemented, the defect density should track the existing trends. Once the SLOCs are known, you can estimate the number of defects and multiply by the average repair time to estimate the effort required to implement the maintenance operations. Typical defect densities at customer ship are 0.8 to 1.0 defects for new code, and decline rapidly in the first year to 0.08 to 0.1 defects per 1,000 SLOC for existing code. This decline is also an indication of the usage of the product.

Use this measure in conjunction with the Defect Trends

The following figure is an example of a defect density report.

 defect density

More Information