Analyze the incidents based on where they are distributed (such as functional area, quality risk, assigned tester, and
assigned developer).
Look for patterns in the distribution (such as functional areas that appear to have above average defects count). Also
look for both developers and testers that may be overworked, and where their quality of work is slipping.
To evaluate test execution coverage, you need to review the Test Logs and determine:
-
The ratio between how many tests (Test Scripts or Test Cases) have been performed in this Test Cycle and a total
number of tests for all intended Target Test Items.
-
The ratio of successfully performed test cases.
The objective is to ensure that a sufficient number of the tests targeted for this Test Cycle have been executed
usefully. If this is not possible, or to augment that execution data, one or more additional test coverage criteria can
be identified, based upon:
-
Quality risk or priority
-
Specification-based coverage (Requirements and so on)
-
Business need or priority
-
Code-based coverage
See Concept: Key Measures of Test.
Record and present the Test Results in a Test Evaluation Report for this Test Cycle.
To analyze defects, you need to review and analyze the measures chosen as part of your defect analysis strategy. The
most common defect measures used include the following different measures (often displayed in the form of a graph):
-
Defect Density: The number of defects are shown as a function of one or two defect attributes (such as
distribution over functional area or quality risk compared to status or severity)
-
Defect Trend: The defect count is shown as a function over time
-
Defect Aging: A special defect density report in which the defect counts are shown as a function of the
length of time a defect remained in a given status (open, new, waiting-for-verification, and so on)
Compare the measures from this Test Cycle to the running totals for the current Iteration, and those from the analysis
of previous iterations, to better understand the emerging trends over time.
It is recommended that you present the results in diagram form, with supporting findings on request.
|