Guideline: Analyzing and Evaluating Test Results
This guideline provides more details on how to analyze and evaluate the test results.
Main Description

Incident distribution

Analyze the incidents based on where they are distributed (such as functional area, quality risk, assigned tester, and assigned developer).

Look for patterns in the distribution (such as functional areas that appear to have above average defects count). Also look for both developers and testers that may be overworked, and where their quality of work is slipping.

Test execution coverage

To evaluate test execution coverage, you need to review the Test Logs and determine:

  • The ratio between how many tests (Test Scripts or Test Cases) have been performed in this Test Cycle and a total number of tests for all intended Target Test Items.
  • The ratio of successfully performed test cases.

The objective is to ensure that a sufficient number of the tests targeted for this Test Cycle have been executed usefully. If this is not possible, or to augment that execution data, one or more additional test coverage criteria can be identified, based upon:

  • Quality risk or priority
  • Specification-based coverage (Requirements and so on)
  • Business need or priority
  • Code-based coverage

See Concept: Key Measures of Test.

Record and present the Test Results in a Test Evaluation Report for this Test Cycle.

Change Requests statistics

To analyze defects, you need to review and analyze the measures chosen as part of your defect analysis strategy. The most common defect measures used include the following different measures (often displayed in the form of a graph):

  • Defect Density: The number of defects are shown as a function of one or two defect attributes (such as distribution over functional area or quality risk compared to status or severity)
  • Defect Trend: The defect count is shown as a function over time
  • Defect Aging: A special defect density report in which the defect counts are shown as a function of the length of time a defect remained in a given status (open, new, waiting-for-verification, and so on)

Compare the measures from this Test Cycle to the running totals for the current Iteration, and those from the analysis of previous iterations, to better understand the emerging trends over time.

It is recommended that you present the results in diagram form, with supporting findings on request.