Planned Versus Actual Test Effort
This metric tracks estimated (planned) test effort against the actual test effort.
Main Description

Purpose

This metric helps the team understand whether the test effort (and associated cost) is on track with the planned test effort.

Definition

Calculate the estimated (planned) effort for all test activities and compare to actual effort recorded for all test activities.

Analysis

A good way to monitor test effort over time is to use a trend line. Plot test effort (typically in hours) on the Y axis, and iterations on the X axis.

Expected trend - Testing effort should meet or be less than plan, while meeting associated quality objectives.

Rising slope - A trend line that shows actual test effort exceeding the plan over time indicates a problem. There may be quality issues that need to be addressed, or perhaps the testing effort was underestimated.

Frequency and reporting

Variance in testing activities should be reviewed at the end of each iteration to identify trends.

Collection and reporting tools

Standard project management tools can be used to capture effort data. IBM® Rational® Quality Manager® captures and reports on this metric.

Assumptions and prerequisites

  • The team must clearly identify what constitutes test activities.
  • Estimates and actuals must be collected at the end of each iteration at a minimum

Pitfalls, advice, and countermeasures

  • Encourage honest reporting (i.e. no repercussion to reporting higher or lower effort)
  • Use quality metrics as a countermeasure to ensure that the testing efforts are sufficient when actuals consistently meet estimates