Ratio of Test Code to System Code
This metric tracks the ratio of developer test code to system code.
Main Description

Purpose

Teams monitor this metric throughout the lifecycle in order to understand their level of developer testing. Defects caught during the development cycle are less costly to fix than those identified later.

The Ratio of Test Code to System Code metric measures the amount of test code relative to system code in terms of the number of methods. Constant developer testing should result in at least one test case for each method, and almost always more than one. As a result, this metric should yield a high value (always more than one, or >1), even in the early stages of the development lifecycle.

Definition

Ratio of Test Code to System Code = Total Number of Test Methods / Total Number of System Methods

Analysis

The number of test methods always continues to grow if developer testing is performed while code is written (rather than waiting to test during integration). The number of test methods should be at least two times the number of system methods, because for each system method developed, there should be at least a positive test case and a negative test case. A higher number for this metric indicates a higher level of developer testing.

The following are example scenarios that you can observe using this metric.

Expected trend - The ratio of test cases to methods should be more than one(1), and should increase over time. In the early stages of development, developers start writing a test case, write code for the particular test case, run the test, find errors, and fix the code. At this time, you should expect the number of test methods to be about the same as the number of system methods. The number of system methods should not be greater than the number of tests (which means that the result of this metric is always greater than one, or >1).

Later in the development lifecycle, the test methods and system methods continue to grow. There are more test cases written for each system method. At this stage, you would presume that the number of test methods is at least two to three times larger than the number of system methods.

Coding without writing a test - At any point during the development lifecycle, if the ratio of test code to system code is less than one (<1), this is a strong indicator that the code has been written without writing the test cases.

At any point during the development lifecycle, if the ratio of test code to system code drops dramatically from the last assessment (for example, from five times to two times), it is a sign that source code has been added without new test cases. The team should investigate ways to improve the ratio of tests to methods for the next assessment.

Frequency and reporting

Teams should review their ratio of test code to system code at the end of every iteration to help identify trends.


Collection and reporting tools

In order to collect the required data, the team must implement an annotation or packaging system to identify code as either "test" or "system" code.

Pitfalls, advice, and countermeasures

  • If the source code is not growing as fast as test code, it may be due to refactoring. New and changed code can then help you determine whether the team is testing continuously. If the new and changed code remains relatively constant, the team is probably refactoring the code.