Purpose
Teams monitor this metric throughout the lifecycle in order to understand their level of developer testing. Defects
caught during the development cycle are less costly to fix than those identified later.
The Ratio of Test Code to System Code metric measures the amount of test code relative to system code in terms of the
number of methods. Constant developer testing should result in at least one test case for each method, and almost
always more than one. As a result, this metric should yield a high value (always more than one, or >1), even in the
early stages of the development lifecycle.
Definition
Ratio of Test Code to System Code = Total Number of Test Methods / Total Number of System Methods
Analysis
The number of test methods always continues to grow if developer testing is performed while code is written (rather
than waiting to test during integration). The number of test methods should be at least two times the number of system
methods, because for each system method developed, there should be at least a positive test case and a negative test
case. A higher number for this metric indicates a higher level of developer testing.
The following are example scenarios that you can observe using this metric.
Expected trend - The ratio of test cases to methods should be more than one(1), and should increase
over time. In the early stages of development, developers start writing a test case, write code for the particular test
case, run the test, find errors, and fix the code. At this time, you should expect the number of test methods to be
about the same as the number of system methods. The number of system methods should not be greater than the number of
tests (which means that the result of this metric is always greater than one, or >1).
Later in the development lifecycle, the test methods and system methods continue to grow. There are more test cases
written for each system method. At this stage, you would presume that the number of test methods is at least two to
three times larger than the number of system methods.
Coding without writing a test - At any point during the development lifecycle, if the ratio of test
code to system code is less than one (<1), this is a strong indicator that the code has been written without writing
the test cases.
At any point during the development lifecycle, if the ratio of test code to system code drops dramatically from the
last assessment (for example, from five times to two times), it is a sign that source code has been added without new
test cases. The team should investigate ways to improve the ratio of tests to methods for the next assessment.
Frequency and reporting
Teams should review their ratio of test code to system code at the end of every iteration to help identify trends.
Collection and reporting tools
In order to collect the required data, the team must implement an annotation or packaging system to identify code as
either "test" or "system" code.
Pitfalls, advice, and countermeasures
-
If the source code is not growing as fast as test code, it may be due to refactoring. New and changed code can then
help you determine whether the team is testing continuously. If the new and changed code remains relatively
constant, the team is probably refactoring the code.
|