One of the most critical factors in the success of a performance test is goal setting. As the saying goes, if you are
not aiming at anything in particular, then you will surely hit it. Performance tests tend to have many stakeholders
that may not be obvious at the beginning of the project. You should ask the direct recipient of the project results who
else will be reviewing and approving actions based on the results of the performance test. Enumerate these additional
stakeholders and make sure their expectations are taken into account in the list of test goals. There has been more
than one performance test project that was restarted or have results rejected because it did not meet the expectations
of someone not involved in the formulation of the original project goals.
There are some fairly standard goals for performance tests that you can use to start from to elicit a solid direction
for the performance testing project. Often several of these are the expected result by various stakeholders. You have
to explicitly state the goals that you are working towards as a way to set both positive expectations of what goals you
are planning to achieve as well as setting negative expectations of the goals you are not attempting to meet.
Examples of performance test goals include:
-
Stress testing the application to prove robustness
-
Measure the overall system capacity in simultaneous user or transaction rate loading
-
Reliability testing of a given work load over a period of days or weeks to gauge an approximate availability metric
or a predicted mean time to failure
-
Response time measurement of critical transactions or user actions during a projection of the production maximum
work load
-
Overload testing to characterize the system's responsiveness as the workload exceeds the system's capacity
-
Load range testing with incremental increases in load to show the system's sensitivity to changing load
-
Tuning, balancing, and optimizing the subsystems and ultimately the complete system to provide maximum balanced
system capacity for the lowest unit cost per user or transaction
By writing down the performance testing goals (usually as part of the workload specification) and reviewing them with
the stakeholders, you can begin your testing project with confidence that you can complete the project with a
satisfactory set of results and conclusions. Without this agreement, the pressure and scrutiny of high profile
stakeholders can make the project difficult if not impossible to complete successfully.
Performance testing is a very high profile and stressful activity as the decision making based on it can have major
business implications. By having everyone in agreement before the test results are reported, there will be less
opportunity to second guess the testing goals, approach, work load definition, or chosen metrics.
|