Guideline: Performance Test Goals
This guideline describes what are the main performance test goals.
Relationships
Main Description

One of the most critical factors in the success of a performance test is goal setting. As the saying goes, if you are not aiming at anything in particular, then you will surely hit it. Performance tests tend to have many stakeholders that may not be obvious at the beginning of the project. You should ask the direct recipient of the project results who else will be reviewing and approving actions based on the results of the performance test. Enumerate these additional stakeholders and make sure their expectations are taken into account in the list of test goals. There has been more than one performance test project that was restarted or have results rejected because it did not meet the expectations of someone not involved in the formulation of the original project goals.

There are some fairly standard goals for performance tests that you can use to start from to elicit a solid direction for the performance testing project. Often several of these are the expected result by various stakeholders. You have to explicitly state the goals that you are working towards as a way to set both positive expectations of what goals you are planning to achieve as well as setting negative expectations of the goals you are not attempting to meet.

Examples of performance test goals include:

  • Stress testing the application to prove robustness
  • Measure the overall system capacity in simultaneous user or transaction rate loading
  • Reliability testing of a given work load over a period of days or weeks to gauge an approximate availability metric or a predicted mean time to failure
  • Response time measurement of critical transactions or user actions during a projection of the production maximum work load
  • Overload testing to characterize the system's responsiveness as the workload exceeds the system's capacity
  • Load range testing with incremental increases in load to show the system's sensitivity to changing load
  • Tuning, balancing, and optimizing the subsystems and ultimately the complete system to provide maximum balanced system capacity for the lowest unit cost per user or transaction

By writing down the performance testing goals (usually as part of the workload specification) and reviewing them with the stakeholders, you can begin your testing project with confidence that you can complete the project with a satisfactory set of results and conclusions. Without this agreement, the pressure and scrutiny of high profile stakeholders can make the project difficult if not impossible to complete successfully.

Performance testing is a very high profile and stressful activity as the decision making based on it can have major business implications. By having everyone in agreement before the test results are reported, there will be less opportunity to second guess the testing goals, approach, work load definition, or chosen metrics.