Guideline: Developing a Performance Test Plan
Relationships
Main Description

Introduction

One of the biggest misconceptions of performance testing is that you can create a high-load performance test specification and just start it running, capture the measurements, and then you have successfully completed your performance test.  Assuming perfect system design and with some additional scrutiny of the test results, this is an idealized result.  However, in practice, this never happens.  This is like suggesting that once a program is written and compiled, it may be shipped since it obviously does not need to be tested as it was written without any product defects.

Create Scheduled Testing Plans

You should create a conservatively scheduled testing plan taking into account the needs for analyzing and tuning major subsystems and newly delivered components that are part of the application environment on the system under test (SUT).  Plan a heavy load test for each of these new subsystems to ensure that they are properly balanced with the remainder of the SUT.  This may include tuning operating system or server parameters, running multiple instances of a web server, J2EE application server, or even changing the connection style to a back-end database server.  Ensuring that front-end load balancing equipment is working properly and is not circumvented by the load testing driver systems is also important.

Plan for Load Testing Sizing

In addition, you must size and deploy an appropriate complex of load testing driver systems and networking capacity so that the system under test can be fully loaded.  Your driver systems must not end up to be the bottleneck preventing further scaling up of the work load.  One simplification that can help guarantee this is using multiple exact duplicates of the driver system and testing the capacity of a single driver system and then de-rating its capacity for the actual performance tests by 10-15% to ensure ample driver capacity.

Once the basic tuning and capacity testing have been done and you are convinced that the test environment is ready for a full load test, you can begin by testing at 10-20% of full load, then 50% of full load, and finally at 100% of full load.  Making 2 or 3 100% full load tests may be important if there are any possibilities of unanticipated system response time variation due to an ill-defined initial system state or a variable test parameter such as network bandwidth availability that might be beyond your control during the test.  Make sure you maintain the 100% load point for a long enough period to ensure that your measurements are taken during a steady state period of system activity.

If overload or excess capacity needs to be tested, schedule a test run that begins as at 100% of full load and then ramps up 5-10% of additional load at a time until the measured responses or verification points begin failing to pass acceptable level.  This is your basic system capacity load level.