Introduction
One of the biggest misconceptions of performance testing is that you can create a high-load performance test
specification and just start it running, capture the measurements, and then you have successfully completed your
performance test. Assuming perfect system design and with some additional scrutiny of the test results, this is
an idealized result. However, in practice, this never happens. This is like suggesting that once a program
is written and compiled, it may be shipped since it obviously does not need to be tested as it was written without any
product defects.
Create Scheduled Testing Plans
You should create a conservatively scheduled testing plan taking into account the needs for analyzing and tuning major
subsystems and newly delivered components that are part of the application environment on the system under test
(SUT). Plan a heavy load test for each of these new subsystems to ensure that they are properly balanced with the
remainder of the SUT. This may include tuning operating system or server parameters, running multiple instances
of a web server, J2EE application server, or even changing the connection style to a back-end database server.
Ensuring that front-end load balancing equipment is working properly and is not circumvented by the load testing driver
systems is also important.
Plan for Load Testing Sizing
In addition, you must size and deploy an appropriate complex of load testing driver systems and networking capacity so
that the system under test can be fully loaded. Your driver systems must not end up to be the bottleneck
preventing further scaling up of the work load. One simplification that can help guarantee this is using multiple
exact duplicates of the driver system and testing the capacity of a single driver system and then de-rating its
capacity for the actual performance tests by 10-15% to ensure ample driver capacity.
Once the basic tuning and capacity testing have been done and you are convinced that the test environment is ready for
a full load test, you can begin by testing at 10-20% of full load, then 50% of full load, and finally at 100% of full
load. Making 2 or 3 100% full load tests may be important if there are any possibilities of unanticipated system
response time variation due to an ill-defined initial system state or a variable test parameter such as network
bandwidth availability that might be beyond your control during the test. Make sure you maintain the 100% load
point for a long enough period to ensure that your measurements are taken during a steady state period of system
activity.
If overload or excess capacity needs to be tested, schedule a test run that begins as at 100% of full load and then
ramps up 5-10% of additional load at a time until the measured responses or verification points begin failing to pass
acceptable level. This is your basic system capacity load level.
|