Performance testing is a well-understood discipline that has been practiced
for more than 30 years. First there were time sharing capabilities on mainframe
computers, then minicomputers with multiple asynchronous terminals, and later
networks of personal computers connected to server systems. Testing all of these
systems revolved around the need to understand the capacity of the shared portions
of the system.
The process of performance testing has not changed significantly since these earlier system types were being tested.
However, the complexities of the system design with more distributed intelligent hardware components and many more
interconnected software subsystems yield more challenges in the analysis and tuning parts of the process. On current
systems, performance testing should be done iteratively and often during the system design and implementation. Tests
should be performed during implementation of critical subsystems, during their integration together into a complete
system, and finally under full capacity work loads before being deployed into production.
The basic steps in the performance testing process are listed here. Each step will be explained in the sections that
follow in terms of the actions taken and results of the step.
- Determine the system performance questions that you need to answer.
- Characterize the workload that you want to apply to the system.
- Identify the important measurements to make within the applied workload.
- Establish success criteria for the measurements to be taken.
- Design the modeled workload, including elements of variation.
- Build the modeled workload elements, and validate each at the various stages
of development (single, looped, parallel, and loaded execution modes).
- Construct workload definitions for each of the experiment load levels to
collect your workload measurements.
- Run the test and monitor the system activities to make sure that the test
is running properly.
- Analyze measurement results and perform system-tuning activities as necessary
to improve the results, and then repeat test runs as necessary.
- Publish the analysis of the measurements to answer the established system
performance questions.
|