Most reasonably sized IT organizations must establish a performance testing
capability as one of their standard functions. Taken as an organizational
function, we discuss the constituent parts of building this capability. Beyond
the obvious organizational elements of budget and staff, you need SWEAT:
Steps, Workload, Equipment, Analysis, and Tools.
Steps refers to the need to follow a standard performance
testing process. By obeying all of the important stages in this process, you
can be assured of a reliable conclusion from your performance test.
Workload refers to the user scenarios modeled, rates of execution,
and the identified measurements used in the performance test. Some performance
tests can yield significant results and system improvement by just running a
stress test without an attempt to model the anticipated production workload
on the system. In other cases, where you are trying to ensure that the system
can handle its intended workload, perform the business analysis or distill it
statistically from the current production system if possible.
Equipment refers to the laboratory environment necessary
to implement a full-scale performance test. Often, the system under test is
undersized or the hardware required to drive the test is not anticipated.
Analysis refers to the need to assemble a multidisciplinary
team, including the system and application architects to participate in performance
testing. They are instrumental in the tuning and subsystem capacity testing,
as well as during the final full-scale testing process. Without their input,
the system tuning would have to be done superficially without the intimate architectural
knowledge of possible optimization,s and it is less likely to result in an acceptable
final result.
Tools refers to the adoption of highly scalable and productive
testing and analysis tools that work with the latest generation of enterprise
servers and their middleware. Today's toolset should include flexible and dynamic
load generation, response time measurement, operating system monitoring, and
distributed application analysis capabilities. Without these functions, today's
highly complex enterprise applications can be difficult to analyze and tune.
Successful performance testing requires a long-term commitment by the organization.
Any enterprise that runs its business on a software-rich infrastructure cannot
only survive, but adapt and thrive by evolving its infrastructure as business
conditions change. Having a capability of repeatedly performance testing new
versions of the software infrastructure is like an insurance policy against
introduction of poor-quality software modifications. This information technology
infrastructure is a strategic asset to businesses competing in today's global
economy. |