Introduction
Online Gardening Supplies is an internet-based outlet selling gardening supplies to the general public. Their web site
generates 100% of their sales and is critical to the survival and success of the business. OGS is re-hosting their
website on IBM Series x346 servers using a three tiered IBM middleware solution that is based on IBM WebSphere
Application Server v6.1, IBM DB2 Database Server, and IBM HTTP Server v6.1.
This test plan serves as the document that describes the lab equipment required to perform the testing and the
scheduled sequence of experiments required to gather the data called out in the workload specification. These tests are
designed to measure the performance results in the presence of the peak commercial user load on the OGS website after
it is placed into production. Performance of the OGS system under this workload will determine whether the system meets
OGS's performance requirements and whether OGS will accept the system as designed.
Environment
Test Environment
The test environment should reflect the production system setup in every way possible. In this case the back-end data
base is currently not the same and does not exhibit the scalability of a separate server running a production IBM DB2
database environment. Without adding this capability to the test system, no formal testing can be performed. Once the
DB2 environment is in place, the system capacity, robustness, and availability testing can be performed.
System Environment
The system under test consists of a front-end IBM HTTP Server v6.1 web server hosted on a dual processor server running
Windows 2003 and an IBM WebSphere Application Server v6.1 hosted on a dual processor server running Windows 2003. In
the production environment, a separate server would be running the database using IBM DB2 Database Server. For these
preliminary tests, the database will reside on the same server as the application server. This will prevent realistic
traffic volumes from being run through this configuration. Performance bottlenecks in the J2EE application code will be
explored prior to deployment of a separate database server. This database deployment will be required to get production
volume traffic and response time estimates can be measured.
The performance testing complex will consist of an RPT console machine and two test agent machines that will be used to
drive the system under test described above. All systems are connected over the 100MB LAN fabric used in the RPT
mini-lab network.
Test Schedule
The overall test schedule consists of the following test phases:
-
Equipment, middleware, and application setup (5 days)
-
Performance test development (3 days)
-
Application tuning of J2EE code (5 days)
-
System capacity testing (2 days)
-
Robustness testing (2 days)
-
Availability testing (10 days)
Each of these test phases requires the completion of the previous phase. The database deployment is required for Phases
4 through 6.
Equipment and Software Setup
Installation of all system hardware and networking must be in place. The operating systems, anti-virus and firewall
software then need to be installed. The web server and application server should be installed next. The RPT test agent
software for data collection should then be installed on the web and application servers. The application server should
be instrumented and the data collection monitors started on both servers. Finally a stable version of the application
code should be deployed on the application server. Browser access to the application through the front end web server
completes the sanity testing for the application environment.
The RPT console and test agent systems need to be installed with the latest product version available making sure that
the test agents installed on all systems are the same version as the version of the RPT console. The RPT console server
needs to be licensed for development and point to an IBM Rational license server hosting virtual tester playback
license packs for use during the multi-user testing of the application.
Deliverable Artifacts
-
Test lab machines configured with OS, middleware, and application software.
-
RPT console and test agent machines configured with latest RPT version.
Performance Test Development
The performance test scenarios described in the OGS Workload Specification shall be captured using the RPT recorder,
verification points added, and input data variation added. The tests shall be debugged for improper operation in single
execution, looped execution, few user parallel execution, and then in a homogeneous stress load environment. Then these
tests will be declared baselined and ready to be used in a multiple scenario workload environment. Any required
datapools that need to be built should be built and tested as part of this debugging process.
Deliverable Artifacts
-
Performance test project
-
Fully debugged and parameterized tests for each of the test scenarios
-
Test schedules used to stress test each individual test
-
Test results from individual test playback and stress test playback for each test
-
Datapool assets to support the required input data variation for each test
Application Server Tuning
First the ramp-up test is performed to identify the basic application server capacity based on response times and
system resource usage. Then perform the transaction breakdown data collection experiment at two/thirds of capacity.
Analyze the transaction data for any slow transactions and work with development to resolve any unexpected bottlenecks
in the application. Finally perform the one hour steady state test with resource monitoring to make sure the
application performance stays steady. Report any suspected memory leaks back to development for analysis.
Deliverable Artifacts
-
Performance test schedule implementing a gradual ramp up to determine application system capacity when response
times become unacceptable
-
Performance test schedule used to tune a single application server at two/thirds server capacity with transaction
breakdown tracing turned on
-
Performance test schedule used to tune a single application server at full load with resource monitoring configured
for tracking server resources
-
Test results from a gradual ramp up past the full load capacity of the application server to identify full capacity
loading
-
Test results from a 15 minute steady state playback at two/thirds server capacity with transaction breakdown
tracing turned on
-
Test results from a 1 hour steady state playback at full load server capacity with resource monitoring to track
java process size growth to verify there are no memory leaks and that performance stays steady
System Capacity Determination
On the complete system including the fully deployed and sized database server, run the ramp up schedule to determine
the system capacity full load point using response times and system resource usage. Verify that the test agents can
handle the full system capacity workload and take accurate measurements by verifying agent resource usage. Take the
ramp up measurement well beyond the system capacity whether it is more or less than the expected production workload.
Deliverable Artifacts
-
Performance test schedule implementing a gradual ramp up to determine system capacity when response times become
unacceptable
-
Test results with resource monitoring for all servers including the test agents
System Capacity Measurement
Run the workload at the measured system capacity for a period of at least one hour while monitoring system resources to
verify that the system can sustain the capacity found during the ramp-up measurement. Verify that system resources on
the test agents are within satisfactory operating region so the measurements can be validated.
Deliverable Artifacts
-
Performance test schedule implementing the maximum system capacity workload
-
Test results with resource monitoring for all servers including the test agents from a steady state run lasting at
least 1 hour
Robustness Testing
Run the workload at 150% of system capacity for at least one hour to prove that you get consistent stable response
times and throughput processing even though the response time is not acceptable. This test is to demonstrate that the
system remains stable even under an overload condition.
Deliverable Artifacts
-
Performance test schedule implementing 150% of expected production workload
-
Test results with resource monitoring for all servers including the test agents from a steady state run lasting at
least 1 hour
Availability Testing
Run the system at expected production workload for 72 hours to demonstrate long term stability of the system without
requiring restarts and to show that system processing throughput and response time can be maintained at peak efficiency
for long periods of time.
Deliverable Artifacts
-
Performance test schedule implementing the expected production workload modified for a 72 hour continuous operation
with reduced data collection
-
Test results from a steady state run lasting at least 72 hours
|