Introduction
Online Gardening Supplies is an internet-based outlet selling gardening supplies to the general public. Their web site
generates 100% of their sales and is critical to the survival and success of the business. OGS is re-hosting their
website on IBM Series x346 servers using a three tiered IBM middleware solution that is based on IBM WebSphere
Application Server v6.1, IBM DB2 Database Server, and IBM HTTP Server v6.1.
This workload specification serves as the requirements and design document for the workload to be implemented and
tested against the pre-production version of the OGS system. It is designed to provide a conservative estimate of the
peak commercial user load on the OGS website after it is placed into production. Performance of the OGS system on this
workload will determine whether the system meets OGS’s performance requirements and whether OGS will accept the system
as designed.
Goals
There are several goals that must be met by the OGS system to pass its performance test. Along with the workload model
specified below, these goals are used in conjunction to test the robustness, availability, responsiveness, and total
throughput of the system.
-
Robustness testing is to be performed to ensure proper execution of transactions and all workload components under
150% of the expected peak load on the system albeit with slower than acceptable response times while maintaining
near peak transaction throughput.
-
Availability testing is to be performed to ensure that the system can continue to perform transactions at peak
efficiency at 100% of expected peak load for a sustained period of 72 hours.
-
Responsiveness testing is to be performed as part of a one hour peak load test to verify successful levels of
response times over the entire test period.
-
Transaction throughput testing is also part of the one hour peak load test to verify that successful average
transaction throughput can be maintained over the entire test period.
Workload Model
Historical statistics show that there are two peak periods of shopping on the OGS website: the third week in May and
the fourth week of August corresponding to the beginning of the summer and fall planting seasons. In each of the three
previous years the August rush was a heavier load in all aspects and will be used as the peak workload.
Shown in the table below are the transactions used to represent the peak workload. The workload consists of two batch
reports: an inventory report and a sales report performed automatically once per hour and two types of user activity:
browsing items for sale on the website and purchasing items from the website.
Business Transaction
|
System-wide Total / hour
|
Simultaneous Users
|
Average Think Time
|
Records Impacted
|
Browse Item
|
50,000
|
5,000
|
3.00 seconds
|
1
|
Buy Items
|
500
|
100
|
5.00 seconds
|
3-5
|
Inventory Rpt
|
1
|
1
|
2.00 seconds
|
200,000
|
Sales Rpt
|
1
|
1
|
2.00 seconds
|
8,000
|
User Scenarios
The four user scenarios are listed for the four business transaction types are listed here.
Browse Item
-
Go to home page of website
-
Select one of three categories of items: flowers, trees, or accessories.
-
Choose at random one of the items from the grid of possibilities and click on it.
-
Add that item to your shopping cart
-
Click continue shopping to return to the grid of items.
-
Repeat steps 3 – 5 ten times and then close the browser (abandoning the cart)
Buy Items
-
Go to home page of website
-
Click the login button
-
Enter a valid username and password which returns you to the home page
-
Select the flowers category of items.
-
Choose at random one of the items from the grid of possibilities and click on it.
-
Add that item to your shopping cart
-
Click continue shopping to return to the grid of flowers
-
Repeat steps 5-7 randomly between 2 and 4 more times
-
Click checkout and fill in the shipping address and payment information.
-
Click submit order to go to the confirmation page.
-
Close your browser after getting confirmation of your order.
Inventory Report
-
Open the administrative console of your website
-
Enter your administrative login and password.
-
Click on run inventory report.
-
When the report posts its results, save the report and close your browser.
Sales Report
-
Open the administrative console of your website
-
Enter your administrative login and password.
-
Click on run sales report.
-
When the report posts its results, save the report and close your browser.
User Characteristics
The user think time characteristics are listed as part of the workload model for each business transaction type based
on the expected experience or care taken when the user executes the scenario.
Input Data Variation
For this workload, the input data that needs to be varied is specified as part of the user scenarios. The other item
that needs to be varied is the user login and password for the buy items scenario. Each simultaneous user will use a
separate login and password. It will be used throughout that virtual user’s entire workload. The virtual user will
re-login every time that user does a buy operation.
Measurements and Success Criteria
There are three priorities is the measurements that must meet the success criteria.
First priority
-
Response times for Home Page, Home Page Logged In, Login, Item Detail, Checkout, Order Summary, and Shipping Pages
must be under 2.0 seconds at the 90th percentile level.
-
Response times for Order Confirmation and Home Page Logged In must be under 4.0 seconds at the 90th percentile
level.
Second priority
-
Response times for Plants, Trees, and Accessory list pages must be under 5.0 seconds at the 90th percentile level.
Third priority
-
Sales report response time must be less than 2.0 minutes.
-
Inventory report response time must be less than 10.0 minutes.
Summary
The OGS performance testing project is the basis for the acceptance of the new e-commerce web site. The specification
contained here represents the workload and success criteria for the system’s performance.
|