IBM Rational Performance Tester Version 6.1 - Release notes

© Copyright IBM Corporation 2005. All Rights Reserved.

Table of contents

1.0 About this release
2.0 Installation information

2.1 Hardware requirements
2.2 Supported software

3.0 Uninstallation information
4.0 Known limitations, problems, and workarounds

4.1 Limitations
4.2 Known problems and workarounds

5.0 Documentation updates
6.0 Contacting customer support
7.0 About Eclipse

1.0 About this release

This release is a complete redesign of the IBM(R) Rational(R) Performance Tester product, which tests system performance. In this new version, Performance Tester supports multiuser testing of Web applications and provides major improvements in ease-of-use, scalability, and multiple platform support. It is hosted in the Eclipse Integrated Development Environment, using a Java-based execution engine.

There are two components to the product: Performance Tester, which is installed on your local computer, and the IBM Rational Agent Controller, which generates load from remote computers. When installed on a remote computer, the agent controller runs a driver that emulates users as part of a load test. When installed on your local computer, as part of Performance Tester, the agent controller can run emulated users from your local computer.

2.0 Installation information

For information about installing and configuring Performance Tester, refer to Installing IBM Rational Performance Tester.

Obtaining a license server

Before using Performance Tester to run a load test consisting of more than five virtual users or containing particular protocols you must check out IBM Rational Common License keys from an IBM Rational License Server. These license keys are separately purchased and installed on a system running the IBM Rational License server. You can obtain the license server from the Passport Advantage download site.

Memory requirements

Do not install Performance Tester on a system with less than 768 MB physical memory. If you proceed with installation, be aware that you might experience poor performance, application crashes due to insufficient memory, or both, and that such a configuration is not supported.

After the install wizard is finished, adjust the JVM heap size for your system. To do this on Windows:

  1. Start Performance Tester.
  2. Click Window > Preferences.
  3. Expand the Test tree, and click Performance Test Workbench.
  4. The recommended JVM heap size is displayed. Type this number into the Java Virtual Machine heap size field, then click OK.

After setting the JVM heap size, it is possible that Performance Tester could fail to start the next time it is launched. This is caused by a low memory or severely fragmented memory condition. Closing some active programs or stopping some services can free up enough memory to allow Performance Tester to start.

If this is not sufficient, reset the JVM heap size by editing the configuration file. This file, RationalSDP.ini, is located in the product install directory (by default, C:\Program Files\IBM\Rational\SDP\6.0). Open the file in a text editor (such as Notepad) and look for a line that begins with VMArgs=-Xmx, followed by a number, followed by m. The number should be what you entered for the Java Virtual Machine heap size field on the Preferences page. Removing the line from the configuration file allows Performance Tester to start.

2.1 Hardware requirements

Refer to Installing IBM Rational Performance Tester for hardware requirements.

2.2 Supported software

The following software is supported:

Operating systems
Web browsers
  • For performance test recording, all HTTP-compliant Web browsers are supported.

    For viewing the online help, the following browsers are supported:
  • Microsoft® Internet Explorer 6 with Service Pack 1.
  • Microsoft Internet Explorer 5.5 with Service Pack 1 or 2.
  • Mozilla 1.6
Application servers
  • All HTTP-compliant servers are supported as the target of performance testing.
Rational ClearCase LT is included with Performance Tester. Refer to Installing IBM Rational Performance Tester for instructions on how to install Clear Case LT.

Other software configuration management (SCM) repositories can be integrated with Performance Tester through SCM team adapters. If you want to work with another SCM provider, SCM team adapters for commercial SCM products are provided by the vendors of those products. To find a list of SCM products and adapters provided by IBM Business Partners, go to www.ibm.com/software/ad/studioappdev/partners/scm.html. From this list, you can access the SCM product vendor Web sites to download an adapter. IBM does not endorse or warrant these adapters. For support, or further information, contact the SCM product vendor directly.

Novell LINUX Desktop Powered by SUSE Linux 9 is not supported.

3.0 Uninstallation information

Refer to Installing IBM Rational Performance Tester for information on uninstalling Performance Tester.

4.0 Known limitations, problems, and workarounds

Release notes cover release-specific information (such as problems and limitations) that did not become available until after the product documentation was final.

For information about known problems and workarounds in the Eclipse base, refer to the readme_eclipse.html file in the eclipse\readme subdirectory of the product installation directory. This information is available only in English.

4.1 Limitations

Problems with shell-sharing

When installing another IBM Rational Software Development Platform product on the same system with Performance Tester, you must run the Rational Product Updater to bring the other products up to the latest version available. This step ensures proper operation of both Performance Tester and the other Rational products.

Multiple applications accessing a single agent controller

Avoid having multiple Performance Tester applications located on different computers concurrently accessing a single agent controller. This configuration may produce unreliable test execution results.

Multiple Performance Tester instances

Do not have multiple concurrent Performance Tester application instances on a single operating system desktop. Browser configuration problems and unreliable test execution can result.

Avoid the # character within test artifact naming

Do not use the # character within test artifact naming. Using this character in an artifact name can cause it to become inaccessible or corrupt.

Windows user group privileges

Customers logging into Windows computers must have Administrative User privileges in order to use Performance Tester.

Error messages during a test or schedule run

During a test or schedule run, you might get a socket write error that resets the connection. To fix this problem, restart the agent controller on the computer that is encountering this problem. Make sure that you start the controller as root.

Schedule deployment hangs at 48%

When you launch a schedule, the schedule never starts to run, the percentage remains at 48%, and nothing is deployed on local or remote agent controllers.

To fix this problem, restart the agent controller on the computer that is encountering this problem.

Cannot install in non-default location that contains double-byte pathname characters

Do not install Performance Tester into a directory whose name contains double-byte characters or special characters such as a dollar sign. Doing so will cause undesirable results.

Eclipse simple projects unsupported

Rational Performance Tester does not support the use of Eclipse-defined simple projects. Simple projects are identified by the absence of a small letter J on the project folder icon within the Test Navigator.

Recording through proxy servers

On Windows, the RPT recorder uses your Internet Explorer configuration settings to perform the recording. Therefore, to record through a company proxy server (or any HTTP proxy servers), you must configure Internet Explorer to use the proxy server before starting a recording. You must specify either "NO HTTP proxy entries" or "BOTH secure and regular HTTP proxy entries." Note that many sites use the same proxy for both. Do not modify these settings during recording.

Classes not recorded on Linux

On Linux, the RPT recorder is unable to record Java classes. This is because Mozilla ignores proxy settings when fetching classes, thus bypassing the recorder. The impact of this is that the classes will not be fetched during playback, providing a less-than-expected load on the server.

Administrator privileges required to run the Rational Product Updater

To run the Rational Product Updater successfully, you need to have Administrator privileges. Otherwise, your updates will fail.

4.2 Known problems and workarounds

Creating tests

Legal filenames when recording a test
When you record a new performance test, the HTTP Proxy Recorder requests a Recording file name. Use only alphanumeric characters in this name. If you include an illegal character, the Finish button remains dimmed.
Recording local Web sites
When recording a test from Internet Explorer at a local Web site (a Web site on your internal LAN), the IWAT3040E Registry Settings Error window might open with the following message:

Your IE configuration setting for Proxy Server Exceptions is not supported.
The recorder only supports either no Proxy or both HTTP and Secure Proxy.

You can get a similar message with Mozilla or another Web browser. The problem is a configuration setting that prevents use of proxies with local Web sites.

To configure Internet Explorer so that it allows the use of a proxy with local Web sites, start Internet Explorer and do the following:

  1. Click Tools > Internet Options.
  2. On the Internet Options window, click the Connections tab.
  3. Click LAN Settings.
  4. Check Use a proxy server for your LAN.
  5. Clear Bypass proxy server for local addresses.
  6. Click Advanced.
  7. In the Exceptions area at the bottom, inspect the Do not use proxy server for addresses beginning with field. Remove any address string that is blocking the Web site you are requiring to record, or clear the field.
  8. Click OK to close the Proxy Settings window.
  9. Clear Use a proxy server for your LAN.
  10. Click OK to close the Local Area Network (LAN) Settings window.
  11. Click OK to close the Internet Options window.

If you get a similar error message with Mozilla or Netscape, the configuration setting causing the problem is the No Proxy for field on the Proxies window. Either clear this field or remove the address that you are trying to record.

"Unable to create part" error with imported datapools
Performance tests (.testsuite files) with datapool associations might not open if the parent performance test project was imported and a project containing the associated datapool was not. An Unable to create part error is shown in the test editor. Removing datapool associations within tests before export or import can minimize this problem. However, if you follow the steps in the Help topic "Copying a project to another workspace" you can avoid this problem entirely.

Editing tests

Renaming a custom code class requires Java refactoring
Changing the name of the Java class for a custom code action in the test editor does not move or refactor the class to use the new name. The previous Java class continues to contain modifications that were made to the generated custom code, and the new Java class will be generated using the name from a template provided in the test editor.

To change the name of the Java class, first refactor the Java class to the new name, then edit the name in the test editor to refer to that class.

Custom code must be located in test package
All custom code must be located within the test package in the test project. It can be placed in a subpackage of test, but the subpackage name must be lowercase (test.custom.VerifyID, for example).

Representing workloads

Edit locations only through the schedule editor
Although you can create a location by clicking New > Other, then expanding the test tree until you see the location icon, this practice is not recommended. A location created by this method does not contain the necessary attributes to work in a schedule. To create a location that works correctly, follow the steps in the Help topic "Running user groups at a remote location."

Running schedules or tests

Memory leak in workbench when running performance tests
There is a memory leak associated with running a performance test. When running a large number of tests, you should periodically restart the Performance Tester workbench.
Playback may fail when tests attempt to upload large files
A test or schedule run might fail when a test attempts to upload files larger than 500 KB. This is because the generated Java test will exceed the maximum allowable size of the Java class file.

To fix this problem, split the uploaded file into smaller parts.

"Test run aborted due to error" message when an application requires a restart
If you need to restart any single application, you should also restart all agent controllers and the local computer. This collective restart is necessary to reset the open network connections created by schedule or test runs. If you do not perform a collective restart of all agent controllers and the local computer, you see the message "Test run aborted due to error."
Cookie cache must be cleared
A cookie is a piece of information that a Web browser allows the Web server to store locally for maintaining useful state information across Web pages or Web sessions. The cookie cache is the dynamic memory of all these states. Performance Tester simulates the Web browser's cookie cache automatically.

In certain instances, you might want Performance Tester to clear all the cookies it has dynamically stored for a particular virtual user. One example is when a loop exists around the invocation of a test. If cookies should not be remembered from one iteration to the next, then the user must indicate to Performance Tester to clear the cookie cache of the current virtual user.

To fix this problem, a Java API has been added to clear the cookie cache of the current virtual user. Add custom code to the test to call this API and clear the cookie cache when the custom code is executed. This can be done in the following steps:

  1. As explained in the help topic "Adding custom code," add custom code to the test at the location where you want the cookie cache to be cleared. For example, at the end of the test.
  2. View the custom code Java file.
  3. Add the following Java import statement
    Import com.ibm.rational.test.lt.execution.http.util.CookieCacheUtil;
  4. Add the following Java code inside the exec method
    CookieCacheUtil.clearCookieCache(log);

The following example shows a custom code addition that clears the cookie cache, with the lines that you need to add to the generated custom code template shown in bold:

package test;
import com.ibm.rational.test.lt.execution.http.util.CookieCacheUtil;
import com.ibm.rational.test.lt.kernel.logging.IKLog;
public class Class1111180036738 implements com.ibm.rational.test.lt.kernel.custom.ICustomCode {
    public Class1111180036738() {
    }
    public String exec(IKLog log, String[] args) {
        CookieCacheUtil.clearCookieCache(log);
        return null;
    }
}

Troubleshooting CodeGen errors
Certain conditions can result in CodeGen errors. A CodeGen error can take the form of an Unable to Persist error view message, an Unable to Launch message, or a Null Pointer window when you run a test or schedule. Altering the .classpath file and removing certain Java sources might resolve the issue.

To alter the .classpath file:

  1. In the Eclipse Navigator view, open the .classpath file. It will look something like this:
    <?xml version="1.0" encoding="UTF-8"?>
    <classpath>
    <classpathentry kind="src" path="src"/>
    <classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER"/>
    <classpathentry kind="lib" path="C:/Program Files/IBM/Rational/SDP/6.0/rpt_prod/eclipse/plugins/com.ibm.rational.test.lt.core_6.1.0/core.jar"/>
    <classpathentry kind="lib" path="C:/Program Files/IBM/Rational/SDP/6.0/rpt_prod/eclipse/plugins/com.ibm.rational.test.lt.datacorrelation.execution.http_6.1.0/datacorrelation.execution.http.jar"/>
    <classpathentry kind="lib" path="C:/Program Files/IBM/Rational/SDP/6.0/rpt_prod/eclipse/plugins/com.ibm.rational.test.lt.kernel_6.1.0/kernel.jar"/>
    ...
    Files/IBM/Rational/SDP/6.0/rpt_prod/eclipse/plugins/com.ibm.rational.test.lt.execution.http_6.1.0/execution.http.jar"/>
    <classpathentry kind="lib" path="C:/Program Files/IBM/Rational/SDP/6.0/rpt_prod/eclipse/plugins/com.ibm.rational.test.lt.provider_6.1.0/provider.jar"/>
    <classpathentry kind="output" path="bin"/>
    </classpath>

  2. Remove all lines with classpathentry kind="lib" at the beginning.
  3. Save the file.
To remove Java sources:
  1. In the Eclipse Navigator view, open all folders containing files like text-string_hex-string.java, where the text string contains either Test or Schedule, and the long hexadecimal string varies. For example, MyTest_Test_C4B984B93C96709D96B4E6C0284611D9.java.
  2. Delete all such files.
Run the test or schedule again. The .classpath and .java files will be re-created in the proper form.

Evaluating results

Execution history may be incomplete if run stops suddenly
When a performance test or schedule is not able to run to completion, the execution history might be incomplete. This can occur when the execution is stopped by the user or when a schedule time limit has been set. When this occurs, some of the stop events (schedule stop, user group stop, virtual user stop, and so on) and some of the verdict rollup events are absent from the execution history.
Execution history may be incomplete if extremely large amounts of data are logged
The amount of data logged during a run is based on the two schedule settings: Execution history log level and Only sample execution history from a subset of users. When running a large workload, the amount of the data logged can be huge, especially the logging level is set to Request or All and there is no sampling (therefore, all users are logged). In these cases, some events near the end of the run are not saved in the execution history. The types of events that can be missing are generally those contained within some executed tests. You should still be able to see hierarchical events including schedule, user group and user-related events.

One workaround is to sample the users. This is generally a good idea when you are emulating large numbers of users. Another workaround is to keep the Execution history log level at the Page level.

Execution history can be misleading when sampling
Performance Tester adds an additional verification point for pages that fail or produce an error. These verification points are required for displaying verdicts within the execution history and are aggregated into the total verification point stats counters. If sampling is enabled (the default), pages that fail might be excluded. If they are, the test results are misleading because the overall verdict is Pass although some pages have failed.
Exporting run statistics does not work if path is entered directly in CSV file field
The performance test run statistics are not exported if you select File > Export and enter the path directly in the CSV file field (without clicking the Browse button).

To work around this problem, click the Browse button and browse to the location you wish to store the file; name the file and click OK.

Run completion sometimes indicated long before data transfer is complete
At the end of a run, the Performance report may indicate that the run is complete before data transfer has completed. In this situation, the CPU will be nearly saturated with seemingly no explanation.

The true test to see if data transfer has completed is to open the execution history. When a run stop time has been placed in the execution history, then data transfer has completed.

A firewall on the local computer may prevent the return of execution history and statistics data
Performance Tester uses network ports to pass data between its components. Network firewalls can disrupt this operation and cause errors during a run. If you cannot see the execution history after the run has completed, this might indicate a firewall issue.

Most network firewalls let you create an exception list for applications that should not be blocked from network access. To ensure that communication between the components of Performance Tester is uninterrupted, add the following applications to your exception list. If the network firewall is installed only on the agent controller, then only the second application listed for each operating system needs to be added to the list.

For Windows:
RPT_install_path\IBM\Rational\SDP\6.0\eclipse\jre\bin\javaw.exe RPT_install_path\IBM\Rational\SDP\6.0\IBM_Agent_Controller\bin\RAService.exe  

For Linux:
RPT_install_path\IBM\Rational\SDP\6.0\eclipse\jre\bin\javaw RPT_install_path\IBM\Rational\SDP\6.0\IBM_Agent_Controller\bin\RAServer

Note: Microsoft Windows XP SP2 installs a restrictive firewall. It must be disabled or adjusted as indicated above to allow proper communication between the components of Performance Tester.

CPU use reaches 100% during a run
During test runs that involve multiple test drivers, the CPU of the local machine might become very busy. If you use the default statistical sample interval of 5 seconds to collect test results, the CPU can become busy and the program might fail to respond.

To fix this problem, change the sample interval to a longer duration; from 30 seconds to 1 minute. This allows the incoming data to be processed without causing the CPU to reach 100%.

Linux

MOZILLA_FIVE_HOME environment variable must be set
If you are running Performance Tester on Red Hat Linux, you must set the MOZILLA_FIVE_HOME environment variable. If you do not set this variable, the online help will not be displayed correctly, execution details will not be displayed in the Protocol Data view, and you will encounter other problems.

To set this environment variable, add the following lines to shortcut_rpt (in the install directory, /opt/IBM/Rational/SPD/6.0 by default):

MOZILLA_FIVE_HOME=/usr/lib/mozilla-1.4.3
export MOZILLA_FIVE_HOME

/usr/lib/mozilla-1.4.3 might be different on your system; it is where libxpcom.so is located.

License client cannot parse the license server information file
When running on Linux, the license server information is placed in the $HOME/.flexlmrc file. If this file contains an EOF, instead of a carriage return or line feed, the license client cannot parse the file. To fix this problem, make sure that a carriage return or line feed marks the end of the file.
Normal users cannot remove the workspace if the agent controller is run as root
When the agent controller is started by the root user, this user owns the files that are created to enable Performance Tester to run. If the workbench and the agent controller are located on the same Linux machine and the agent controller was started by the root user, files are created in a workspace directory that can be deleted only by the root user.
Mozilla must be installed from the SuSE install discs
With SuSE Linux, installing the Mozilla browser from an internet download does not create the appropriate configuration files. To avoid this problem, be sure to install the Mozilla browser from the SuSE install discs using the install tools available on SuSE. The Mozilla install on SuSE has been customized for the operating system.

5.0 Documentation updates

Both the product home page and IBM developerWorks have Library sections that provide updates to tutorial documents and other documentation.

The IBM Rational zone at www.ibm.com/developerworks/rational offers technical articles, "how to" information, tutorials, samples, and previews of new technology.

Updates to Installing IBM Rational Performance Tester

Step 6 in the section "Installing Rational Performance Tester from the CD-ROM" states that the Integrated Development Environment (IDE), Agent Controller, and Common Licensing are selected by default. This is true only if Common Licensing has not been installed. If the Common Licensing client has been installed, only the IDE and the Agent Controller are selected by default.

The section "Creating a network installation image" explains how to copy the product installation CDs to a network server so that users can install the product across the network rather than from the CDs. The instructions are correct but the "Note" underneath the instructions is misleading. Replace this note with the following text:

To install the product across the network on a Windows client, do the following:

  1. On the client machine, map a drive letter to the location of the installation image on the network (for example, x:/install_image).
    Note that if you fail to map a drive letter to the network location , the Agent Controller will not be installed with the product, and you might get an error message alerting you that there is a problem with the installation.
  2. Change to the mapped drive letter (for example, x:) and enter the disk1 folder (cd rpt_win\disk1).
  3. Double-click launchpad.exe.

The section "Installing license keys on Linux" is incorrect. This section should read:

To install license keys on a Linux client, create the file $HOME/.flexlmrc, containing a line with the computer name or the IP address of the Rational license server. For example, add the lines RATIONAL_LICENSE_FILE=@server-name.com or RATIONAL_LICENSE_FILE=@ip-address to the .flexlmrc file. The .flexlmrc file must be created in the home directory of every user on the target system who will run more than five virtual testers.

Flash plug-in requirement

In order to properly view multimedia user assistance, such as Tours, Tutorials, and Show me viewlets, you need to have Flash Player version 6.0r65 or later installed.

6.0 Contacting customer support

Refer to the "Troubleshooting and support" section of the online help for information on how to contact IBM Software Support.

The product home page for Performance Tester is www.ibm.com/software/awdtools/rationalappdev. For FAQs, lists of known problems and fixes, and other support information, visit the Support page on the product home page.

7.0 About Eclipse

Eclipse is an open-source project that creates royalty-free technology and a universal platform for tools integration. Eclipse-based tools give developers freedom of choice in a multilanguage, multiplatform environment. Eclipse delivers a framework for developing plug-ins that make it easier to create, integrate, and use software tools. By collaborating and sharing core integration technology, tool producers can concentrate on their areas of expertise and the creation of new development technology. The Eclipse platform is written in the JavaTM language and comes with extensive toolkits and examples for construction plug-ins. It has already been deployed on a range of operating system environments including Linux, MAC OS X, QNX, and Windows-based systems. Full details of the Eclipse community and white papers documenting the design of the Eclipse platform are available at www.eclipse.org.

Plug-in tools from other vendors, also built using the open-source Eclipse platform, integrate with the Rational environment to provide additional value.

Copyright and notices