Introduction
After running a test, all the test failures need to be located, isolated and diagnosed. RQM proposes multiple ways of
conducting this analysis. Once understood, those test failures need to be appropriately documented and communicated to
the development team.
Tool Steps
1. Open the Log files
In order to analyze any test failure using RQM, the first step that you must perform is to open a test log attached to
a specific test execution record. Rational Quality Manager offers multiple possible paths to open a test log for
a test execution record.
From the Planning section:
-
Start by opening the test plan containing the test execution record that you want to open the log for: point
to the Planning icon and click All Test Plans (or My Test
Plans). A list of test plan(s) is then displayed on the screen.
-
Click on the test plan's name containing the targeted failed test case(s). At this point, you can see the test plan
displayed on the screen.
-
In the Table of Contents section, click on Test Cases. It will bring you to the
list of test cases associated with that test plan.
-
Click on the test case for which you want to investigate the failure for. At this point, the test case is displayed
on the screen.
-
In the Table of Contents section, click Test Execution Records. This action will
bring you to the list of executions for that item.
-
In the Test Execution Records section, click the Status link attached to one of
the executions. By clicking on that link, you will be directed to the result history for that specific item. The
status that is displayed in that column is related to the last test run. Thus, you may have to click on a status
link different than Failed to access the result history for that specific item.
-
You can now click log icon of the targeted failed execution.
The second way of accessing a test log is after a test execution, as RQM brings you by default to the log
result of that run.
From the Execution section:
-
Start by opening the Test Execution Records list: point to the Execution
icon and then click All Test Execution Records (or My Test Execution Records).
-
Please refer to steps 6 and 7 of the following above section.
After following one of those two paths, you will now have a displayed test log containing specific information, such as
the test verdict, test environment and test result details.
2. Investigate each Failure
You now need to investigate each failure. It is important to look for similarities in the failures (Did they happened
under the same test configuration? Did they have similar symptoms? Etc ...). It's important to understand that a single
failure may have cause several subsequent failures (maybe all of the subsequent failures!). Make sure to investigate
the failure(s) in a sequential order: from the beginning to the end of the execution.
Depending on which type of test script is attached to the test (except for manual script type), you will have to
open the external tool log file by clicking on the log file link in the Result Details section.
For more information about how to interpret the different external log types, please refer to the online help for their
respective product.
Example of investigating a test failure coming from a manual test script
For a test implemented with a manual script, you can start the investigation by looking at the Result
Details section at the bottom of the test log:
-
Look for Failed status in the steps and verification points listing section.
-
For each of those Failed verification point(s), in the Result Details section,
look for the comment icon and the attachment icon. Those icons will be your main
source of information to investigate the failure. If you point on those icons, any comment or file attachment that
was created during the execution for a specific step will be displayed. Those comment(s) and attachment(s) may
contain a description of the failure with possible screenshots of the problem. Use this information to diagnose the
problem.
-
For any of the found failure(s), you may enter a defect, from RQM, in a supported defect tracking tool.
|