Inspect the Test Logs and other output for completeness and accuracy. Identify where errors have occurred, and inspect
them.
When test automation is being employed, there are two categories of halted tests of which it is important to be aware:
-
Fatal errors: The system fails (network failures, hardware crashes, and so on)
-
Test failures: This is when some part of a Test within a Test Suite cannot be executed as intended
When either category of abnormal behavior occurs during test execution, tests may exhibit the following symptoms:
-
A large number of (ongoing occurrence) unexpected actions or unexpected windows occur while the Test Suite is
executing
-
The test environment appears unresponsive, is slow, or is in an undesirable state (such as hung or crashed)
Work through the symptoms until you can determine the root cause of the problem.
Errors may be found in the input data consumed by the test, the test itself, or other aspects of the test (such as the
test environment or runtime tool settings). It is common for a fix to an error in one aspect of the test to require the
correct state to be present in all often aspects of the test.
Once you have finished investigating problems, you may have discovered one or more faults that need correction. To make
permanent corrections to the environment, test data, or the test itself, it is a good practice to restore each aspect
of the test again to a known state before applying any permanent corrections. This ensures that no additional unwanted
or invalid changes find their way into the known-state environment.
After making the necessary changes, save the Test, and then backup or save the accompanying input data and test
environment as required.
Reschedule and re-execute the Test Suite. Depending on what recovery process is available (if any), you may be able to
restart the test suite from an interim point, rather than starting from the beginning. Note that the ability to enable
recovery of test execution from a point part-way through the test run typically necessitates the implementation and
ongoing maintenance of some form of partial recovery procedure.
Confirm that the Test Suite now runs to completion. If there are still problems, work through these subsections that
make up Recover from halted tests again until all of the problems are resolved.
|