Guideline: Recovering from Halting Tests
This guideline provides more details on how to recover from halted tests.
Main Description

Inspect the Test Logs and other output

Inspect the Test Logs and other output for completeness and accuracy. Identify where errors have occurred, and inspect them.

When test automation is being employed, there are two categories of halted tests of which it is important to be aware:

  • Fatal errors: The system fails (network failures, hardware crashes, and so on)
  • Test failures: This is when some part of a Test within a Test Suite cannot be executed as intended

When either category of abnormal behavior occurs during test execution, tests may exhibit the following symptoms:

  • A large number of (ongoing occurrence) unexpected actions or unexpected windows occur while the Test Suite is executing
  • The test environment appears unresponsive, is slow, or is in an undesirable state (such as hung or crashed)

Work through the symptoms until you can determine the root cause of the problem.

Correct errors

Errors may be found in the input data consumed by the test, the test itself, or other aspects of the test (such as the test environment or runtime tool settings). It is common for a fix to an error in one aspect of the test to require the correct state to be present in all often aspects of the test.

Once you have finished investigating problems, you may have discovered one or more faults that need correction. To make permanent corrections to the environment, test data, or the test itself, it is a good practice to restore each aspect of the test again to a known state before applying any permanent corrections. This ensures that no additional unwanted or invalid changes find their way into the known-state environment.

After making the necessary changes, save the Test, and then backup or save the accompanying input data and test environment as required.

Schedule and execute the Test Suite again

Reschedule and re-execute the Test Suite. Depending on what recovery process is available (if any), you may be able to restart the test suite from an interim point, rather than starting from the beginning. Note that the ability to enable recovery of test execution from a point part-way through the test run typically necessitates the implementation and ongoing maintenance of some form of partial recovery procedure.

Reevaluate execution of Test Suite

Confirm that the Test Suite now runs to completion. If there are still problems, work through these subsections that make up Recover from halted tests again until all of the problems are resolved.