Impact of not having |
Without this or similar documentation, there is no record of which tests
were run, what variances were discovered, and what action was taken. If this
information is not available:
- There is no way to know which tests passed and which failed
- There is no way to assess the status of testing and the quality
of the product at that level of testing
- It is difficult to know how many tests remain outstanding
- Contractual and legal issues might arise
|
Reasons for not needing | When you execute automated tests, test logs are automatically produced.
Typically, the issue is not whether to produce the test log, but whether
to keep a record, and where to keep the records. For manual testing, the issue
is whether to keep a separate test log or to summarize the test results in
another form. |
Representation Options | Because this is a collection of raw data for subsequent analysis, it can
be represented in a number of ways:
- For manual tests, log the actual results on a copy
of the manual Test Script
- For automated tests, direct the output to log files that you
can trace back to the automated Test Script
- Track raw results data in a test management tool
Automation tools often provide their own Test Log facilities, which can be extended or supplemented with additional
logging provided both through custom user-routines and the use of additional tools.
The output may take one single, or many different, forms. Typically, Test Logs have a tabular or spreadsheet-like
appearance, with each entry comprising some form of date and time stamp, a description of the event logged, some
indication of the observed status, and possibly some additional contextual information.
If you are using automated test tools, such as those found in the IBM® Rational Suite® family of products, much of the above
functionality is provided by default with the tool. These Test Log facilities typically provide the ability for the
capture, filtering, and sorting (and the analysis of) the information contained in the log. This allows the Test Log to be
expanded in detail, or collapsed to a summary view, as required. The tools also offer the ability to customize and retain
views of the Test Log for reporting purposes.
Where the logic that produces an automated Test Log simply appends new information to an existing log file, it will be
necessary to provide sufficient storage to retain the Test Log file. An alternative solution to this approach is to use
a ring buffer. A good explanation of Using Ring Buffer Logging to help find bugs is presented in a pattern catalog by Brian Marick. (Get
Adobe reader.) This catalog provides
an overview of other classic problems with using automated Test Logs.
|