PreviousDebugging Tutorials

Chapter 6: FaultFinder

FaultFinder is an aid to debugging production applications. It enables you to to obtain a text-based report on the state of an application at the time of any abnormal application termination. The report is a snapshot of the application at the time of failure and identifies items such as open files, file buffers, loaded programs, and so on.

6.1 Operation

FaultFinder is controlled by tunables. To turn on FaultFinder set the run-time tunable faultfind_level to a positive value. Then, if the application terminates abnormally, the run-time-system automatically invokes FaultFinder which in turn generates the report. For details of the report provided, see the section FaultFinder Report.

The following run-time tunables control the operation and output of FaultFinder:

faultfind_level=n Turns on FaultFinder and enables tracing of events. n can be any positive value less than or equal to 1024 and enables the last n events to be dumped. The default value is 0.
faultfind_outfile="filename" The file to which the analysis is written. The default filename is ffnd_out in the current working directory. If, for some reason, the specified file cannot be opened, output is directed to the screen.
faultfind_recsize="record_size" A numeric value, in bytes, that specifies the amount of extra data that is to be kept for each trace operation. record_size can be any non-negative value less than or equal to 1024. The default value is 64.

For detailed information on a tunable, click the tunable in the preceding table. For information on setting run-time tunables, see the chapter Run-time Configuration in your Server Express User Guide.

Example:
set faultfind_level=5
set faultfind_outfile="applflt.log"
set faultfind_recsize=1024

6.2 FaultFinder Report

The FaultFinder report contains information on the:

This information is contained in titled sections in the report. You can search for sections in the report by their titles.

For threaded programs (run with the multi-threading run-time system), additional information is output; this is discussed in the relevant sections.

6.2.1 Environment

The Environment section of the report describes:

6.2.2 Call Stack

The Call Stack section of the report contains, for each thread running at the time of failure, a call stack traceback. Programs are listed with full pathname and extension (where available). Entry points are not listed. The information for each thread is listed in a reverse calling hierarchy, so that:

If the program contains Local-storage and/or Linkage sections, these sections are dumped with the corresponding call-stack instance. For these sections, the following information is dumped:

For .int files, the current program-counter is also dumped.

6.2.3 Loaded Programs

The Loaded Programs section of the report contains all currently loaded programs, with their reentrant and non-reentrant data areas. The non-reentrant data area has separate areas for working-storage and each instance of user-allocated data (using CBL_ALLOC_MEM, flag=0). A separate reentrant data area exists for each thread executing that program and has separate areas for thread-local storage and thread-local allocated data (using CBL_ALLOC_MEM, flag=8). Each chunk of data states the hexadecimal address of the data item and the dump (in hexadecimal and ASCII with duplicate lines not repeated but replaced by a single "*") of the data.


Note: The working-storage section includes all of the non-user program specific data areas, as in the file-control and file sections, as well as system needed non-reentrant space. To better define the start of the program's actual working-storage section, it is recommended to insert a special data item as the first item in working-storage and give it a value that can easily be searched for.


6.2.4 Logically Cancelled Programs

This section of the report contains a list of logically cancelled programs, with full pathname and filename extensions.

Files that have been logically cancelled are still known to the run-time system, so can be loaded and run directly without having to be loaded from disk. Files that are physically cancelled release all their memory and no longer exists to the run-time section.

No other information is available in this section of the report.

6.2.5 Local Open Files

This section of the report contains all non-external opened files. For each file the information listed is:

The entire last record and the file control descriptor are both dumped (with address) in hexadecimal and ASCII.

The operation trace records the basic operation (READ, WRITE, OPEN and so on) as well as the:

The last two items are only recorded when applicable.

6.2.6 External Open Files

The External Open Files section of the report lists all externally opened files. For each file the information list is:

The entire last record and the file control descriptor are both dumped (with address) in hexadecimal and ASCII.

The operation trace records the basic operation (READ, WRITE, OPEN and so on) as well as the:

The last two items are only recorded when applicable.

6.2.7 Trace Back

The Trace Back section of the report lists the last n events for each thread, where n is defined by the run-time tunable faultfind_level. For each thread, the operations traced are:

This section is broken down by thread; it records all operations for thread 1, followed by all operations recorded for thread 2, and so on. There are typically two events per operation: one as the operation begins, and another as it finishes. A start event is prefaced with a < while an end event is prefaced with a >.  The last event to have happened is printed first.


Copyright © 1999 MERANT International Limited. All rights reserved.
This document and the proprietary marks and names used herein are protected by international law.
PreviousDebugging Tutorials