You can investigate waits in a CICS® system by online inquiry, by tracing, or by analysis of the formatted CICS system dump. The last two techniques are, to some extent, complementary.
Online inquiry is the least powerful technique, and it can only tell you what resource a suspended user task is waiting for. This is enough information to locate the failing area, but you often need to do more investigation before you can solve the problem. The advantage of online inquiry is that you can find out about the waiting task as soon as you detect the problem, and so you capture the data early.
Tracing can give you much more detail than online inquiry, but it involves significant processing overhead. It must also be running with the appropriate options selected when the task first enters a wait state, so this usually means you need to reproduce the problem. However, the information it gives you about system activity in the period leading up to the wait is likely to provide much of the information you need to solve the problem.
A CICS system dump can give you a picture of the state of the CICS system at an instant during the wait. You can request the dump as soon as you notice that a task has entered a wait state, so it gives you early data capture for the problem. However, the dump is unlikely to tell you anything about system activity in the period leading up to the wait, even if you had internal tracing running with the correct selectivity when the task entered the wait. This is because the trace table has probably wrapped before you have had a chance to respond. However, the formatted dump might contain much of the information you need to solve the problem.
If you are able to reproduce the problem, consider using auxiliary tracing and dumping in combination.
Online, you can use either CEMT INQ TASK or EXEC CICS INQUIRE TASK to find out what resource a user task is waiting on. EXEC CICS INQUIRE TASK can be executed under CECI, or from a user program. Whatever online method you use, you need to supply the task ID of the suspended user task.
If the task is suspended, the information that is returned to you includes the resource type and/or the resource name identifying the unavailable resource. CEMT INQ TASK displays the resource type of the unavailable resource in the HTYPE field. The HVALUE field displays the resource name of the unavailable resource. EXEC CICS INQUIRE TASK returns values in the SUSPENDTYPE and SUSPENDVALUE fields which correspond to the resource type and resource name of the unavailable resource.
HTYPE and SUSPENDTYPE, and HVALUE and SUSPENDVALUE correspond to the values in the resource type and resource name fields of the dispatcher task summary.
Table 9 gives a list of all the resource types and resource names that user tasks might be suspended on, and references showing where to look next for guidance about solving the wait.
You probably need a system dump of the appropriate CICS region to investigate the wait. If you do not yet have one, you can get one using CEMT PERFORM SNAP or CEMT PERFORM DUMP--but make sure the task is still in a wait state when you take the dump. You subsequently need to format the dump using keywords for the given resource type. Advice on which keywords to use is given, where appropriate, in the individual sections.
You can find detailed information about the suspension and resumption of tasks during a run of CICS by studying the trace table. Tracing must, of course, be running when the task in question is suspended or resumed, and the tracing options must be selected correctly.
When you look at the trace table, you can find trace entries relating to a particular task from the task numbers that the entries contain. Each is unique to a task so you can be sure that, for any run of CICS, trace entries having the same task number belong to the same task.
For general guidance about setting tracing options and interpreting trace entries, see Using traces in problem determination.
Gate DSSR of dispatcher domain provides the major functions associated with the suspension and resumption of tasks. (See How tasks are made to wait.) The level-1 trace points DS 0004 and DS 0005 are produced on entry to, and exit from, the gate.
You need, therefore, to select tracing to capture the DS level-1 trace entries to investigate any wait problem. You need to capture trace entries for other components as well, when you know what functional areas are involved. The functional area invoking the task wait might, for example, be terminal control (TC), or file control (FC). Level-1 tracing is often enough for these components. However, there are cases, such as those relating to VSAM I/O errors where level-2 trace is needed to examine the RPL as it is passed to VSAM.
Next, you need to ensure that tracing is done for the task that has the wait problem. At first select special tracing for just that task, and disable tracing for all other tasks by setting the master system trace flag off. Subsequently, you can select special tracing for other tasks as well if it becomes clear that they are implicated in the wait.
For new-style trace entries, which include those for point IDs DS 0004 and DS 0005, the function being traced is shown explicitly in the interpretation string. The functions that can cause a task to enter a wait state are identified in the table. Look out for these in particular in the trace entries for any waiting task you are investigating.
Each function has its own set of input and output parameters, and these, too, are shown in the interpretation strings of the formatted trace entries. Input parameters are shown in the trace entries made from point ID DS 0004, and output parameters in the trace entries made from point ID DS 0005.
The values of the parameters can provide valuable information about task waits, so pay particular attention to them when you study the trace table.
If you are suitably authorized, you can request a CICS system dump using CEMT PERFORM DUMP, CEMT PERFORM SNAP, or EXEC CICS PERFORM DUMP. Make sure that the task in question is waiting when you take the dump, so that you capture information relevant to the wait.
You need to use the dump formatting keyword DS to format the dispatcher task summary. You probably need to look at other areas of the dump as well, so keep the dump on the dump data set.
The dispatcher task summary gives you information like that shown in Figure 5.
===DS: DISPATCHER DOMAIN - SUMMARY
KEY FOR SUMMARY
TY = TYPE OF TASK SY=SYSTEM NS=NON-SYSTEM
S = STATE OF TASK DIS=DISPATCHABLE SUS=SUSPENDED RUN=RUNNING REE=RESUMED EARLY
P = PURGEABLE WAIT/SUSPEND Y=YES N=NO
PS = PURGE STATUS OK=NO PURGE PU=PURGED PP=PURGE PENDING
TT = TIMEOUT TYPE IN=INTERVAL DD=DEADLOCK DELAYED DI=DEADLOCK IMMEDIATE
ST = SUSPEND TYPE MVS=WAIT_MVS SUSP=SUSPEND OLDC=WAIT_OLDC OLDW=WAIT_OLDW
DTA= DISPATCHER TASK AREA
AD = ATTACHING DOMAIN
MO = TASK MODE CO=CONCURRENT QR=QUASI-REENTRANT RO=RESOURCE OWNING SZ=FEPI OWNING
RP=ONC/RPC OWNING FO=FILE OWNING
DS_TOKEN KE_TASK TY S P PS TT RESOURCE RESOURCE ST TIME OF TIMEOUT DTA AD ATTACHER MO SUSPAREA XM_TXN_TOKEN
TYPE NAME SUSPEND DUE (DSTSK) TOKEN
00000001 06A23900 SY SUS N OK - ENF NOTIFY MVS 17:50:10.056 - 06B69080 DM 06C33640 RO 06C33658
00020003 06A23580 SY SUS N OK - SUSP 17:51:23.515 - 06B69180 XM 06C06690 QR 06B69180 06C066900000022C
00040003 06A23200 SY SUS N OK - TIEXPIRY DS_NUDGE SUSP 17:50:35.325 - 06B69280 TI 003D0003 QR 06B6A530
00820003 06A30900 SY SUS N OK - ICEXPIRY DFHAPTIX SUSP 17:50:35.326 - 06B89180 XM 06C06360 QR 06B89180 06C063600000006C
00900003 06A30C80 SY SUS N OK - ICMIDNTE DFHAPTIM SUSP 17:50:29.437 - 06B89880 XM 06C06250 QR 06B89880 06C062500000005C
00940003 06B5FC80 SY SUS N OK - TCP_NORM DFHZDSP OLDW 17:51:46.369 - 06B89A80 XM 06C06470 QR 0004FC10 06C064700000008C
00980001 06C3D080 SY SUS N OK IN SMSYSTEM SUSP 17:50:10.081 17:55:10.081 06B89C80 SM 00000002 QR 06B6A580
0200000B 07128B00 NS SUS Y OK - ZCIOWAIT DFHZARQ1 SUSP 17:51:39.616 - 06B9C080 XM 06C06580 QR 06B9C080 06C065800000029C
02020009 07135080 NS RUN 06B9C180 XM 06C06140 QR 06C061400000031C
A brief explanation of the summary information is given in the dump. A more detailed explanation is given in the section that follows.
Detailed descriptions of the fields in the dispatcher task summary are given in Table 3. Some of the fields relate to all tasks known to the dispatcher, and some (identified in the table) relate only to suspended tasks. Values are not provided in fields of the latter type for tasks that are not suspended.
Field | Description |
---|---|
AD | The 2-character domain index identifying the domain that attached the task to the dispatcher. |
ATTACHER TOKEN | A token provided by the domain that attached the task. This token uniquely identifies the task to the attaching domain. |
DS_TOKEN | A token given by the dispatcher to a domain that attaches a task. It identifies the attached task uniquely to the dispatcher. |
DTA | An address used internally by the dispatcher. |
KE_TASK | A value that uniquely identifies to the kernel a task that has been created. |
MO | The dispatching mode for the task. There is an MVS™ TCB for each
mode. All tasks in a given mode are running, or will run next, under the
TCB corresponding to the mode. The possible values are:
|
P | Whether at the time of the suspend call the control block
containing the definition of the transaction specified SPURGE(YES) or SPURGE(NO).
SPURGE(NO) inhibits deadlock time-out, CEMT SET TASK PURGE, and EXEC CICS
SET TASK PURGE.
The possible values are:
|
PS | The purge status of the task. The possible values are:
|
RESOURCE NAME (suspended tasks only) | The name of the resource that a suspended task is waiting for. A value is given only if RESOURCE_NAME has been included as an input parameter on the suspend call. |
RESOURCE TYPE (suspended tasks only) | The type of the resource that the task is waiting for. A value is given only if RESOURCE_TYPE has been included as an input parameter on the suspend call. |
S | The state of the task within the dispatcher. Possible
values are:
|
ST (suspended tasks only) | The type of function that was invoked to suspend a currently
suspended task. Possible values include:
|
SUSPAREA (suspended tasks only) | Either an address used internally by the dispatcher, or
an ECB address, or an ECB list address. These are the cases:
|
TIME OF SUSPEND (suspended tasks only) | The time when a currently suspended task was suspended.
The format is hh:mm:ss.mmm (hours, minutes, seconds, milliseconds), GMT. |
TIMEOUT DUE (suspended tasks only) | The time that a suspended task is due to timeout, if a
timeout interval has been specified. A suspended task only times out if it
is not resumed before this time arrives.
The format is hh:mm:ss.mmm (hours, minutes, seconds, milliseconds). |
TT (suspended tasks only) | The time-out type for the task. The possible values,
where one is given, are:
|
TY | Whether this is a system task or a non-system task. Possible
values are:
|
Many of the values shown in the dispatcher task summary are provided directly by parameters included on calls to and from the dispatcher. If you are using trace, you can see the values of the parameters in the trace entries, and this can be useful for debugging. For details of how you can use trace to investigate waits, see Investigating waits--using trace .
Table 4 shows the parameters that set task summary fields, the functions that use those parameters, and the domain gates that provide the functions. Task summary fields that are not set by parameters are also identified (by none in "Related parameter" column).
Field | Related parameter | Function | Input or output | Gate |
---|---|---|---|---|
AD | DOMAIN_INDEX |
INQUIRE_TASK GET_NEXT |
IN OUT |
DSBR |
DTA | ATTACH_TOKEN | CREATE_TASK | IN | KEDS |
DS_TOKEN | TASK_TOKEN |
ATTACH CANCEL_TASK PURGE_INHIBIT_QUERY SET_PRIORITY TASK_REPLY GET_NEXT INQUIRE_TASK |
OUT IN IN IN IN OUT OUT |
DSAT DSBR |
KE_TASK | TASK_TOKEN |
CREATE_TASK CREATE_TCB PUSH_TASK TASK_REPLY TCB_REPLY |
OUT OUT IN IN IN |
KEDS |
MO | MODE |
ATTACH CHANGE_MODE GET_NEXT INQUIRE_TASK |
IN IN OUT OUT |
DSAT DSBR |
P | PURGEABLE |
SUSPEND WAIT_MVS WAIT_OLDC WAIT_OLDW |
IN IN IN IN |
DSSR |
PS | none | |||
RESOURCE NAME | RESOURCE_NAME |
ADD_SUSPEND SUSPEND WAIT_MVS WAIT_OLDC WAIT_OLDW GET_NEXT INQUIRE_TASK |
IN IN IN IN IN OUT OUT |
DSSR DSBR |
RESOURCE TYPE | RESOURCE_TYPE |
ADD_SUSPEND SUSPEND WAIT_MVS WAIT_OLDC WAIT_OLDW GET_NEXT INQUIRE_TASK |
IN IN IN IN IN OUT OUT |
DSSR DSBR |
S (see note 1) |
STATE |
GET_NEXT INQUIRE_TASK |
OUT OUT |
DSBR |
SUSPAREA (see note 2) |
ECB_ADDRESS or ECB_LIST_ADDRESS (see note 3) |
WAIT_MVS WAIT_OLDC WAIT_OLDW |
IN IN IN |
DSSR |
TIME OF SUSPEND | none | |||
TASKNO | none | |||
TIMEOUT DUE (see note 4) |
none | |||
TT |
INTERVAL and DEADLOCK_ACTION |
SUSPEND WAIT_MVS WAIT_OLDW WAIT_OLDC |
IN IN IN IN |
DSSR |
TY | none | |||
ATTACHER TOKEN | USER_TOKEN |
ATTACH PURGE_INHIBIT_QUERY TASK_REPLY GET_NEXT INQUIRE_TASK |
OUT OUT |
DSAT DSBR |
ST | none |