Implementing an event store for the application

An event store is a persistent cache in the application where event records are saved until the connector can process them. The event store might be a database table, application event queue, email inbox, or any type of persistent store. If the connector is not operational, a persistent event store enables the application to detect and save event records until the connector becomes operational.

This section provides the following information about an event store:

Standard contents of an event record

Event records must encapsulate everything a connector needs to process an event. Each event record should include enough information that the connector poll method can retrieve the event data and build a business object that represents the event.

Note:
Although different event retrieval mechanisms might exist, this section describes event records in the context of the most common mechanism, polling.

If the application provides an event detection mechanism that writes event records to an event store, the event record should provide discrete detail on the object and verb. If the application does not provide sufficient detail, it might be possible to configure it to provide this level of detail.

Table 39 lists the standard elements for event records. The sections that follow include more information on certain fields.

Table 39. Standard elements of an event record

Element Description For more information
Event identifier (ID)
A unique identifier for the event. "Event identifier"
Business object name
The name of the business object definition as it appears in the repository. "Business object name"
Verb
The name of the verb, such as Create, Update, or Delete. "Event verb"
Object key The primary key for the application entity. "Object key"
Priority
The priority of the event in the range 0 - n, where 0 is the highest priority. "Processing events by event priority"
Timestamp
The time at which the application generated the event. None.
Status
The status of the event. This is used for archiving events. "Event status"
Description
A text string describing the event. None
Connector identifier (ID)
An identifier for the connector that will process the event. "Event distribution"
Note:
A minimal set of information in an event record includes the event ID, business object name, verb, and object key. You may also want to set a priority for an event so that if large numbers of events are queued in the event store, the connector can select events in order of priority.

Business object name

You can use the name of the business object definition to check for event subscriptions. Note that the event record should specify the exact name of the business object definition, such as SAP_Customer rather than Customer.

Event verb

The verb represents the kind of event that occurred in the application, such as Create, Update, or Delete. You can use the verb to check for event subscriptions.

Note:
Events that represent deletion of application data should generate event records with the Delete verb. This is true even for logical delete operations, where the delete is an update of a status value to inactive. For more information, see "Processing Delete events".

The verb that the connector sets in the business object should be same verb that was specified in the event record.

Object key

The entity's object key enables the connector to retrieve the full set of entity data if the object has subscribing events.

Note:
The only data from the application entity that event records should include are the business object name, active verb, and object key. Storing additional entity data in the event store requires memory and processing time that might be unneeded if no subscriptions exist for the event.

The object key column must use name/value pairs to set data in the event record. For example, if ContractId is the name of an attribute in the business object, the object key field in the event record would be:

ContractId=45381
 

Depending on the application, the object key may be a concatenation of several fields. Therefore, the connector should support multiple name/value pairs that are separated by a delimiter, for example ContractId=45381:HeaderId=321. The delimiter should be configurable as set by the PollAttributeDelimiter connector configuration property. The default value for the delimiter is a colon (:).

Event identifier

Each event must have a unique identifier. This identifier can be an number generated by the application or a number generated by a scheme that your connector uses. As an example of an event ID numbering scheme, the event may generate a sequential identifier, such as 00123, to which the connector adds its name. The resulting object event ID is ConnectorName_00123. Another technique might be to generate a timestamp, resulting in an identifier such as ConnectorName_06139833001001.

Your connector can optionally store the event ID in the ObjectEventId attribute in a business object. The ObjectEventId attribute is a unique value that identifies each event in the IBM WebSphere business integration system. Because this attribute is required, the connector framework generates a value for it if the application-specific connector does not provide a value. If no values for ObjectEventIds are provided for hierarchical business objects, the connector framework generates values for the parent business object and for each child. If the connector generates ObjectEventId values for hierarchical business objects, each value must be unique across all business objects in the hierarchy regardless of level.

Event status

IBM recommends that a C++ connector use the event-status values that Table 40 lists. To improve readability of your C++ code, you might want to define constants for each of these event-status values.

Table 40. Suggested event-status values for a C++ connector

Event-status value Description
0 Ready for poll
1 Sent to the integration broker
2 No subscriptions for event
3 Event is in progress
-1 Error in processing the event. A description of the error can be appended to the event description in the event record.
-2 Error in sending the event to the integration broker. A description of the error can be appended to the event description in the event record.

Possible implementations of an event store

The application might use any of the following as the event store:

Note:
Some applications might provide multiple ways of keeping track of changes to application entities. For example, an application might provide workflow for some database tables and user exits for other tables. If this is the case, you may have to piece together an event notification mechanism that handles events in one way for some business objects and another way for other business objects.

Event inbox

Some applications have a built-in inbox mechanism. This inbox mechanism can be used to transfer information about application events to the connector, as follows:

Figure 45 illustrates this interaction.

Figure 45. An event inbox as an event store

Event table

An application can use its application database to store event information. It can create a special event table in this database to use as the event store for event records. This table is created during the installation of the connector. With an event table as an event store:

Figure 46 illustrates this interaction.

Figure 46. An event table as an event store

Note:
Avoid full table scans of existing application tables as a way of determining whether application tables have changed. The recommended approach is to populate an event table with event information and poll the event table.

If your connector supports archiving of events, you can also create an archive table in the application database to hold the archived events. Table 41 shows a recommended schema for event and archive tables. You can extend this schema as needed for your application.

Table 41. Recommended schema for event and archive tables

Column name Type Description
event_id Use appropriate type for database The unique key for the event. System constraints determine format.
object_name Char 80 Complete name of the business object.
object_verb Char 80 Event verb.
object_key Char 80 The primary key of the object.
event_priority Integer The priority of the event, where 0 is the highest priority.
event_time DateTime The timestamp for the event (time at which the event occurred).
event_processed DateTime For the archive table only. The time at which the event was handed to the connector framework.
event_status Integer For possible status values, see Event status.
event_description Char 255 Event description or error string
connector_id Integer Id for the connector (if applicable)

Email

You can use an email system as an event store:

Figure 47 illustrates this interaction.

Figure 47. A mailbox as an event store

For an email-based event store, the mailbox used for a connector must be configurable, and the actual name of the inbox used should reflect its usage. The following list specifies the format and recommended names for fields in event messages.

Flat files

If no other event detection mechanism is available, it might be possible to set up an event store using flat files. With this type of event store:

If the file is not directly accessible by the connector (if, for example, it was generated on a mainframe system), the file must be transferred to a location that the connector can access. One way of transferring files is to use File Transfer Protocol (FTP). This can be done either internally in the connector or using an external tool to copy the file from one location to another. There are other ways to transfer information between files; the approach that you choose depends on your application and connector.

Figure 48 illustrates event detection and retrieval using flat files. In this example, FTP is used to transfer the event information to a location accessible by the connector.

Figure 48. Retrieving event records from flat files


Copyright IBM Corp. 1997, 2003