How the ABAP Extension Module works

Most of the functionality provided by the ABAP Extension Module occurs inside of the SAP application. For most of the virtual functions that every connector must implement, there is a corresponding ABAP function module in the SAP application. However, SAP does not provide ABAP function modules that support the specific requirements of the init(), doVerbFor(), and pollForEvents() methods, so these function modules have been developed and delivered as part of the connector module. While the Java components provide some functionality, the majority of the processing for these methods is done by the ABAP components in the SAP application.

Table 6 shows the virtual Java methods that the connector module implements and their corresponding ABAP components. Keep in mind that this is not a complete list of the ABAP components used by the connector.

Table 6. Java components and their corresponding ABAP components

Java components ABAP components
doVerbFor() /CWLD/RFC_DO_VERB_NEXTGEN
getVersion() No implementation required
getBOHandlerForBO No implementation required
init() /CWLD/RFC_LOGON
pollForEvents() /CWLD/RFC_EVENT_REQUEST /CWLD/RFC_EVENT_RETURN
terminate() No implementation required

Together, these ABAP function modules are the core of the ABAP Extension Module. The following sections describe connector initialization, business object processing, and how the connector handles event notification.

The implemented functions are discussed in the rest of this chapter.

Initialization

The init() method calls the ABAP function module /CWLD/RFC_LOGON to validate that the destination SAP application is running and that the RFC library can be used to execute ABAP function modules. The /CWLD/RFC_LOGON function module is also called to process all of the in-progress events. All events in the event table that are marked with a status of event retrieved (status marked as R in the event table) will be processed based on the InDoubtEvents Connector Property. The default property value is Ignore. When event distribution is being used only the events belonging to that particular connector and server with a status of 'R' will be handled according to the connector property. If event distribution is not being used then all events with a status of 'R' will be handled according to the connector property. If the connector property equals reprocess, these events will be changed to a status of queued (marked as Q in the event table). When the connector polls for events, all events of 'Q' status will be processed, using /CWLD/RFC_E VENT_REQUEST function module. If the connector property is equal to FailOnStartUp, a fatal error is logged within the SAP log and the local log file and the connector will shut down. An email is also sent notifying the user a fatal error has occurred. If the connector property is equal to LogError an error is logged within the SAP log and the local log file. The in-progress events are not processed and the connector does not shut down. If the connector property is equal to Ignore, the in-progress events are ignored and the connector polls as if there weren't any in-progress events in the event table.

If the function module does not execute successfully, the connector terminates.

Business object processing

All service call requests for SAP are initiated by the doVerbFor() method in the Java component of the connector module. The connector's ABAP function module /CWLD/RFC_DO_VERB_NEXTGEN and an ABAP handler in the ABAP component of the connector module handle the requests.

Figure 5 illustrates business object processing.

Figure 5. Business object processing of doVerbFor()

doVerbFor()

In the Java component of the connector module, the doVerbFor() method of a single business object handler implementation handles all of the business object requests from the integration broker and all business object events from the pollForEvents() method. In either case, doVerbFor() executes in the following manner:

  1. Converts an instance of a WebSphere business object for SAP to a single, predefined flat structure that contains the business object data.
  2. Calls the ABAP function module /CWLD/RFC_DO_VERB_NEXTGEN, passes the business object data to it, and then waits for business object data to be returned.
  3. Converts the returned business object data back into a WebSphere business object.

The doVerbFor() method passes business object data to function module /CWLD/RFC_DO_VERB_NEXTGEN and then creates an entirely new business object structure from the returned business object data.

/CWLD/RFC_DO_VERB_NEXTGEN

In the ABAP component of the connector module, the connector's ABAP function module /CWLD/RFC_DO_VERB_NEXTGEN is responsible for handling all WebSphere business object processing in the SAP application. Specifically, it routes business object data to the appropriate ABAP handler. In this sense, function module /CWLD/RFC_DO_VERB_NEXTGEN can be thought of as a business object router. It executes in the following manner:

  1. Receives a business object.
  2. Dynamically calls an ABAP handler to process the business object data and passes the business object data as a parameter.
  3. Receives business object data from an ABAP handler and returns it to the requesting call.

/CWLD/RFC_DO_VERB_NEXTGEN uses ABAP handlers to fulfill each object type and verb-specific request. /CWLD/RFC_DO_VERB_NEXTGEN uses the value in a business object's verb application-specific information to determine which ABAP handler to call. It also checks for the archive status. /CWLD/RFC_DO_VERB_NEXTGEN can be thought of as a router from the doVerbFor() method to an ABAP handler.

ABAP handlers

ABAP handlers are unique to the connector module in that they extend the business object handler functionality from the Java component of the connector module. ABAP handlers reside in the SAP application as ABAP function modules and communicate directly with /CWLD/RFC_DO_VERB_NEXTGEN. ABAP handlers are needed to get business object data into or out of the SAP application database.

Figure 6 illustrates the business object processing components of the ABAP Extension Module and their relationship to one another. Notice that for a single business object handler (doVerbFor()) and business object router (/CWLD/RFC_DO_VERB_NEXTGEN), there are multiple ABAP handlers.

Figure 6. Adapter-provided business object processing components

ABAP handlers are responsible for adding business object data into the SAP application database (Create, Update, Delete) or for using the business object data as the keys to retrieving data from the SAP application database (Retrieve).

The adapter provides generic ABAP handlers. For example, function module /CWLD/DYNAMIC_TRANSACTION supports flat business objects for Create, Update, Delete, and Retrieve operations.

The WebSphere business integration system provides a metadata repository and the adapter provides a generic ABAP handler to support flat business objects. The adapter also provides an ABAP handler (/CWLD/IDOC_HANDLER) to support hierarchical business objects; however, you must develop an additional business-object-specific ABAP handler for each hierarchical business object that you need to support.

The WebSphere business integration system provides tools that facilitate the development process. For more information on developing business objects and ABAP handlers, see Developing business objects for the ABAP Extension module and Appendix E, Generating business object definitions using SAPODA.

Event notification

Event notification refers to the collection of processes that notify the connector of SAP application object events. Notification includes, but is not limited to the type of the event (object and verb) and the data key required for the external system to retrieve the associated data.

Figure 7 illustrates the event notification process, which uses the pollForEvents() method.

Figure 7. Event notification process

Event notification for the connector consists of two functions:

Event polling

Event polling consists of three functions that are carried out by the pollForEvents() method:

Note:
The roles of these functions are distributed in the Java and ABAP components. However, the Java component always initiates event polling.
Event request

Event request is the process of polling and retrieving events from the event table in the SAP application. The event request mechanism of the Java component has a counterpart function module in the SAP application, /CWLD/RFC_EVENT_REQUEST. This function retrieves events from the connector's ABAP event table, /CWLD/EVT_CUR.

Every triggered event enters the event table with an initial status of prequeued (status marked as P in the event table) and a default event priority of zero. Before an event can be processed, its status must be changed to queued (Q in the event table). The priority of an event must be zero before the connector retrieves the full object that it represents. For more information on event priority, see ***.

The status of an event changes from prequeued to queued if there are no database locks for the combination of the user who created the event and the event's key. After the event has been retrieved from the event table the status of the event is updated to event retrieved (R in the event table. If locks exist, the status of the event is set to locked (L in the event table) and the event is requeued. An ABAP constant, C_MAXIMUM_REQUEUE, defines of the number of times that an event can be requeued. If the maximum number (defaulted to 100) is attained, then the event is archived to the event archive table.

Note:
Every event with a prequeued or locked status is updated with every poll. You can run into performance issues when events are triggered in batches. You can configure the polling frequency using the PollFrequency configuration property. For more information, see Appendix B, Standard configuration properties for connectors.

After preprocessing all prequeued events, the ABAP function module /CWLD/RFC_EVENT_REQUEST selects the events to return to the event request method in the Java component of the connector module (only events with a status of queued can be selected). The connector-specific configuration property PollQuantity (defaulted to 20) determines the maximum number of events returned for a single poll. For more information, see Installing and configuring the connector.

The event request mechanism performs the event selection process in two steps:

  1. Selects events dedicated to the connector and the integration broker.

    Events are dedicated to a specific integration broker in the event distribution table (/CWLD/EVT_DIS). The name of the integration broker specified in this table must match the name specified in the shortcut that starts the connector. For example, the standard shortcut for an SAP connector running on Windows has the format:

    ...\start_SAP.bat SAPconnectorName integrationBrokerName -cConfigFileName
     
    

    When a message broker is the integration broker, the WebSphere business integration system identifies the integration broker specified in the event distribution table by getting values from the connector's startup command:

  2. If fewer than the maximum number of events have been selected, pulls the balance from the events that are not configured for event distribution.

    For example, if the connector-specific configuration property PollQuantity is kept at 20 and there are 8 events dedicated to the specific connector and the integration broker, the mechanism selects 12 additional events.

    When a message broker is the integration broker and only one Queue Manager has been configured, the names of the queues must be unique for each instance of the integration broker. When a message broker is the integration broker and a cluster has been configured, the names of the queues must be unique for each integration broker within the cluster.

    If desired, you can incorporate the name of the broker (as specified in the integrationBrokerName parameter of the startup command) or the name of the connector into the names of the queues. For example, if two brokers are named WMQI1 and WMQI2, their respective ADMINOUTQUEUEs might be named ADMINOUTQUEUE_MQI1 and ADMINOUTQUEUE_MQI2, respectively.

    Important:
    If you set up multiple connectors to poll, you must configure every event to be processed by only one connector. Otherwise the connector may send duplicate events, or may archive events instead of retrieving them.
Event processing

The event request function produces an array of events to be processed from the /CWLD/EVT_CUR event table. It passes these events to the event processing function, which handles them one at a time in the following manner:

  1. Evaluates if the event is in the connector subscription list using the object.verb value.

    If an event is not in the subscription list, sets the status of the event to not subscribed.

  2. Creates a parentObjectOnly.Retrieve business object if an event is in the subscription list. The event processing function sets the key value in one of the following ways:
  3. Invokes doVerbFor() and passes the business object data to it. Once the business object is passed, event processing waits for business object data to return.
  4. Updates the status of the event array based on the doVerbFor() processing.
  5. Delivers the business object data to the integration broker if the business object data is successfully retrieved.
Event return

After each event is processed by event request, it is returned to the SAP application using function module /CWLD/RFC_EVENT_RETURN. This function module makes a copy of the processed event, adds it to the event archive table (/CWLD/EVT_ARC), and then deletes the original entry from the event table.

Note:
Events with their new status are all updated after each event is processed.

Archived events include successfully processed events, events that were processed but terminated in an error, and unsubscribed events. Each event has a status that can indicate one of the following conditions:

Use the IBM CrossWorlds Station tool in the SAP application to administer the event archive table. IBM CrossWorlds Station enables an administrator to display and truncate the archive table and to resubmit events for processing. For more information about maintaining the archive table and setting up log truncation, see Managing the ABAP Extension module.

Event triggering

The connector is event-driven. In order to get events out of the SAP application, you need to implement an event triggering mechanism for each IBM WebSphere-supported business object. Event triggering for the connector comprises three functions:

Event detection

Event detection is the process of identifying that an event was generated in the SAP application. Typically, connectors use database triggers to detect an event. However, because the SAP application is tightly integrated with the SAP database, SAP allows very limited access for direct modifications to its database. Therefore, the event detection mechanisms are implemented in the application transaction layer above the database.

The IBM WebSphere Business Integration Adapter for mySAP.com commonly uses four mechanisms to detect an event in the SAP application:

All of these event detection mechanisms support real-time triggering and retrieval of objects. In addition, code enhancements and batch programs provide functionality to delay the retrieval of events. An event whose retrieval is delayed is called a future event. For more information on triggering future events, see Event triggering.

Note:
Each event detection mechanism has advantages and disadvantages that need to be considered when designing and developing a business object trigger. For more information on implementing an event detection mechanism, see Developing event detection for the ABAP Extension module.

Keep in mind that these are only a few examples of event detection mechanisms. There are many different ways to detect events.

Event triggering

Once an event is identified by one of the event detection mechanisms, it is triggered using one of the adapter-delivered event triggers.

Note:
Both functions are for real-time triggering. /CWLD/ADD_TO_QUEUE processes events immediately and /CWLD/ADD_TO_QUEUE processes events at a later time

If the event will be triggered in real-time, then /CWLD/ADD_TO_QUEUE commits the event to the current event table (/CWLD/EVT_CUR). Specifically, it adds a row of data for the object name, verb, and key that represents the event.

Figure 8 illustrates events triggered by /CWLD/ADD_TO_QUEUE.

Figure 8. /CWLD/ADD_TO_QUEUE

If an event needs to be processed at a future date, then /CWLD/ADD_TO_QUEUE_IN_FUTURE commits the event to the future event table (/CWLD/EVT_FUT). Specifically, it adds a row of data for the object name, verb, and key that represents the event. In addition, it adds a Date row which is read by the adapter-delivered batch program /CWLD/SUBMIT_FUTURE_EVENTS. This batch program can be scheduled to retrieve events from the future event table. Once it retrieves an event, it calls /CWLD/ADD_TO_QUEUE to trigger the event to the current event table.

Note:
/CWLD/ADD_TO_QUEUE_IN_FUTURE uses the system date as the current date when it populates the Date row of the future event table.

Figure 9 illustrates events triggered by /CWLD/ADD_TO_QUEUE_IN_FUTURE.

Figure 9. /CWLD/ADD_TO_QUEUE_IN_FUTURE

For more information on triggering events for the future event table, see Developing event detection for the ABAP Extension module.

All events are added to the current event table using /CWLD/ADD_TO_QUEUE. In addition to adding a row of data to the current event table, /CWLD/ADD_TO_QUEUE can be set up for:

Event filtering, event distribution, and event priority are executed as part of the event trigger and by no other program. They result in either the event's restriction (filtering), or modification (event distribution and event prioritization).

Event filtering
The event trigger can be used to filter out events that you do not want added to the event table. The adapter provides an ABAP include program (/CWLD/TRIGGERING_RESTRICTIONS) that enables you to restrict specific events for this purpose.
Event distribution
Load balancing can be used to distribute event processing across multiple connectors allowing you to process multiple events at the same time. The event trigger provides this capability through the event distribution table (/CWLD/EVT_DIS). You can dedicate business objects to be retrieved by a specific connector. Also, event distribution can take a single event and replicate it one or more times for each subscribed combination of connector and integration broker.

Attention: If you are using multiple connectors to poll, you must dedicate every subscribed event to a specific connector. Failure to do so may result in duplicate events delivered. You must guarantee that there is no dependency between objects dedicated to different connectors, because this may result in events being delivered out of sequence.

For example, assume you have a single integration broker named CrossWorlds1 that subscribes to two different business objects, BO_A and BO_B. The BO_A business object is small and can be retrieved quickly whereas BO_B is large and takes much longer to retrieve. With two connectors polling, SAP1connector and SAP2connector, you can set up the event distribution table so that SAP1connector retrieves BO_A and SAP2connector retrieves BO_B. SAP1connector can continuously poll small objects of type A, while SAP2connector focuses on the larger type B objects.

Note:
For information on how the WebSphere business integration system identifies each unique instance of a message broker integration broker, see Event request.
Important:
If the event distribution table is not configured for a specific object, then each event triggered for that object is available for any combination of connector and integration broker.
Event priority
You can set the event priority for each combination of business object, connector, and integration broker by delaying the retrieval of events. An event's priority indicates the number of polls that are needed before the event is picked up for delivery. For example, if you set the priority of an event to 10, then the connector polls the event table ten times before the event is retrieved. Each time the connector polls, the priority value is reduced by one until it reaches zero.

By default, all events are given a priority of zero. An object's priority is configured in the same ABAP table as event distribution.

Figure 10 illustrates the event triggering functionality inside the SAP application. The events E1, E2, and E3 are received by the event trigger /CWLD/ADD_TO_QUEUE. E1 represents a Customer event and E3 represents an Order event. Event distribution is set up so that all Customer objects are handled by SAP1connector and all Order objects are handled by SAP2connector. In this environment, both connectors use the same integration broker. Because E1 is a Customer object, it is polled by SAP1connector and because E3 is an Order object, it is polled by SAP2connector. E2 is an Inventory object that is filtered out by code in the restriction program /CWLD/TRIGGERING_RESTRICTIONS that restricts inventory objects to a specific warehouse.

Figure 10. Event priority with function module /CWLD/ADD_TO_QUEUE

Event persistence

Once the event trigger inserts an event into the event table, the event is committed to the database with its event distribution and event priority values set. At this time, only polling can modify the event. When the event polling processes is completed, meaning the event was retrieved from the SAP application and processed by the Java component of the connector, a copy of the processed event is added to the event archive table (/CWLD/EVT_ARC). The original event is then deleted from the event table.

Note:
You can resubmit an event from the archive table. Keep in mind that the event is simply moved to the event table and is not triggered again. Specifically, it does not pass back through event filtering, event distribution, and event priority.

Copyright IBM Corp. 1997, 2004