Batch data stream framework and patterns

Compute Grid provides a batch data stream (BDS) framework that includes pre-built code to work with popular streams like text, byte, database, datasets, and so on. You can implement an interface where the business logic for processing the stream is added. The pre-built code manages the opening, closing, externalizing/internalizing checkpoints, and so on.

newfeat: You can use this feature if you have Version 6.1.0.3 or later installed.

BDS framework patterns

A BDS framework pattern is a simple Java TM interface for a particular type of data stream that a user implements to insert business logic. The BDS framework has several supporting classes for each pattern that do most of the mundane tasks related to stream management. The following table shows the patterns that Compute Grid provides:

Pattern name Description Supporting classes

JDBCReaderPattern

Used to retrieve data from a database using a JDBC connection.

  • LocalJDBCReader
  • JDBCReader
  • CursorHoldableJDBCReader

JDBCWriterPattern

Used to write data to a database using a JDBC connection.

  • LocalJDBCWriter
  • JDBCWriter

ByteReaderPattern

Used to read byte data from a file.

FileByteReader

ByteWriterPattern

Used to write byte data from a file.

FileByteWriter

FileReaderPattern

Used to read a text file.

TextFileReader

FileWriterPattern

Used to write to a text file.

TextFileWriter

[For z/OS operating system] RecordOrientedDatasetReaderPattern

Used to read a z/OS dataset.

  • ZFileStreamOrientedTextReader
  • ZFileStreamOrientedByteReader
  • ZFileRecordOrientedDataReader

[For z/OS operating system] RecordOrientedDataSetWriterPattern

Used to write to a z/OS dataset.

  • ZFileStreamOrientedTextWriter
  • ZFileStreamOrientedByteWriter
  • ZFileRecordOrientedDataReader

JPAReaderPattern

Used to retrieve data from a database using OpenJPA

JPAReader

JPAWriterPattern

Used to write data to a database using a Java Persistence API (JPA) connection.

JPAWriter

BDS framework steps

BDS framework steps minimize the amount of work needed to create a new batch step by performing the bookkeeping tasks related to step management and delegating the business logic to a class implemented by the user.

Step Description

Generic batch step (GenericXDBatchStep)

A simple step that uses one input and one output stream.

Error Tolerant step

A simple step that uses one input, one output stream, and one error stream..

ThresholdPolicies

Step Description

RecordBasedThresholdPolicy

This policy provides a Compute Grid implementation of the ThresholdPolicy interface..

PercentageBasedThresholdPolicy

This policy provides a Compute Grid implementation of the ThresholdPolicy interface