gtpc3m24Concepts and Structures

TPF Collection Support

TPF collection support (TPFCS) is a service for managing the storage and retrieval of data on a TPF database. The data is stored in units known as collections. Collections are abstract representations of data having common attributes and functions. Persistent collections maintain their state after the entry control block (ECB) that creates them exits.

TPFCS can be considered an application development tool that integrates database functionality with the application. Potentially complicated data manipulation routines are not needed in application programs because their functionality is already included in the TPF 4.1 system. Furthermore, the TPF 4.1 system does not need to have any knowledge of the format of the data, so more control is given to the application and taken away from the TPF system. TPFCS provides a single, client-level API for storing data elements in a database regardless of the format of the data.

The following table lists topics related to TPFCS and where this information is found:

Table 9. TPFCS Information

Type of TPFCS Information Publication
Application programming TPF Application Programming
Application programming interfaces (APIs) TPF C/C++ Language Support User's Guide
Database information TPF Database Reference
Commands TPF Operations
General overview information TPF Concepts and Structures
Terms and definitions TPF Library Guide
Messages (online, system) Messages (System Error and Offline) and Messages (Online)
Migration information TPF Migration Guide: Program Update Tapes

Benefits of TPFCS

TPFCS primarily allows you to facilitate the implementation of problem solutions. This generic benefit can be broken down into the following, and will be discussed in more detail:

First, several different abstract data representations are provided, and for each representation, many predefined data manipulation routines are available. Together, all of these routines form a standardized C (C++) library of APIs. As a result, time is not needed on designing new data models and corresponding functions from the beginning. This allows an application to be completed much faster than it would be otherwise. Code is reused over and over, saving time and money.

Second, application quality can be greatly improved by using the TPFCS library. It is less likely that problems will be found with code that has been used several times before than if new code was written. Furthermore, TPFCS APIs hide the low-level TPF interfaces, including ECB usage, data event control block (DECB) usage, and file address information. Because less TPF-specific skills are needed, education time and costs are also reduced.

Next, using TPFCS can improve database integrity. With different collection types (see Types of Collections) and corresponding data manipulation operations available, you are able to find an abstract representation that best matches your data and use a set of consistent APIs with that data. Furthermore, by using this support, it is possible for TPFCS database users to access and retrieve information from the collection without having to retrieve the entire collection.

In addition, TPFCS provides several different locking mechanisms for concurrency control (see Concurrency Controls) to ensure that the database remains in a consistent state at all times. Furthermore, utilities are available for database maintenance and testing. This includes utilities to examine data, repair data that may be corrupted, and capture and restore data.

Finally, TPFCS enables applications to control how data is recouped instead of requiring recoup descriptors to be set up in advance. Applications can create and modify recoup indexes that describe the layout of embedded chain-chase information dynamically.

TPFCS Database

TPFCS is a service that contains all the components that you need to access collections. The TPFCS database consists of user-defined data stores. In those data stores are collections. Each collection consists of elements. Elements can contain character data, binary data, structures, or references to other collections or TPF files.

To relate the concept to familiar terms, you might say that an element could be a record, a collection could be a file, and a data store could be a related set of files. All of these files put together are the TPFCS database. All files that the TPFCS database uses are created as collections and accessed as collections using the TPFCS APIs.

Figure 56 shows the general layout of a TPFCS database:

Figure 56. General Layout of a TPFCS Database


Data Stores

A data store (DS) contains collections. Some data stores are created when TPFCS is initialized; others are user-defined and user-named by entering the ZOODB command (see ZOODB Commands). See TPF Database Reference for more information about data stores.

Collection Overview

A collection is an abstract representation of data that allows you to manage a group of data elements that have common attributes. Collections are used to store and manage elements. Different collections have different internal structures and different functions for element storage and retrieval. See Types of Collections for information on collections supported by TPFCS.

All persistent collections are assigned an identification token called a persistent identifier (PID). The PID is architected as a 32-byte token consisting of a format indicator and other information used to locate the collection.

Collection Characteristics

Table 10 describes the four basic characteristics that are used to help you distinguish between the different collections:

Table 10. Collection Characteristics

Characteristic Description
Ordering Used when a next or previous relationship exists between elements.
Access by key Used when a separate key or a predefined part of the element is relevant for accessing an element in the collection.
Equality of elements Used when you need to test if an entire element value is included in a collection.
Uniqueness of entries Used when all elements in a collection must have a unique key or value.

Ordering of Collection Elements

The elements of a collection can be ordered in two ways:

Access by Key

A given collection can have a key defined for its elements. A key is an identifier that helps to organize and access elements in a collection. A key can be a data field of the element or it can also be assigned by the application program using some function. A primary key is a value that is separate from the data component of an element (although the same value could be stored in the element). An alternate key can be a data field within the element. Keys let you:

Key Path Support

There are two types of key paths: primary and alternate. Primary key paths are sorted in ascending binary value by the primary key and they can be either unique or nonunique depending on the collection type. Alternate key paths support nonunique key path fields (keys) and are sorted in ascending binary value based on the key path field. A maximum of 16 alternate key paths (in addition to the primary key path) can be defined one at a time for each collection.

Key path support enables you to search for and access data in a collection by using the value of a specified data field or key (known as the key path field). TPFCS automatically builds a key path based on the field values, which enables you to access data elements using that specific field.

Key path support provides the capability for TPFCS to support not only the primary, but alternate key paths for the following persistent keyed and sorted collections:

Note:
Sorted bag and sorted set collections do not have primary keys; they only have sort fields.

For information about using key paths, see TPF Application Programming.

Equality of Elements

A collection can have an equality relation defined for its elements. The default equality relation is based on the element as a whole, not just on one or more of its data members (for example, the key). For two elements to be equal, all data members of both elements must be equal. The equality relation is needed for functions such as those that locate or remove a given element. A collection that has an equality relation has element equality.

Uniqueness of Entries

For collections with a key, the terms unique and nonunique apply to the key; for collections with no key, the terms unique and nonunique apply to the element. In some collections such as key sorted set, set, key set, and sorted set, two element data components may be equal, but no two keys are equal. Such collections are called unique collections. Other collections, including sorted bag, bag, key bag, and key sorted bag can have two equal elements or elements with equal keys. Such collections are called nonunique collections.

Types of Collections

There are different collection types, each with predefined functions. Some functions work on all collection types, while other functions only work on a specific collection type.

The following table provides information about the types of collections provided by the collection library and explains some of the key concepts:

Table 11. Collection Type Summary

Collection Description
Array An array is an ordered, nonunique collection of elements with no key. Elements can be added but not inserted or removed.
Bag A bag is an unordered, nonunique collection of elements with element equality and no key.
BLOB A binary large object (BLOB) is a special array collection of elements with an element size of 1 byte but act as contiguous data. Operations can be performed on a range of elements. A BLOB is also known as a byte array.
Key Bag A key bag is an unordered, nonunique collection of elements that have a key.
Key Set A key set is an unordered collection of elements that have a unique key.
Key Sorted Bag A key sorted bag is an ordered collection of elements that have a nonunique key.
Key Sorted Set A key sorted set is an ordered collection of elements that have a unique key. A key sorted set is also known as a dictionary.
Keyed Log A keyed log is ordered by arrival sequence; when the collection is full, the collection will wrap and start overlaying elements at the start of the collection. The elements in a keyed log collection have a key.
Log A log is ordered by arrival sequence; when the collection is full, the collection wraps and starts overlaying elements at the start of the collection.
Sequence A sequence is an ordered, nonunique collection of elements with no key. Elements can be added, inserted, and removed.
Set A set is an unordered collection of elements with unique values, element equality, and no key.
Sorted Bag A sorted bag is an ordered, nonunique collection of elements with no key.
Sorted Set A sorted set is an ordered collection of elements with unique values and no key.

For more information about creating, deleting, and managing collections, see TPF Application Programming. For examples of each collection type, see TPF Database Reference.

Collection Access Support

The TPFCS database provides a set of services that allows collections to be created, accessed, stored, updated, and deleted. The TPFCS database can handle any collection from 0 to 2 147 483 648 elements in size. The TPFCS database handles the collection as a binary byte string. The collection size is limited only by the available amount of DASD 4-KB long-term pool space. The TPFCS database relies heavily on TPF long-term pool records.

The TPFCS database requires a single TPF record type, which is used to contain the actual collection store directory.

Data Definitions

A data definition (DD) is an integral part of a collection definition. A collection is created using a DD defined for a particular data store (DS). A DD provides attributes for collections in a DS. They are also used to assign record IDs (RIDs) to collections and to specify shadowing (see Shadowing).

See TPF Database Reference for more information about data definitions.

Properties

The TPFCS database supports a property service for persistent collections that are created. Properties are essentially typed named values that you can dynamically associate with an already existing persistent collection. Once these properties are defined, they can be named, their values can be set and obtained, their access modes can be set, and they can also be deleted. See TPF Database Reference for more information about properties.

Collection Lifetimes

TPFCS provides the following collection lifetimes:

Persistent long-term
Exists beyond the life of the creating ECB, resides on DASD in 4-KB long-term pools, and can survive a re-IPL. TPFCS reuses its own long-term pool addresses so applications do not flush through pool records too quickly and cause the TPF system to run out of long-term pool records.

Persistent short-term
Exists beyond the life of the creating ECB, resides on DASD in short-term pools, and can survive a re-IPL. Because the collection resides in short-term pools, it will be deleted when the short-term pools are recycled.

Temporary
Resides in the private heap area of the ECB, overflows to DASD in short-term pools, and cannot survive a re-IPL. It is deleted when the ECB exits because the private heap area of the ECB is reclaimed by the system when the ECB exits.

Cursors

A cursor is an internal structure that is used to reference an element in a collection. Cursors are used to iterate through collections and to establish locks on the target collection. Cursors can be used with any collection. Using a cursor provides several advantages:

To use the collections effectively, you often need cursor functions, iterator functions, or both. Cursor functions are used to reference an element in a collection. Iterator functions let you iterate over all elements of a collection and, for example, apply a certain function to all elements or iterate over the collection until you have found a certain element.

Cursors are also used to prevent concurrent updating of a collection while you are accessing elements in the collection.

For complete information about using cursors, see TPF Application Programming.

Database Integrity

This section provides information about TPFCS services that maintain database integrity.

Database Access

There are no restrictions on which users can access the TPFCS database. Database access is denied only if the data store you requested does not exist.

Concurrency Controls

TPFCS provides three levels of concurrency control:

  1. None (using nonlocking cursors)

    This type of concurrency control uses a nonlocking cursor. The cursor is used for read-only operations on elements in a collection without creating any type of interlock on the target collection. Because the information can change underneath the cursor, it is considered a dirty-read cursor.

  2. Optimistic (using update sequence counters)

    As a form of database integrity for those collections that allow element updating, an update sequence counter is stored in each element in a given collection and optimistic concurrency is enforced on the element during update processing. Optimistic concurrency is a way of controlling database access; a user can read and update a collection element without exclusive access to the collection.

  3. Pessimistic (using locking cursors)

    Pessimistic concurrency uses a locking cursor to create an exclusive lock on the collection. Write operations are only allowed by the ECB creating the locking cursor, and requests by other ECBs to create a lock or write data will be delayed until the locking cursor is deleted.

For more information about concurrency controls, see TPF Application Programming.

Dirty-Reader Protection

TPFCS provides dirty-reader protection by using the FILNC macro. FILNC will write all new or updated records to DASD before any referencing record is written. This is done to ensure that another user who is attempting to read the collection using a nonlocking cursor will be able to follow a whole chain. If the updates were filed in the wrong order, it could then be possible for the reader to follow a chain with either holes in it or the chain could point to records that were not even part of the collection.

For more information about the FILNC macro, see TPF General Macros.

TPF Transaction Services

TPFCS uses TPF transaction services to establish a commit scope on behalf of the caller for all update requests with the exception of cursor update requests. The application is responsible for commit scopes when using cursors. TPF transaction services will then either commit or roll back the change depending on the success of the request.

Note:
To determine which TPFCS functions use commit and rollback protocols, see the TPF C/C++ Language Support User's Guide.

Shadowing

TPFCS provides an option when a collection is created that allows you to specify that the collection is to be shadowed. When TPFCS shadows a collection, two copies of the collection are maintained, the prime and the shadow. When using normal TPF duplicate records for the collection, shadowing means that TPFCS is actually maintaining four copies of the data.

Validation

TPFCS validation involves performing a check of collection structures to ensure that they are built correctly. Validation is used with the reconstruction function to detect, isolate, and correct internal structural errors.

Reconstruction

Reconstruction involves rebuilding a control record of a collection and the chains that the collection anchors. Such reconstruction would be necessary if data becomes inaccessible as a result of data or control record corruption, logic errors, I/O errors, or a user error.

Database Archives

This section provides information about TPFCS database archive services.

External Device Support

External device support and archiving provide interfaces that will allow you to read and write data from external devices such as tape, general data sets, communications, and any other devices supported by the TPF system.

Archiving

For archiving, the application calls the TPFxd_archiveStart function instead of the TPFxd_externalStart function. The TPFxd_archiveStart function will return a token that the application passes to TPFCS as a parameter on the TO2_capture or TO2_restore function.

Capture and Restore

TPFCS provides functions to write collections to external storage and retrieve collections from external storage. By using these TPFCS functions and the archiving support of the external device support, collections will be able to be moved in and out of archived storage.

See TPF Database Reference for more information about external device support.

TPFCS APIs

The provided functions (APIs) that make up the collection library are divided into three types:

See TPF Application Programming and the TPF C/C++ Language Support User's Guide for more information.

Environment Block

A TPFCS environment block must be created by each application to access a data store. A pointer to that block must be passed on every subsequent TPFCS function call.

Error Handling

When TPFCS finds an error attempting to process a function, it sets an error code in the environment block and returns a 0 to the application program. The application can interrogate the environment block to determine the exact error and retrieve the associated text.

See the TPF C/C++ Language Support User's Guide for information about all of the supported TPFCS APIs.

Maintaining TPFCS

This section provides information about the commands used to initialize and maintain the TPFCS database.

Note:
Most of the functionality achieved by using the commands can also be implemented by writing applications that use the TPFCS APIs.

ZOODB Commands

The ZOODB commands are used to initialize the TPFCS database, define, change, and display a data store or data definition, and delete a data definition.

The following describes the ZOODB commands:

Function
Description

ZOODB CHANGE
Changes a data definition or data store.

ZOODB DEFINE
Defines a data definition or data store.

ZOODB DELETE
Deletes a data definition or data store.

ZOODB DISPLAY
Displays a data definition or data store.

ZOODB INIT
Initializes support.

ZOODB MIGRATE
Migrates a data store.

ZOODB RECREATE
Re-creates a data store.

ZOODB SET
Sets on or sets off the method trace table or set dump creation on a TO2_getErrorText function call.

For more information about the ZOODB commands, see TPF Operations.

ZBROW Commands

TPFCS provides browsing support that allows classes, methods, and collections to be located, interrogated, validated, displayed, and dumped. This support is provided by using the ZBROW commands and TPFCS function calls.

The following describes the ZBROW commands:

Function
Description

ZBROW ALTER
Changes the contents or access mode of a specified collection.

ZBROW CLASS
Displays class name information.

ZBROW COLLECTION
Performs maintenance on a collection.

ZBROW DISPLAY
Displays information about a collection or its contents.

ZBROW KEYPATH
Adds, displays, or removes a key path.

ZBROW NAME
Alters or displays collection name information.

ZBROW PATH
Displays path information for a collection structure.

ZBROW PROPERTY
Alters or displays a property.

ZBROW QUALIFY
Qualifies ZBROW command requests for a data store.

ZBROW RECOUP
Manages recoup indexes.

See TPF Operations for more information about the ZBROW commands. See the TPF C/C++ Language Support User's Guide for more information about the TPFCS C function calls.