IBM Tivoli Software IBM Tivoli Software

[ Bottom of Page | Previous Page | Next Page | Contents | Index ]


ETL Processes

The warehouse pack has the following processes:

EC2_m05_ETL2_Initialization_Process

This process contains the initialization step that extracts the translated event severity and status values from the TWG.TRANSLATED_TERM tables and loads the data into the EC2.D_SEVERITY and EC2.D_STATUS tables.

The process step must be manually run only once before the EC2_c05_ETL1_Process and EC2_m10_ETL2_Process processes are run. Do not schedule this process to run.

You must create the event filter table and the extended attribute filter table before you run the ETLs for the first time. For more information, see Problem determination.

This process has the following step:

EC2_m05_s010_mart_initialization
This step loads the translated event severity and status values into the D_SEVERITY and D_STATUS tables.

EC2_c05_ETL1_Process

This process contains the steps that are responsible for extracting raw events from the Tivoli Enterprise Console(R) event database, transforming this data to fit into the schema, and loading the data into Tivoli(R) Data Warehouse.

This process should be scheduled to run on a nightly basis. The process steps can run manually also. For information about how to run the steps manually, see Installing and Configuring Tivoli Data Warehouse.

This process has the following steps:

EC2_c05_s010_src_pre_extract
This step drops and recreates the staging tables.
EC2_c05_s020_src_extract
This step updates the Extract_control table and extracting the relevant event information from the Tivoli Enterprise Console event database using the event class filter and the extended attribute filter. This step populates the temporary tables that are created in the previous step with the extracted information.
EC2_c05_s030_src_transform
This step transforms the temporary event information and relationship information into a format that is comparable to the central data warehouse schema. It also creates any new attribute update events or repeat count events and the necessary relationships.
EC2_c05_s040_src_load
This step moves the data from the staging or translation tables into the central data warehouse tables.

If errors occur in this process, they are displayed in the Work In Progress window in the DB2(R) Data Warehouse Center. A red X is displayed for the step that failed during this process. To determine more information about the error, see the log files for the step that are located in the $TIVOLI_COMMON_DIR/cdw/logs/etl directory under the root directory of your DB2 installation. For more information about the error information, see Installing and Configuring Tivoli Data Warehouse.

EC2_m10_ETL2_Process

This process contains five steps that extract event data from the central data warehouse, transform the data to fit into the Tivoli Enterprise Console 3.9 data mart schema, load the data into the data mart, and prune data from the data mart.

Note:
Do not schedule or manually run the EC2_m10_ETL2 process. It is started automatically when the EC2_c05_ETL1_Process sucessfully completes running.

This process has the following steps:

EC2_m10_s010_mart_pre_extract
This step drops and recreates the staging tables in the data mart.
EC2_m10_s020_mart_extract
This step extracts the relevant event information from the central data warehouse based on the updated Extract_control table and puts the data into temporary tables in the data mart.
EC2_m10_s030_mart_transform
This step transforms the temporary event information and relationships into a format for the star schema for the data mart.
EC2_m10_s040_mart_load
This step moves the data from the staging tables into the static data mart tables, which makes the data available for the Crystal reports. This step also updates the extract_control table in the central data warehouse.
EC2_m10_s050_mart_prune
This step removes data that is based on a pruning duration value as specified by the customer.

For information about pruning data from the central warehouse database and data mart database see Pruning data.

If errors occur in this process, they are displayed in the Work In Progress window in the DB2 Data Warehouse Center. A red X is displayed for the step that failed during this process. To determine more information about the error, see the log files for the step that are located in the $TIVOLI_COMMON_DIR/cdw/logs/etl directory under the root directory of your DB2 installation. For more information about the error information, see Installing and Configuring Tivoli Data Warehouse.


[ Top of Page | Previous Page | Next Page | Contents | Index ]