===================================================================
Licensed Materials - Property of IBM

5724-L62

(C) Copyright IBM Corp. 2002, 2010  All Rights Reserved

US Government Users Restricted Rights - Use, duplication or
disclosure restricted by GSA ADP Schedule Contract with
IBM Corp.
===================================================================
  IBM Tivoli Composite Application Manager for WebSphere V6.1.0.5

                      IBM iSeries Data Collector

                    Interim Fix 0009 Readme

                            April, 2010

===================================================================
Contents

Section 1. General Description
Section 2. Defects Fixed
Section 3. Prerequisites
Section 4. Customizable Files
Section 5. Installation Instructions
Section 6. Uninstallation Instructions
Appendix A PK81336
Appendix B WPS 6.2 and ESB 6.2 support
Appendix C Support for token-less tracking for CTG TT
Appendix D Additional Platform Support for TTAPI
Appendix E Data collector configuration changes
Appendix F PK83088: Method Profiling changes
Appendix G PK93126: URL Request Mapper
Appendix H Additional install instructions if upgrading from FP3 level 
          (FP3 + any IFIX)
Appendix I PK98307: WebSphere Portal does not start on IBM iSeries
 
====================================================================
Section 1. General Description
--------------------------------------

This is an Interim Fix 0009 for ITCAM for WebSphere V6.1.0.5,
IBM iSeries Data Collector.

For supported platform information about ITCAM IBM iSeries Platform Data
Collector, please visit the following URL.

http://www-306.ibm.com/software/sysmgmt/products/support/Tivoli_Supported_Platforms.html

Note: If this is the first time you are trying to access our supported
platform information, you will be prompted to register with IBM with
an IBM ID and password.

For additional ITCAM for WebSphere documentation, best practices, troubleshooting tips, 
tuning options, etc. please visit the Wiki page at:
 
https://www.ibm.com/developerworks/wikis/display/tivolimonitoring/Tivoli+Composite+Application+Manager+for+WebSphere

====================================================================
Section 2. Defects Fixed
--------------------------------------

Defects Fixed in 6.1.0.5-TIV-ITCAMfWAS_MP-IF0009

--------------------------------------------------------------------------------
DEFECT                   DESCRIPTION
--------------------------------------------------------------------------------

PM08267/205439  Need means to exclude classes from LoadedClassMap
PM06301/205562  Duplicate entries in the classinfo.text.data file
PM00225         ITCAM FOR WEBSPHERE CAUSES RMI PORT CONFLICT WITH APPLICATION
PM02947         DEADLOCK WHEN MULTIPLE THREADS ARE STARTED PRIOR TO ITCAM DC   
PM04782         JAVACORE FAILS BECAUSE OF EBCDIC NAMES
PM09233         CLASSCASTEXCEPTION ERROR WHEN ACCESSING PORTLET FACTORY

--------------------------------------------------------------------------------

Defects Fixed in 6.1.0.5-TIV-ITCAMfWAS_MP-IF0007

--------------------------------------------------------------------------------
DEFECT                   DESCRIPTION
--------------------------------------------------------------------------------

203674          TTAPI: Support for new platform: Solaris SPARC 64 bit JVM
PM03122/197308  cyn_collector gives wrong results when started with '-a' key
PM03131/203124  TTAPI: extra event is generated for uncaught exception
PM03135/203430  TTAPI: DC should check if GPS token is valid for Web Services provider
PM03138/204498  TTAPI: events do not have Horizontal link for Web Services request
PM03021/204889  DC status is not correct in MOD with Port Consolidator
PK93890         ERROR CYNVE0739E AND SUN JDK 1.5 HEAP DUMP NOT ENABLED  
PK99591         CYNVE1568E ERROR MESSAGE DISPLAYED WHEN RETRIEVING JMX DATA    
PM00366         ITCAM DOES NOT DISPLAY APPLICATIONS OUTSIDE THE DEFAULT        
PM02078         Document steps to avoid use of ITCAM plugins for FP4 upgrade 
                to avoid NoClassDefFound errors
PK98307         WebSphere Portal does not start on IBM iSeries

--------------------------------------------------------------------------------

Defects Fixed in 6.1.0.5-TIV-ITCAMfWAS_MP-IF0005
--------------------------------------------------------------------------------
DEFECT                   DESCRIPTION
--------------------------------------------------------------------------------

193695    Error in DC log: Could not read cyn-cclog.properties    
193929    Supporting Sun JCAPS jax-rpc 1.1 style webservice    
195443    Interceptor not filtering properly    
195851    TTAPI should not be enabled until DC is fully initialized    
196142    Stack corruption caused by JMS callback    
196648    Duplicate trace components only have 1 notification object    
196668    Get rid of dynamic loading of jakarta regexp for GCCollector    
197121    Not show the message for SecurityConfig if not available    
197149    Missing JMXConnectorServerTrigget MBean for WAS7       
197470    Request Aggregator leaking ThreadContextData objects    
200170    No misbehaving trace after trap is reactivated    
200327    DC ClassLoader subordinate to wrong WAS CL    
200622    Memory Leak  during long run
200654    DC WAS 6.0 insufficient memory message for heap dump    
200708    NPE PmiAbstractModule.java - stopping server    
201676    Port the URL remapping facility from v7.1    
201795    JMS Provider URL is empty      
201981    Request Data Integrity after Servlet exception    
PK83088   Optimize method profiling feature
PK83949   ITCAM DC floods trace log with JVMTI error message    
PK86928   No GC data due to TEMAGCCollector.gclog.path setting during silent config
PK87412   Average Pool Size more then Maximum Pool Size on TEP console    
PK90807   Script config_dc to support WAS variable syntax $()  
PK93126   TEMA crashes due to large number of monitored URL requests
PK93465   During DC configuration, do not increase heap size by 128MB automatically.   
PK94210   Do not add -server option during DC configuration for Appplication Server
PK94243   Slow Response Time for Server Overview Page with ITCAM 6.1 DC
PK95323   Changes in GLS component to prevent OOM
PK96138   Data Collector unable to handle WAS encrypted passwords
PK97216   Corruption of classinfo.text.data causes OOME

--------------------------------------------------------------------------------

Defects Fixed in 6.1.0.5-TIV-ITCAMfWAS_MP-IF0003
--------------------------------------------------------------------------------
DEFECT                   DESCRIPTION
--------------------------------------------------------------------------------

193725    Support for ESB 6.2 and WPS 6.2 (See Appendix B)
193908    Support for token-less tracking for CTG TT (See Appendix C)
194647    DC can not send ttapi data to TTAS on window 64 bit (See Appendix D)
194787    memory leak in native code    
195182    Wrong behavior in WAS6.1 for MBean Calls on zOS 
195801    TEMA cant discover newly deployed application    
196027    ConcurrentModificationExec in trace-dc-ParentLast.log    
196137    TEMA doesn't show Dynamic Cache stats
196153    Update Installer should support TTAPI Native lib renaming
196225    TEMA cant get request data in workspaces for SUN JDK domain      
196227    TEMA Application Health has no data in WAS5.1  
196373    TEMA Backend tier analysis data isn't consistent    
196601    TEMA connection to DC WAS5.1 is broken    
197436    TEMA wrong Datasource name in WAS5.1     
197707    Unnecessary ClassLoader instrumentation    
198051    Classes from parentless Class Loaders not instrumented    
198122    WAS failed to restart with modified context on linux     
198245    Set property am.jmxe.use.object=true      
198306    Change max heap size setting method    
198615    cyn_collector.sh No such file or directory error
PK75797   The graphic'JVM CPU UTILIZATION (%, LAST HOUR)' doesn't show data
PK77456   Log parser stops when file is redirected to dev/null 
PK78324   Install fails even if space exists in server directory
PK79330   DC 6.1 Configurator does not handle multiple DMGR in WAS XD
PK80213   WSADMIN authentication fails when password contains % char
PK80634   JSR168 portlets added to ITCAM functionality
PK81283   Potential memory leak in RA, keeping Thread objects alive
PK81336   JVMs are experiencing OOM especially with large number of resources (See Appendix A)
PK82763   Can't see data in MQI reports for MQ7 from ITCAM MS VE
PK82770   Warning messages for harmless notices in trace_dc_bcm.log
PK83442   L3 Method Trace for Misbehaving Trap is incomplete
PK88014   A wrong PID is displayed in ITM interface for an application
PK89977   Problem with the cyn_collector.sh script in FP4 on Solaris
PK90510   Memory Leak of ThreadContextData objects in WAS 5.1

====================================================================
Section 3. Prerequisites
--------------------------------------

1. IBM Tivoli Composite Application Manager for WebSphere V6.1,
   Fix Pack 5, IBM iSeries Data Collector
   (6.1.0-TIV-ITCAMfWAS_MP-FP0005_iSeries)

2. IBM Tivoli Composite Application Manager for WebSphere V6.1,
   FixPack 5, Managing Server (6.1.0-TIV-ITCAMfWAS_SVR-FP0005)

3. IBM Tivoli Composite Application Manager for WebSphere V6.1,
   Fix Pack 5, Interim Fix 0009, Managing Server
   (6.1.0.5-TIV-ITCAMfWAS_SVR-IF0009)
   
NOTE: 
This Interim Fix is cumulative, it includes all fixes published to
date, at Fix Pack 5 level.

====================================================================
Section 4. Customizable Files
-----------------------------
This interim fix will overwrite following script. If you have customized this file, 
please make a backup copy of the file before proceeding with the installation and reapply the customization.

itcamdc/bin/proxyserverctrl_ws.sh

====================================================================
Section 5. Installation Instructions
--------------------------------------

Note:
The Update Installer is packaged in the Interim Fix tar file.

Complete the following steps to install the update:

    1. Upload the update tar 6.1.0.5-TIV-ITCAMfWAS_MP-IF0009_iSeries.tar
       to the iSeries host.

    2. Login IBM iSeries and run QSH.

    3. Extract the Interim Fix 0009 tar file:

       /qopensys/usr/bin/tar -xf 6.1.0.5-TIV-ITCAMfWAS_MP-IF0009_iSeries.tar

    4. Install according to the installation instructions in
       ITCAM_V61_UpdateInstaller.pdf or ITCAM_V61_UpdateInstaller.html.

    5. If the upgrade is being done from ITCAM 6.1 FP3 level, follow the additional instructions
       listed in Appendix H.

    6. Restart the Data Collector WebSphere Application Server process.

This Interim Fix also includes fix for the IBM iSeries Data Collector installer. 
Refer to Appendix I on how to apply installer fix.

====================================================================
Section 6. Uninstallation Instructions
--------------------------------------

Complete the following steps to uninstall the update:

    1. Uninstall according to the uninstallation instructions in
       ITCAM_V61_UpdateInstaller.pdf or ITCAM_V61_UpdateInstaller.html.
      
    2. Restart the Data Collector WebSphere Application Server process.

====================================================================
Appendix A
----------
PK81336: JVMs are experiencing OOM especially with large number of resources
 
To fix this problem, apply  Fix Pack 5, Interim Fix 0003 or higher, to the Managing Server and the Data Collector.

The following setting is enabled in the toolkit.properties file in the 
DC/toolkit/etc/toolkit.properties file:

am.jmxe.use.object=true

This setting instructs the Data Collector to use a new protocol for transfering JMX data to the Managing Server
that does not allocate large sized objects.

Therefore it is required that the Managing Server has the corresponding fix which will be able to receive data 
in the new protocol.

Note that the Managing Server at Fix Pack 5, IF0003 and higher is able to process JMX data in the older mode (which
is equivalent to the setting of am.jmxe.use.object=false). As a result it can still display data from Data Collectors 
at Fix Pack 5 and earlier levels.

Appendix B
----------
This Interim Fix provides support for the following:

- Support for WebSphere Process Server (WPS) Version 6.2 
- Support for WebSphere Enterprise Service Bus (WESB)  Version 6.2

If you have an existing Data Collector configured on WPS 6.2 or WESB 6.2 Servers, and want to add the 
full support for V6.2 which is provided in this Interim Fix, complete the following steps:

For the Network Deployment (ND) environment, the Deployment Manager and the Node Agent must be running during 
configuration/unconfiguration.

1. Unconfigure the Data Collector using config_dc.sh (config_dc.bat); the server(s) do not need to be 
   restarted after the Data Collector is unconfigured.
2. Stop the server(s). 
3. Install FP5-IF0003 or higher.
4. Configure the server(s) using config_dc.sh (config_dc.bat).
5. Start the server(s) to take advantage of the new configuration

For a stand alone environment, the server must be running during configuration/unconfiguration.

1. Unconfigure the Data Collector using config_dc.sh (config_dc.bat); the server(s) do not need to be 
   restarted after the Data Collector is unconfigured.
2. Stop the server(s) 
3. Install FP5-IF0003 or higher.
4. Start the server(s) 
5. Configure the server(s) using config_dc.sh (config_dc.bat).
6. Stop and restart the server(s) to take advantage of the new configuration


Appendix C
----------    
This Interim Fix provides support for token-less tracking of CICS Transaction Gateway (CTG).

In order to enable token-less tracking feature:

1. Add the following properties to  DC/runtime/{platform.node.server}/custom/toolkit_custom.properties:
 
   # 1  Enable integration with Transaction Tracking
   com.ibm.tivoli.itcam.dc.ttapi.enable=true
   com.ibm.tivoli.itcam.dc.ttapi.ttas.transport=tcp:<ip_addr>:<port_number> 

   # 2  Enable CTG instrumentation  
   com.ibm.tivoli.itcam.dc.ctg.enablectg=true

   # 3  Enable integration with token-less CTG-Transaction Tracking feature
   com.ibm.tivoli.itcam.dc.ttapi.ctg.tokenless.enabled=true

2. Customize the DC/runtime/{platform.node.server}/custom/ctg.filters
   file to define which CTG requests are to be tracked without imbedded tokens.
   These requests will be processed using this tokenless feature.
   
   For example, following filter configuration enables token-less tracking for requests flowing
   to CICS3101 server:

     Type=E,Server=CICS3101

   Please check the ctg.filter file for detailed instructions on how to configure filters.


Appendix D
----------
This Interim Fix adds following platforms for Data Collector and ITCAM for Transactions integration through 
the Transaction Tracking Application Programming Interface (TTAPI):

- 64 bit Windows 
- 64 bit AIX
- xLinux 64 bit
- zLinux 31 bit and 64 bit

To enable Data Collector and TTAPI integration, add the following properties to 
DC/runtime/{platform.node.server}/custom/toolkit_custom.properties:
 
com.ibm.tivoli.itcam.dc.ttapi.enable=true
com.ibm.tivoli.itcam.dc.ttapi.ttas.transport=tcp:<ip_addr>:<port_number> 

Appendix E
----------
Starting from ITCAM 6.1 Fixpack 5 + Interim fix 5, the following Data Collector Configuration changes
have been made:

PK93465: The JVM maximum heap size will not be increased automatically by 128 MB in any new
Datacollector configuration. This is now optional and the WebSphere system administrator must decide 
whether to increase the heap size based on system and other limitations.  It is strongly recommended   
that the heap size be increased by 128 MB to account for additional monitoring overhead using ITCAM.

PK94210: The "-server" JVM option will not be inserted in any new Data collector configuration
on Solaris platform running HotSpot VM.  This is left to the decision of the WebSphere system administrator.
Having the "-server" option turns on the optimization for long running Java applications like
applications servers on HotSpot VMs, and is strongly recommended.


Appendix F
----------
PK83088: Method Profiling optimization

The Method Profiling feature in the Data Collector has been optimized to consume
less memory.  Previously, if the L3 entry/exit instrumentation is turned on
in toolkit_custom.properties, Method profiling will consume a fixed amount of Java heap
amounting to a default size of 18MB.  This optimization will allocate heap memory 
on an as required basis.

The default value of the following method profiling properties have been changed 
in the datacollector.properties file:

am.mp.cpuThreshold=30
am.mp.clockThreshold=30

These properties have a 30 millisecond value. Only those methods that exceed one of these
values (cpu or clock) will be displayed in the method profiling reports. The purpose of 
method profiling is to identify methods that have high execution time. Having a 
threshold limit allows to reduce the number of methods monitored, thus reducing the
monitoring overhead, by tracking only the most time consuming methods. 

Change the above values if you wish to track methods with other thresholds. Note that
lowering the value will most likely monitor more number of methods, in which case 
you may also want to consider increasing the am.mp.leaguetablesize property value 
mentioned below.

am.mp.leaguetablesize=1000

The default value is to monitor 1000 methods through method profiling during one polling cycle.
If you wish to monitor additional methods, this value needs to be increased.  Note that only
those methods that exceed am.mp.cpuThreshold or am.mp.clockThreshold will be included in this
count.

Appendix G
----------
PK93126: URL Request Mapper

Some applications have thousands of unique URLs to be monitored.  Most of them have a base URI
and additional paths appended to it.  For example: 
  http://com.myco.server/bankapp/acct/create
  http://com.myco.server/bankapp/acct/update
  http://com.myco.server/bankapp/acct/delete
  http://com.myco.server/bankapp/loan/rates
  http://com.myco.server/bankapp/loan/apply
  http://com.myco.server/tradeapp/invest/info
  http://com.myco.server/tradeapp/invest/acct
  http://com.myco.server/tradeapp/quotes
  and so on.
  
Monitoring such unique URLs has a monitoring overhead in terms of Java heap size and
could lead to OutOfMemory condition in Data Collector as well as in TEMA and other components
if there are thousands of such unique URLs.

To overcome this issue, ITCAM 6.1 Fixpack Interim Fix 5 introduces a new URL mapping feature.
By using this facility, the user can cause HTTP requests to be renamed based upon an  
initial portion of the request URI, so that individual URIs that begin with a common prefix will 
be lumped together and given the  same request name, and essentially treated as two instances 
of the same request.  The reduction in number of URLs tracked reduces the memory overhead for 
monitoring.

Implementation:
--------------
Step 1: Create a XML file

To implement this, a XML file needs to be created with the following format:

<gpe>
   <runtimeConfiguration>

      <requestMapperDefinition type="servlet">
      
           <httpRequestMapper>
                <uri>.....</uri>
                <uri>.....</uri>
                ....
                <uri>.....</uri>
           </httpRequestMapper>
      
      </requestMapperDefinition>
   </runtimeConfiguration>        
</gpe>

<httpRequestMapper> 
   This tag defines a series of HTTP URI prefix strings that define how URIs and URLs are to be remapped.
   The <httpRequestMapper> consists of a series of <uri> tags, each of which identifies a URI initial    
   substring.  When the Data Collector detects the start of an HTTP request, it locates the most  
   specific URI initial sub-string within the set of <uri> tags, and remaps the original URI to the tag value.  
   This allows a group of URIs that have a common prefix to be remapped to the prefix, allowing them to be 
   grouped together, and resulting in a smaller set of unique URIs.  The URL is similarly remapped, 
   by taking the resulting URI prefix string, and adding it to the end of the network location portion 
   of the original URL.
   
<requestMapperDefinition type="servlet">
   Currently, this is the only requestMapperDefinition.  The type="servlet" applies to URIs that access
   both servlets and JSPs.
   
<uri> 
   This tag defines an initial URI substring within the <httpRequestMapper> tag.  When an HTTP request starts, 
   the request mapper finds the most specific <uri> tag that is a match for the URI passed by the caller, 
   and remaps the URI and URL to the value of the <uri>.
   
For example, to remap the URIs shown above to only two types, the XML file will look like the following:

    <?xml version="1.0"?>
    <gpe>
        <runtimeConfiguration>

            <requestMapperDefinition type="servlet">
      
                <httpRequestMapper>
                    <uri>bankapp</uri>
                    <uri>tradeapp</uri>
                </httpRequestMapper>
      
            </requestMapperDefinition>
        </runtimeConfiguration>        
     </gpe>

Using this, URLs like http://com.myco.server/bankapp/acct/create and http://com.myco.server/bankapp/acct/update.
will be mapped to http://com.myco.server/bankapp.

Similarly,   http://com.myco.server/tradeapp/invest/acct and http://com.myco.server/tradeapp/quotes
will be mapped to http://com.myco.server/tradeapp

Step 2:  DataCollector changes

Place the  XML file (the file can be called anything you want, as long as it has a suffix of ".xml") 
in the DCHOME/runtime/<server-name>/custom/ directory.

Then, update the DCHOME/runtime/<server-name>/custom/toolkit_custom.properties file, and add the following property 
to the end:

       am.camtoolkit.gpe.customxml.reqMap=<requestMapper xml file name>

where "<requestMapper xml file name>" is the simple (unqualified) file name of the request mapper XML file.

Appendix H
----------
Additional install instructions if upgrading from ITCAM 6.1 FP3 level (FP3 + any IFIX) to the current version.

If a ITCAM 6.1 FP3 level install is being upgraded to the current FP5 IFIX version, it is imperative that
the ITCAM sub directories in the WAS plugins directory be deleted, and the WAS plugins be refreshed 
for all defined profiles.

The ITCAM for WAS Data Collector was modified with FP4 so that it no longer requires plugins in the 
WAS 6.1 environment, as it did prior to FP4.  However, to avoid problems with potential conflicts between 
FP4 and ITCAM jar files that might still exist in the WAS plugins, these plugins should be removed prior to 
installing FP4 or FP5, or immediately after installing these fix packs but prior to restarting the WAS server.

Because all servers in a WAS installation directory share the same plugins directory, this implies that 
all servers that have a configured ITCAM Data Collector must be upgraded from a pre-FP4 environment 
to a post-FP4 environment at the same time.

Do the following to remove the plugins:

1. From a command line interface, make the <WASHOME>/plugins directory the current directory.

2. Check whether the following directories exist: itcam_6.1.0, itcammqjms_6.1.0, and itcamsib_6.1.0.  
   If they exist, delete the contents of the directory, and then delete the directory itself:
        > cd <directory-name>
        > rm *
        > cd ..
        > rmdir <directory-name> 
        
3. Run through the following steps for each WAS profile:

        a. Make sure that all server instances in the profile, including nodeagents, are stopped.

        b. cd <WASHOME>/profiles/<profile name>/bin

        c. Run the ./osgiCfgInit script.
           If this script runs successfully, it does not produce any messages.
        
        d. Server instances within the profile can now be restarted.

Appendix I 
----------
PK98307: WebSphere Portal does not start on IBM iSeries

After installation and configuration of "ITCAM for WebSphere V6.1.0.4, 
i5/OS Data Collector" to the WebSphere Portal server the server may fail to start.

To prevent this problem apply the fix to "ITCAM for WebSphere V6.1.0.4, 
i5/OS Data Collector" installer before installation as described below:

  - After you untarred DC installation (i.e. C1RK0EN.tar) to the installation 
    working directory, from which you will install DC, copy PK98307.tar there

  - Untar PK98307.tar in the installation working directory using command appropriate 
    for platform:

    - For remote attached Linux client:
    $ tar -xf PK98307.tar

    - For remote attached Windows client:
    You can use any archive management programs which supports tar format to
    extract it. Remeber to specify the installation working directory as the target 
    directory to extract to.

    - For local installation from i5/OS:
    Run next command from the QSHELL:
    $ /qopensys/usr/bin/tar -xf PK98307.tar

  - Follow with DC installation and configuration.

Note: when you will install FP5 to IBM iSeries DC then you will need to skip the 
updateQualDir action as follows:

  - add the following line to the bottom of silentUpdate.properties:

       install.actions.skip=updateQualDir

====================================================================
