- Job fails in the Verify step.
This indicates that a configuration
error was detected before beginning the migration process. This can
be due to either incorrect data entered when you created the migration
jobs or a configuration problem. Review the log output for the error
detected, then correct and rerun. The logs are located in temporary_directory_location/nnnnn,
where temporary_directory_location is the value
that you specified when you created the migration jobs (where the
default is /tmp/migrate) and nnnnn is
a unique number that is generated and displayed during the creation
of your migration jobs as well as displayed in the JESOUT DDNAME
of the WROUT and WRERR steps of your batch job stream.
- Job fails after the Verify step.
In the event of failure in
the migration job after the Verify step, you can rerun the migration
job; but first, you must delete the WebSphere Application Server for
z/OS configuration home directory created in the CRHOME step. This
corresponds to the home directory that you entered when you created
the migration jobs, and it can also be found in the migration JCL
environment variable V6_HomeDir. Because the migration
procedure creates a new configuration file system for each node being
migrated, it is a simple process to delete the configuration and start
from scratch.
- Problems occur with a managed (federated) node migration.
A
federated node is the most complex node to migrate because it is essentially
two migrations rolled into one. A federated node requires a migration
of the node configuration information contained in the deployment
manager's master repository as well as the configuration information
contained in the federated node. Federated node migration requires
an active connection to the deployment manager. If you have security
enabled, it is essential that you follow the instructions that were
generated when you created the migration jobs. The migration job must
be submitted with a WebSphere Administrator's user ID that has been
properly configured for obtaining secure connections.
Version
5.x node agents might display as not synchronized or not available
when you change the deployment manager node name in a mixed cell during
migration to the Version 6.1 deployment manager. Version 5.x node
agents maintain a link to the Version 5.x deployment manager until
they are restarted; therefore, they might fail to synchronize with
the new deployment manager. The discovery problem, which prevents
automatic synchronization, occurs because the node agent is not yet
aware of the deployment manager name change that occurred during the
migration. If you experience this problem, perform these steps on
the node.
- Stop the node.
- Run the syncNode command.
- Restart the node.
- Job fails during the application-installation phase of migration.
If
you select the option for the migration process to install the enterprise
applications that exist in the Version 5.x or Version 6.0.x configuration
into the new Version 6.1 configuration, you might encounter error
messages during the application-installation phase of migration.
The
applications that exist in the Version 5.x or Version 6.0.x configuration
might have incorrect deployment information—usually, invalid XML documents
that were not validated sufficiently in previous WebSphere Application
Server runtimes. The runtime now has an improved application-installation
validation process and will fail to install these malformed EAR files.
This results in a failure during the application-installation phase
of WASPostUpgrade and produces an "E:" error
message. This is considered a "fatal" migration error.
If migration
fails in this way during application installation, you can do one
of the following:
- Fix the problems in the Version 5.x or Version 6.0.x applications,
and then remigrate.
- Proceed with the migration and ignore these errors.
- Restart the migration job in the FINISHUP step to allow the remaining
migration functions to be performed.
Do this by adding the RESTART=FINISHUP
parameter to the job card and resubmitting the job.
- Later, fix the problems in the applications and then manually
install them in the new Version 6.1 configuration using the administrative
console or an install script.
- Out of space errors occur.
The migration logs are located in temporary_directory_location/nnnnn,
where temporary_directory_location is the value
that you specified when you created the migration jobs (where the
default is /tmp/migrate) and nnnnn is
a unique number that was generated during the creation of your migration
jobs. Normally, the space requirements for the migration logs are
small. If you enable tracing, however, the log files can be quite
large. The best practice is to enable tracing only after problems
have been found. If tracing is required, try to only enable tracing
related to the step in the process that is being debugged. This will
help to reduce the space requirements.
Tracing can be enabled
when you create the migration jobs or by changing the variables in
the migration JCL from disabled to enabled:
TraceState=enabled
profileTrace=disabled
preUpGradeTrace=disabled
postUpGradeTrace=enabled
During migration, a
backup copy of your Version 5.x or 6.0.x configuration is made. This
backup becomes the source of the information being migrated. The default
backup location is /tmp/migrate/nnnnn.
This location can be changed when you create the migration jobs. Depending
on the size of the node being migrated, this backup can be quite large.
If your temporary space is inadequate, then you will need to relocate
this backup.
- Batch job time is exceeded.
Each z/OS installation is different
with respect to job classes and time limitations. Make sure you have
specified appropriate job classes and timeout values on your job card.
- Migration of a deployment manager or standalone application server
completes but no applications are installed.
The log file might
show messages similar to the following:
MIGR0339I: Application WISO_wisoadmin_war.ear is deploying using the wsadmin command.
MIGR0241I: Output of wsadmin.
Error: unable to allocate 268435456 bytes for GC in j9vmem_reserve_memory.
JVMJ9VM015W Initialization error for library j9gc23(2): Failed to instantiate heap. 256M requested
Could not create the Java virtual machine.
The problem
is that the WASPostUpgrade script launched from bbomigrt2.sh does
not have enough remaining address space to initialize the Java virtual
machine (JVM). Typically, this indicates that the spawned process
is running in the same address space as the WASPostUpgrade JVM.
You
can use the environment variable _BPX_SHAREAS to tell the underlying
process whether or not spawned processes should share the same address
space as the parent process. The default value (null) is NO, but
administrators can change this to YES or MUST to get a performance
benefit because the address space does not need to be copied during
fork or spawn actions.
If your system is experiencing the problem
described here, run the migration job with the environment variable
_BPX_SHAREAS explicitly set to NO. You can set this either in the
system profile (
/etc/profile) or in the user
profile for the user running the migration job. Use the following
entry to set this variable to NO:
export _BPX_SHAREAS = NO
After
the migration job completes, you can update the profile to reset _BPX_SHAREAS
to its original value.
- Failures occur during server startup after migration.
Review
the instructions that were generated when you created the migration
jobs. Verify that the JCL procedures have been copied over correctly
to your PROCLIB, the RACF definitions have been created, the Version
6.1 libraries have been authorized, and, if required, your STEPLIB
statements to the Version 6.1 libraries have been specified. Make
sure that the daemon process associated with your cell is at the appropriate
level. The daemon process must be at the highest WebSphere Application
Server for z/OS version level of all servers that it manages within
the cell.
After migrating to a Version 6.1 cell that contains
or interoperates with Version 6.0.x nodes that are not at Version
6.0.2.11 or later, the cluster function might fail. When starting
these Version 6.0.x application servers, you might see the following
problems:
- You might see a first failure data capture (FFDC) log that shows
a ClassNotFoundException error message. This exception is thrown from
the RuleEtiquette.runRules method and looks something like the following
example:
Exception = java.lang.ClassNotFoundException
Source = com.ibm.ws.cluster.selection.SelectionAdvisor.<init>
probeid = 133
Stack Dump = java.lang.ClassNotFoundException: rule.local.server
at java.net.URLClassLoader.findClass(URLClassLoader.java(Compiled Code))
at com.ibm.ws.bootstrap.ExtClassLoader.findClass(ExtClassLoader.java:106)
at java.lang.ClassLoader.loadClass(ClassLoader.java(Compiled Code))
at java.lang.ClassLoader.loadClass(ClassLoader.java(Compiled Code))
at java.lang.Class.forName1(Native Method)
at java.lang.Class.forName(Class.java(Compiled Code))
at com.ibm.ws.cluster.selection.rule.RuleEtiquette.runRules(RuleEtiquette.java:154)
at com.ibm.ws.cluster.selection.SelectionAdvisor.handleNotification(SelectionAdvisor.java:153)
at com.ibm.websphere.cluster.topography.DescriptionFactory$Notifier.run(DescriptionFactory.java:257)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1462)
- You might see a java.io.IOException that looks something like
the following example:
Exception = java.io.IOException
Source = com.ibm.ws.cluster.topography.DescriptionManagerA. update probeid = 362
Stack Dump = java.io.IOException
at com.ibm.ws.cluster.topography.ClusterDescriptionImpl.importFromStream(ClusterDescriptionImpl.java:916)
at com.ibm.ws.cluster.topography.DescriptionManagerA.update(DescriptionManagerA.java:360)
Caused by: java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java(Compiled Code))
at java.io.DataInputStream.readUTF(DataInputStream.java(Compiled Code))
at com.ibm.ws.cluster.topography.KeyRepositoryImpl.importFromStream(KeyRepositoryImpl.java:193)
During migration, Version 6.1 cluster information is distributed
throughout the cell. Version 6.0.x nodes that are not at Version 6.0.2.11
or later fail to read this information. To avoid this problem, upgrade
all Version 6.0.x nodes that will be contained in or interoperating
with a Version 6.1 cell to Version 6.0.2.11 or later before migrating
your deployment managers to Version 6.1.
After migration, carefully review the job output and log files
for errors.
If you migrate a node
to Version 6.1 then discover that you need to revert back to Version
5.x or 6.0.x, see Rolling back your environment.
If
you do not see a problem that resembles yours or if the information
provided does not solve your problem, contact IBM support for further
assistance. For current information available from IBM Support on
known problems and their resolution, see the IBM Support page. IBM Support also has documents
that can save you time gathering information needed to resolve this
problem. Before opening a PMR, see the IBM Support page.