Chapter 5
IBM 2109 and SilkWorm 2000 FC Switches

This document supports the newest release and the previous release of OEM firmware qualified for use on IBM FC Switches with the DYNIX/ptx V4.6.x operating systems.

New Release: a2.2.1a, Section 5.2

Previous Release: V2.1.7, Section 5.3

See Table 1-1 for the latest recipe for the best set of SAN FC firmware to run on a given operating system.

The family of FC Switches used on IBM xSeries 430 and IBM NUMA-Q servers includes the following models:

These switches are supplied by an OEM, Brocade Communications Systems, Incorporated.

The switch models are identical except for their nomenclature and exterior color scheme. Switches with IBM nomenclature are all black and switches with the Brocade SilkWorm nomenclature are have purple accents on the front panel.

New system and add-on orders are shipped with IBM 2109 switches. Older systems being upgraded to a new software release are most likely to be using the SilkWorm-nomenclated models.


ATTENTION

On DYNIX/ptx V4.6.x, all FC switches must have a license key enabled. Otherwise, EES events, which are captured by in-band communication with the FC switch, will not be logged.

Additionally, zoning cannot be enabled without a valid license key. While the zoning commands are enabled by the presence of a ZONING license key, IBM does not support general usage of the zoning commands on xSeries 430 or IBM NUMA-Q servers by customers. Because these commands can easily disable a system if used incorrectly, IBM requires deployment of the zoning feature only by IBM Professional Services.



5.1 FC Switch Naming Requirements

The name of the switch:

Additionally, it is suggested that switch names be unique in a cascaded environment.

See IBM 2109 and SilkWorm 2000 FC Switch Administration for procedures to set or change the name of a switch.


5.2 New Release a2.2.1a

Version a2.2.1a is the June 2001 release of firmware for the Fabric Operating System of the IBM 2109 and SilkWorm 2000 families of FC Switches for use in servers running DYNIX/ptx V4.6.1 operating systems.

The xSeries 430 and IBM NUMA-Q servers use most, but not all, of the new features present in the new release. See the discussion in "Changes Since the Previous Release," Section 5.2.2

This New Release section contains the following subsections:

To load the software distribution files, see "Install Software for IBM 2109 and SilkWorm 2000 FC Switches," Section 5.4.

To download firmware to the switches, see "Download to IBM 2109 and SilkWorm 2000 Firmware," Section 5.5.


5.2.1 Compatibility Information

When running on an IBM 2109 or SilkWorm 2000 FC Switches, version a2.2.1a of this OEM firmware is compatible with the following:


5.2.2 Changes Since the Previous Release

Version a2.2.1a of the Fabric Operating System is being released at this time to synchronize IBM 2109 FC Switch firmware on xSeries 430 and IBM NUMA-Q servers with IBM 2109 switches used by other IBM servers.


ATTENTION

With this release of its Fabric OS, the OEM vendor Brocade Communications Systems, Inc., has implemented a scheme to differentiate its OEM software distribution channels, such as IBM, from its direct-market distribution channels, such as their own support website for software downloads. The version name of this particular OEM release has a different form than previous releases from this vendor. The first character in this new form is a lowercase "a" instead of the usual uppercase "V." There is no content difference in the distribution types, but the OEM type can be installed on an OEM-sourced FC Switch without needing to install a new license key. A direct-sourced distribution would require a specific license or else all ports on the switch would be disabled following an upgrade and reboot.

DYNIX/ptx system tools such as ptx/INSTALL and other listing utilities do not accommodate a lowercase character in the first position of a label. System tools will substitute an uppercase V preceding the version numbers of this software package, for instance, "V2.2.1." When inspecting a log file such as /etc/versionlog for installation history, the version numbers are the significant information.

When an IBM-supported FC Switch is queried with the version command during a telnet session, the switch's operating system should respond with a version label that shows that the operational firmware came from the OEM distribution, such as "a2.2.1a."


When this release is installed, a new directory, /ibm_2109 is added in the /usr/ssw/fw/fc_switch directory to contain the new binary file. The new binary file that contains the release software is named fc_sw_ibm2109.bin.

The existing directory named /brocade2000 and its binary file fc_sw_217.bin remain.

The xSeries 430 and IBM NUMA-Q servers do not use all of the features present in this release. See the following descriptions of the new commands that are used by those servers:


5.2.3 a2.2.1a Firmware Problem Reports

Most problems described in this section have workarounds that allow the system to recover and continue operating without loss of data. The problems without workarounds do not jeopardize data I/O integrity and are mostly internal administrative functions within the switch.

The following problems have been found in a2.2.1a. See "FC I/O Subsystem-Level Problem Reports," Chapter 2 for possible I/O subsystem-level problems involving FC Switches.

Empty Switchname String After Reboot

If the Ethernet IP address field is set to "none," the switch's name will be displayed as an empty string following the next reboot.

Workaround: Set the Ethernet IP address and reboot the switch.

Zoning Configuration Changes Not Propagated After Reboot

Sometimes, in a cascade, changes in the enabled zoning configurations are not propagated to other switches after a switch reboot.

Workaround: Assuming that the system is at the single-user level and there is no I/O activity, do the following sequence of telnet commands on each switch in the cascade that did not get updated with the changed configuration:

  1. Take each switch offline (switchDisable) except the switch where the zone configuration change was first created and enabled.

  2. Clear any existing zone information from the database of each switch (cfgClear).

  3. Save that state in each switch (cfgSave).

  4. Put each switch back online(switchEnable).

  5. Reboot each switch (reboot.). As each rebooted switch comes up, the new enabled zoning configuration will be automatically propogated to it from the source switch (now the Principal switch) over the ISLs of the cascade.

Spurious "Timeout" Error Received

Periodically, an "fcpCommand: iu 0x10ec4f60 iu_status 0x18 (timed out)" error will be reported.

Workaround: Ignore this error report.

RAID Device Symbolic Name Missing in Name Server and from nsShow Command

Sometimes, when a RAID device logs in, the inquiry data (symbolic name) is missing from the name server information table and does not show in the nsShow command.

Workaround: Remove the device from the switch and then add it back in.

Excessive firmwareDownload Time

Sometimes the firmwareDownload command takes a very long time to complete.

Workaround: Power cycle the switch and perform another download operation.

Unexpected Diagnostics Pages Returned

When the online diagnostics of the DYNIX/ptx V4.6.x operating system requests status information using the SES protocol, sometimes more pages than the expected Page 0 are returned.

Workaround: No known workaround.


5.2.4 Unused a2.2.1a Features and Commands

Even though extensive instructions on the following switch management features appear in the hard copy User Guides, xSeries 430 or IBM NUMA-Q servers do not support nor use commands associated with:

Of the commands available from the telnet interface of the IBM 2109 and SilkWorm 2000 FC Switches, xSeries 430 or IBM NUMA-Q servers do not use nor support the following specific commands:

agtcfgDefault
aliasShow
tempShow
gbicShow
msConfigure

Of the menu items that appear on the Control-Panel interface of the 16-port switches, IBM 2109 S16 and SilkWorm 2800, xSeries 430 and IBM NUMA-Q servers do not use nor support the following commands:

Disable Device Probing
Disable RLS Probing
Suppress Class F Traffic
Frame Collection
Always Send RSCN
Disable FAN Frames
VC Priorities
VC Multicast
VC Class 3
VC Class 2
VC Link Ctl
Temperature

5.3 Previous Release V2.1.7

V2.1.7 is the December 2000 release of firmware for the Fabric Operating System of FC Switches running in a DYNIX/ptx V4.5.2 system.

This section contains the following:

To load the software distirbution files, see "Install Software for SilkWorm 2400/IBM 2109 S08 and SilkWorm 2800/IBM 2109 S16 FC Switches," Section 5.4.

To download firmware to the switches, see "Download to SilkWorm 2400/IBM 2109 S08 and SilkWorm 2800/IBM 2109 S16 Firmware," Section 5.5.


5.3.1 V2.1.7 Compatibility Information

When running on a SilkWorm 2000, version 2.1.7 of this OEM firmware is compatible with the following:


5.3.2 Changes Since the Previous Release

V2.1.7 is a maintenance release with the following new features and improvements:


5.3.3 V2.1.7 Firmware Problem Reports

Most problems described in this section have workarounds that allow the system to recover and continue operating without loss of data. The problems without workarounds do not jeopardize data I/O integrity and are mostly internal administrative functions within the switch.

The following problems have been found in V2.1.7. See "FC I/O Subsystem-Level Problem Reports," Chapter for possible I/O subsystem-level problems involving FC Switches.

Excessive firmwareDownload Time

Sometimes the firmwareDownload command takes a very long time to complete.

Workaround: Power cycle the switch and perform another download operation.

Unexpected Diagnostics Pages Returned

When the online diagnostics of the DYNIX/ptx V4.6.x operating system requests status information using the SES protocol, sometimes more pages than the expected Page 0 are returned.

Workaround: No known workaround.


5.3.4 Unused V2.1.7 Commands and Features

While the zoneHelp menu of commands are enabled by the presence of a license key, IBM NUMA-Q does not support general usage of the zoning commands by customers. Because these commands can easily disable a system if used incorrectly, NUMA Systems requires deployment of the zoning feature only by IBM Professional Services.

Of the commands available from the telnet interface of the IBM 2109 and SilkWorm 2000 FC Switches, IBM NUMA-Q servers do not use nor support the following specific commands and features:

agtcfgDefault
aliasShow
parityCheck
portCfgMcastLoopback
tempShow
All Quick Loop commands
SNMP switch-management commands
Web-based switch-management commands

Of the menu items that appear on the Control-Panel interface of the 16-port switches, IBM 2109 S16 and SilkWorm 2800, IBM NUMA-Q servers do not use nor support the following commands:

Disable Device Probing
Disable RLS Probing
Suppress Class F Traffic
Frame Collection
Always Send RSCN
Disable FAN Frames
VC Priorities
VC Multicast
VC Class 3
VC Class 2
VC Link Ctl
Temperature

5.4 Install Software for IBM 2109 and SilkWorm 2000 FC Switches

The software can be installed at anytime.

Use the distribution CD-ROM labeled "DYNIX/ptx V4.6.1 OS and Layered Products Software, Vol. 1, March 2001, Revision A"


ATTENTION

In the DYNIX/ptx INSTALL tool and all other DYNIX/ptx listing utilities, the software package named "V2.2.1" contains the OEM software that will be reported by the FC Switch version command as "a2.2.1.a."


  1. Insert the distribution CD-ROM into the CD-ROM drive in the Bootbay connected to Quad 0 of the host system.

  2. Log in to the host as root.


    ATTENTION

    There are two families of FC Switch firmware listed during the installation process, one for SilkWorm 1000s and one for IBM 2109 or SilkWorm 2000s. Be sure to read the descriptions and load only the correct switch software package from the installation menus.


  3. Use the ptx/ADMIN® menu system and the ptx/INSTALL utility to load the "V2.2.1a" FC Switch software into the /usr/ssw/fw/fc_switch/ibm_2109 directory.

    Verify that all directories in that path and the binary file itself have read, write, and execute permissions for all. If any one does not, the firmware download command will fail.

  4. Remove the distribution CD-ROM.

To download and flash the software to firmware, use the procedure in the next section.


5.5 Download to IBM 2109 or SilkWorm 2000 FC Switch Firmware


ATTENTION

The following procedures will require the system to be at the single-user level during the firmware upgrade.



5.5.1 Required System Information

The following information is required before beginning the downloading procedure:

host_IP_address
The IP address of the host system holding the new distribution software file.
hostname
The name of the host system holding the new distribution software file.
username
A valid, password-protected, user login account on the host system that contains the file to be downloaded.
filename
The path to the FC-Switch software file on the host system.
Switch Address
The IP address of each Switch to be upgraded
Switch Password
The Switch-login password for admin, the level required to execute the downloading command. (The factory-set default is password. The password for admin could be different after initial installation, depending on the installer's or customer's preference.)
Fabric Domain ID
The domain ID of each switch to be upgraded.
Site-specific Configuration Variables
Such as:

Collect this list before beginning the upgrade.


5.5.2 Check the Host System Configuration

Software downloading is by TCP connection between the Switch and the host system containing the source file. Before attempting to download software to a Switch, use the following checklist to make sure that the host system was properly configured when the system or Switch was initially installed. If these host-system files are not configured as described, you may get the "Permission denied" and the "rcmd fails" error messages. The configuration procedures are described in Section 8.5 of the Fibre Channel Subsystem Installation. Here is the checklist:

[  ]

On the host system containing the new distribution software file, the /etc/hosts file must contain a name and IP address for each Switch to be upgraded.

[  ]

On the host system, there must be a user account set up, with a password and a basic login shell, for administering FC Switches.

[  ]

On the host system, the .rhosts file in that user's account must contain IP address entries for each Switch to be upgraded.

[  ]

On the host system, permissions for all to read and execute must be set for all the directories and the binary file in the path /usr/ssw/fw/fc_switch/ibm_2109/filename.bin.

For instance:

#cd /usr/ssw
#chmod a+rx fw
#cd fw
#chmod a+rx fc_switch
#cd fc_switch
#chmod a+rx ibm_2109
#cd ibm_2109
#chmod a+r filename.bin

[  ]

On the host system, test the reshd daemon from the user account set up to administer Switches:

username$ resh localhost ls -a

If the resh command fails, check the shell entry in the /etc/inetd.conf file and make sure that it is set up to be a stream socket and not a tli socket.

If the /etc/inetd.conf file must be modified, kill and restart the inetd daemon.

[  ]

On each Switch, the Switch name and Ethernet IP address must match the name and IP address entered for it in the host system's /etc/hosts file.


5.5.3 Check the Network Connectivity

From the host system, verify that it can communicate with each Switch to be upgraded. Issue the following command:

username$ /etc/ping Switch_IP_address


5.5.4 Download the New Firmware

Do upgrade procedures with the host system at the single-user level.

The following procedures are performed from the VCS PTX Console window via a remote login (telnet) over the Ethernet LAN connected to the IBM 2109 or SilkWorm 2000 FC Switch to be upgraded.

  1. Warn all users and then use the shutdown command to bring the host system down to the single-user level .

  2. Enable basic networking and the inetd daemon from the single-user level with the following commands:

    # /etc/ifconfigall 2
    # /etc/inetd

  3. Verify network operation. For example:

    # netstat -in
    Name Mtu Network Address Ipkts Opkts Odrop
    loop 0 <Link> 0 0 0
    loop 0 127/8 127.0.0.1 0 0 0
    pe0 1502 <Link>00.00.bc.0f.04.ee 57 12 0
    pe0 1502 138.95.103.24 138.95.103.77 57 12 0

    If the Switch is not on the same side of a network router as the host, use the route command on the host system. For example:

    # route fill
    add net default: gateway 138.95.103.215

  4. Initiate a telnet login session with the first Switch to be upgraded and log in as admin. For example:

    (Host_prompt): telnet switch_IP_address
    (Switch_name) login: admin
    Password: admin_password
    sw3:admin>

  5. If this switch is one of a cascaded pair making up a fabric, determine its current domain ID at this time. The domain ID is the first item in the field named "Switch ID" in the following display:

    sw3:admin> fabricShow
    Switch ID Worldwide Name Enet IP Addr FC IP Addr Name
    ------------------------------------------------------------------
    1: fffc41 10:00:00:60:69:00:0a:12 192.168.1.2 0.0.0 >"sw2"
    2: fffc42 10:00:00:60:69:00:01:b4 192.168.1.3 0.0.0 "sw3"

    After the firmware has been upgraded, the ID must be restored to this value before returning the switch to service.


    ATTENTION

    The ">" in the display indicates the Principal switch in the fabric.


  6. Put the Switch into its offline state with the following command:

    sw3:admin> switchDisable

    Observe that the yellow LEDs of all the ports flash steadily, indicating all ports unavailable.

  7. Download the new software into the Switch's flash memory with the following firmwareDownload command:


    ATTENTION

    The syntax of double quotes and commas to separate each argument is important.

    The username in the following command is the account set up on the host system for administering Switches.


    sw3:admin> firmwareDownload "host_IP_address", "username",
    "/usr/ssw/fw/fc_switch/ibm_2109/fc_sw_ibm2109.bin"

    1159644+195092+876080
    writing flash 0 ................
    writing flash 1 ................
    download complete
    value 0 = 0x0
    sw3:admin>

  8. Make the new firmware effective at this time by rebooting the Switch.

    sw3:admin> reboot

    Observe that the green LEDs of all the cabled ports glow steadily, indicating ports available.


    ATTENTION

    During the reboot, the telnet session will time out and be closed by the remote host.


  9. Start a new telnet session with the Switch and log in as admin.

  10. Verify the currently-executing firmware version. For example:

    sw3:admin> version
    VxWorks version: 5.3.1
    Firmware version: a2.2.1a
    Made on: Fri Aug 25 15:47:31 PDT 2000
    Flash: Fri Aug 25 15:47:58 PDT 2000
    Boot PROM: Thu Jun 17 15:20:39 PDT 1999

    where:

    VxWorks version:
    The current version of the Switch's internal operating system. IBM NUMA-Q does not use this information.
    Firmware version:
    The OEM's version ID for the firmware currently executing on the Switch.
    Made on:
    The date and time that the OEM firmware was compiled into a release.
  11. Since the reboot cycle enables the Switch, you must disable it again before issuing a configuration command:

    sw3:admin> switchDisable
    value = 0 = 0x0
  12. Issue the following command to ensure that all configuration values are reset to the factory defaults.

    sw3:admin> configDefault
    value = 0 = 0x0

  13. Turn on in-order frame delivery:

    sw3:admin> iodSet
    Committing configuration ...done.

  14. Turn off the dynamic load-sharing for routing:

    sw3:admin> dlsReset
    Committing configuration ...done.


    ATTENTION

    If the switch being upgraded is part of a DYNIX/ptx V4.6.x system that has only IBM 2109 or SilkWorm 2000-family switches making up the fabrics, then skip the following step and go directly to Step 17.

    If the switch is part of an older system that is being upgraded to DYNIX/ptx V4.6.x and it is cascaded with a SilkWorm 1000-family switch in the same fabric, then additional configuration parameters must be set in the IBM 2109 or SilkWorm 2000-family switch to ensure total interoperability with the older Silkworm 1000-family switch. Continue with the next step in this procedure.


  15. The configure command used in this step presents each configurable parameter one line at a time. The range of values will be shown in the parentheses following the colon of the prompt. Following the range of values, the current value will be shown in the square brackets. Enter a new value at the end of the line and press the Return key. To leave the value as is, make no entry and press the Return key.

    Configure only the "Fabric Parameters." At the end of the list of Fabric Parameters, press Ctrl-D to write the changes and exit from the command. IBM NUMA-Q servers do not use the Virtual Channel and Arbitrated Loop Parameters or System services listed in the rest of the command's parameters. They will retain their default settings.

    Set only these two Fabric Parameters to a value of 1:

    VC Encoded Address Mode
    Disable Translative Mode

    ATTENTION

    If the VC Encoded Address Mode value is left as "0," then the associated parameter Disable Translative Mode will not be displayed by the configure command. The VC Encoded Address Mode must be set to "1" to activate the Disable Translative Mode parameter.



    Leave all other Fabric Parameters unchanged.

    For example:

    sw3:admin> configure

    Configure...
    Fabric parameters (yes, y, no, n): [no] yes
    Domain: (1..239) [1]
    BB credit: (1..16) [16]
    R_A_TOV: (4000..12000) [10000]
    E_D_TOV: (1000..5000) [2000]
    Data field size: (256..212) [2112]
    Non-SCSI Tachyon Mode: (0..1) [0]
    Disable Device Probing: (0..1) [0]
    VC Encoded Address Mode: (0..1) [0] 1
    Disable Translative Mode: (0..1) [0] 1
    Per-frame Routing Priority: (0..1) [0]
    Virtual Channel (yes, y, no, n): [no] Crtl-D
    Committing configuration ...done.
    sw3:admin#
  16. Issue the configure command again and verify that the values are changed. Exit the command with a Crtl-C command after the Fabric Parameters have been reviewed.

    For instance:


    ATTENTION

    Enabling the VC Encoded Address Mode and Disable Translative Modes causes the range for domain IDs to be restricted and changed to 0-31.


    sw3:admin> configure

    Configure...
    Fabric parameters (yes, y, no, n): [no] yes
    Domain: (0..31) [1]
    BB credit: (1..16) [16]
    R_A_TOV: (4000..12000) [10000]
    E_D_TOV: (1000..5000) [2000]
    Data field size: (256..212) [2112]
    Non-SCSI Tachyon Mode: (0..1) [0]
    Disable Device Probing: (0..1) [0]
    VC Encoded Address Mode: (0..1) [1]
    Disable Translative Mode: (0..1) [1]
    Per-frame Routing Priority: (0..1) [0]
    Virtual Channel (yes, y, no, n): [no] Crtl-C
    No changes.
    sw3:admin>
  17. Set the domain ID:


    ATTENTION

    The default domain ID for the V2.x Fabric Operating System family is 1 after a specific "Reset to Default" such as the configDefault command.

    For serviceability reasons, we recommend that each switch have a unique domain ID even though they may be in opposite fabrics. Since the DYNIX/ptx V4.6.x operating system employs the simple name service, it can identify a switch by any ID. This technique allows for easier correlation from the system dumpconf listings to the physical switches.


    If this one switch makes up the entire fabric, set it to an ID different than the switch in the opposite fabric, within the range 1-239.

    If this switch is cascaded to another IBM 2109 or SilkWorm 2000-family switch, match the value determined back in Step 5. This method keeps the new system dumpconf listings the same as the old listings.

    If this switch is cascaded to a SilkWorm 1000-family switch in a mixed fabric, then make its new domain ID unique and within the range 0-31.

    1. Issue the configure command again.

    2. If the default value is the desired value, then use Crtl-C command to quit and exit the command with no changes. For example:

      sw3:admin> configure

      Configure...
      Fabric parameters (yes, y, no, n): [no] yes
      Domain: (0..31) [1] Crtl-C
      No changes.

    3. If the default value inside of the square brackets is not the desired value, then enter the preferred value and use Crtl-D to write the new value into flash and exit the command. For example:

      sw3:admin> configure

      Configure...
      Fabric parameters (yes, y, no, n): [no] yes
      Domain: (0..31) [1] 3
      BB credit: (1..16) [16] Crtl-D
      Committing configuration ... done.

  18. Restore other switch configuration values such as site-specific passwords, usernames, and site-specific identities at this time:

    1. Use the passwd command to recreate any non-default passwords for the username accounts.

    2. Use the passwd command also to rename any of the username accounts.

    3. Use the errDispSet command to restore the error-severity level for reporting to the display panel of a 16-port switch.

    4. Use the agtcfgSet command to restore the variables for sysDescr:, sysLocation:, and sysContact: only.

  19. Power cycle the Switch at this point to enable it and to put the restored values into effect:


    ATTENTION

    Since Switches do not get power cycled very often in a customer production environment, it is a good idea to take advantage of the upgrade opportunity to make a switch go through its entire POST sequence.


  20. Repeat this process for each IBM 2109 or SilkWorm 2000 switch in the system.

  21. When all Switches in a system have been upgraded, boot the system back up to the multiuser level with the following command:

    # init 6

  22. Verify that all cabled port LEDs on the upgraded Switches show a steady green, indicating that each upgraded Switch has re-established FC links to the system's host adapters and to FC-ported storage subsystems that are connected to each Switch.