Back to Contents Page

Broadcom Advanced Server Program (BASP) Driver Software (Linux): Broadcom NetXtreme® 57XX User Guide

NOTES:

Overview

Packaging

Installing Broadcom Advanced Server Program

BASP Files

Configuring Teaming

Installing Broadcom NICE Patches

Uninstalling the BASP RPM Package

Removing a Physical Interface in Generic Trunking and IEEE 802.3ad Mode

Installing BASP SNMP Agent

Known Problems


Overview

Broadcom Advanced Server Program is a kernel module designed for Linux 2.4.x and 2.6.x kernels. See Broadcom Advanced Server Program Overview in Teaming for a detailed description.

Broadcom Advanced Server Program also provides remote management through the SNMP protocol, and this package is installed separately (see Installing BASP SNMP Agent).

Packaging

Broadcom Advanced Server Program is released in 2 packaging formats: source RPM and compressed tar formats. The file names for the 2 packages are basplnx-version.src.arch.rpm and basplnxversion.arch.tgz, respectively. Identical source files to build the driver are included in both RPM and TAR source packages.

Installing Broadcom Advanced Server Program

Broadcom Advanced Server Program for Linux is shipped in mixed forms; the platform and kernel-specific files are in source code, and the core file is in object form. Four packages are shipped in this release: three RPM packages and one tar archive. The tar archive for i386 platform is basplnx-version.i386.tgz.

Installing the BASP Source RPM Package

  1. Run the following:
  2. % rpm -i basplnx-version.src.archrpm
    
  3. Change to the directory containing the RPM package and build the binary driver for the kernel (use rpmbuild for Red Hat Linux 2.1 and 3.0 or later). The path to the RPM package is different for different Linux distributions.
  4. % CD /usr/src/redhat
    
    % rpm -bb SPECS/basplnx.spec
    

    or

    rpmbuild -bb SPECS/basplnx.spec
    

    NOTE: During the installation process, the following message may be displayed:

    error: cannot create %sourcedir /usr/src/redhat/SOURCES 
    
The most likely cause of the error is that the rpm-build package has not been installed. To install the rpm-build package, see Installing the BASP Source RPM Package.
  1. Install the driver and other required files:
  2. % rpm -i RPMS/i386/basplnx-version.archrpm
    
  3. Load the driver:
  4. % modprobe basp
    

See Configuring Teaming for Linux Red Hat Distributions to set up the teams.

Installing the BASP TAR Archive

Uncompress and expand the tar archive:

% tar xvfz basplnx-version.arch.tgz
To install the BASP TAR archive
  1. Change to the directory where the BASP source files are located:
  2. % CD basplnx-version
    
  3. Build the kernel module, basp.o:
  4. % make
    

    NOTE: The Make process automatically builds the correct module for different kernel options, for example, symbol version and SMP support. There is no need to define -DMODVERSIONS in the Makefile.

  5. Create a device file and copy files:
  6. % make install
    
  7. Update the module reference:
  8. % depmod -a
    
  9. Load the driver:
  10. % modprobe basp
    

See Configuring Teaming for Other Linux Distributions to set up the teams.

BASP Files

Table 1: BASP Files
File Name
Description
makefile
makefile
baspcfg
Precompiled configuration utility
bcmtype.h
Commonly used type header file
blf.c
BASP module entry points
blf.h
IOCTL interface
blfcore.h
Core interface
blfcore.o
Precompiled core object
blfopt.h
Automatically generated header file from Make
blfver.h
Version header file
nicext.h
NICE header file
pal.c
Platform abstraction implementation
pal.h
Header for platform abstraction
release.txt
Release notes for BASP driver for Linux 2.4x and 2.6x kernels
nice-2.2.16
NICE enabled driver for Linux 2.2 kernel
nice-2.4.16
NICE enabled driver for Linux 2.4 kernel
scripts
Contains sample scripts
scripts/basp
Initialization script, goes to /etc/rc.d/init.d
scripts/baspteam
Start/stop script, goes to /etc/basp
scripts/baspif
Start/stop network, interface, goes to /etc/basp
scripts/team-sample
Sample script of an SLB team with three adapters
scripts/team-gec
Sample script of GEC team with three adapters
scripts/team-vlan
Sample script of an SLB team with 2 VLANs
basp.4
man page
baspcfg.8
man page for baspcfg utility

Configuring Teaming

Configuring Teaming for Linux Red Hat Distributions

NOTES:

Because Red Hat distribution installations do not automatically load drivers for network devices unless the device is configured with an IP address, you must manually configure a network script file for all of the physical (installed) adapters that are potential team members. Network script files are located in the /etc/sysconfig/network-scripts file. The file name of the script file must be prefixed with ifcfg- and followed by the physical adapter alias. For interface eth0, for example, create a file with the name ifcfg-eth0,and then add the content, as shown in the following example.

Example:

DEVICE = eth0
BOOTPROTO = static
ONBOOT = yes

Broadcom Advanced Server Program includes several scripts for configuring teams for Linux Red Hat distributions.

NOTE: Enabling Dynamic Host Configuration Protocol (DHCP) is not recommended for members of an SLB team

To configure a team

  1. Copy a configuration script from the /etc/basp/samples directory to the /etc/basp directory. The configuration script name must be prefixed with team-.
  2. Modify the configuration script:
    1. Change the team type.
    2. Add/delete a physical network interface.
    3. Add/delete a virtual network interface.
    4. Assign an IP address to each virtual network interface.

See Using BASP Scripts for Configuring Teams for the syntax of the configuration script or in the /etc/basp/sample/team-sample script file. When configuring teaming, you must designate at least one adapter as a primary team member.

  1. Manually start the team for the first time:
  2. % /etc/init.d/basp start
    

    NOTES:

  3. This step is required only for the initial installation; the team configuration is automatically started on subsequent reboots.
  4. If not all the virtual network interfaces are configured with an IP address, an error in starting the BASP team results. When this happens, modify the configuration script to assign an IP address for all of the virtual network interfaces.
  5. Forming multiple teams is possible by copying the sample files to the /etc/basp/team-name file and modifying this file as described in the sample file.
  6. For instructions on how to create more that one virtual interface (VLAN) for each team, refer to the appropriate section in the sample files.
Using BASP Scripts for Configuring Teams

The team-sample configuration script creates an SLB team named Team1. The team consists of 3 network interfaces, eth0, eth1, and eth2. All 3 interfaces are primary adapters. One virtual interface named sw0 is added to the team and the VLAN is not enabled. This script is part of the Broadcom Advanced Server Program driver for Linux distributions.

The syntax used in the sample scripts, team-sample and team-gec (see BASP Files) is the same as the syntax shown in the following table:

Table 2: BASP Scripts for Configuring Teams  
Configurable Property
Description
TEAM_ID
This number uniquely identifies a team.
TEAM_TYPE
0 = SLB
1 = Generic Trunking/GEC/FEC
2 = 802.3ad
3 = SLB (Auto-Fallback Disable)
LIVELINK_ENABLE
1 = SLB (with LiveLink)
0 = SLB (without LiveLink)
LiveLink is supported for SLB teams only. A value of 1 is ignored when TEAM_TYPE is not 0 (SLB).
TEAM_NAME
ASCII name of the team.
TEAM_PAx_NAME
ASCII name of the physical interface x, where x can be 0 to 7.
TEAM_PAx_ROLE
Role of the physical interface x:
0 = Primary
1 = Hot standby.
This field must be 0 for Generic Trunking/GEC/FEC teams and 802.3ad teams.
TEAM_PAx_IP
Unique IP address to be used by each adapter when LiveLink is enabled. The format should be x.x.x.x.
If an IP address unique to the subnet is not assigned, the adapter will not be used when LiveLink is enabled.
TEAM_VAx_NAME
ASCII name of the virtual interface x, where x can be 0 to 63.
TEAM_VAx_VLAN
802.1p VLAN ID of the virtual interface x.
For an untagged virtual interfaces (without VLAN enabled) , set the VLAN ID to 0. The valid VLAN ID can be 0 to 4094.
TEAM_VAx_IP
IP address of the virtual interface x. The format should be aa.bb.cc.dd.
TEAM_VAx_NETMASK
Subnet mask of the virtual interface x. The format should be mm.nn.oo.pp.
TEAM_VAx_BROADCAST
Optional broadcast address of the virtual interface x. The format should be qq.rr.ss.tt.
TEAM_VAx_GW
Optional default gateway. The format should be ww.xx.yy.zz. Usually one default gateway is specified for the system, and it should be reachable from one network interface.
PROBE_TARGET_IPx
Target IP addresses for the LiveLink option. The format shoud be x.x.x.x. The first LiveLink probe target is required when LiveLink is enabled. Up to 3 additional probe targets may be defined.
PROBE_INTERVAL
The interval in seconds between sending LiveLink packets to probe targets. Allowable values are 1, 2, 5, 10, 20. 30 and 60.
MAX_RETRY
The number of consecutively missed responses from a LiveLink probe target before a failover is triggered. The value is n * 5 (for example, an n setting of 5 results in a value of 25 (5 * 5)). Allowable values of n are 1 to 10.

NOTE: Teaming scripts are intended for Red Hat distributions only. Do not use teaming scripts with other Linux distributions.

Configuring Teaming for Other Linux Distributions

Broadcom Advanced Server Program Configuration (baspcfg) is a command-line tool to configure the BASP teams, add/remove NICs, and add/remove virtual devices. This tool can be used in custom initialization scripts. Read your distribution-specific documentation for more information on startup procedures.

Example:

baspcfg v6.2.7 — Broadcom Advanced Server Program Configuration Utility Copyright (c) 2000–2004 Broadcom Corporation. All rights reserved.

Usage: baspcfg command

BASP Configuration Commands

Table 3: BASP Configuration Commands
Command
Action
addteam tid type tname
Create a team.
addteam tid type tname target ip 0–3 max interval max retries
Create a LiveLink team.
delteam tid
Delete a team.
txoffld tid y|n
Enable or disable tx offload features
addva tid vlan_id vname [macaddr]
Add a virtual adapter to a team.
delva tid vlan_id
Delete a virtual adapter from a team.
bind tid role device
Bind a physical adapter to a team.
bind tid role device nic ip
Bind a LiveLink physical adapter to a team.
unbind tid device
Unbind a physical adapter from a team.
show tid
Display team configurations.

BASP Configuration Command Placeholders

Table 4: BASP Configuration Command Placeholders
Placeholder
Description
tid
A unique ID for each team, starting from 0.
type
Team type:
0 = SLB
1 = FEC/GEC
2 = 802.3ad
3 = SLB (Auto-Fallback Disable)
tname
ASCII string of the team.
vlan_id
VLAN ID: from 1 to 4094, 0 = untagged or no VLAN.
vname
ASCII string of the virtual device.
macaddr
MAC address (optional), for example, 00:10:18:00:11:44.
role
Role of the physical device:
0 = primary
1 = hot-standby
device
ASCII string of the physical device, for example. eth0.
target/nic ip
Probe target IP address (for example, 192.168.1.1)
probe interval
The interval in seconds between sending LiveLink packets to probe targets. Allowable values are 1, 2, 5, 10, 20. 30 and 60.
max retry
The number of consecutively missed responses from a LiveLink probe target before a failover is triggered. The value is n * 5 (for example, an n setting of 5 results in a value of 25 (5 * 5)). Allowable values of n are 1 to 10.

NOTE: The baspcfg command can be run only in Super User mode. Attempting to run baspcfg as a standard user returns the following error message:

Error in communicating to BASP Module. Is it loaded?

When configuring teaming, you must designate at least one adapter as a primary team member.

Configuring LiveLink

Read the following notes before you attempt to configure LiveLink™.

NOTES:

To configure LiveLink

  1. Open the BASP directory.
  2. /etc/basp
    
  3. Change to the SAMPLES directory.
  4. cd samples
    
  5. Copy the team-sample file to the BASP directory.
  6. cp /etc/basp/samples/team-sample /etc/basp/
    
  7. Edit the team-sample script to enable LiveLink, assign IP addresses to probe targets, assign IP addresses to team members, specify the time interval for sending link packets to probe targets, and to specify the maximum number of probe retries. You must delete the # sign at the beginning of each respective line item.
    1. Go to #LIVELINK_ENABLE=0 and change the value from 0 to 1.
    2. Go to #PROBE_TARGET_IP0= and type the target IP address.
    3. NOTE: Only the first probe target is required. You can specify up to 3 additional probe targets to serve as backups by assigning IP addresses to the other #PROBE_TARGET_IPx= line items

    4. Go to #TEAM_PA0_IP= (the first physical Interface in the team) and type the team member IP address.
    5. NOTE: You must assign an IP address to each team member for that member interface to be used by the team.

    6. Go to #PROBE_INTERVAL=5 and accept the default value (recommended) by deleting the # sign. To specify a different value, type the desired value in seconds within the predefined range.
    7. Go to #MAX_RETRY=5 and accept the default value (recommended) by deleting the # sign. To specify a different value, type the desired value in seconds within the predefined range.
  8. Save the file.
  9. NOTES:

  10. Before you start the team, it is recommended that you use the ifconfig command to temporarily bring up one of the member interfaces using the IP address that you specified for that member in step 4.C. Then ping the probe target that you specified in step 4.B to verify connectivity. After you confirm that there is connectivity, type the ifconfig down command to bring down the interface. Now you are ready to start the team.
  11. To view the status of LiveLink activity (the number of probes sent and the number of probe retries) for the team, use the baspcfg show command.
  12. CAUTION! If the IP address of the probe target is changed, the value of PROBE_TARGET_IPx must be updated, and the team must be restarted.

To configure LiveLink in VLAN-tagged environments

CAUTION! For the teams with VLANs (on which LiveLink is enabled): to be able to communicate with the probe target, both the probe target and the team must be on VLAN ID 0 (untagged). Otherwise, the team loses connectivity.

  1. Ensure that the team has an untagged VLAN (VLAN ID 0).
  2. Ensure there is network connectivity between the team and the probe target on the untagged VLAN.
  3. Configure LiveLink as specified in Configuring LiveLink.

Installing Broadcom NICE Patches

Also included in this release are network device drivers patched with Broadcom NICE support. These drivers are originally taken from the Linux 2.4.16 kernel distribution. To install patched drivers:

  1. Copy the Broadcom NICE header file (nicext.h) to the appropriate Linux kernel include directory:
  2. % cp /usr/src/nice-2.4.16/nicext.h /usr/src/linux/include/linux
    
  3. Rename the original network device driver under the Linux kernel source tree (/usr/src/linux/drivers/net).
  4. Copy the patched drivers to the Linux kernel network driver source directory (for example, /usr/src/linux/drivers/net).
  5. Follow the kernel rebuild instructions to configure kernel support for these drivers:
  6. % CD /usr/src/linux
    
    % make config
    
  7. If the patched drivers are configured into the kernel, go to step 7. If the patched drivers are configured as modules, go to step 6.
  8. When supporting only the module version of these drivers, it is possible to run the following to compile patched drivers and to install them into the proper module directory:
  9. % make modules
    
    % make modules_install
    

There is no need to compile the complete kernel. Go to step 8.

  1. Rebuild the kernel to compile these patched drivers:
  2. % make clean 
    
    % make dep 
    
    % make
    
  3. Either reboot the system or unload/load the patched modules. Then run the configuration scripts to test the patch.

Uninstalling the BASP RPM Package

  1. Uninstall the BASP RPM package:
  2. % rpm -e basplnx
    
  3. Reboot the system:
  4. % reboot
    

Removing a Physical Interface in Generic Trunking and IEEE 802.3ad Mode

In Generic Trunking and IEEE 802.3ad mode, all of the physical and virtual interfaces belonging to a team have the same MAC address. This MAC address is the same address as that of the first physical interface that is bound to the team. Removing the first physical interface from the team using baspcfg and binding it directly to the protocol could lead to having a duplicate MAC address problem on the network. If the removed physical interface does not participate in any traffic, no problem occurs.

To properly remove a physical interface

  1. Backup the original team configuration script:
  2. % cp /etc/basp/team-gec /etc/basp/backup-gec
    

    NOTE:

  3. team-gec is the name of the configuration script.
  4. backup-gec is the name of the backup script. The name of the backup script must not be prefixed with team-.
  5. Modify the team configuration script to remove the physical interface.
  6. Stop the team that is running:
  7. % /etc/basp/baspif /etc/basp/backup-gec stop
    % /etc/basp/baspteam /etc/basp/backup-gec del
  8. Restart the team.
  9. % /etc/basp/baspteam /etc/basp/team-gec add
    % /etc/basp/baspif /etc/basp/team-gec start

Installing BASP SNMP Agent

This BASP SNMP agent is designed to support the configuration and statistics information pertaining to the BASP driver. The BASP SNMP agent is available in two packaging formats: TAR archive and RPM. Both packages include the same script and MIB files.

Installing BASP SNMP Agent from the TAR Archive
  1. Uncompress and expand the tar archive:
  2. % tar xvfz baspsnmp-version.tar
    
  3. Install the TAR archive.
  4. Copy the getBaspInfo and genBaspTraps script files to the /usr/bin directory.
  5. Copy the BASP-Config-MIB.txt, BASP-Statistics-MIB.txt and Brcm-BSAPTrap-MIB.txt files to the /usr/share/snmp/mibs directory.
  6. Locate the snmpd.conf file (it is normally located at: /etc/snmp or /usr/lib/snmp or $HOME/.snmp) and add the following lines to the snmpd.conf file:
  7. pass .1.3.6.1.4.1.4413.1.2.1 /usr/bin/getBaspInfo
    
    pass .1.3.6.1.4.1.4413.1.2.2.1 /usr/bin/getBaspInfo
    
    pass .1.3.6.1.4.1.4413.1.2.2.2 /usr/bin/getBaspInfo
    
    pass .1.3.6.1.4.1.4413.1.2.2.3 /usr/bin/getBaspInfo
    
  8. Stop the snmpd daemon and restart it:
  9. % /etc/init.d/snmpd stop
    % /etc/init.d/snmpd start
  10. Run the genBaspTraps script to allow monitoring of the BASP trap events:
  11. % genBaspTraps 
    

If BASP trap event monitoring is no longer needed, terminate this script by pressing CTRL+C.

The snmpget and snmpgetnext commands can be used to receive the BASP SNMP objects such as:

% snmpget localhost public BASP-Config-MIB::btTeamNumber
% snmpgetnext localhost public BASP-Config-MIB::btTeamNumber

BASP SNMP objects are provided in the following text files:

Installing BASP SNMP Agent from the RPM Package
  1. Install BASP SNMP Agent from the RPM package:
  2. % rpm -i baspsnmp-version.i386.rpm
    

    This modifies the snmpd.conf configuration file to add support for the BASP SNMP agent.

    NOTE: The current RPM installation fails to append the additional directives to the snmpd.conf file that are needed to support BASP objects. To modify the snmpd.conf file, see step 3 in Installing BASP SNMP Agent from the TAR Archive.

BASP SNMP Files

Table 5: BASP SNMP Files
Name
Description
genBaspTrap
Script monitoring the BASP trap events
getBaspInfo
Script to process SNMP get/getnext inquiries
BASP-Config-MIB.txt
SNMP MIB file for BASP configuration objects
BASP-Statistics-MIB.txt
SNMP MIB file for BASP statistics objects
Brcm-BSAPTrap-MIB.txt
SNMP MIB file for BASP trap objects
release.txt
Release notes about the BASP SNMP agent

Uninstalling BASP SNMP Agent from the RPM Package
  1. Uninstall BASP SNMP.
  2. % rpm -e baspsnmp-version.i386.rpm
    
  3. Reboot the system.
  4. % reboot
    

Known Problems

IEEE 802.3ad team-member links disconnect and reconnect continuously when they are connected to the HP2524 switch. This is a third-party issue. It is seen only when configuring an IEEE 802.3ad team with greater than 2 members on the system and connecting an HP2524 switch with LACP enabled as passive or active. The HP switch shows an LACP channel being brought up successfully with only 2 members. All other member's links disconnect and reconnect. This does not occur with a Cisco Catalyst 6500.


Back to Contents Page