Back to Contents Page

Linux Driver Software: Broadcom NetXtreme® 57XX User Guide

TG3 Driver Software

Broadcom BCM57XX Linux Driver


TG3 Driver Software

Limitations

Packaging

Installing TG3 Driver Software

Network Installations

Unloading/Removing the TG3 Driver

Driver Messages

Teaming with Channel Bonding

Limitations

Refer to the DISTRIB.TXT file on the installation CD for a list of the specific Linux distributions on which the driver has been tested.

Packaging

The Linux driver is released in the following packaging formats (file names):

Identical source files to build the driver are included in both RPM and TAR source packages. The tar file contains additional utilities such as patches and driver disk images for network installation.

Installing TG3 Driver Software

Building the Driver from the Source TAR File
  1. Create a directory (tg3-version) and extract the TAR files to the directory.
  2. tar xvzf tg3-version.tgz
    
  3. Build the driver tg3.o as a loadable module for the running kernel.
  4. CD tg3-version
    
    make clean
    
    make; make install
    
  5. Test the driver by loading it.
  6. rmmod tg3
    
    modprobe tg3
    

    No message should be returned if this command runs properly.

    NOTE: See the RPM instructions above for the location of the installed driver.

  7. To configure network protocol and address, refer to the manuals supplied with your operating system.

Network Installations

For network installations through NFS, FTP, or HTTP (using a network boot disk or PXE), a driver disk that contains the tg3 driver may be needed. The driver disk images for the most recent Red Hat versions are included. Boot drivers for other Linux versions can be compiled by modifying the Makefile and the make environment. Further information is available from the Red Hat website, http://www.redhat.com.

To create the driver disk, select the appropriate image file (located in tg3_sup-version.tar.gz) and type the following:

dd if=dd.img of=/dev/fd0

Unloading/Removing the TG3 Driver

Unloading/Removing the Driver from an RPM Installation

Removing the Driver from a TAR Installation

Unloading/Removing the Driver from an RPM Installation

To unload the driver, use ifconfig to bring down all ethX interfaces opened by the driver, and then type the following:

rmmod tg3

If the driver was installed using rpm, do the following to remove it:

rpm -e tg3-<version>
Removing the Driver from a TAR Installation

If the driver was installed using make install from the tar file, the tg3.o driver file has to be manually deleted from the operating system. See Installing the Source RPM Package for the location of the installed driver.

If there is an interface configuration that is related to the tg3 driver, then bring the interface down first by using ifconfig ethx down and then rmod tg3.

Driver Messages

The following are the most common sample messages that may be logged in the /var/log/messages file. Use dmesg -nlevel to control the level at which messages appear on the console. Most systems are set to level 6 by default.

Driver Signon

tg3.c:version (date) 

NIC Detected

eth#: Tigon3 [partno (BCM95xxx) rev 4202 PHY (57xx) (PCI Express) 10/100/1000BaseT Ethernet :00:xx:xx:xx:xx:xx
eth#: RXcsums [1] LinkChg REG [0] MIirq [0] ASF [0] Split [0] Wirespeed [1]TSOcap [1]
eth#: dma_rwctrl [76180000]
ACPI : PCI interrupt 0000:02:02.0 [A] -> GSI 26 (level,low) -> IRQ 233 

Flow Control

tg3: eth#: Flow control is configured for TX and for RX.

Link Up and Speed Indication

tg3: eth#: Link is up at 1000 Mbps, full duplex.

Link Down Indication

tg3: eth#: Link is down.

Teaming with Channel Bonding

With the TG3 driver, you can team adapters together using the bonding kernel module and a channel bonding interface. Information on channel bonding can be found at http://www.redhat.com/docs/manuals/enterprise/RHEL-3-Manual/ref-guide/s1-modules-ethernet.html.

Broadcom BCM57XX Linux Driver

Limitations

Packaging

Installing Linux Driver Software

Patching PCI Files (Optional)

Patching the Driver Into the Kernel (Optional)

Network Installations

Unloading/Removing the Linux Driver

Setting Values for Optional Properties

Driver Messages

Statistics

Broadcom Advanced Server Program Driver Software for Linux

Configuring Teaming for Linux Red Hat Distributions

Limitations

Refer to the DISTRIB.TXT file on the installation CD for a list of the specific Linux distributions on which the adapter driver has been tested .

Packaging

The Linux driver is released in the following packaging formats (file names):

Identical source files to build the driver are included in both RPM and TAR source packages. The tar file contains additional utilities such as patches and driver diskette images for network installation.

Installing Linux Driver Software

Installing the Source RPM Package
  1. Install the source RPM package:
  2. rpm -ivh bcm5700-version.src.rpm
    
  3. Change the directory to the RPM path and build the binary driver for your kernel (the RPM path is different for different Linux distributions):
  4. cd /usr/src/redhat,OpenLinux,turbo,packages,rpm ...
    
    rpm -bb SPECS/bcm5700.spec or rpmbuild -bb SPECS/bcm5700.spec
    
    rpmbuild -bb SPECS/bcm5700.spec (for RPM version 4.x.x)
    

    NOTE: During your attempt to install a source RPM package, the following message may be displayed:

    error: cannot create %sourcedir /usr/src/redhat/SOURCES
    

    The most likely cause of the error is that the rpm-build package has not been installed. Locate the rpm-build package on the Red Hat installation media and install it using the following command:

    rpm -ivh rpm-build-version.i386.rpm
    

    Then complete the installation of the source RPM.

  5. Install the newly-built package (driver and man page):
  6. rpm -ivh RPMS/i386/bcm5700-version.i386.rpm
    

    The --force option is needed if installing over an existing distribution that may already contain an older version of the driver.

    Depending on the kernel, the driver is installed to one of the following paths:

    2.4.x kernels:

    /lib/modules/kernel_version/kernel/drivers/net/bcm5700.o

    2.4.x kernels with the bcm5700 driver patched in:

    /lib/modules/kernel_version/kernel/drivers/net/bcm/bcm5700.o

    -or-

    /lib/modules/kernel_version/kernel/drivers/addon/bcm5700/bcm5700.o

    2.6.0 kernels:

    /lib/modules/kernel_version/kernel/drivers/net/bcm5700.ko

    2.6.0 kernels with bcm5700 driver patched in:

    /lib/modules/kernel_version/kernel/drivers/net/bcm/bcm5700.ko

  7. Load the driver:
  8. modprobe bcm5700
    

To configure the network protocol and address, refer to the Linux version-specific documentation.

Building the Driver from the TAR File
  1. Create a directory and extract the TAR files to the directory:
  2. tar xvzf bcm5700-version.tar.gz
    
  3. Build the driver bcm5700.o as a loadable module for the running kernel:
  4. CD src
    
    make
    

    NOTE: If you are loading the driver on Red Hat 7.3, 2.1 AS, or other newer kernels that have the tg3 driver, refer to "Remove tg3 Driver" in the Distrib.txt file before loading the driver.

  5. Test the driver by loading it:
  6. insmod bcm5700.o
    

    -or-

    for Linux 2.6 kernels:

    insmod bcm5700.ko
    

    No message should be returned if this command runs properly.

  7. Install the driver and man page:
  8. make install
    

    NOTE: See the RPM instructions above for the location of the installed driver.

  9. To configure network protocol and address, refer to the manuals supplied with your operating system.

Patching PCI Files (Optional)

To use the Red Hat kudzu hardware detection utility, a number of files containing PCI vendor and device information need to be patched with information on the BCM57XX series NICs. The most recent Red Hat distributions are included. Apply the appropriate patch by running the patch command. For example, on Red Hat Enterprise Linux 2.1 and 3.0 for i386, apply the patch by doing the following:

patch -N -p1 -d /usr < pci-rh80-i386.patch

Run kudzu:

kudzu

Patching the Driver Into the Kernel (Optional)

Patch files are included for patching the driver into some of the latest 2.4.x and 2.6.x kernel source trees. This step is optional and should only be done by users familiar with configuring and building the kernel. The patch modifies the original kernel source code.

To patch the driver into the kernel

  1. Select the patch file that matches your kernel and apply the patch:
  2. patch -p1 -d kernel_src_root < bcm5700-version-2.x.x.patch
    

    where version is the version of the BCM57XX driver and 2.x.x is the version of the kernel to patch (for example, 2.4.10).

    NOTE: kernel_src_root is usually /usr/src/linux or /usr/src/linux-2.x.x.

  3. Configure the kernel to include the bcm570x driver. It can be found under Network Device Support > Ethernet (1000 Mbps) > Broadcom BCM5700 support when make menuconfig is run. Select built-in or module for the driver:
  4. cd kernel_src_root
    
    make menuconfig 
    
  5. Compile the kernel:
  6. make dep
    
    make clean
    
    ....
    
    ....
    

Network Installations

For network installations through NFS, FTP, or HTTP (using a network boot disk or PXE), a driver disk that contains the BCM57XX driver may be needed. The driver disk images for the most recent Red Hat versions are included. Boot drivers for other Linux versions can be compiled by modifying the Makefile and the make environment. Further information is available from the Red Hat website, http://www.redhat.com.

To create the driver disk, select the appropriate image file and type the following:

dd if=dd.img of=/dev/fd0H1440

Unloading/Removing the Linux Driver

Unloading/Removing the Driver from an RPM Installation

To unload the driver, use ifconfig to bring down all eth# interfaces opened by the driver, and then type the following:

rmmod bcm5700

If the driver was installed using rpm, do the following to remove it:

rpm -e bcm5700-<version>
Removing the Driver from a TAR Installation

If the driver was installed using make install from the tar file, the bcm5700.o driver file has to be manually deleted from the operating system. See Installing the Source RPM Package for the location of the installed driver.

Setting Values for Optional Properties

Values for optional properties can be set for each installed adapter using command line arguments in the modprobe command. Typically, the values for the properties are set in the /etc/modules.conf file (see the man page for the modules.conf file). These properties take the following form:

property=value[,value,...]

Multiple values for the same property are for multiple adapters installed in the system.

NOTES:

line_speed

The line_speed property selects the line speed of the link. This property is used together with the full_duplex and auto_speed properties to select the speed and duplex operation of the link and the setting of auto-negotiation.

0 = Auto-negotiates for the highest speed supported by link partner (default)

10 = Sets the speed at 10 Mbit/s

100 = Sets the speed at 100 Mbit/s

1000 = Sets the speed at 1000 Mbit/s

If line_speed is set to 10, 100, or 1000 Mbit/s, the adapter auto-negotiates for the selected speed (and selected duplex mode) if auto_speed is set to 1. If auto_speed is set to 0, the selected speed and duplex mode are set without auto-negotiation. The 1000 Mbit/s speed must be negotiated for copper twisted-pair links. Forced 1000 Mbit/s speed is not needed for copper links; auto-negotiation is the proper supported configuration under the IEEE Ethernet specification.

auto_speed

Enables or disables auto-negotiation. This property is ignored and is assumed to be 1 if line_speed is set to 0.

0 = Disables auto-negotiation

1 = Enables auto-negotiation (default)

full_duplex

Selects the duplex mode of the link. This property is used together with the line_speed property to select the speed and duplex mode of the link. The full_duplex property is ignored if the line_speed property is set to 0.

0 = Sets the mode to half-duplex

1 = Sets the mode to full-duplex (default)

rx_flow_control

Enables or disables receiving flow control (PAUSE) frames. This property is used together with the auto_flow_control property.

0 = Disables receiving PAUSE frames

1 = Enables receiving PAUSE frames if the auto_flow_control property is set to 0, or advertises PAUSE frame receive if the auto_flow_control property is set to 1 (default)

tx_flow_control

Enables or disables transmitting flow control (PAUSE) frames. This property is used together with the auto_flow_control property.

0 = Disables the transmission of PAUSE frames

1 = Enables the transmission of PAUSE frames if the auto_flow_control property is set to 0, or advertises PAUSE frame transmit if the auto_flow_control property is set to 1 (default)

auto_flow_control

Enables or disables the auto-negotiation of flow control. This property is used together with the rx_flow_control and tx_flow_control properties to determine the advertised flow control capability.

0 = Disables flow control auto-negotiation

1 = Enables flow control auto-negotiation with the capability specified in the rx_flow_control and tx_flow_control properties (only valid if the line_speed property is set to 0 or the auto_speed property is set to 1) (default)

mtu

Enables jumbo frames up to the specified MTU size. The valid range for this property is 1500 to 9000. The default value is 1500, which is standard Ethernet (non-jumbo) MTU size. Note that the MTU size excludes the Ethernet header size of 14 bytes. The actual frame size is MTU size + 14 bytes. Jumbo MTU sizes are not supported on BCM5705/BCM5721/BCM5751 chips.

The MTU size can also be changed using ifconfig after the driver is loaded. Refer to the ifconfig man page for details.

tx_checksum

Enables or disables hardware transmit TCP/UDP checksum.

0 = Disables hardware transmit TCP/UDP checksum

1 = Enables hardware transmit TCP/UDP checksum (default)

rx_checksum

Enables or disables hardware receive TCP/UDP checksum.

0 = Disables hardware receive TCP/UDP checksum

1 = Enables hardware receive TCP/UDP checksum (default)

scatter_gather

Enables or disables scatter/gather and 64-bit DMA on IA32. This option is only useful when running on TUX-enabled kernels or kernels with zero-copy TCP.

0 = Disables scatter/gather and 64-bit DMA on IA32

1 = Enables scatter/gather and 64-bit DMA on IA32 (default)

tx_pkt_desc_cnt

Configures the number of transmit descriptors. The default value is 100, and the valid range of values is from 1 to 600. Depending on kernel and system architecture, the driver may require up to 268 bytes per descriptor. The driver may not be able to allocate the required amount of memory if this property is set too high. This property should not be set to a value of less than 80 if the adaptive_coalesce property is enabled.

rx_std_desc_cnt

Configures the number of receive descriptors for frames up to 1528 bytes. The default value is 200, and the valid range of values is from 1 to 511. This property should not be set with a value less than 80 on systems with high network traffic. Setting the value of this property higher allows the adapter to buffer larger bursts of network traffic without dropping frames, especially on slower systems. Note that the driver may not be able to allocate the required amount of memory if the value of this property is set too high. This property should not be set to a value of less than 50 if the adaptive_coalesce property is enabled.

rx_jumbo_desc_cnt

Configures the number of receive descriptors for jumbo frames larger than 1528 bytes. The default value is 128 and the valid range of values is from 1 to 255. When jumbo frames larger than 1528 bytes are used, the value of this property should not be set lower than 60 on systems with high network traffic. Setting the value of this property higher allows the adapter to buffer larger bursts of jumbo traffic without dropping frames, especially on slower systems. Depending on kernel and system architecture, the driver may require up to 268 bytes per descriptor. Each descriptor also requires a buffer the size of a maximum jumbo frame. On systems with insufficient memory, it may be necessary to set a lower value for this property. When the maximum frame size is less than 1528 (MTU size less than 1514), this property is not used and always has a value of 0.

adaptive_coalesce

Enables or disables adaptive adjustments to the various interrupt coalescing properties. Enabling this property allows the driver to dynamically adjust the interrupt coalescing properties to achieve high throughput during heavy traffic and low latency during light traffic. The value of the rx_std_desc_cnt property (and the rx_jumbo_desc_cnt property if using jumbo frames) should not be set less than 50, and the value of the tx_pkt_desc_cnt property should not be set to a value less than 80 when this property is enabled.

0 = Disables adaptive adjustments to the various interrupt coalescing properties

1 = Enables adaptive adjustments to the various interrupt coalescing properties (default)

rx_coalesce_ticks

Configures the number of 1-microsecond ticks before the adapter generates a receive interrupt after receiving a frame. This property works in conjunction with the rx_max_coalesce_frames property. An interrupt is generated when either of these thresholds is exceeded. A 0 means this property is ignored and an interrupt is generated when the rx_max_coalesce_frames threshold is reached. The valid range of values is from 0 to 500, and the default value is 80. This property is not used and is adjusted automatically if the adaptive_coalesce property is set to 1.

rx_max_coalesce_frames

Configures the number of received frames before the adapter generates a receive interrupt. The valid range of values is from 0 to 100, and the default value is 15. The values of both this property and rx_coalesce_ticks property cannot be set to 0; otherwise, no receive interrupts are generated. The value should also be set significantly lower than the value of rx_std_desc_cnt property (and the rx_jumbo_desc_cnt property if using jumbo frames). This property is not used and is adjusted automatically if the adaptive_coalesce property is set to 1.

tx_coalesce_ticks

Configures the number of 1-microsecond ticks before the adapter generates a transmit interrupt after transmitting a frame. This property works in conjunction with the tx_max_coalesce_frames property. An interrupt is generated when either of these thresholds is exceeded. A 0 means this property is ignored and an interrupt is generated when the tx_max_coalesce_frames threshold is reached. The valid range of values is from 0 to 500, and the default value is 200. This property is not used and is adjusted automatically if the adaptive_coalesce property is set to 1.

tx_max_coalesce_frames

Configures the number of transmitted frames before the adapter generates a transmit interrupt. The valid range of values is from 0 to 100, and the default value is 35. The values of both this property and the tx_coalesce_ticks property cannot be set to 0. Otherwise, no transmit completion interrupt can be generated. This property should always be set to a value lower than the value of the tx_pkt_desc_cnt property. This property is not used and is adjusted automatically if the adaptive_coalesce property is set to 1.

stats_coalesce_ticks

Configures the number of 1-microsecond ticks between periodic statistic block DMAs. The valid range of values is from 0 to 3 600 000 000, and the default value is 1 000 000 microseconds (1 second). Setting this property to 0 disables statistics updates. This property is not used and is set to the default value if the adaptive_coalesce property is set to 1.

enable_wol

Enables or disables Magic Packet™ Wake on LAN when the system is shut down. Note that not all systems support Wake on LAN.

0 = Disables Magic Packet Wake on LAN (default)

1 = Enables Magic Packet Wake on LAN

enable_tso

Enables or disables the TCP Segmentation Option (TSO) when you are using kernels that support it.

0 = Disables TSO (default)

1 = Disables TSO

vlan_tag_mode

Controls the stripping of VLAN tags on incoming packets and is used to allow VLAN tagged ASF or IPMI packets to be received properly.

0 = Auto mode (default)

1 = Normal strip mode

2 = Forced strip mode

In normal mode, VLAN tags are stripped only if VLANs are registered by the IEEE 802.1q VLAN module or BASP. In forced strip mode, VLAN tags are always stripped. Auto mode selects normal strip mode if ASF/IPMI is disabled, or selects forced strip mode if ASF/IPMI is enabled.

delay_link

If the delay_link property is set to 1, the driver returns -EOPNOTSUPP when the SIOCGMIIREG or ETHTOOL_GLINK I/O controls are called during the first 6 seconds after driver reset. When the driver resets the adapter during ifconfig, the link is dropped, and it may take several seconds for the link to come up after auto-negotiation is completed. Some applications, such as ifup, may not wait long enough for the link before giving up. Setting this property to 1 may resolve such problems. The default value is 0, which means that the driver always return true link states to all I/O control calls, when applicable.

disable_d3hot

If the disable_d3hot property is set to 1, the driver never puts the adapter in the D3Hot power state when the adapter is shut down or is suspended. If set, this property also disables the Wake on Lan setting. A rare D3Hot related problem was seen during repeated shutdown of PCI Express devices on systems running 2.6 kernels.

Driver Messages

The following are the most common sample messages that may be logged in the /var/log/messages file. Use dmesg -nlevel to control the level at which messages appear on the console. Most systems are set to level 6 by default.

Driver Signon

Broadcom Gigabit Ethernet Driver bcm5700 with Broadcom NIC Extension (NICE) version (date)

NIC Detected
eth#: Broadcom BCM5701 1000Base-T found at mem faff0000, IRQ 16, node addr 0010180402d8
eth#: Broadcom BCM5701 Integrated Copper transceiver found
eth#: Scatter-gather ON, 64-bit DMA ON, Tx Checksum ON, Rx Checksum ON
Link Up and Speed Indication
bcm5700: eth# NIC Link is Up, 1000 Mbps full duplex
Link Down Indication
bcm5700: eth# NIC Link is Down

Statistics

Detailed statistics and configuration information can be viewed in the /proc/net/nicinfo/eth#.info file.

Broadcom Advanced Server Program Driver Software for Linux

For information on Broadcom Advanced Server Program (BASP), see Broadcom Advanced Server Program (BASP) Driver Software (Linux).

Configuring Teaming for Linux Red Hat Distributions

For information on configuring teaming for Linux Red Hat distributions, see Configuring Teaming.


Back to Contents Page