Bug 89113

Summary: (NET TG3) tg3 driver will not receive packets if MTU is (dynamically) set to greater than 1500
Product: Red Hat Enterprise Linux 3 Reporter: Mark Harig <maharig>
Component: kernelAssignee: David Miller <davem>
Status: CLOSED ERRATA QA Contact:
Severity: high Docs Contact:
Priority: medium    
Version: 3.0CC: benlu, binand, jgarzik, k.georgiou, mvoelker, tim
Target Milestone: ---   
Target Release: ---   
Hardware: i686   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2009-12-02 19:02:56 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Fix for MTU changing bug. none

Description Mark Harig 2003-04-17 19:12:27 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0.1) Gecko/20021003

Description of problem:
When a Broadcom5703 Ethernet card is running with the tigon3 (i.e., 'tg3')
driver, and the MTU is (dynamically) set to a value greater then 1500 (e.g.,
1504), the driver loses the ability to receive packets of any length.  Only
unloading and reloading the driver restores it.  No error messages are sent to
/var/log/dmesg or /var/log/messages.


Version-Release number of selected component (if applicable):
kernel-2.4.18-27.7.x

How reproducible:
Always

Steps to Reproduce:
1. Connect a server that has the tg3 driver installed and running the Broadcom
5703 Ethernet card to a network.  Configure the card as necessary for your network.

2. Establish that you can communicate with the server, for example, send pings
to the IP address.

3. Log on as 'root' to the server that has the Broadcom Ethernet card installed.

4. Configure the appropriate Ethernet interface to set the MTU to a value
greater than 1500, for example:

   # ifconfig eth0 mtu 1504

5. From another server, attempt to ping the Broadcom Ethernet server.

6. Uninstall the 'tg3' driver on that server:

   # service network stop
   # rmmod tg3
   # service network start

7. Attempt to ping the Broadcom Ethernet from a second server.  This should now
work.


Actual Results:  Described in the 'Steps to Reproduce' section, i.e., no packets
are received after the MTU was set to a value above 1500.

Expected Results:  Packets should have been received and acknowledged after the
MTU was set to a value above 1500.

Additional info:

The Broadcom Ethernet card is installed on an IBM xSeries 335 server.

Device driver: /lib/modules/2.4.18-27.7.x/kernel/drivers/net/tg3.o

Comment 1 Tim Gladding 2003-06-25 21:06:13 UTC
Have you tried setting the MTU to anything LOWER than 1500?

I'm seeing this issue setting the MTU to anything.

Card is "Broadcom Corporation NetXtreme BCM5701 Gigabit Ethernet (rev 15)", as
found in the Compaq DL360 G2.

Comment 2 David Miller 2004-03-06 07:19:57 UTC
No more 7.x updates, so this is unlikely to be fixed.


Comment 3 Mark Harig 2004-03-08 18:20:58 UTC
Can someone tell me whether this bug has been fixed in a later Red 
Hat release?  (Red Hat 9, ES?)


Comment 4 José Carlos Ramírez Pérez 2004-04-29 11:03:37 UTC
The same happens to me with a pair of Broadcom 5704 1000Base-T 
ethernet cards, when I try to set up a fixed value for MTU, with 
driver bcm5700 from a 2.4.9-e.34smp kernel (from RHEL 2.1AS).

In this case I try to lower MTU to 1400 (to workaround a problem 
related to tunneling with ADSL lines), with the same results: the 
network interface stops working (i.e. doesn't send or receive 
anything).

The driver version is 'Broadcom Gigabit Ethernet Driver bcm5700 with 
Broadcom NIC Extension (NICE) ver. 6.0.2 (02/14/03)' as shown in 
dmesg.

May it be a problem with hardware/kernel driver...

Regards,


Comment 5 Mark Harig 2004-04-29 22:48:37 UTC
This appears to be a problem with the Broadcom ethernet card firmware.

The following configuration did NOT work for me:

# rpm -q kernel-smp
kernel-smp-2.4.21-9.EL

# ethtool -i eth0
driver: tg3
version: 2.3
firmware-version: 
bus-info: 02:01.0

And, running the Hardware Browser tool in the System Tools menu, the 
Network devices entry lists the devices as "NetXtreme BCM5703 Gigabit 
Ethernet".

I then updated the ethernet cards' firmware from version 2.25 to 
2.35.  I can now run 'ifconfig eth0 mtu 1504' or even 'ifconfig eth0 
mtu 1600' without any loss of communication.

Because 'ethtool' does not display the firmware version, I do not 
know how to determine which version is on the BCM5703 card.

Two BIG caveats:

1. Unfortunately, this behavior is flaky.  Sometimes when I ran the 
command:

   # ifconfig eth0 mtu 1504
or
   # ifconfig eth0 mtu 1600

the communication was lost and I had to reset the interface using:

   # service network stop
   # rmmod tg3
   # service network start

that is, sometimes I could reset the interface MTU value and 
sometimes I could not.  I do not know how to reproduce this 
consistently.

2. When I could reset the MTU value without losing the interface, 
setting it subseqently to a smaller value ALWAYS caused the interface 
to fail, i.e., the following combination always causes the interface 
to shut down:

   # ifconfig eth0 mtu 1600
   # ifconfig eth0 mtu 1504

As can be seen from the command 'rpm -q kernel-smp', above, this 
problem is occurring on RH Enterprise Linux 3.


Comment 6 Mark Harig 2004-04-29 22:55:19 UTC
Because I am seeing this problem on the latest version of Red Hat 
Enterprise Linux, I am re-opening this problem.  

I am changing the Product to 'Red Hat Enterprise Linux'.

Comment 7 Mark Harig 2004-04-29 22:56:34 UTC
Note: There is no 'kernel-smp' Component provided by Bugzilla, even 
though the rpm package is 'kernel-smp'.


Comment 8 José Carlos Ramírez Pérez 2004-05-07 12:34:35 UTC
You can check your Broadcom card's firmware version using the 
following command:

# cat /proc/net/nicinfo/eth0.info | grep Bootcode_Version

Mine says (bcm5704):

Bootcode_Version                5704-v3.20


Comment 9 Mark Harig 2004-05-07 20:16:42 UTC
> You can check your Broadcom card's firmware version using the 
> following command:
> 
> # cat /proc/net/nicinfo/eth0.info | grep Bootcode_Version

Unfortunately, Red Hat Enterprise Linux 3 does not appear to have 
that file, or even the /proc/net/nicinfo directory.  I have a server 
with all RH EL 3 RPM packages installed.



Comment 10 Mark T. Voelker 2004-07-07 19:40:47 UTC
I have now seen this on two different ethernet cards that also use the
tg3 driver.  One uses a BCM5704 chip on the motherboard, the other is
a Netgear GA302T card (we've tried several GA302T's, actually, and all
have the same issue).  Relevant output from lspci:

02:09.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5704
Gigabit Ethernet (rev 03)
02:09.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5704
Gigabit Ethernet (rev 03)

And:

00:0b.0 Ethernet controller: Altima (nee Broadcom) AC9100 Gigabit
Ethernet (rev 15) 0

Until the problem is fixed, some folks who have Broadcom BCM57XX-based
cards might want to consider using the GPL'd Broadcom bcm5700 driver
available from http://www.broadcom.com/drivers/downloaddrivers.php as
a temporary workaround.  I understand that this driver has it's own
set of bugs and that Red Hat will not support it or include it in
RHEL, but we need to be able to up the MTU on our machines and bcm5700
allows us to do so.  We haven't had any negative fallout from
switching drivers thus far, so we're using it as a temporary
workaround until this bug is fixed.  We're out of luck for the GA302T
card though, since it does not use a BCM57XX chip and therefor
Broadcom's driver won't work for it.

Note that a co-worker of mine was unable to reproduce this bug on a
BCM5782 card.  Output from his lspci:

05:02.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5782 
Gigabit Ethernet (rev 03)
        Subsystem: Hewlett-Packard Company: Unknown device 12bf
        Flags: bus master, 66Mhz, medium devsel, latency 64, IRQ 17
        Memory at f8400000 (64-bit, non-prefetchable) [size=64K]
        Capabilities: [48] Power Management version 2
        Capabilities: [50] Vital Product Data
        Capabilities: [58] Message Signalled Interrupts: 64bit+
Queue=0/3 Enable

Comment 11 David Miller 2004-08-17 01:29:43 UTC
Created attachment 102776 [details]
Fix for MTU changing bug.

This will fix the problem.

Comment 12 Don Howard 2009-12-02 19:02:56 UTC
Closing this old bug that is in modified state.  If this bug has not been resolved, please reopen it with new information.