Bug 624386 - IBM BladeCenter HS22 stops responding, messes up network switch
Summary: IBM BladeCenter HS22 stops responding, messes up network switch
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: 12
Hardware: x86_64
OS: Linux
low
high
Target Milestone: ---
Assignee: Kernel Maintainer List
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-08-16 09:37 UTC by Jukka Laurila
Modified: 2010-12-03 12:50 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2010-12-03 12:50:06 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
Ping log from the crash event displaying the network latency problem and disconnect. (8.54 KB, text/plain)
2010-08-16 09:37 UTC, Jukka Laurila
no flags Details
lspci -v output on the affected machine (16.32 KB, text/plain)
2010-08-17 14:05 UTC, Jukka Laurila
no flags Details

Description Jukka Laurila 2010-08-16 09:37:38 UTC
Created attachment 438913 [details]
Ping log from the crash event displaying the network latency problem and disconnect.

Description of problem:

We have a number of HS22 blades in an IBM BladeCenter H, which also has a Nortel layer 2/3 gigabit switch (IBM product code 32R1860), to which all the blades are connected via chassis internal links. All the blades are running CentOS 5.5, except for one which is running Fedora 12. 

Several times now the Fedora 12 blade has locked up. It doesn't respond to network requests, the screen is blank when viewed via the Advanced Management Module (a separate HW component in the chassis for managing the blades, connected via a separate network connection). In about a minute or so the AMM will report that management network connectivity has been lost to the Fedora 12 blade. After this happens, network traffic to other blades in the same chassis will first show increased latency and then stop entirely. Network functionality is restored immediately when the Fedora 12 blade is powered off.

Version-Release number of selected component (if applicable):

2.6.32.16-141.fc12.x86_64 

How reproducible:

This happens relatively infrequently, every few weeks. Today's crash occurred during moderate network activity (running yum update).

Comment 1 Chuck Ebbert 2010-08-17 11:22:21 UTC
All that log shows is that something is wrong.

What kind of network adapter is in that machine? Is there anything in the system log when that happens?

Comment 2 Jukka Laurila 2010-08-17 14:05:40 UTC
Created attachment 439123 [details]
lspci -v output on the affected machine

Comment 3 Jukka Laurila 2010-08-17 14:07:53 UTC
Dual Broadcom Corporation NetXtreme II BCM5709S Gigabit Ethernet (rev 20). lspci -v attached.

At least /var/log/messages shows just normal looking activity, and then the messages after reboot, any other files I should check?

Comment 4 Jukka Laurila 2010-08-17 14:10:22 UTC
Also, the problem is not related to broken hardware, since the crashes moved to a different machine after swapping the Fedora 12 RAID-1 system drives to it. Under CentOS 5.5 all the blades seem to work fine.

Comment 5 Chuck Ebbert 2010-08-19 01:50:44 UTC
Can you try booting with the kernel option "pcie_aspm=off"? You could also try "pci=nomsi" in addition to that but it's doubtful that MSI is causing any problems.

Comment 6 Jukka Laurila 2010-09-17 11:55:27 UTC
Sorry, I didn't notice your comment before now. I thought Bugzilla would automatically send updates to the bug reporter, but apparently I need to add myself manually to the CC list.

The crash happened again this week with the same kernel. I'll try rebooting with that option and we'll see if it helps. It may take another month though before it crashes again.

Comment 7 Bug Zapper 2010-11-03 10:58:25 UTC
This message is a reminder that Fedora 12 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 12.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '12'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 12's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 12 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 8 Bug Zapper 2010-12-03 12:50:06 UTC
Fedora 12 changed to end-of-life (EOL) status on 2010-12-02. Fedora 12 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.