Created attachment 419311 [details] Console messages from th ebnx2x driver Description of problem: The bnx2x driver dumps verbose messages to the console prefixed with "bnx2x_panic_dump". After that network becomes unusable. This causes the Symantec clusterware to eventually panic nodes. Version-Release number of selected component (if applicable): (Linux)(c1062-hpblade1) ~{1} uname -a Linux c1062-hpblade1.engba.symantec.com 2.6.18-164.el5 #1 SMP Tue Aug 18 15:51:48 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux (Linux)(c1062-hpblade1) ~{2} modinfo bnx2x filename: /lib/modules/2.6.18-164.el5/kernel/drivers/net/bnx2x.ko version: 1.48.105 license: GPL description: Broadcom NetXtreme II BCM57710/57711/57711E Driver author: Eliezer Tamir srcversion: 6D030D52DFD981356EEC2BE alias: pci:v000014E4d00001650sv*sd*bc*sc*i* alias: pci:v000014E4d0000164Fsv*sd*bc*sc*i* alias: pci:v000014E4d0000164Esv*sd*bc*sc*i* depends: vermagic: 2.6.18-164.el5 SMP mod_unload gcc-4.1 parm: multi_mode: Use per-CPU queues (int) parm: disable_tpa: Disable the TPA (LRO) feature (int) parm: int_mode: Force interrupt mode (1 INT#x; 2 MSI) (int) parm: poll: Use polling (for debug) (int) parm: mrrs: Force Max Read Req Size (0..3) (for debug) (int) parm: debug: Default debug msglevel (int) module_sig: 883f3504a8b7cb4bd273d74512bb112f59309f6c2877f55cb62445a8c8af7cdaf1e559983bdb09f5b1b3fa9aca58a8f88267866b046e346ebe3bbaa (Linux)(c1062-hpblade1) ~{3} How reproducible: Occurs in relatively heavy network load. Easily reproducible with Symantec clusterware with large configurations. Actual results: Machine shows bnx2x messages on console. Network goes unusable. Additional info: Attaching the messages seen. Symantec contact: udas
This bug is most likely the same issue as we have in bug 516090. It is already fixed by driver update. Try up-to-date kernel like 2.6.18-194.3.1.el5, or kernels from http://people.redhat.com/jwilson/el5/
Any comments on above?
The latest kernel from the above link did not work with us. Similar log entries were observed. Symantec contact: udas
We have another bnx2x panic, happens on RHEL5.4 and on up-to-date RHEL5 kernels.
@Veritas, did blacklisting cnic and bnx2i modules help?
Please try kernel 204.el5, it include cnic fix for bug that can cause bnx2x panic. If it does not help we will probably need more info. I'm not sure if this bnx2x crash dump contains all information to allow Broadcom to fix the issue. @Broadcom, do you want any more info to track down this bnx2x panic?
Created attachment 427988 [details] debug tool debugging tool
Please provide us result of "grcDump" command using attached debug tool Building it: > tar xf edebug_linux_ver_0.1.4.tar.gz > cd edebug_0.1.4/ > make >./load.sh Using it The tool will show list of BCM5771x device on the system select one caused the crash by "device X" (you can recognize it by pci bus address or MAC address). Then apply command "grcDump regs.dump". Exit application by "exit" command. Upload generated regs.dump file. Thanks
Created attachment 427990 [details] example of running the debug tool
Our repro setup is no longer available for this. We will get back to you whenever they become available again. However, the issue is easily reproduce on HP blades.
This BZ has had the needinfo? flag set for more than 3 years. Closing.
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days