Hide Forgot
Description of problem: Emulex 10G card cant be initilized properly when booting with 6.1 RC kernel. Version-Release number of selected component (if applicable): How reproducible: 2.6.32-131.0.13.el6.x86_64 Steps to Reproduce: 1. boot intel-mccreary-01.lab.bos.redhat.com 2. 3. Actual results: failed to init, and there is no interface after booting Expected results: interface eth1/2 functions normal Additional info: kernel log: be2net 0000:01:00.1: PCI INT B -> GSI 17 (level, low) -> IRQ 17 be2net 0000:01:00.1: setting latency timer to 64 ttyS0: 3 input overrun(s) alloc irq_desc for 27 on node -1 alloc kstat_irqs on node -1 be2net 0000:01:00.1: irq 27 for MSI/MSI-X alloc irq_desc for 28 on node -1 alloc kstat_irqs on node -1 be2net 0000:01:00.1: irq 28 for MSI/MSI-X alloc irq_desc for 29 on node -1 alloc kstat_irqs on node -1 be2net 0000:01:00.1: irq 29 for MSI/MSI-X alloc irq_desc for 30 on node -1 alloc kstat_irqs on node -1 be2net 0000:01:00.1: irq 30 for MSI/MSI-X alloc irq_desc for 31 on node -1 alloc kstat_irqs on node -1 be2net 0000:01:00.1: irq 31 for MSI/MSI-X alloc irq_desc for 32 on node -1 alloc kstat_irqs on node -1 be2net 0000:01:00.1: irq 32 for MSI/MSI-X be2net 0000:01:00.1: Error in cmd completion - opcode 90, compl 2, extd 30 be2net 0000:01:00.1: PCI INT B disabled be2net 0000:01:00.1: ServerEngines BladeEngine2 10Gbps NIC initialization failed be2net: probe of 0000:01:00.1 failed with error -1 be2net 0000:01:00.3: PCI INT D -> GSI 19 (level, low) -> IRQ 19 be2net 0000:01:00.3: setting latency timer to 64 be2net 0000:01:00.3: irq 27 for MSI/MSI-X be2net 0000:01:00.3: irq 28 for MSI/MSI-X be2net 0000:01:00.3: irq 29 for MSI/MSI-X be2net 0000:01:00.3: irq 30 for MSI/MSI-X be2net 0000:01:00.3: irq 31 for MSI/MSI-X be2net 0000:01:00.3: irq 32 for MSI/MSI-X be2net 0000:01:00.3: Error in cmd completion - opcode 90, compl 2, extd 30 be2net 0000:01:00.3: PCI INT D disabled be2net 0000:01:00.3: ServerEngines BladeEngine2 10Gbps NIC initialization failed
This is a regression. with 6.0 kernel, the interfaces can be brought up, and ssh works fine with it. These lines show -71 kernel's log: eth1: Link up (beiscsi_async_link_state_process():163):Link UP on Physical Port 0 be2net 0000:01:00.1: Error in cmd completion - opcode 102, compl 2, extd 30
[root@intel-mccreary-01 ~]# lspci | grep Emulex 01:00.0 Mass storage controller: Emulex Corporation BladeEngine2 10Gb Gen2 PCIe iSCSI Adapter (rev 01) 01:00.1 Ethernet controller: Emulex Corporation BladeEngine2 10Gb Gen2 PCIe Network Adapter (rev 01) 01:00.2 Mass storage controller: Emulex Corporation BladeEngine2 10Gb Gen2 PCIe iSCSI Adapter (rev 01) 01:00.3 Ethernet controller: Emulex Corporation BladeEngine2 10Gb Gen2 PCIe Network Adapter (rev 01)
Hushan, I have reserved the affected machine from beaker and I'm going to try to reproduce the issue.
So, I found the reason... This is commit causing this issue: cc4ce02 be2net: Changes to use only priority codes allowed by f/w Problem description: In the log above there is a message "Error in cmd completion - opcode 90, compl 2, extd 30". According opcodes defined in be_cmds.h the opcode 90 is OPCODE_COMMON_MCC_CREATE_EXT and this opcode is used in function be_cmd_mccq_create that is called on init. The commit itself is not so problematic, but switches the opcode OPCODE_COMMON_MCC_CREATE (21) to OPCODE_COMMON_MCC_CREATE_EXT (90). The problem is that (probably) this opcode is unknown for certain revision of firmware in the adapter. So it is probably the firmware issue instead of HW issue, in other words the firmware in the adapter is too old. (CC-ing Subbu from Emulex). Subbu, could you please look on it? Is this a firmware issue? If so, could it be updated? Firmware in the mentioned adapter: # ethtool -i eth0 | grep firmware firmware-version: 2.0.415.15
We have seen this issue in the past, and a firmware upgrade resolved it. Our firmware version is : ethtool -i eth3 | grep firmware firmware-version: 2.103.397.31
Ivan, Can you try to install the firmware and then re-test?
Ivan is right. That is indeed the problem. We can change driver to use the old command in older f/w; but I guess it is too late for RH 6.1. Also, 2.0.415.15 is very old and needs to be updated anyways. Any version later than 2.102.235.0 is fine. If you need the image, you can download from http://www.emulex.com/downloads/emulex.html Click on OCe11102 or OCe10102 depending on whether you have BE3 or BE2 card. You can use ethtool -f to update. Since this requires the driver loaded, you will need to use Rh 6.0.
Subbu, I made an F/W update, but after reboot the driver failed to initialize card. dmesg output: [root@intel-mccreary-01 ~]# dmesg | grep be2net be2net 0000:01:00.1: PCI INT B -> GSI 17 (level, low) -> IRQ 17 be2net 0000:01:00.1: setting latency timer to 64 be2net 0000:01:00.1: POST error; stage=0x100 be2net 0000:01:00.1: PCI INT B disabled be2net 0000:01:00.1: ServerEngines BladeEngine2 10Gbps NIC initialization failed be2net: probe of 0000:01:00.1 failed with error -1 be2net 0000:01:00.3: PCI INT D -> GSI 19 (level, low) -> IRQ 19 be2net 0000:01:00.3: setting latency timer to 64 be2net 0000:01:00.3: POST error; stage=0x100 be2net 0000:01:00.3: PCI INT D disabled be2net 0000:01:00.3: ServerEngines BladeEngine2 10Gbps NIC initialization failed be2net: probe of 0000:01:00.3 failed with error -1