Bug 757743

Summary: mlx4 card not being named by biosdevname
Product: [Fedora] Fedora Reporter: Robert Kennedy <rt>
Component: biosdevnameAssignee: Narendra K <narendra_k>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 16CC: dwmw2, harald, jordan_hargrave, matt_domsch, mebrown, narendra_k, praveen_paladugu
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-07-26 22:32:43 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
dmidecode from a host with a Mellanox card in it
none
biosdecode from a host with a Mellanox card in it
none
lspci -tv from a host with a Mellanox card in it
none
lspci -vvvxxxx from a host with a Mellanox card in it
none
Patch to consider serial/infiniband PCI devices for biosdevname ethernet devices
none
ls -lh /sys/class/net/* from a host with a Mellanox in it
none
grep . /sys/class/net/*/ifindex from a host with a Mellanox in it none

Description Robert Kennedy 2011-11-28 15:20:19 UTC
Description of problem:

A Mellanox ConnectX-2 PCI card has no BIOS device and biosdevname gives it an eth* name instead of a p#p# name.

Version-Release number of selected component (if applicable):

biosdevname-0.3.11-5.fc16.x86_64

How reproducible:

Always.


Steps to Reproduce:

# biosdevname --policy physical -d

Actual results:

BIOS device: 
Kernel name: eth4
Permanent MAC: 00:02:C9:0F:52:D7
Assigned MAC : 00:02:C9:0F:52:D7
Driver: mlx4_en
Driver version: 1.5.4.1 (March 2011)
Firmware version: 2.9.1000
Bus Info: 0000:04:00.0
  

Expected results:

Same, but with p1p1 as the BIOS device and kernel name (It is the first PCI card).

Additional info:

I am unsure if this is a problem with the mlx4_{core,en} driver or biosdevname.

Comment 1 Narendra K 2011-11-29 18:09:44 UTC
Hi Robert, could you please attach the output of the following -

dmidecode
biosdecode
lspci -tv
lspci -vvvxxxx

Comment 2 Robert Kennedy 2011-11-29 18:17:43 UTC
Created attachment 538159 [details]
dmidecode from a host with a Mellanox card in it

Comment 3 Robert Kennedy 2011-11-29 18:18:39 UTC
Created attachment 538160 [details]
biosdecode from a host with a Mellanox card in it

Comment 4 Robert Kennedy 2011-11-29 18:19:17 UTC
Created attachment 538161 [details]
lspci -tv from a host with a Mellanox card in it

Comment 5 Robert Kennedy 2011-11-29 18:19:43 UTC
Created attachment 538162 [details]
lspci -vvvxxxx from a host with a Mellanox card in it

Comment 6 Narendra K 2011-12-01 14:43:45 UTC
The attached lspci -vvv shows that the ConnectX card is at '0c:00.0'

0c:00.0 InfiniBand: Mellanox Technologies MT26428 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE] (rev b0)

The 'Bus info' shown by biosdevname -d is '0000:04:00.0' and lspci shows that '0000:04:00.0'  it a PCI bridge

04:00.0 PCI bridge: Intel Corporation 6311ESB/6321ESB PCI Express Upstream Port (rev 01) (prog-if 00 [Normal decode])

So biosdevname returns without suggesting any name as '04:00.0' is not a 'network class' device.

It seems like the mlx4_en is returning incorrect information for  ethtool 'getdriverinfo' command. 

Hi Robert,

Could you please post the output from 'ethtool -i eth4' ?

Comment 7 Robert Kennedy 2011-12-01 15:18:35 UTC
Hmmm. I'm wondering if I didn't accidentally give you info from 2 different hosts between the initial bug report and the attachments.  When I went to check, this is what I found on the same host:

ethtool -i:

driver: mlx4_en
version: 1.5.4.1 (March 2011)
firmware-version: 2.9.1000
bus-info: 0000:0c:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: no
supports-register-dump: no

biosdevname -d:

BIOS device: 
Kernel name: eth4
Permanent MAC: 00:02:C9:0D:74:6D
Assigned MAC : 00:02:C9:0D:74:6D
Driver: mlx4_en
Driver version: 1.5.4.1 (March 2011)
Firmware version: 2.9.1000
Bus Info: 0000:0c:00.0

and lspci:

0c:00.0 InfiniBand: Mellanox Technologies MT26428 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE] (rev b0)

So the bus info matches up but no still no "BIOS device". Sorry for the confusion.

Comment 8 Narendra K 2011-12-05 15:01:00 UTC
Ok. Thanks. Could you please post the output from 

cat /sys/class/net/eth4/device/class

It could be that the class of the device is not "network" (0x020000),so  biosdevname is not considering it as a network device.

Comment 9 Robert Kennedy 2011-12-05 15:16:18 UTC
/sys/class/net/eth4/device/class is 0x0c0600.

Comment 10 Robert Kennedy 2011-12-09 15:59:48 UTC
Sooo. I guess this combo card thinks of itself as a Serial device (Infiniband) first.

Comment 11 Narendra K 2011-12-13 19:33:24 UTC
It seems like this adapter is not being added to the 'bios devices list' because 

In file "src/bios_device.c", function 'match_all' 

1.the following check fails, as it is not PCI class network.

 if (!is_pci_network(p))
        continue;

2.  if (!is_ethernet(n)) /* for virtual interfaces */
                        continue;

I will try to see if i can find this adapter and reproduce this issue.

Comment 12 Robert Kennedy 2011-12-14 13:11:06 UTC
Created attachment 546719 [details]
Patch to consider serial/infiniband PCI devices for biosdevname ethernet devices

Well, the most straightforward thing to do is to consider 0x0c06 devices...

It looks like there is further probing to determine if such a thing is actually an ethernet device, so hopefully this won't break the world for the IB folks. It doesn't seem to affect the ib0 interface of the Mellanox card in our environment.

BIOS device: p2p1
Kernel name: eth4
Permanent MAC: 00:02:C9:4B:E3:AF
Assigned MAC : 00:02:C9:4B:E3:AF
Driver: mlx4_en (MT_0DD0120009)
Driver version: 1.5.3 (Jan 2011)
Firmware version: 2.9.1000
Bus Info: 0000:0c:00.0
PCI name      : 0000:0c:00.0
PCI Slot      : 2
SMBIOS Label: PCI2
Index in slot: 1

Comment 13 jordan hargrave 2012-01-04 01:31:20 UTC
can you attach output of :
ls -Hl /sys/class/net/*
cat /sys/class/net/*/ifindex

Comment 14 Robert Kennedy 2012-01-06 13:01:38 UTC
Created attachment 551144 [details]
ls -lh /sys/class/net/* from a host with a Mellanox in it

Comment 15 Robert Kennedy 2012-01-06 13:03:08 UTC
Created attachment 551145 [details]
grep . /sys/class/net/*/ifindex from a host with a Mellanox in it

Yes, these are the same files from case 756164, both troublesome cards are in the same host.

Comment 16 Robert Kennedy 2012-02-14 20:51:18 UTC
Has there been any discussion in the last month over whether or not Serial/Infiniband devices should be considered for biosdevnaming? I can certainly maintain a locally patched copy but I'd like to know the general consensus, or if there is a better place to put the lever.

Comment 17 Fedora Update System 2012-07-18 13:36:20 UTC
biosdevname-0.4.1-1.fc17 has been submitted as an update for Fedora 17.
https://admin.fedoraproject.org/updates/biosdevname-0.4.1-1.fc17

Comment 18 Fedora Update System 2012-07-19 09:02:20 UTC
Package biosdevname-0.4.1-1.fc17:
* should fix your issue,
* was pushed to the Fedora 17 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing biosdevname-0.4.1-1.fc17'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-10778/biosdevname-0.4.1-1.fc17
then log in and leave karma (feedback).

Comment 19 Fedora Update System 2012-07-26 22:32:43 UTC
biosdevname-0.4.1-1.fc17 has been pushed to the Fedora 17 stable repository.  If problems still persist, please make note of it in this bug report.