RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1693299 - virsh nodedev-list --cap net can not list all the network device when modprobe ib_umad
Summary: virsh nodedev-list --cap net can not list all the network device when modprob...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: libvirt
Version: 8.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: 8.1
Assignee: Pavel Hrdina
QA Contact: yalzhang@redhat.com
URL:
Whiteboard:
: 1707757 (view as bug list)
Depends On: 1398111
Blocks: 1574709 1660905
TreeView+ depends on / blocked
 
Reported: 2019-03-27 13:33 UTC by Pavel Hrdina
Modified: 2020-11-14 10:04 UTC (History)
12 users (show)

Fixed In Version: libvirt-4.5.0-25.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1398111
Environment:
Last Closed: 2019-11-05 20:48:28 UTC
Type: Bug
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2019:3345 0 None None None 2019-11-05 20:49:10 UTC

Description Pavel Hrdina 2019-03-27 13:33:47 UTC 
+++ This bug was initially created as a clone of Bug #1398111 +++

Description of problem:
virsh nodedev-list --cap net can not list all the network devices when modprobe ib_umad

Version-Release number of selected component (if applicable):
kernel-3.10.0-514.el7.x86_64
libvirt-2.0.0-10.el7.x86_64

How reproducible:
100%

Steps to Reproduce:

1.# virsh nodedev-list --cap net
net_eno1_34_40_b5_b1_16_68
net_eno2_34_40_b5_b1_16_6a
net_ens2f0_00_0f_53_0c_7f_00
net_ens2f1_00_0f_53_0c_7f_01

2.
# ll /sys/class/net
total 0
lrwxrwxrwx. 1 root root 0 Nov 16 05:35 eno1 -> ../../devices/pci0000:00/0000:00:01.0/0000:04:00.0/net/eno1
lrwxrwxrwx. 1 root root 0 Nov 16 05:35 eno2 -> ../../devices/pci0000:00/0000:00:01.0/0000:04:00.1/net/eno2
lrwxrwxrwx. 1 root root 0 Nov 16 05:35 enp0s29f0u2 -> ../../devices/pci0000:00/0000:00:1d.0/usb6/6-2/6-2:1.0/net/enp0s29f0u2
lrwxrwxrwx. 1 root root 0 Nov 16 05:35 ens2f0 -> ../../devices/pci0000:80/0000:80:01.0/0000:86:00.0/net/ens2f0
lrwxrwxrwx. 1 root root 0 Nov 16 05:35 ens2f1 -> ../../devices/pci0000:80/0000:80:01.0/0000:86:00.1/net/ens2f1
lrwxrwxrwx. 1 root root 0 Nov 16 05:35 lo -> ../../devices/virtual/net/lo
lrwxrwxrwx. 1 root root 0 Nov 16 05:35 virbr0 -> ../../devices/virtual/net/virbr0
lrwxrwxrwx. 1 root root 0 Nov 16 05:35 virbr0-nic -> ../../devices/virtual/net/virbr0-nic


3,# ip l
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether 34:40:b5:b1:16:68 brd ff:ff:ff:ff:ff:ff
3: eno2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
    link/ether 34:40:b5:b1:16:6a brd ff:ff:ff:ff:ff:ff
4: ens2f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
    link/ether 00:0f:53:0c:7f:00 brd ff:ff:ff:ff:ff:ff
    vf 0 MAC 00:00:00:00:00:00, spoof checking off, link-state auto
5: ens2f1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
    link/ether 00:0f:53:0c:7f:01 brd ff:ff:ff:ff:ff:ff
    vf 0 MAC 00:00:00:00:00:00, spoof checking off, link-state auto
6: enp0s29f0u2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN mode DEFAULT qlen 1000
    link/ether 36:40:b5:b2:96:6b brd ff:ff:ff:ff:ff:ff
7: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT qlen 1000
    link/ether 52:54:00:7d:8d:cc brd ff:ff:ff:ff:ff:ff
8: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN mode DEFAULT qlen 1000
    link/ether 52:54:00:7d:8d:cc brd ff:ff:ff:ff:ff:ff



Actual results:
"virsh nodedev-list --cap net" can not list all the network devices as above


Expected results:
"virsh nodedev-list --cap net" can  list all the network devices


Additional info:
With the help of lhuang' debugging, we found that if the the module ib_umad modprobed, it will create some pathes :
/sys/class/infiniband
/sys/class/infiniband_cm
/sys/class/infiniband_srp
/sys/class/infiniband_verbs

It will cause libvirt to the path /sys/devices/pci0000:00/0000:00:1d.0/usb6/6-2/6-2:1.0 to find the file "resource", but it can not be found, it will can not 
distinguish this NIC


--- Additional comment from Pavel Hrdina on 2019-03-27 14:31:45 CET ---

Upstream commit:

commit f38ef0fac0582ac0cbb749af9d3f8ba515a6084a
Author: Pavel Hrdina <phrdina>
Date:   Thu Mar 7 17:52:55 2019 +0100

    util: skip RDMA detection for non-PCI network devices
Comment 1 Pavel Hrdina 2019-05-10 08:20:58 UTC 
*** Bug 1707757 has been marked as a duplicate of this bug. ***
Comment 2 Nanda Kishore Chinnaram 2019-05-13 13:09:18 UTC 
Pavel,
In which build can we expect the fix?
Comment 3 Pavel Hrdina 2019-05-20 08:09:48 UTC 
Hi Nanda,

That's hard to tell, this will be fixed in RHEL-8.1.0.

You can watch this BZ and once the "Status" is MODIFIED there will
be libvirt package version in "Fixed in Version".
Comment 6 yalzhang@redhat.com 2019-07-03 09:19:51 UTC 
Reproduce on libvirt-4.5.0-24.module+el8.1.0+3205+41ff0a42.x86_64

# rpm -q libvirt
libvirt-4.5.0-24.module+el8.1.0+3205+41ff0a42.x86_64

# lsmod | grep ib_umad
ib_umad                28672  4
ib_core               282624  13 rdma_cm,ib_ipoib,rpcrdma,mlx4_ib,ib_srpt,ib_srp,iw_cm,ib_iser,ib_umad,ib_isert,rdma_ucm,ib_uverbs,ib_cm

# ll /sys/class/net
total 0
lrwxrwxrwx. 1 root root 0 Jun 21 19:03 eno1 -> ../../devices/pci0000:00/0000:00:03.1/0000:01:00.0/net/eno1
lrwxrwxrwx. 1 root root 0 Jun 21 19:03 eno2 -> ../../devices/pci0000:00/0000:00:03.1/0000:01:00.1/net/eno2
lrwxrwxrwx. 1 root root 0 Jun 21 19:03 eno3 -> ../../devices/pci0000:00/0000:00:03.0/0000:02:00.0/net/eno3
lrwxrwxrwx. 1 root root 0 Jun 21 19:03 eno4 -> ../../devices/pci0000:00/0000:00:03.0/0000:02:00.1/net/eno4
lrwxrwxrwx. 1 root root 0 Jun 21 19:03 enp4s0f0 -> ../../devices/pci0000:00/0000:00:02.0/0000:04:00.0/net/enp4s0f0
lrwxrwxrwx. 1 root root 0 Jun 21 19:03 enp4s0f1 -> ../../devices/pci0000:00/0000:00:02.0/0000:04:00.1/net/enp4s0f1
lrwxrwxrwx. 1 root root 0 Jun 21 19:03 ib0 -> ../../devices/pci0000:00/0000:00:02.2/0000:06:00.0/net/ib0
lrwxrwxrwx. 1 root root 0 Jun 21 19:03 ib1 -> ../../devices/pci0000:00/0000:00:02.2/0000:06:00.0/net/ib1
lrwxrwxrwx. 1 root root 0 Jun 21 19:03 lo -> ../../devices/virtual/net/lo
lrwxrwxrwx. 1 root root 0 Jul  3 16:59 macvtap0 -> ../../devices/virtual/net/macvtap0
lrwxrwxrwx. 1 root root 0 Jul  1 15:07 virbr0 -> ../../devices/virtual/net/virbr0
lrwxrwxrwx. 1 root root 0 Jul  1 15:07 virbr0-nic -> ../../devices/virtual/net/virbr0-nic
lrwxrwxrwx. 1 root root 0 Jul  3 17:01 vnet0 -> ../../devices/virtual/net/vnet0

# virsh nodedev-list --cap net
net_eno1_18_66_da_e6_83_98
net_eno2_18_66_da_e6_83_99
net_eno3_18_66_da_e6_83_9a
net_eno4_18_66_da_e6_83_9b
net_enp4s0f0_a0_36_9f_95_0a_08
net_enp4s0f1_a0_36_9f_95_0a_0a
net_ib0_80_00_02_08_fe_80_00_00_00_00_00_00_e4_1d_2d_03_00_48_23_31
net_ib1_80_00_02_09_fe_80_00_00_00_00_00_00_e4_1d_2d_03_00_48_23_32

The non-PCI network devices like macvtap0, virbr0-nic, vnet0 do not show in nodedev-list outputs.

Test on libvirt-4.5.0-29.x86_64, the result is as expected:

# rpm -q libvirt 
libvirt-4.5.0-29.module+el8.1.0+3567+07fd3313.x86_64

# virsh nodedev-list --cap net
net_eno1_18_66_da_e6_83_98
net_eno2_18_66_da_e6_83_99
net_eno3_18_66_da_e6_83_9a
net_eno4_18_66_da_e6_83_9b
net_enp4s0f0_a0_36_9f_95_0a_08
net_enp4s0f1_a0_36_9f_95_0a_0a
net_ib0_80_00_02_08_fe_80_00_00_00_00_00_00_e4_1d_2d_03_00_48_23_31
net_ib1_80_00_02_09_fe_80_00_00_00_00_00_00_e4_1d_2d_03_00_48_23_32
net_lo_00_00_00_00_00_00
net_macvtap0_52_54_00_76_91_49
net_virbr0_nic_52_54_00_da_3f_87
net_vnet0_fe_54_00_77_92_da

The non-PCI network devices like vnet0, virbr0-nic, lo, macvtap0 can list out in nodedev-list outputs.
Comment 7 Nanda Kishore Chinnaram 2019-07-15 04:53:52 UTC 
Pavel,
is this going to make into RHEL-8.0 Z-Stream?
Comment 8 Pavel Hrdina 2019-07-18 11:59:12 UTC 
(In reply to Nanda Kishore Chinnaram from comment #7)
> Pavel,
> is this going to make into RHEL-8.0 Z-Stream?

Hi, there was no z-stream fix requested so we don't have any plans to fix
it there.  It will be fixed in RHEL-8.1, if you would have to request
z-stream fix.
Comment 10 errata-xmlrpc 2019-11-05 20:48:28 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:3345
Note You need to log in before you can comment on or make changes to this bug.