Bug 1398111 - virsh nodedev-list --cap net can not list all the network device when modprobe ib_umad
Summary: virsh nodedev-list --cap net can not list all the network device when modprob...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: libvirt
Version: 8.1
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Pavel Hrdina
QA Contact: yalzhang@redhat.com
URL:
Whiteboard:
Depends On:
Blocks: 1693299
TreeView+ depends on / blocked
 
Reported: 2016-11-24 07:29 UTC by Jingjing Shao
Modified: 2020-11-14 06:03 UTC (History)
7 users (show)

Fixed In Version: libvirt-5.0.0-9.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1693299 (view as bug list)
Environment:
Last Closed: 2019-08-07 10:41:09 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:2395 0 None None None 2019-08-07 10:41:35 UTC

Description Jingjing Shao 2016-11-24 07:29:04 UTC
Description of problem:
virsh nodedev-list --cap net can not list all the network devices when modprobe ib_umad

Version-Release number of selected component (if applicable):
kernel-3.10.0-514.el7.x86_64
libvirt-2.0.0-10.el7.x86_64

How reproducible:
100%

Steps to Reproduce:

1.# virsh nodedev-list --cap net
net_eno1_34_40_b5_b1_16_68
net_eno2_34_40_b5_b1_16_6a
net_ens2f0_00_0f_53_0c_7f_00
net_ens2f1_00_0f_53_0c_7f_01

2.
# ll /sys/class/net
total 0
lrwxrwxrwx. 1 root root 0 Nov 16 05:35 eno1 -> ../../devices/pci0000:00/0000:00:01.0/0000:04:00.0/net/eno1
lrwxrwxrwx. 1 root root 0 Nov 16 05:35 eno2 -> ../../devices/pci0000:00/0000:00:01.0/0000:04:00.1/net/eno2
lrwxrwxrwx. 1 root root 0 Nov 16 05:35 enp0s29f0u2 -> ../../devices/pci0000:00/0000:00:1d.0/usb6/6-2/6-2:1.0/net/enp0s29f0u2
lrwxrwxrwx. 1 root root 0 Nov 16 05:35 ens2f0 -> ../../devices/pci0000:80/0000:80:01.0/0000:86:00.0/net/ens2f0
lrwxrwxrwx. 1 root root 0 Nov 16 05:35 ens2f1 -> ../../devices/pci0000:80/0000:80:01.0/0000:86:00.1/net/ens2f1
lrwxrwxrwx. 1 root root 0 Nov 16 05:35 lo -> ../../devices/virtual/net/lo
lrwxrwxrwx. 1 root root 0 Nov 16 05:35 virbr0 -> ../../devices/virtual/net/virbr0
lrwxrwxrwx. 1 root root 0 Nov 16 05:35 virbr0-nic -> ../../devices/virtual/net/virbr0-nic


3,# ip l
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether 34:40:b5:b1:16:68 brd ff:ff:ff:ff:ff:ff
3: eno2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
    link/ether 34:40:b5:b1:16:6a brd ff:ff:ff:ff:ff:ff
4: ens2f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
    link/ether 00:0f:53:0c:7f:00 brd ff:ff:ff:ff:ff:ff
    vf 0 MAC 00:00:00:00:00:00, spoof checking off, link-state auto
5: ens2f1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
    link/ether 00:0f:53:0c:7f:01 brd ff:ff:ff:ff:ff:ff
    vf 0 MAC 00:00:00:00:00:00, spoof checking off, link-state auto
6: enp0s29f0u2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN mode DEFAULT qlen 1000
    link/ether 36:40:b5:b2:96:6b brd ff:ff:ff:ff:ff:ff
7: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT qlen 1000
    link/ether 52:54:00:7d:8d:cc brd ff:ff:ff:ff:ff:ff
8: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN mode DEFAULT qlen 1000
    link/ether 52:54:00:7d:8d:cc brd ff:ff:ff:ff:ff:ff



Actual results:
"virsh nodedev-list --cap net" can not list all the network devices as above


Expected results:
"virsh nodedev-list --cap net" can  list all the network devices


Additional info:
With the help of lhuang' debugging, we found that if the the module ib_umad modprobed, it will create some pathes :
/sys/class/infiniband
/sys/class/infiniband_cm
/sys/class/infiniband_srp
/sys/class/infiniband_verbs

It will cause libvirt to the path /sys/devices/pci0000:00/0000:00:1d.0/usb6/6-2/6-2:1.0 to find the file "resource", but it can not be found, it will can not 
distinguish this NIC

Comment 3 Pavel Hrdina 2019-03-27 13:31:45 UTC
Upstream commit:

commit f38ef0fac0582ac0cbb749af9d3f8ba515a6084a
Author: Pavel Hrdina <phrdina>
Date:   Thu Mar 7 17:52:55 2019 +0100

    util: skip RDMA detection for non-PCI network devices

Comment 8 yalzhang@redhat.com 2019-07-02 09:33:32 UTC
Reproduce the bug on libvirt-2.0.0-10.el7_3.14.x86_64 with below steps:

1. fresh boot os, ib_umad is not loaded, the bug can not be reproduced:
# rpm -q libvirt qemu-kvm-rhev
libvirt-2.0.0-10.el7_3.14.x86_64
qemu-kvm-rhev-2.6.0-28.el7_3.18.x86_64

# virsh nodedev-list --cap net
net_enp0s29u1u1u5_42_f2_e9_32_6d_9f
net_enp11s0_40_f2_e9_32_6d_9d
net_enp6s0_40_f2_e9_32_6d_9c
net_lo_00_00_00_00_00_00
net_virbr0_nic_52_54_00_3a_6d_8e

#  lsmod | grep ib_umad
==> no outputs

2. load the ib_umad module, the bug can be reproduced after restart libvirtd:
# modprobe ib_umad
# lsmod | grep ib_umad
ib_umad                22093  0 
ib_core               255469  12 rdma_cm,ib_cm,iw_cm,rpcrdma,ib_srp,ib_iser,ib_srpt,ib_umad,ib_uverbs,rdma_ucm,ib_ipoib,ib_isert

# virsh nodedev-list --cap net
net_enp0s29u1u1u5_42_f2_e9_32_6d_9f
net_enp11s0_40_f2_e9_32_6d_9d
net_enp6s0_40_f2_e9_32_6d_9c
net_lo_00_00_00_00_00_00
net_virbr0_nic_52_54_00_3a_6d_8e

# systemctl restart libvirtd 

# virsh nodedev-list --cap net
net_enp11s0_40_f2_e9_32_6d_9d
net_enp6s0_40_f2_e9_32_6d_9c

# grep -i enp0s29u1u1u5 /var/log/libvirt/libvirtd.log
...
2019-07-02 09:29:42.419+0000: 2374: debug : udevAddOneDevice:1135 : Discarding device -1 0x7f191c170eb0 /sys/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.1/2-1.1.5/2-1.1.5:1.0/net/enp0s29u1u1u5
2019-07-02 09:29:42.419+0000: 2374: debug : udevProcessDeviceListEntry:1157 : Failed to create node device for udev device '/sys/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.1/2-1.1.5/2-1.1.5:1.0/net/enp0s29u1u1u5'
...
After load the ib_umad module and restart libvirtd, the device enp0s29u1u1u5 and lo, virbr0-nic all disappeared in the output of nodedev-list.

Comment 9 yalzhang@redhat.com 2019-07-03 08:21:53 UTC
verify this bug on libvirt-5.0.0-11.module+el8.0.1+3459+e357ef2f.x86_64

# rpm -q libvirt
libvirt-5.0.0-11.module+el8.0.1+3459+e357ef2f.x86_64

1. Make sure the rdma is enabled by checking the ib_umad is loaded:
# lsmod | grep ib_umad
ib_umad                28672  4
ib_core               282624  13 rdma_cm,ib_ipoib,rpcrdma,mlx4_ib,ib_srpt,ib_srp,iw_cm,ib_iser,ib_umad,ib_isert,rdma_ucm,ib_uverbs,ib_cm

2. Start a guest with network and direct type interface, then check the nodedev-list outputs:
# ll /sys/class/net
total 0
lrwxrwxrwx. 1 root root 0 Jun 21 19:03 eno1 -> ../../devices/pci0000:00/0000:00:03.1/0000:01:00.0/net/eno1
lrwxrwxrwx. 1 root root 0 Jun 21 19:03 eno2 -> ../../devices/pci0000:00/0000:00:03.1/0000:01:00.1/net/eno2
lrwxrwxrwx. 1 root root 0 Jun 21 19:03 eno3 -> ../../devices/pci0000:00/0000:00:03.0/0000:02:00.0/net/eno3
lrwxrwxrwx. 1 root root 0 Jun 21 19:03 eno4 -> ../../devices/pci0000:00/0000:00:03.0/0000:02:00.1/net/eno4
lrwxrwxrwx. 1 root root 0 Jun 21 19:03 enp4s0f0 -> ../../devices/pci0000:00/0000:00:02.0/0000:04:00.0/net/enp4s0f0
lrwxrwxrwx. 1 root root 0 Jun 21 19:03 enp4s0f1 -> ../../devices/pci0000:00/0000:00:02.0/0000:04:00.1/net/enp4s0f1
lrwxrwxrwx. 1 root root 0 Jun 21 19:03 ib0 -> ../../devices/pci0000:00/0000:00:02.2/0000:06:00.0/net/ib0
lrwxrwxrwx. 1 root root 0 Jun 21 19:03 ib1 -> ../../devices/pci0000:00/0000:00:02.2/0000:06:00.0/net/ib1
lrwxrwxrwx. 1 root root 0 Jun 21 19:03 lo -> ../../devices/virtual/net/lo
lrwxrwxrwx. 1 root root 0 Jul  3 16:17 macvtap0 -> ../../devices/virtual/net/macvtap0
lrwxrwxrwx. 1 root root 0 Jul  1 15:07 virbr0 -> ../../devices/virtual/net/virbr0
lrwxrwxrwx. 1 root root 0 Jul  1 15:07 virbr0-nic -> ../../devices/virtual/net/virbr0-nic
lrwxrwxrwx. 1 root root 0 Jul  3 16:07 vnet0 -> ../../devices/virtual/net/vnet0

# virsh nodedev-list --cap net
net_eno1_18_66_da_e6_83_98
net_eno2_18_66_da_e6_83_99
net_eno3_18_66_da_e6_83_9a
net_eno4_18_66_da_e6_83_9b
net_enp4s0f0_a0_36_9f_95_0a_08
net_enp4s0f1_a0_36_9f_95_0a_0a
net_ib0_80_00_02_08_fe_80_00_00_00_00_00_00_e4_1d_2d_03_00_48_23_31
net_ib1_80_00_02_09_fe_80_00_00_00_00_00_00_e4_1d_2d_03_00_48_23_32
net_lo_00_00_00_00_00_00
net_macvtap0_52_54_00_76_91_49
net_virbr0_nic_52_54_00_da_3f_87
net_vnet0_fe_54_00_77_92_da

The non-PCI network devices like virbr0-nic, vnet0, macvtap0 list out, the result is as expected.

Comment 11 errata-xmlrpc 2019-08-07 10:41:09 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2395


Note You need to log in before you can comment on or make changes to this bug.