Bug 1402951

Summary: libvirt nodedev doesn't rescan the pci_devices
Product: Red Hat Enterprise Linux 7 Reporter: Robin Cernin <rcernin>
Component: libvirtAssignee: Laine Stump <laine>
Status: CLOSED DUPLICATE QA Contact: Jingjing Shao <jishao>
Severity: high Docs Contact:
Priority: unspecified    
Version: 7.3CC: alex.williamson, berrange, dyuan, jsuchane, laine, rbalakri, rcernin, vcojot, vromanso, xuzhang
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-12-04 18:11:39 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1420851    

Description Robin Cernin 2016-12-08 17:37:40 UTC
Description of problem:

[root@server2 nova]# sudo virsh nodedev-dumpxml pci_0000_81_00_1
<device>
  <name>pci_0000_81_00_1</name>
  <path>/sys/devices/pci0000:80/0000:80:02.0/0000:81:00.1</path>
  <parent>pci_0000_80_02_0</parent>
  <driver>
    <name>ixgbe</name>
  </driver>
  <capability type='pci'>
    <domain>0</domain>
    <bus>129</bus> 
    <slot>0</slot> 
    <function>1</function>
    <product id='0x10f8'>82599 10 Gigabit Dual Port Backplane Connection</product>
    <vendor id='0x8086'>Intel Corporation</vendor>
    <capability type='virt_functions' maxCount='63'>
      <address domain='0x0000' bus='0x81' slot='0x10' function='0x1'/>
      <address domain='0x0000' bus='0x81' slot='0x10' function='0x3'/>
      <address domain='0x0000' bus='0x81' slot='0x10' function='0x5'/>
      <address domain='0x0000' bus='0x81' slot='0x10' function='0x7'/>
      <address domain='0x0000' bus='0x81' slot='0x11' function='0x1'/>
      <address domain='0x0000' bus='0x81' slot='0x11' function='0x3'/>
      <address domain='0x0000' bus='0x81' slot='0x11' function='0x5'/>
      <address domain='0x0000' bus='0x81' slot='0x11' function='0x7'/>
      <address domain='0x0000' bus='0x81' slot='0x12' function='0x1'/>
      <address domain='0x0000' bus='0x81' slot='0x12' function='0x3'/>
      <address domain='0x0000' bus='0x81' slot='0x12' function='0x5'/>
      <address domain='0x0000' bus='0x81' slot='0x12' function='0x7'/>
      <address domain='0x0000' bus='0x81' slot='0x13' function='0x1'/>
      <address domain='0x0000' bus='0x81' slot='0x13' function='0x3'/>
      <address domain='0x0000' bus='0x81' slot='0x13' function='0x5'/>
      <address domain='0x0000' bus='0x81' slot='0x13' function='0x7'/>
      <address domain='0x0000' bus='0x81' slot='0x14' function='0x1'/>
      <address domain='0x0000' bus='0x81' slot='0x14' function='0x3'/>
      <address domain='0x0000' bus='0x81' slot='0x14' function='0x5'/>
      <address domain='0x0000' bus='0x81' slot='0x14' function='0x7'/>
      <address domain='0x0000' bus='0x81' slot='0x15' function='0x1'/>
      <address domain='0x0000' bus='0x81' slot='0x15' function='0x3'/>
      <address domain='0x0000' bus='0x81' slot='0x15' function='0x5'/>
      <address domain='0x0000' bus='0x81' slot='0x15' function='0x7'/>
      <address domain='0x0000' bus='0x81' slot='0x16' function='0x1'/>
      <address domain='0x0000' bus='0x81' slot='0x16' function='0x3'/>
      <address domain='0x0000' bus='0x81' slot='0x16' function='0x5'/>
      <address domain='0x0000' bus='0x81' slot='0x16' function='0x7'/>
      <address domain='0x0000' bus='0x81' slot='0x19' function='0x1'/>
      <address domain='0x0000' bus='0x81' slot='0x19' function='0x3'/>
      <address domain='0x0000' bus='0x81' slot='0x19' function='0x5'/>
      <address domain='0x0000' bus='0x81' slot='0x19' function='0x7'/>
      <address domain='0x0000' bus='0x81' slot='0x1a' function='0x1'/>
      <address domain='0x0000' bus='0x81' slot='0x1a' function='0x3'/>
      <address domain='0x0000' bus='0x81' slot='0x1a' function='0x5'/>
      <address domain='0x0000' bus='0x81' slot='0x1a' function='0x7'/>
      <address domain='0x0000' bus='0x81' slot='0x1b' function='0x1'/>
      <address domain='0x0000' bus='0x81' slot='0x1b' function='0x3'/>
      <address domain='0x0000' bus='0x81' slot='0x1b' function='0x5'/>
      <address domain='0x0000' bus='0x81' slot='0x1b' function='0x7'/>
      <address domain='0x0000' bus='0x81' slot='0x1c' function='0x1'/>
      <address domain='0x0000' bus='0x81' slot='0x1c' function='0x3'/>
      <address domain='0x0000' bus='0x81' slot='0x1c' function='0x5'/>
      <address domain='0x0000' bus='0x81' slot='0x1c' function='0x7'/>
      <address domain='0x0000' bus='0x81' slot='0x1d' function='0x1'/>
      <address domain='0x0000' bus='0x81' slot='0x1d' function='0x3'/>
      <address domain='0x0000' bus='0x81' slot='0x1d' function='0x5'/>
      <address domain='0x0000' bus='0x81' slot='0x1d' function='0x7'/>
      <address domain='0x0000' bus='0x81' slot='0x1e' function='0x1'/>
      <address domain='0x0000' bus='0x81' slot='0x1e' function='0x3'/>
      <address domain='0x0000' bus='0x81' slot='0x1e' function='0x5'/>
      <address domain='0x0000' bus='0x81' slot='0x1e' function='0x7'/>
      <address domain='0x0000' bus='0x81' slot='0x1f' function='0x1'/>
      <address domain='0x0000' bus='0x81' slot='0x1f' function='0x3'/>
      <address domain='0x0000' bus='0x81' slot='0x1f' function='0x5'/>
    </capability>  
    <iommuGroup number='25'>
      <address domain='0x0000' bus='0x81' slot='0x00' function='0x1'/>
    </iommuGroup>  
    <numa node='1'/>
    <pci-express>  
      <link validity='cap' port='0' speed='5' width='8'/>
      <link validity='sta' speed='5' width='8'/>
    </pci-express> 
  </capability>
</device>
 
    </pci-express> 
 
[root@server2 ~]# lspci|grep -i 81:1d.3
81:1d.3 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
 
[root@server2 ~]# virsh nodedev-dumpxml pci_0000_81_1d_3
error: Could not find matching device 'pci_0000_81_1d_3'
error: Node device not found: no node device with matching name 'pci_0000_81_1d_3'

[root@server2 ~]# systemctl restart libvirtd.service

[root@server2 ~]# virsh nodedev-dumpxml pci_0000_81_1d_3
< I didn't kept the output but this time it worked > 

Isn't that any nodedev command should rescan the PCI devices?

Version-Release number of selected component (if applicable):
libvirt-2.0.0-10.el7.x86_64                                 Mon Oct 31 21:35:13 2016
libvirt-client-2.0.0-10.el7.x86_64                          Mon Oct 31 21:29:40 2016
libvirt-daemon-2.0.0-10.el7.x86_64                          Mon Oct 31 21:29:41 2016
libvirt-daemon-config-network-2.0.0-10.el7.x86_64           Mon Oct 31 21:29:46 2016
libvirt-daemon-config-nwfilter-2.0.0-10.el7.x86_64          Mon Oct 31 21:29:43 2016
libvirt-daemon-driver-interface-2.0.0-10.el7.x86_64         Mon Oct 31 21:29:42 2016
libvirt-daemon-driver-lxc-2.0.0-10.el7.x86_64               Mon Oct 31 21:29:45 2016
libvirt-daemon-driver-network-2.0.0-10.el7.x86_64           Mon Oct 31 21:29:41 2016
libvirt-daemon-driver-nodedev-2.0.0-10.el7.x86_64           Mon Oct 31 21:29:43 2016
libvirt-daemon-driver-nwfilter-2.0.0-10.el7.x86_64          Mon Oct 31 21:29:41 2016
libvirt-daemon-driver-qemu-2.0.0-10.el7.x86_64              Mon Oct 31 21:31:59 2016
libvirt-daemon-driver-secret-2.0.0-10.el7.x86_64            Mon Oct 31 21:29:43 2016
libvirt-daemon-driver-storage-2.0.0-10.el7.x86_64           Mon Oct 31 21:31:58 2016
libvirt-daemon-kvm-2.0.0-10.el7.x86_64                      Mon Oct 31 21:35:01 2016
libvirt-python-2.0.0-2.el7.x86_64                           Mon Oct 31 21:29:42 2016
erlang-kernel-18.3.4.1-1.el7ost.x86_64                      Mon Oct 31 21:25:47 2016
kernel-3.10.0-514.el7.x86_64                                Mon Oct 31 20:42:03 2016
kernel-devel-3.10.0-514.el7.x86_64                          Mon Oct 31 21:39:15 2016
kernel-headers-3.10.0-514.el7.x86_64                        Mon Oct 31 21:39:00 2016
kernel-tools-3.10.0-514.el7.x86_64                          Mon Oct 31 20:42:14 2016
kernel-tools-libs-3.10.0-514.el7.x86_64                     Mon Oct 31 20:37:46 2016

Comment 1 Daniel Berrangé 2016-12-08 17:47:16 UTC
(In reply to Robin Cernin from comment #0)
> 
> Isn't that any nodedev command should rescan the PCI devices?

No, libvirt is supposed to be notified of any changes by udev, so its view ought to be fully up2date without need for any manual refresh command.

Comment 2 Laine Stump 2016-12-08 18:55:50 UTC
The reproduction steps in the Description don't match what I understood on IRC. In IRC it was explained that the host had a PF with 5 VFs when libvirtd started, then the PF was changed to have 63 VFs but libvirt still only showed 5 of them in the dumpxml for the PF. What I see here is that the dumpxml of the PF *does* show the new VFs, but that the VFs themselves can't be dumped.

If that's the case, and if the host journal is showing errors like the following right after the VF count is changed:

   libvirtd[1605]: Device 0000:03:00.0 not found: could not access
   /sys/bus/pci/devices/0000:03:00.0/config: No such file or directory

then I think this actually *is* the problem reported in Bug 1376907 (in spite of our best attempts to convince you otherwise :-)

Comment 3 Jaroslav Suchanek 2016-12-09 08:32:42 UTC
Can you please confirm/deny Laine's suspicions in comment 2? Thanks.

Comment 6 Laine Stump 2017-12-04 18:11:39 UTC
Lacking other feedback, I'm assuming this is the same problem as Bug 1409957, and mark it as a duplicate. If you learn otherwise, feel free to reopen.

*** This bug has been marked as a duplicate of bug 1409957 ***