Bug 1047818

Summary: VFs can not be listed by net-dumpxml directly after starting the hostdev network
Product: Red Hat Enterprise Linux 7 Reporter: Hu Jianwei <jiahu>
Component: libvirtAssignee: Laine Stump <laine>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: low Docs Contact:
Priority: low    
Version: 7.0CC: dyuan, honzhang, lcheng, mzhan, rbalakri, xuzhang
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-1.2.8-1.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-03-05 07:29:23 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Hu Jianwei 2014-01-02 09:58:32 UTC
Description of problem:
VFs can not be listed by net-dumpxml directly after starting the hostdev network

Version-Release number of selected component (if applicable):
libvirt-1.1.1-17.el7.x86_64
qemu-kvm-1.5.3-30.el7.x86_64
kernel-3.10.0-64.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Setup vfio env
modprobe vfio_pci
echo 1 > /sys/module/vfio_iommu_type1/parameters/allow_unsafe_interrupts

2. Dump that hostdev network.
[root@ibm-x3850x5-05 jiahu]# virsh net-list --all
 Name                 State      Autostart     Persistent
----------------------------------------------------------
 default              active     yes           yes
 vfio-passthrough     inactive   no            yes

[root@ibm-x3850x5-05 jiahu]# virsh net-dumpxml vfio-passthrough
<network>
  <name>vfio-passthrough</name>
  <uuid>fea392f0-281a-473a-8c45-b63a6b529f09</uuid>
  <forward mode='hostdev' managed='yes'>
    <driver name='vfio'/>
    <pf dev='p2p1'/>
  </forward>
</network>

[root@ibm-x3850x5-05 jiahu]# lspci | grep 86:00.0
86:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)
[root@ibm-x3850x5-05 jiahu]# virsh nodedev-dumpxml pci_0000_86_00_0
<device>
  <name>pci_0000_86_00_0</name>
  <path>/sys/devices/pci0000:80/0000:80:01.0/0000:86:00.0</path>
  <parent>pci_0000_80_01_0</parent>
  <driver>
    <name>ixgbe</name>
  </driver>
  <capability type='pci'>
    <domain>0</domain>
    <bus>134</bus>
    <slot>0</slot>
    <function>0</function>
    <product id='0x10fb'>82599EB 10-Gigabit SFI/SFP+ Network Connection</product>
    <vendor id='0x8086'>Intel Corporation</vendor>
    <capability type='virt_functions'>
      <address domain='0x0000' bus='0x86' slot='0x10' function='0x0'/>
      <address domain='0x0000' bus='0x86' slot='0x10' function='0x2'/>
      <address domain='0x0000' bus='0x86' slot='0x10' function='0x4'/>
      <address domain='0x0000' bus='0x86' slot='0x10' function='0x6'/>
      <address domain='0x0000' bus='0x86' slot='0x11' function='0x0'/>
      <address domain='0x0000' bus='0x86' slot='0x11' function='0x2'/>
      <address domain='0x0000' bus='0x86' slot='0x11' function='0x4'/>
      <address domain='0x0000' bus='0x86' slot='0x11' function='0x6'/>
    </capability>
    <iommuGroup number='27'>
      <address domain='0x0000' bus='0x86' slot='0x00' function='0x0'/>
      <address domain='0x0000' bus='0x86' slot='0x00' function='0x1'/>
    </iommuGroup>
  </capability>
</device>

3. Start that hostdev network and check the dump xml
[root@ibm-x3850x5-05 jiahu]# virsh net-start vfio-passthrough
Network vfio-passthrough started

[root@ibm-x3850x5-05 jiahu]# virsh net-list --all
 Name                 State      Autostart     Persistent
----------------------------------------------------------
 default              active     yes           yes
 vfio-passthrough     active     no            yes

[root@ibm-x3850x5-05 jiahu]#
[root@ibm-x3850x5-05 jiahu]# virsh net-dumpxml vfio-passthrough
<network>
  <name>vfio-passthrough</name>
  <uuid>fea392f0-281a-473a-8c45-b63a6b529f09</uuid>
  <forward mode='hostdev' managed='yes'>
    <driver name='vfio'/>
    <pf dev='p2p1'/>
  </forward>                                    <=================No any VFs were printed.
</network>                          

[root@ibm-x3850x5-05 jiahu]# virsh dumpxml rhel7 | grep interface -A5
    <interface type='network'>
      <mac address='52:54:00:e5:91:32'/>
      <source network='vfio-passthrough'/>
      <model type='rtl8139'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </interface>
...
 
4. Do something to trigger off it, then check the dump xml again.
[root@ibm-x3850x5-05 jiahu]# virsh start rhel7
Domain rhel7 started

[root@ibm-x3850x5-05 jiahu]# virsh net-dumpxml vfio-passthrough
<network connections='1'>
  <name>vfio-passthrough</name>
  <uuid>fea392f0-281a-473a-8c45-b63a6b529f09</uuid>
  <forward mode='hostdev' managed='yes'>
    <driver name='vfio'/>
    <pf dev='p2p1'/>
    <address type='pci' domain='0x0000' bus='0x86' slot='0x10' function='0x0'/>   <========They are displayed.
    <address type='pci' domain='0x0000' bus='0x86' slot='0x10' function='0x2'/>
    <address type='pci' domain='0x0000' bus='0x86' slot='0x10' function='0x4'/>
    <address type='pci' domain='0x0000' bus='0x86' slot='0x10' function='0x6'/>
    <address type='pci' domain='0x0000' bus='0x86' slot='0x11' function='0x0'/>
    <address type='pci' domain='0x0000' bus='0x86' slot='0x11' function='0x2'/>
    <address type='pci' domain='0x0000' bus='0x86' slot='0x11' function='0x4'/>
    <address type='pci' domain='0x0000' bus='0x86' slot='0x11' function='0x6'/>
  </forward>
</network>

Actual results:
As shown above, can not find its VF in dump xml.

Expected results:
All associated VFs should be displayed when starting that hostdev network, and do not take any other actions to enable it.

Comment 1 Laine Stump 2014-06-18 10:24:47 UTC
Yeah, I'm not sure why the author chose to do a "lazy" algorithm for this rather than just populating it immediately. I agree that it would be better to fill in the VF list as soon as the network is started.

Comment 4 Laine Stump 2014-08-12 03:56:41 UTC
Fix pushed upstream:

commit 2a193f64580094021745c7507ac2ad22d3c28127
Author: Laine Stump <laine>
Date:   Tue Aug 5 17:15:31 2014 -0400

    network: populate interface pool immediately when network is started
    
    When a network is defined with "<pf dev='xyz'/>", libvirt will query
    sysfs to learn the list of all virtual functions (VF) associated with
    that Physical Function (PF) then populate the network's interface pool
    accordingly. This action was previously done only when the first guest
    actually requested an interface from the network. This patch changes
    it to populate the pool immediately when the network is started. This
    way any problems with the PF or its VFs will become apparent sooner.
    
    Note that we can't remove the old calls to networkCreateInterfacePool
    that happen whenever a guest requests an interface - doing so would be
    asking for failures on hosts that had libvirt upgraded with a network
    that had been started but not yet used.

Comment 7 lcheng 2014-11-19 06:55:34 UTC
Verify it as follows. The result is expected.


Version:
libvirt-1.2.8-7.el7.x86_64
qemu-kvm-1.5.3-79.el7.x86_64
qemu-kvm-rhev-2.1.2-8.el7.x86_64


Steps:
# virsh net-define vfio-pt.xml 
Network vfio-pt defined from vfio-pt.xml

# virsh net-list --all
 Name                 State      Autostart     Persistent
----------------------------------------------------------
 default              active     yes           yes
 vfio-pt              inactive   no            yes

# virsh net-dumpxml vfio-pt
<network>
  <name>vfio-pt</name>
  <uuid>fea392f0-281a-473a-8c45-b63a6b529f09</uuid>
  <forward mode='hostdev' managed='yes'>
    <driver name='vfio'/>
    <pf dev='ens1f1'/>
  </forward>
</network>


# virsh nodedev-dumpxml pci_0000_03_00_1
<device>
  <name>pci_0000_03_00_1</name>
  <path>/sys/devices/pci0000:00/0000:00:01.0/0000:03:00.1</path>
  <parent>pci_0000_00_01_0</parent>
  <driver>
    <name>igb</name>
  </driver>
  <capability type='pci'>
    <domain>0</domain>
    <bus>3</bus>
    <slot>0</slot>
    <function>1</function>
    <product id='0x10c9'>82576 Gigabit Network Connection</product>
    <vendor id='0x8086'>Intel Corporation</vendor>
    <capability type='virt_functions'>
      <address domain='0x0000' bus='0x03' slot='0x10' function='0x1'/>
      <address domain='0x0000' bus='0x03' slot='0x10' function='0x3'/>
      <address domain='0x0000' bus='0x03' slot='0x10' function='0x5'/>
      <address domain='0x0000' bus='0x03' slot='0x10' function='0x7'/>
      <address domain='0x0000' bus='0x03' slot='0x11' function='0x1'/>
      <address domain='0x0000' bus='0x03' slot='0x11' function='0x3'/>
      <address domain='0x0000' bus='0x03' slot='0x11' function='0x5'/>
    </capability>
    <iommuGroup number='14'>
      <address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
      <address domain='0x0000' bus='0x03' slot='0x00' function='0x1'/>
    </iommuGroup>
    <pci-express>
      <link validity='cap' port='0' speed='2.5' width='4'/>
      <link validity='sta' speed='2.5' width='4'/>
    </pci-express>
  </capability>
</device>

# virsh net-start vfio-pt
Network vfio-pt started

# virsh net-dumpxml vfio-pt
<network>
  <name>vfio-pt</name>
  <uuid>fea392f0-281a-473a-8c45-b63a6b529f09</uuid>
  <forward mode='hostdev' managed='yes'>
    <driver name='vfio'/>
    <pf dev='ens1f1'/>
    <address type='pci' domain='0x0000' bus='0x03' slot='0x10' function='0x1'/>
    <address type='pci' domain='0x0000' bus='0x03' slot='0x10' function='0x3'/>
    <address type='pci' domain='0x0000' bus='0x03' slot='0x10' function='0x5'/>
    <address type='pci' domain='0x0000' bus='0x03' slot='0x10' function='0x7'/>
    <address type='pci' domain='0x0000' bus='0x03' slot='0x11' function='0x1'/>
    <address type='pci' domain='0x0000' bus='0x03' slot='0x11' function='0x3'/>
    <address type='pci' domain='0x0000' bus='0x03' slot='0x11' function='0x5'/>
  </forward>
</network>

Comment 9 errata-xmlrpc 2015-03-05 07:29:23 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0323.html