Bug 1409957 - hot-plugged VF can‘t be used again after reset SRIOV-NIC's kernel mod
Summary: hot-plugged VF can‘t be used again after reset SRIOV-NIC's kernel mod
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt
Version: 7.3
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Laine Stump
QA Contact: Jingjing Shao
URL:
Whiteboard:
: 1402951 (view as bug list)
Depends On:
Blocks: 1420851
TreeView+ depends on / blocked
 
Reported: 2017-01-04 02:13 UTC by Han Han
Modified: 2018-02-15 21:09 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-02-15 21:09:58 UTC


Attachments (Terms of Use)
libvirtd log (7.64 MB, text/plain)
2017-01-04 02:13 UTC, Han Han
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Bugzilla 1376907 None None None 2019-09-29 06:17:13 UTC
Red Hat Bugzilla 1402951 None CLOSED libvirt nodedev doesn't rescan the pci_devices 2019-09-29 06:17:13 UTC

Internal Links: 1376907 1402951

Description Han Han 2017-01-04 02:13:24 UTC
Created attachment 1237047 [details]
libvirtd log

Description of problem:
As subject

Version-Release number of selected component (if applicable):
kernel-3.10.0-514.6.1.el7.x86_64
libvirt-2.0.0-10.el7_3.3.x86_64
qemu-kvm-rhev-2.6.0-28.el7_3.2.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Reload SRIOV kernel mod and create VF
# modprobe -r ixgbe
# modprobe ixgbe max_vfs=2

2. Prepare an VM and hot-plug the VF
# DOM=V
# virsh start $DOM && sleep 20
Domain V started

# virsh attach-device V hostdev.xml
Device attached successfully

3. Reset kernel mod and try to attach/detach the VF
# modprobe -r ixgbe
# modprobe ixgbe max_vfs=2
# virsh detach-device $DOM hostdev.xml
error: Failed to detach device from hostdev.xml
error: operation failed: no device matching mac address 02:24:6b:89:bc:e0 found

# virsh attach-device $DOM hostdev.xml
error: Failed to attach device from hostdev.xml
error: internal error: Not detaching active device 0000:86:10.1

Actual results:
As step3

Expected results:
VF coulde be attached after resetting the kernel mod.


Additional info:
After restart libvirtd, VF could be attached in step3.

Comment 1 Han Han 2017-01-04 02:15:24 UTC
The hostdev.xml file:
<interface type='hostdev' managed='yes'>
<mac address='02:24:6b:89:bc:e0'/>
<source>
<address type='pci' domain='0x0000' bus='0x86' slot='0x10' function='0x1'/>
</source>
</interface>

Comment 2 Jingjing Shao 2017-01-04 07:04:56 UTC
Additional info:

After the steps3 in the Description,

4, check the dumpxml of Domain V
#virsh dumpxml V | grep hostdev -A6

5, check the nodedev-dumpxml of VF
# virsh nodedev-dumpxml pci_0000_86_10_1
<device>
  <name>pci_0000_86_10_1</name>
  <path>/sys/devices/pci0000:80/0000:80:01.0/0000:86:10.1</path>
  <parent>pci_0000_80_01_0</parent>
  <driver>
    <name>ixgbevf</name>
  </driver>
  <capability type='pci'>
    <domain>0</domain>
    <bus>134</bus>
    <slot>16</slot>
    <function>1</function>
    <product id='0x10ed'>82599 Ethernet Controller Virtual Function</product>
    <vendor id='0x8086'>Intel Corporation</vendor>
    <capability type='phys_function'>
      <address domain='0x0000' bus='0x86' slot='0x00' function='0x1'/>
    </capability>
    <iommuGroup number='83'>
      <address domain='0x0000' bus='0x86' slot='0x10' function='0x1'/>
    </iommuGroup>
    <numa node='1'/>
    <pci-express>
      <link validity='cap' port='0' width='0'/>
      <link validity='sta' width='0'/>
    </pci-express>
  </capability>
</device>


6, shutddown the domain
# virsh destroy V
Domain V destroyed

7.detach the VF from host, then get error

# virsh nodedev-detach pci_0000_86_10_1 
error: Failed to detach device pci_0000_86_10_1
error: Requested operation is not valid: PCI device 0000:86:10.1 is in use by driver QEMU, domain V

8. restart libvirtd 
service libvirtd restart
Redirecting to /bin/systemctl restart  libvirtd.service



After restart libvirtd, all the things turn to be right

9.# virsh nodedev-detach pci_0000_86_10_1
Device pci_0000_86_10_1 detached

# virsh nodedev-reattach pci_0000_86_10_1
Device pci_0000_86_10_1 re-attached


10.# virsh start V
Domain V started

# virsh attach-device V interface.xml
Device attached successfully

Comment 3 Jiri Denemark 2017-01-04 08:15:43 UTC
Is this a regression introduced by libvirt-2.0.0-10.el7_3.3.x86_64 or can you reproduce even with libvirt-2.0.0-10.el7_3.2.x86_64 or libvirt-2.0.0-10.el7.x86_64?

Comment 4 Han Han 2017-01-04 10:02:36 UTC
Hi Jiri,
The bug can be reproduced on libvirt-2.0.0-10.el7_3.2.x86_64 and libvirt-2.0.0-10.el7.x86_64 as commment0 and comment2.
It is not a regression on RHEL7.3.

Comment 6 Jaroslav Suchanek 2017-01-06 13:40:43 UTC
Dup (or just different result) of bug 1402951?

Comment 7 Han Han 2017-01-10 02:36:41 UTC
The bug can be reproduced on libvirt-2.5.0-1.el7.x86_64 as comment0 and commment2

Comment 10 Laine Stump 2017-12-04 18:11:39 UTC
*** Bug 1402951 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.