Bug 1721403

Summary: After hot unplug virtio-net and vfio nic, hot plug vfio-pci device fails in Win2019 guest
Product: Red Hat Enterprise Linux 7 Reporter: Lei Yang <leiyang>
Component: qemu-kvm-rhevAssignee: Julia Suvorova <jsuvorov>
Status: CLOSED ERRATA QA Contact: Yanghang Liu <yanghliu>
Severity: high Docs Contact:
Priority: high    
Version: 7.7CC: ailan, chayang, jinzhao, juzhang, mrezanin, pezhang, virt-maint, yanghliu, yvugenfi
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: qemu-kvm-rhev-2.12.0-45.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1710667 Environment:
Last Closed: 2020-08-03 07:23:25 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1710667    
Bug Blocks: 1797441    

Comment 3 Lei Yang 2019-06-18 08:48:44 UTC
Version-Release:
qemu-kvm-rhev-2.12.0-32.el7.x86_64
kernel-3.10.0-1057.el7.x86_64
virtio-win-1.9.7-0.el7.noarch

Comment 7 Yanghang Liu 2019-11-12 14:08:44 UTC
Hi,

I will still encounter this issue using the latest qemu-kvm-rhev version and kernel version in RHEL7.7


Version-Release number of selected component (if applicable):
qemu-kvm-rhev-2.12.0-33.el7_7.5.x86_64
3.10.0-1062.4.1.el7.x86_64
virtio-win-1.9.8-6.el7.noarch


How reproducible:
100%


Steps to Reproduce
(1)start the Win2019 guest with a virtio-net-pci device and a vfio-pci device.

The qemu cmd line is as followed:
/usr/libexec/qemu-kvm -name Win2019 \
-M q35,kernel-irqchip=split -m 4G \
-cpu Haswell-noTSX \
-nodefaults \
-device intel-iommu,intremap=on,caching-mode=on \
-smp 4,sockets=1,cores=4,threads=1 \
-device pcie-root-port,id=root.1,chassis=1 \
-device pcie-root-port,id=root.2,chassis=2 \
-device pcie-root-port,id=root.3,chassis=3 \
-device pcie-root-port,id=root.4,chassis=4 \
-device pcie-root-port,id=root.5,chassis=5 \
-blockdev driver=file,cache.direct=off,cache.no-flush=on,filename=/home/images/win2019.qcow2,node-name=my_file \
-drive id=drive_cd1,if=none,snapshot=off,aio=native,cache=none,media=cdrom,file=/home/images/en_windows_server_2019_x64_dvd_4cb967d8.iso \
-blockdev driver=qcow2,node-name=my,file=my_file \
-device virtio-blk-pci,drive=my,id=virtio-blk0,bus=root.1 \
-device ide-cd,id=cd1,drive=drive_cd1,bus=ide.0,unit=0 \
-drive id=drive_winutils,if=none,snapshot=off,aio=native,cache=none,media=cdrom,file=/usr/share/virtio-win/virtio-win-1.9.8.iso \
-device ide-cd,id=winutils,drive=drive_winutils,bus=ide.1,unit=0 \
-vnc :0 \
-vga qxl \
-monitor stdio \
-qmp tcp:0:5555,server,nowait \
-usb -device usb-tablet \
-boot menu=on \
-device vfio-pci,host=0000:04:00.0,id=pf,bus=root.3 \
-device virtio-net-pci,netdev=nic1,id=vnet0,mac=54:43:00:1a:11:33,bus=root.2 \
-netdev tap,id=nic1,script=/etc/qemu-ifup,vhost=on \

(2)hot unplug virtio-net-pci device

hmp:
(qemu) device_del vnet0
(qemu) netdev_del nic1
qmp:
# telnet 10.73.73.73 5555
{"execute":"qmp_capabilities"}
{"return": {}}
{"execute": "device_del", "arguments": {"id": "vnet0"}}
{"return": {}}
{"timestamp": {"seconds": 1573566784, "microseconds": 650482}, "event": "DEVICE_DELETED", "data": {"path": "/machine/peripheral/vnet0/virtio-backend"}}
{"execute": "netdev_del", "arguments": {"id": "nic1"}}
{"return": {}}

(3)Hot unplug vfio-pci device
hmp:
(qemu) device_del pf
qmp:
{"execute": "device_del", "arguments": {"id": "pf"}}   
{"return": {}}

(4)Hot plug vfio-pci device
hmp:
(qemu)  device_add vfio-pci,host=0000:04:00.0,id=pf,bus=root.3
Duplicate ID 'pf' for device
Try "help device_add" for more information
qmp:
{"execute": "device_add", "arguments": {"driver":"vfio-pci","host":"04:00.0","id":"pf","bus":"root.3"}}
{"error": {"class": "GenericError", "desc": "Duplicate ID 'pf' for device"}}


Actual results:
The vfio-pci can not be hotplugged to the Win2019 guest after the Win2019 guest hotunplug a virtio-net nic and hotunplug a vfio nic, 

Expected results:
The vfio-pci device can be hotpluged to a Win2019 guest successfully.

Additional info:
# lspci | grep -i 04:00.0
04:00.0 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02)

Comment 14 Yvugenfi@redhat.com 2020-03-03 16:54:59 UTC
This BZ is solved by BZ#1708480

Comment 20 Yanghang Liu 2020-04-13 03:46:32 UTC
Verification:

Versions:
3.10.0-1133.el7.x86_64
qemu-kvm-rhev-2.12.0-46.el7.x86_64
virtio-win-1.9.6-1.el7.noarch


Steps:
(1)start the Win2019 guest with a virtio-net-pci device and a vfio-pci device.
/usr/libexec/qemu-kvm -name Win2019 \
-M q35,kernel-irqchip=split \
-m 4G \
-nodefaults \
-cpu Haswell-noTSX \
-smp 4,sockets=1,cores=4,threads=1 \
-device pcie-root-port,id=root.1,chassis=1,addr=0x2.0,multifunction=on \
-device pcie-root-port,id=root.2,chassis=2,addr=0x2.1 \
-device pcie-root-port,id=root.3,chassis=3,addr=0x2.2 \
-device pcie-root-port,id=root.4,chassis=4,addr=0x2.3 \
-device pcie-root-port,id=root.5,chassis=5,addr=0x2.4 \
-device pcie-root-port,id=root.6,chassis=6,addr=0x2.5 \
-device pcie-root-port,id=root.7,chassis=7,addr=0x2.6 \
-device pcie-root-port,id=root.8,chassis=8,addr=0x2.7 \
-device intel-iommu,intremap=on,caching-mode=on \
-blockdev driver=file,cache.direct=off,cache.no-flush=on,filename=/home/images/win2019.qcow2,node-name=my_file \
-blockdev driver=qcow2,node-name=my,file=my_file \
-device virtio-blk-pci,drive=my,id=virtio-blk0,bus=root.1 \
-drive id=drive_winutils,if=none,snapshot=off,aio=native,cache=none,media=cdrom,file=/usr/share/virtio-win/virtio-win-1.9.6.iso \
-device ide-cd,id=winutils,drive=drive_winutils,bus=ide.1,unit=0 \
-vnc :0 \
-vga qxl \
-monitor stdio \
-usb -device usb-tablet \
-boot menu=on \
-qmp tcp:0:5555,server,nowait \
-device virtio-net-pci,netdev=nic1,id=vnet0,mac=54:43:00:1a:11:33,bus=root.2 \
-netdev tap,id=nic1,script=/etc/qemu-ifup,vhost=on \
-device vfio-pci,host=0000:04:00.0,id=pf,bus=root.3 \


(2)hot unplug virtio-net-pci device
qmp:
# telnet 10.73.73.73 5555
{"execute":"qmp_capabilities"}
{"return": {}}
{"execute": "device_del", "arguments": {"id": "vnet0"}}
{"return": {}}
{"timestamp": {"seconds": 1586748999, "microseconds": 984905}, "event": "DEVICE_DELETED", "data": {"path": "/machine/peripheral/vnet0/virtio-backend"}}
{"timestamp": {"seconds": 1586749000, "microseconds": 43700}, "event": "DEVICE_DELETED", "data": {"device": "vnet0", "path": "/machine/peripheral/vnet0"}}
{"execute": "netdev_del", "arguments": {"id": "nic1"}}
{"return": {}}




(3)hot unplug vfio-pci device
qmp:
{"execute": "device_del", "arguments": {"id": "pf"}}   
{"return": {}}
{"timestamp": {"seconds": 1586749031, "microseconds": 847422}, "event": "DEVICE_DELETED", "data": {"device": "pf", "path": "/machine/peripheral/pf"}}


(4)hot plug vfio-pci device
qmp:
{"execute": "device_add", "arguments": {"driver":"vfio-pci","host":"04:00.0","id":"pf","bus":"root.3"}}
{"return": {}}


(5)check vfio-pci device status in guest/on host
The vfio-pci device can be hotpluged to a Win2019 guest successfully.





According to the test result,this bug has been fixed well.

According to comment 18 and comment 19 , move the bug status from MODIFIED to VERIFIED.

Comment 23 errata-xmlrpc 2020-08-03 07:23:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:3267