Description of problem: hotplug device under hotplug=off controller, the device do not really hotplug in guest, there is no relevant devices in guest. but using qmp command query-pci may get the pci. Version-Release number of selected component (if applicable): host: 4.18.0-193.el8.x86_64 qemu-kvm-core-5.0.0-0.scrmod+el8.2.0+6151+7a36dcb6.wrb200401.x86_64 How reproducible: 100% Steps to Reproduce: 1.create images with different size qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg1.qcow2 1G qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg2.qcow2 2G ... qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg8.qcow2 8G 2.boot vm /usr/libexec/qemu-kvm \ -name 'avocado-vt-vm1' \ -machine q35 \ -nodefaults \ -device VGA,bus=pcie.0,addr=0x1 \ -device pcie-root-port,id=pcie.0-root-port-2,slot=2,chassis=2,addr=0x2,bus=pcie.0 \ -device qemu-xhci,id=usb1,bus=pcie.0-root-port-2,addr=0x0 \ -object iothread,id=iothread0 \ -device pcie-root-port,id=pcie.0-root-port-3,slot=3,chassis=3,addr=0x3,bus=pcie.0 \ -device virtio-scsi-pci,id=virtio_scsi_pci,iothread=iothread0,bus=pcie.0-root-port-3,addr=0x0 \ -drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/rhel820-64-virtio-scsi.qcow2 \ -device scsi-hd,id=image1,drive=drive_image1,bootindex=0 \ -device pcie-root-port,id=pcie.0-root-port-4,slot=4,chassis=4,addr=0x4,bus=pcie.0 \ -device virtio-net-pci,mac=9a:ab:fa:09:62:1c,id=idLVRSKI,netdev=idfMN0PL,bus=pcie.0-root-port-4,addr=0x0 \ -netdev tap,id=idfMN0PL,vhost=on \ -m 8G \ -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \ -vnc :6 \ -qmp tcp:0:5956,server,nowait \ -rtc base=localtime,clock=host,driftfix=slew \ -boot order=cdn,once=c,menu=off,strict=off \ -enable-kvm \ -monitor stdio \ -blockdev node-name=file_stg1,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/stg1.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_stg1,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg1 \ -blockdev node-name=file_stg2,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/stg2.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_stg2,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg2 \ \ -blockdev node-name=file_stg3,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/stg3.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_stg3,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg3 \ -blockdev node-name=file_stg4,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/stg4.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_stg4,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg4 \ \ -blockdev node-name=file_stg5,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/stg5.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_stg5,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg5 \ -blockdev node-name=file_stg6,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/stg6.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_stg6,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg6 \ \ -blockdev node-name=file_stg7,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/stg7.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_stg7,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg7 \ -blockdev node-name=file_stg8,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/stg8.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_stg8,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg8 \ \ -device pcie-root-port,id=pcie_port_1,slot=5,chassis=5,addr=0x5,bus=pcie.0,hotplug=off \ -device virtio-blk-pci,id=stg1,drive=drive_stg1,bus=pcie_port_1,addr=0x0 \ \ -device pcie-root-port,id=pcie_port_2,slot=6,chassis=6,addr=0x6,bus=pcie.0,hotplug=on \ -device virtio-blk-pci,id=stg2,drive=drive_stg2,bus=pcie_port_2,addr=0x0 \ \ -device pcie-root-port,id=pcie_port_3,slot=7,chassis=7,addr=0x7,bus=pcie.0,hotplug=off \ -device pcie-root-port,id=pcie_port_4,slot=8,chassis=8,addr=0x8,bus=pcie.0,hotplug=on \ 3.check pci in guest [root@bootp-73-131-155 ~]# lspci 00:00.0 Host bridge: Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller 00:01.0 VGA compatible controller: Device 1234:1111 (rev 02) 00:02.0 PCI bridge: Red Hat, Inc. QEMU PCIe Root port 00:03.0 PCI bridge: Red Hat, Inc. QEMU PCIe Root port 00:04.0 PCI bridge: Red Hat, Inc. QEMU PCIe Root port 00:05.0 PCI bridge: Red Hat, Inc. QEMU PCIe Root port 00:06.0 PCI bridge: Red Hat, Inc. QEMU PCIe Root port 00:07.0 PCI bridge: Red Hat, Inc. QEMU PCIe Root port 00:08.0 PCI bridge: Red Hat, Inc. QEMU PCIe Root port 00:1f.0 ISA bridge: Intel Corporation 82801IB (ICH9) LPC Interface Controller (rev 02) 00:1f.2 SATA controller: Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode] (rev 02) 00:1f.3 SMBus: Intel Corporation 82801I (ICH9 Family) SMBus Controller (rev 02) 01:00.0 USB controller: Red Hat, Inc. QEMU XHCI Host Controller (rev 01) 02:00.0 SCSI storage controller: Red Hat, Inc. Virtio SCSI (rev 01) 03:00.0 Ethernet controller: Red Hat, Inc. Virtio network device (rev 01) 04:00.0 SCSI storage controller: Red Hat, Inc. Virtio block device (rev 01) 05:00.0 SCSI storage controller: Red Hat, Inc. Virtio block device (rev 01) 4.hot plug new disks #qmp {"execute":"qmp_capabilities"} {"execute": "device_add", "arguments": {"driver": "virtio-blk-pci", "id": "stg3", "drive": "drive_stg3", "bus": "pcie_port_3"}, "id": "t07OBwF3"} {"execute": "device_add", "arguments": {"driver": "virtio-blk-pci", "id": "stg4", "drive": "drive_stg4", "bus": "pcie_port_4"}, "id": "t07OBwF4"} 5.check pci in guest [root@bootp-73-131-155 ~]# lspci 00:00.0 Host bridge: Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller 00:01.0 VGA compatible controller: Device 1234:1111 (rev 02) 00:02.0 PCI bridge: Red Hat, Inc. QEMU PCIe Root port 00:03.0 PCI bridge: Red Hat, Inc. QEMU PCIe Root port 00:04.0 PCI bridge: Red Hat, Inc. QEMU PCIe Root port 00:05.0 PCI bridge: Red Hat, Inc. QEMU PCIe Root port 00:06.0 PCI bridge: Red Hat, Inc. QEMU PCIe Root port 00:07.0 PCI bridge: Red Hat, Inc. QEMU PCIe Root port 00:08.0 PCI bridge: Red Hat, Inc. QEMU PCIe Root port 00:1f.0 ISA bridge: Intel Corporation 82801IB (ICH9) LPC Interface Controller (rev 02) 00:1f.2 SATA controller: Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode] (rev 02) 00:1f.3 SMBus: Intel Corporation 82801I (ICH9 Family) SMBus Controller (rev 02) 01:00.0 USB controller: Red Hat, Inc. QEMU XHCI Host Controller (rev 01) 02:00.0 SCSI storage controller: Red Hat, Inc. Virtio SCSI (rev 01) 03:00.0 Ethernet controller: Red Hat, Inc. Virtio network device (rev 01) 04:00.0 SCSI storage controller: Red Hat, Inc. Virtio block device (rev 01) 05:00.0 SCSI storage controller: Red Hat, Inc. Virtio block device (rev 01) 07:00.0 SCSI storage controller: Red Hat, Inc. Virtio block device (rev 01) 6.check pci in qmp {"execute":"query-pci"} Actual results: Find unexpected info {"bus": 0, "pci_bridge": {"bus": {"prefetchable_range": {"limit": 4232052735, "base": 4229955584}, "memory_range": {"limit": 4261412863, "base": 4259315712}, "secondary": 6, "io_range": {"limit": 4095, "base": 61440}, "number": 0, "subordinate": 6}, "devices": [{"bus": 6, "qdev_id": "stg3", "irq": 0, "slot": 0, "class_info": {"class": 256, "desc": "SCSI controller"}, "id": {"device": 4162, "subsystem-vendor": 6900, "vendor": 6900, "subsystem": 4352}, "function": 0, "regions": [{"prefetch": false, "mem_type_64": false, "bar": 1, "size": 4096, "address": -1, "type": "memory"}, {"prefetch": true, "mem_type_64": true, "bar": 4, "size": 16384, "address": -1, "type": "memory"}]}]}, "qdev_id": "pcie_port_3", "irq": 11, "slot": 7, "class_info": {"class": 1540, "desc": "PCI bridge"}, "id": {"device": 12, "vendor": 6966}, "function": 0, "regions": [{"prefetch": false, "mem_type_64": false, "bar": 0, "size": 4096, "address": 4271988736, "type": "memory"}]}, Expected results: stg3 not added in guest, it should get sync info with guest via query-block. Additional info:
Assigned to Amnon for initial triage per bz process and age of bug created or assigned to virt-maint > 5d
Posted upstream qemu-devel: [PATCH] hw/pci/pcie: Forbid hot-plug via QMP if it's disabled on the slot Now you should get an error when trying to hotplug to slot to avoid confusion. > stg3 not added in guest, it should get sync info with guest via query-block. No it shouldn't, query-pci reflects the status of devices inside QEMU, not inside the guest.
*** Bug 1820532 has been marked as a duplicate of this bug. ***
Applied upstream.
Test with virtio-net device on Win10-32 and win2019 Got the right error info in qmp {"execute":"device_del","arguments":{"id":"idotmiWW"}} {"error": {"class": "GenericError", "desc": "Hot-unplug failed: unsupported by the port device 'pcie.0-root-port-4'"}} cmd: -device pcie-root-port,id=pcie.0-root-port-4,slot=4,chassis=4,addr=0x4,bus=pcie.0,hotplug=off \ -device virtio-net-pci,mac=9a:57:da:4f:cb:b2,id=idotmiWW,mq=on,vectors=14,netdev=idT6Wphl,bus=pcie.0-root-port-4,addr=0x0 \ -netdev tap,id=idT6Wphl,vhost=on,queues=6 qemu-kvm-4.2.0-21.module+el8.2.1+6586+8b7713b9.x86_64 Thanks Yu Wang
(In reply to Yu Wang from comment #9) > Test with virtio-net device on Win10-32 and win2019 > Got the right error info in qmp > > {"execute":"device_del","arguments":{"id":"idotmiWW"}} > {"error": {"class": "GenericError", "desc": "Hot-unplug failed: unsupported > by the port device 'pcie.0-root-port-4'"}} > > cmd: > -device > pcie-root-port,id=pcie.0-root-port-4,slot=4,chassis=4,addr=0x4,bus=pcie.0, > hotplug=off \ > -device > virtio-net-pci,mac=9a:57:da:4f:cb:b2,id=idotmiWW,mq=on,vectors=14, > netdev=idT6Wphl,bus=pcie.0-root-port-4,addr=0x0 \ > -netdev tap,id=idT6Wphl,vhost=on,queues=6 > > qemu-kvm-4.2.0-21.module+el8.2.1+6586+8b7713b9.x86_64 > > Thanks > Yu Wang try to hotplug virtio-net device, the qmp event is: { "execute": "netdev_add", "arguments": { "type":"tap","id":"hostnet0","vhost":"on","script":"/etc/qemu-ifup"}} {"return": {}} {"execute": "device_add", "arguments": { "driver":"virtio-net-pci","netdev":"hostnet0","mac":"00:1a:4a:42:0b:01","id": "net0","bus":"pcie.0-root-port-4"}} {"timestamp": {"seconds": 1589253395, "microseconds": 373115}, "event": "DEVICE_DELETED", "data": {"path": "/machine/peripheral/net0/virtio-backend"}} {"error": {"class": "GenericError", "desc": "Hot-plug failed: unsupported by the port device 'pcie.0-root-port-4'"}} It will prompt a extra event: {"timestamp": {"seconds": 1589253395, "microseconds": 373115}, "event": "DEVICE_DELETED", "data": {"path": "/machine/peripheral/net0/virtio-backend"}} Pls help to check if it is normal, thanks
works with virtio serial and balloon device. qmp: {"execute":"device_del","arguments":{"id":"virtio-serial1"}} {"error": {"class": "GenericError", "desc": "Hot-unplug failed: unsupported by the port device 'pci.6'"}} {"execute":"device_del","arguments":{"id":"balloon0"}} {"error": {"class": "GenericError", "desc": "Hot-unplug failed: unsupported by the port device 'pci.5'"}} guest: no vioser and balloon device can be removed.
Test with virtio-net device on RHEL.8.2_guest. Test Version: qemu-kvm-4.2.0-20.module+el8.2.1+6467+49dc3278.x86_64 hot unplug cmd: -device pcie-root-port,id=pcie-root-port-3,hotplug=off,port=0x3,addr=0x1.0x3,bus=pcie.0,chassis=4 \ -device virtio-net-pci,mac=9a:82:59:0b:01:17,id=idVn2ph4,mq=on,vectors=10,netdev=idb8GVnn,bus=pcie-root-port-3,addr=0x0 \ -netdev tap,id=idb8GVnn,vhost=on \ 1. hot unplug nic {"execute":"device_del", "arguments": {"id": "idVn2ph4"}} {"return": {}} 2.check the guest,nic still exists.This is expect result. hot plug cmd: -device pcie-root-port,id=pcie-root-port-3,hotplug=off,port=0x3,addr=0x1.0x3,bus=pcie.0,chassis=4 \ 1.hot plug nic { "execute": "netdev_add", "arguments": { "type":"tap","id":"idb8GVnn","vhost":"on","script":"/etc/qemu-ifup"}} {"return": {}} {"execute": "device_add", "arguments": { "driver":"virtio-net-pci","netdev":"idb8GVnn","mac":"00:1a:4a:42:0b:01","id": "idVn2ph4","bus":"pcie-root-port-3"}} {"return": {}} 2.Guest didn't get a nic,this is expect result.
Test on: qemu-kvm-4.2.0-21.module+el8.2.1+6586+8b7713b9.x86_64 Guest:4.18.0-193.el8.x86_64 Test steps: 1.Create images (only execute one time) qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg1.qcow2 1G qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg2.qcow2 2G qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg3.qcow2 3G qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg4.qcow2 4G qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg5.qcow2 5G qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg6.qcow2 6G qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg7.qcow2 7G qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg8.qcow2 8G 2.boot vm /usr/libexec/qemu-kvm \ -name 'avocado-vt-vm1' \ -machine q35 \ -nodefaults \ -device VGA,bus=pcie.0,addr=0x1 \ -device pcie-root-port,id=pcie.0-root-port-2,slot=2,chassis=2,addr=0x2,bus=pcie.0 \ -device qemu-xhci,id=usb1,bus=pcie.0-root-port-2,addr=0x0 \ -object iothread,id=iothread0 \ -device pcie-root-port,id=pcie.0-root-port-3,slot=3,chassis=3,addr=0x3,bus=pcie.0 \ -device virtio-scsi-pci,id=virtio_scsi_pci,iothread=iothread0,bus=pcie.0-root-port-3,addr=0x0 \ -drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/rhel820-64-virtio-scsi.qcow2 \ -device scsi-hd,id=image1,drive=drive_image1,bootindex=0 \ -device pcie-root-port,id=pcie.0-root-port-4,slot=4,chassis=4,addr=0x4,bus=pcie.0 \ -device virtio-net-pci,mac=9a:ab:fa:09:62:1c,id=idLVRSKI,netdev=idfMN0PL,bus=pcie.0-root-port-4,addr=0x0 \ -netdev tap,id=idfMN0PL,vhost=on \ -m 8G \ -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \ -vnc :6 \ -qmp tcp:0:5956,server,nowait \ -rtc base=localtime,clock=host,driftfix=slew \ -boot order=cdn,once=c,menu=off,strict=off \ -enable-kvm \ -monitor stdio \ -blockdev node-name=file_stg1,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/stg1.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_stg1,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg1 \ -blockdev node-name=file_stg2,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/stg2.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_stg2,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg2 \ \ -blockdev node-name=file_stg3,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/stg3.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_stg3,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg3 \ -blockdev node-name=file_stg4,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/stg4.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_stg4,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg4 \ \ -blockdev node-name=file_stg5,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/stg5.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_stg5,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg5 \ -blockdev node-name=file_stg6,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/stg6.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_stg6,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg6 \ \ -blockdev node-name=file_stg7,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/stg7.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_stg7,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg7 \ -blockdev node-name=file_stg8,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/stg8.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_stg8,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg8 \ \ -device pcie-root-port,id=pcie_port_1,slot=5,chassis=5,addr=0x5,bus=pcie.0,hotplug=off \ -device virtio-blk-pci,id=stg1,drive=drive_stg1,bus=pcie_port_1,addr=0x0 \ \ -device pcie-root-port,id=pcie_port_2,slot=6,chassis=6,addr=0x6,bus=pcie.0,hotplug=on \ -device virtio-blk-pci,id=stg2,drive=drive_stg2,bus=pcie_port_2,addr=0x0 \ \ -device pcie-root-port,id=pcie_port_3,slot=7,chassis=7,addr=0x7,bus=pcie.0,hotplug=off \ -device pcie-root-port,id=pcie_port_4,slot=8,chassis=8,addr=0x8,bus=pcie.0,hotplug=on \ 3.hotplug disks {'execute':'qmp_capabilities'} {"execute": "device_add", "arguments": {"driver": "virtio-blk-pci", "id": "stg3", "drive": "drive_stg3", "bus": "pcie_port_3"}, "id": "t07OBwF3"} {"execute": "device_add", "arguments": {"driver": "virtio-blk-pci", "id": "stg4", "drive": "drive_stg4", "bus": "pcie_port_4"}, "id": "t07OBwF4"} {"timestamp": {"seconds": 1589272385, "microseconds": 684270}, "event": "DEVICE_DELETED", "data": {"path": "/machine/peripheral/stg3/virtio-backend"}} {"id": "t07OBwF3", "error": {"class": "GenericError", "desc": "Hot-plug failed: unsupported by the port device 'pcie_port_3'"}} 4.check output with query command and guest {"execute":"query-block"} {"execute":"query-pci"} lsblk 5.hotunplg disks {"execute":"device_del","arguments":{"id":"stg1"}} {"execute":"device_del","arguments":{"id":"stg2"}} {"error": {"class": "GenericError", "desc": "Hot-unplug failed: unsupported by the port device 'pcie_port_1'"}} {"return": {}} {"timestamp": {"seconds": 1589272639, "microseconds": 252528}, "event": "DEVICE_DELETED", "data": {"path": "/machine/peripheral/stg2/virtio-backend"}} {"timestamp": {"seconds": 1589272639, "microseconds": 307080}, "event": "DEVICE_DELETED", "data": {"device": "stg2", "path": "/machine/peripheral/stg2"}} 6.check output with query command and guest {"execute":"query-block"} {"execute":"query-pci"} lsblk It looks like the hotplug off works .but there is confuse message on step3 : {"timestamp": {"seconds": 1589272385, "microseconds": 684270}, "event": "DEVICE_DELETED", "data": {"path": "/machine/peripheral/stg3/virtio-backend"}} hotplug operation why generate DEVICE_DELETED event? please confirm it.
(In reply to qing.wang from comment #13) > Test on: > qemu-kvm-4.2.0-21.module+el8.2.1+6586+8b7713b9.x86_64 > Guest:4.18.0-193.el8.x86_64 > > > Test steps: > 1.Create images (only execute one time) > qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg1.qcow2 1G > qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg2.qcow2 2G > qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg3.qcow2 3G > qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg4.qcow2 4G > qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg5.qcow2 5G > qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg6.qcow2 6G > qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg7.qcow2 7G > qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg8.qcow2 8G > > 2.boot vm > > /usr/libexec/qemu-kvm \ > -name 'avocado-vt-vm1' \ > -machine q35 \ > -nodefaults \ > -device VGA,bus=pcie.0,addr=0x1 \ > -device > pcie-root-port,id=pcie.0-root-port-2,slot=2,chassis=2,addr=0x2,bus=pcie.0 \ > -device qemu-xhci,id=usb1,bus=pcie.0-root-port-2,addr=0x0 \ > -object iothread,id=iothread0 \ > -device > pcie-root-port,id=pcie.0-root-port-3,slot=3,chassis=3,addr=0x3,bus=pcie.0 \ > -device > virtio-scsi-pci,id=virtio_scsi_pci,iothread=iothread0,bus=pcie.0-root-port-3, > addr=0x0 \ > -drive > id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=qcow2, > file=/home/kvm_autotest_root/images/rhel820-64-virtio-scsi.qcow2 \ > -device scsi-hd,id=image1,drive=drive_image1,bootindex=0 \ > -device > pcie-root-port,id=pcie.0-root-port-4,slot=4,chassis=4,addr=0x4,bus=pcie.0 \ > -device > virtio-net-pci,mac=9a:ab:fa:09:62:1c,id=idLVRSKI,netdev=idfMN0PL,bus=pcie.0- > root-port-4,addr=0x0 \ > -netdev tap,id=idfMN0PL,vhost=on \ > -m 8G \ > -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \ > -vnc :6 \ > -qmp tcp:0:5956,server,nowait \ > -rtc base=localtime,clock=host,driftfix=slew \ > -boot order=cdn,once=c,menu=off,strict=off \ > -enable-kvm \ > -monitor stdio \ > -blockdev > node-name=file_stg1,driver=file,aio=threads,filename=/home/kvm_autotest_root/ > images/stg1.qcow2,cache.direct=on,cache.no-flush=off \ > -blockdev > node-name=drive_stg1,driver=qcow2,cache.direct=on,cache.no-flush=off, > file=file_stg1 \ > -blockdev > node-name=file_stg2,driver=file,aio=threads,filename=/home/kvm_autotest_root/ > images/stg2.qcow2,cache.direct=on,cache.no-flush=off \ > -blockdev > node-name=drive_stg2,driver=qcow2,cache.direct=on,cache.no-flush=off, > file=file_stg2 \ > \ > -blockdev > node-name=file_stg3,driver=file,aio=threads,filename=/home/kvm_autotest_root/ > images/stg3.qcow2,cache.direct=on,cache.no-flush=off \ > -blockdev > node-name=drive_stg3,driver=qcow2,cache.direct=on,cache.no-flush=off, > file=file_stg3 \ > -blockdev > node-name=file_stg4,driver=file,aio=threads,filename=/home/kvm_autotest_root/ > images/stg4.qcow2,cache.direct=on,cache.no-flush=off \ > -blockdev > node-name=drive_stg4,driver=qcow2,cache.direct=on,cache.no-flush=off, > file=file_stg4 \ > \ > -blockdev > node-name=file_stg5,driver=file,aio=threads,filename=/home/kvm_autotest_root/ > images/stg5.qcow2,cache.direct=on,cache.no-flush=off \ > -blockdev > node-name=drive_stg5,driver=qcow2,cache.direct=on,cache.no-flush=off, > file=file_stg5 \ > -blockdev > node-name=file_stg6,driver=file,aio=threads,filename=/home/kvm_autotest_root/ > images/stg6.qcow2,cache.direct=on,cache.no-flush=off \ > -blockdev > node-name=drive_stg6,driver=qcow2,cache.direct=on,cache.no-flush=off, > file=file_stg6 \ > \ > -blockdev > node-name=file_stg7,driver=file,aio=threads,filename=/home/kvm_autotest_root/ > images/stg7.qcow2,cache.direct=on,cache.no-flush=off \ > -blockdev > node-name=drive_stg7,driver=qcow2,cache.direct=on,cache.no-flush=off, > file=file_stg7 \ > -blockdev > node-name=file_stg8,driver=file,aio=threads,filename=/home/kvm_autotest_root/ > images/stg8.qcow2,cache.direct=on,cache.no-flush=off \ > -blockdev > node-name=drive_stg8,driver=qcow2,cache.direct=on,cache.no-flush=off, > file=file_stg8 \ > \ > -device > pcie-root-port,id=pcie_port_1,slot=5,chassis=5,addr=0x5,bus=pcie.0, > hotplug=off \ > -device virtio-blk-pci,id=stg1,drive=drive_stg1,bus=pcie_port_1,addr=0x0 > \ > \ > -device > pcie-root-port,id=pcie_port_2,slot=6,chassis=6,addr=0x6,bus=pcie.0, > hotplug=on \ > -device virtio-blk-pci,id=stg2,drive=drive_stg2,bus=pcie_port_2,addr=0x0 > \ > \ > -device > pcie-root-port,id=pcie_port_3,slot=7,chassis=7,addr=0x7,bus=pcie.0, > hotplug=off \ > -device > pcie-root-port,id=pcie_port_4,slot=8,chassis=8,addr=0x8,bus=pcie.0, > hotplug=on \ > > > 3.hotplug disks > {'execute':'qmp_capabilities'} > > {"execute": "device_add", "arguments": {"driver": "virtio-blk-pci", "id": > "stg3", "drive": "drive_stg3", "bus": "pcie_port_3"}, "id": "t07OBwF3"} > {"execute": "device_add", "arguments": {"driver": "virtio-blk-pci", "id": > "stg4", "drive": "drive_stg4", "bus": "pcie_port_4"}, "id": "t07OBwF4"} > > {"timestamp": {"seconds": 1589272385, "microseconds": 684270}, "event": > "DEVICE_DELETED", "data": {"path": > "/machine/peripheral/stg3/virtio-backend"}} > {"id": "t07OBwF3", "error": {"class": "GenericError", "desc": "Hot-plug > failed: unsupported by the port device 'pcie_port_3'"}} > > 4.check output with query command and guest > {"execute":"query-block"} > {"execute":"query-pci"} > lsblk > > 5.hotunplg disks > {"execute":"device_del","arguments":{"id":"stg1"}} > {"execute":"device_del","arguments":{"id":"stg2"}} > > {"error": {"class": "GenericError", "desc": "Hot-unplug failed: unsupported > by the port device 'pcie_port_1'"}} > {"return": {}} > {"timestamp": {"seconds": 1589272639, "microseconds": 252528}, "event": > "DEVICE_DELETED", "data": {"path": > "/machine/peripheral/stg2/virtio-backend"}} > {"timestamp": {"seconds": 1589272639, "microseconds": 307080}, "event": > "DEVICE_DELETED", "data": {"device": "stg2", "path": > "/machine/peripheral/stg2"}} > > > 6.check output with query command and guest > {"execute":"query-block"} > {"execute":"query-pci"} > lsblk > > It looks like the hotplug off works .but there is confuse message on step3 : > {"timestamp": {"seconds": 1589272385, "microseconds": 684270}, "event": > "DEVICE_DELETED", "data": {"path": > "/machine/peripheral/stg3/virtio-backend"}} > > hotplug operation why generate DEVICE_DELETED event? please confirm it. Good catch. This seems to happen with all virtio devices if hotplug is failed for any reason. Since it wasn't introduced by this patch, doesn't affect the behaviour and can happen in other situations too, open another bug, please. Best regards, Julia Suvorova.
(In reply to Julia Suvorova from comment #15) > (In reply to qing.wang from comment #13) > > Test on: > > qemu-kvm-4.2.0-21.module+el8.2.1+6586+8b7713b9.x86_64 > > Guest:4.18.0-193.el8.x86_64 > > > > > > Test steps: > > 1.Create images (only execute one time) > > qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg1.qcow2 1G > > qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg2.qcow2 2G > > qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg3.qcow2 3G > > qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg4.qcow2 4G > > qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg5.qcow2 5G > > qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg6.qcow2 6G > > qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg7.qcow2 7G > > qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg8.qcow2 8G > > > > 2.boot vm > > > > /usr/libexec/qemu-kvm \ > > -name 'avocado-vt-vm1' \ > > -machine q35 \ > > -nodefaults \ > > -device VGA,bus=pcie.0,addr=0x1 \ > > -device > > pcie-root-port,id=pcie.0-root-port-2,slot=2,chassis=2,addr=0x2,bus=pcie.0 \ > > -device qemu-xhci,id=usb1,bus=pcie.0-root-port-2,addr=0x0 \ > > -object iothread,id=iothread0 \ > > -device > > pcie-root-port,id=pcie.0-root-port-3,slot=3,chassis=3,addr=0x3,bus=pcie.0 \ > > -device > > virtio-scsi-pci,id=virtio_scsi_pci,iothread=iothread0,bus=pcie.0-root-port-3, > > addr=0x0 \ > > -drive > > id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=qcow2, > > file=/home/kvm_autotest_root/images/rhel820-64-virtio-scsi.qcow2 \ > > -device scsi-hd,id=image1,drive=drive_image1,bootindex=0 \ > > -device > > pcie-root-port,id=pcie.0-root-port-4,slot=4,chassis=4,addr=0x4,bus=pcie.0 \ > > -device > > virtio-net-pci,mac=9a:ab:fa:09:62:1c,id=idLVRSKI,netdev=idfMN0PL,bus=pcie.0- > > root-port-4,addr=0x0 \ > > -netdev tap,id=idfMN0PL,vhost=on \ > > -m 8G \ > > -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \ > > -vnc :6 \ > > -qmp tcp:0:5956,server,nowait \ > > -rtc base=localtime,clock=host,driftfix=slew \ > > -boot order=cdn,once=c,menu=off,strict=off \ > > -enable-kvm \ > > -monitor stdio \ > > -blockdev > > node-name=file_stg1,driver=file,aio=threads,filename=/home/kvm_autotest_root/ > > images/stg1.qcow2,cache.direct=on,cache.no-flush=off \ > > -blockdev > > node-name=drive_stg1,driver=qcow2,cache.direct=on,cache.no-flush=off, > > file=file_stg1 \ > > -blockdev > > node-name=file_stg2,driver=file,aio=threads,filename=/home/kvm_autotest_root/ > > images/stg2.qcow2,cache.direct=on,cache.no-flush=off \ > > -blockdev > > node-name=drive_stg2,driver=qcow2,cache.direct=on,cache.no-flush=off, > > file=file_stg2 \ > > \ > > -blockdev > > node-name=file_stg3,driver=file,aio=threads,filename=/home/kvm_autotest_root/ > > images/stg3.qcow2,cache.direct=on,cache.no-flush=off \ > > -blockdev > > node-name=drive_stg3,driver=qcow2,cache.direct=on,cache.no-flush=off, > > file=file_stg3 \ > > -blockdev > > node-name=file_stg4,driver=file,aio=threads,filename=/home/kvm_autotest_root/ > > images/stg4.qcow2,cache.direct=on,cache.no-flush=off \ > > -blockdev > > node-name=drive_stg4,driver=qcow2,cache.direct=on,cache.no-flush=off, > > file=file_stg4 \ > > \ > > -blockdev > > node-name=file_stg5,driver=file,aio=threads,filename=/home/kvm_autotest_root/ > > images/stg5.qcow2,cache.direct=on,cache.no-flush=off \ > > -blockdev > > node-name=drive_stg5,driver=qcow2,cache.direct=on,cache.no-flush=off, > > file=file_stg5 \ > > -blockdev > > node-name=file_stg6,driver=file,aio=threads,filename=/home/kvm_autotest_root/ > > images/stg6.qcow2,cache.direct=on,cache.no-flush=off \ > > -blockdev > > node-name=drive_stg6,driver=qcow2,cache.direct=on,cache.no-flush=off, > > file=file_stg6 \ > > \ > > -blockdev > > node-name=file_stg7,driver=file,aio=threads,filename=/home/kvm_autotest_root/ > > images/stg7.qcow2,cache.direct=on,cache.no-flush=off \ > > -blockdev > > node-name=drive_stg7,driver=qcow2,cache.direct=on,cache.no-flush=off, > > file=file_stg7 \ > > -blockdev > > node-name=file_stg8,driver=file,aio=threads,filename=/home/kvm_autotest_root/ > > images/stg8.qcow2,cache.direct=on,cache.no-flush=off \ > > -blockdev > > node-name=drive_stg8,driver=qcow2,cache.direct=on,cache.no-flush=off, > > file=file_stg8 \ > > \ > > -device > > pcie-root-port,id=pcie_port_1,slot=5,chassis=5,addr=0x5,bus=pcie.0, > > hotplug=off \ > > -device virtio-blk-pci,id=stg1,drive=drive_stg1,bus=pcie_port_1,addr=0x0 > > \ > > \ > > -device > > pcie-root-port,id=pcie_port_2,slot=6,chassis=6,addr=0x6,bus=pcie.0, > > hotplug=on \ > > -device virtio-blk-pci,id=stg2,drive=drive_stg2,bus=pcie_port_2,addr=0x0 > > \ > > \ > > -device > > pcie-root-port,id=pcie_port_3,slot=7,chassis=7,addr=0x7,bus=pcie.0, > > hotplug=off \ > > -device > > pcie-root-port,id=pcie_port_4,slot=8,chassis=8,addr=0x8,bus=pcie.0, > > hotplug=on \ > > > > > > 3.hotplug disks > > {'execute':'qmp_capabilities'} > > > > {"execute": "device_add", "arguments": {"driver": "virtio-blk-pci", "id": > > "stg3", "drive": "drive_stg3", "bus": "pcie_port_3"}, "id": "t07OBwF3"} > > {"execute": "device_add", "arguments": {"driver": "virtio-blk-pci", "id": > > "stg4", "drive": "drive_stg4", "bus": "pcie_port_4"}, "id": "t07OBwF4"} > > > > {"timestamp": {"seconds": 1589272385, "microseconds": 684270}, "event": > > "DEVICE_DELETED", "data": {"path": > > "/machine/peripheral/stg3/virtio-backend"}} > > {"id": "t07OBwF3", "error": {"class": "GenericError", "desc": "Hot-plug > > failed: unsupported by the port device 'pcie_port_3'"}} > > > > 4.check output with query command and guest > > {"execute":"query-block"} > > {"execute":"query-pci"} > > lsblk > > > > 5.hotunplg disks > > {"execute":"device_del","arguments":{"id":"stg1"}} > > {"execute":"device_del","arguments":{"id":"stg2"}} > > > > {"error": {"class": "GenericError", "desc": "Hot-unplug failed: unsupported > > by the port device 'pcie_port_1'"}} > > {"return": {}} > > {"timestamp": {"seconds": 1589272639, "microseconds": 252528}, "event": > > "DEVICE_DELETED", "data": {"path": > > "/machine/peripheral/stg2/virtio-backend"}} > > {"timestamp": {"seconds": 1589272639, "microseconds": 307080}, "event": > > "DEVICE_DELETED", "data": {"device": "stg2", "path": > > "/machine/peripheral/stg2"}} > > > > > > 6.check output with query command and guest > > {"execute":"query-block"} > > {"execute":"query-pci"} > > lsblk > > > > It looks like the hotplug off works .but there is confuse message on step3 : > > {"timestamp": {"seconds": 1589272385, "microseconds": 684270}, "event": > > "DEVICE_DELETED", "data": {"path": > > "/machine/peripheral/stg3/virtio-backend"}} > > > > hotplug operation why generate DEVICE_DELETED event? please confirm it. > > Good catch. This seems to happen with all virtio devices if hotplug is > failed for any reason. > Since it wasn't introduced by this patch, doesn't affect the behaviour and > can happen in other situations too, open another bug, please. Ok, I found a more serious problem: it deletes drive_stg3 too. Looks like it can happen if we failed to realize virtio-blk or virtio-net even without q35 and pci, we just didn't have control on it previously.
Open new bug to track DEVICE_DELETED issue https://bugzilla.redhat.com/show_bug.cgi?id=1835528
Posted upstream: [PATCH] hw/pci/pcie: Move hot plug capability check to pre_plug callback
*** Bug 1832727 has been marked as a duplicate of this bug. ***
*** Bug 1832749 has been marked as a duplicate of this bug. ***
Retest with qemu-kvm-4.2.0-27.module+el8.2.1+7092+9d345e72, it is normal now. 1 tried to hotplug a virtio-net device with "-device pcie-root-port,id=pcie.0-root-port-4,slot=4,chassis=4,addr=0x4,bus=pcie.0,hotplug=off" {"execute": "qmp_capabilities"} {"return": {}} { "execute": "netdev_add", "arguments": { "type":"tap","id":"hostnet0","vhost":"on","script":"/etc/qemu-ifup"}} {"return": {}} {"execute": "device_add", "arguments": { "driver":"virtio-net-pci","netdev":"hostnet0","mac":"00:1a:4a:42:0b:01","id": "net0","bus":"pcie.0-root-port-4"}} {"error": {"class": "GenericError", "desc": "Hot-plug failed: unsupported by the port device 'pcie.0-root-port-4'"}} There is no "{"timestamp": {"seconds": 1589253395, "microseconds": 373115}, "event": "DEVICE_DELETED", "data": {"path": "/machine/peripheral/net0/virtio-backend"}}" event now 2 Got the right error info in qmp when hot-unplug a virtio-net device when hotplug=off {"execute":"device_del","arguments":{"id":"idotmiWW"}} {"error": {"class": "GenericError", "desc": "Hot-unplug failed: unsupported by the port device 'pcie.0-root-port-4'"}} cmd: -device pcie-root-port,id=pcie.0-root-port-4,slot=4,chassis=4,addr=0x4,bus=pcie.0,hotplug=off \ -device virtio-net-pci,mac=9a:57:da:4f:cb:b2,id=idotmiWW,mq=on,vectors=14,netdev=idT6Wphl,bus=pcie.0-root-port-4,addr=0x0 \ -netdev tap,id=idT6Wphl,vhost=on,queues=6 So maybe device delete event problem of bz1835528 is also fixed. Thanks Yu Wang
Test on: Host: qemu-kvm-common-4.2.0-27.module+el8.2.1+7092+9d345e72.x86_64 4.18.0-193.10.1.el8_2.x86_64 Guest:4.18.0-193.el8.x86_64 Test steps: 1.Create images (only execute one time) qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg1.qcow2 1G qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg2.qcow2 2G qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg3.qcow2 3G qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg4.qcow2 4G qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg5.qcow2 5G qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg6.qcow2 6G qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg7.qcow2 7G qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg8.qcow2 8G 2.boot vm /usr/libexec/qemu-kvm \ -name 'avocado-vt-vm1' \ -machine q35 \ -nodefaults \ -device VGA,bus=pcie.0,addr=0x1 \ -device pcie-root-port,id=pcie.0-root-port-2,slot=2,chassis=2,addr=0x2,bus=pcie.0 \ -device qemu-xhci,id=usb1,bus=pcie.0-root-port-2,addr=0x0 \ -object iothread,id=iothread0 \ -device pcie-root-port,id=pcie.0-root-port-3,slot=3,chassis=3,addr=0x3,bus=pcie.0 \ -device virtio-scsi-pci,id=virtio_scsi_pci,iothread=iothread0,bus=pcie.0-root-port-3,addr=0x0 \ -drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/rhel820-64-virtio-scsi.qcow2 \ -device scsi-hd,id=image1,drive=drive_image1,bootindex=0 \ -device pcie-root-port,id=pcie.0-root-port-4,slot=4,chassis=4,addr=0x4,bus=pcie.0 \ -device virtio-net-pci,mac=9a:ab:fa:09:62:1c,id=idLVRSKI,netdev=idfMN0PL,bus=pcie.0-root-port-4,addr=0x0 \ -netdev tap,id=idfMN0PL,vhost=on \ -m 8G \ -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \ -vnc :6 \ -qmp tcp:0:5956,server,nowait \ -rtc base=localtime,clock=host,driftfix=slew \ -boot order=cdn,once=c,menu=off,strict=off \ -enable-kvm \ -monitor stdio \ -blockdev node-name=file_stg1,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/stg1.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_stg1,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg1 \ -blockdev node-name=file_stg2,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/stg2.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_stg2,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg2 \ \ -blockdev node-name=file_stg3,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/stg3.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_stg3,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg3 \ -blockdev node-name=file_stg4,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/stg4.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_stg4,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg4 \ \ -blockdev node-name=file_stg5,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/stg5.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_stg5,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg5 \ -blockdev node-name=file_stg6,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/stg6.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_stg6,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg6 \ \ -blockdev node-name=file_stg7,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/stg7.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_stg7,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg7 \ -blockdev node-name=file_stg8,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/stg8.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_stg8,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg8 \ \ -device pcie-root-port,id=pcie_port_1,slot=5,chassis=5,addr=0x5,bus=pcie.0,hotplug=off \ -device virtio-blk-pci,id=stg1,drive=drive_stg1,bus=pcie_port_1,addr=0x0 \ \ -device pcie-root-port,id=pcie_port_2,slot=6,chassis=6,addr=0x6,bus=pcie.0,hotplug=on \ -device virtio-blk-pci,id=stg2,drive=drive_stg2,bus=pcie_port_2,addr=0x0 \ \ -device pcie-root-port,id=pcie_port_3,slot=7,chassis=7,addr=0x7,bus=pcie.0,hotplug=off \ -device pcie-root-port,id=pcie_port_4,slot=8,chassis=8,addr=0x8,bus=pcie.0,hotplug=on \ 3.hotplug disks {'execute':'qmp_capabilities'} {"execute": "device_add", "arguments": {"driver": "virtio-blk-pci", "id": "stg3", "drive": "drive_stg3", "bus": "pcie_port_3"}, "id": "t07OBwF3"} {"execute": "device_add", "arguments": {"driver": "virtio-blk-pci", "id": "stg4", "drive": "drive_stg4", "bus": "pcie_port_4"}, "id": "t07OBwF4"} {"id": "t07OBwF3", "error": {"class": "GenericError", "desc": "Hot-plug failed: unsupported by the port device 'pcie_port_3'"}} 4.check output with query command and guest {"execute":"query-block"} {"execute":"query-pci"} lsblk 5.hotunplg disks {"execute":"device_del","arguments":{"id":"stg1"}} {"execute":"device_del","arguments":{"id":"stg2"}} {"error": {"class": "GenericError", "desc": "Hot-unplug failed: unsupported by the port device 'pcie_port_1'"}} {"timestamp": {"seconds": 1593418237, "microseconds": 633700}, "event": "DEVICE_DELETED", "data": {"path": "/machine/peripheral/stg2/virtio-backend"}} {"timestamp": {"seconds": 1593418237, "microseconds": 922377}, "event": "DEVICE_DELETED", "data": {"device": "stg2", "path": "/machine/peripheral/stg2"}} 6.check output with query command and guest {"execute":"query-block"} stg2 not in output {"execute":"query-pci"} lsblk No issue found. It also passed test on guest win2019
Test PASS with virtio-rng device. Host: qemu-kvm-common-4.2.0-27.module+el8.2.1+7092+9d345e72.x86_64 4.18.0-193.10.1.el8_2.x86_64 Guest:4.18.0-193.6.3.el8.x86_64 Test steps: 1. Boot VM with: /usr/libexec/qemu-kvm \ ... -device pcie-root-port,id=pcie_port_1,slot=5,chassis=5,addr=0x5,bus=pcie.0,hotplug=off \ -device virtio-rng-pci,rng=obj0,id=rng0,bus=pcie_port_1,addr=0x0 \ -device pcie-root-port,id=pcie_port_2,slot=6,chassis=6,addr=0x6,bus=pcie.0,hotplug=on \ -device virtio-rng-pci,rng=obj1,id=rng1,bus=pcie_port_2,addr=0x0 \ -device pcie-root-port,id=pcie_port_3,slot=7,chassis=7,addr=0x7,bus=pcie.0,hotplug=off \ -device pcie-root-port,id=pcie_port_4,slot=8,chassis=8,addr=0x8,bus=pcie.0,hotplug=on \ ... 2. Check in the qmp and guest: {"execute":"query-pci"} # lspci 3. Hotplug virtio-rng-pci devices: {"execute": "device_add", "arguments": {"driver": "virtio-rng-pci", "id": "rng2","bus": "pcie_port_3"}} {"error": {"class": "GenericError", "desc": "Hot-plug failed: unsupported by the port device 'pcie_port_3'"}} {"execute": "device_add", "arguments": {"driver": "virtio-rng-pci", "id": "rng3","bus": "pcie_port_4"}} {"return": {}} 4. Check in the qmp and guest: {"execute":"query-pci"} # lspci 5. Hotunplug virtio-rng-pci devices: {"execute":"device_del","arguments":{"id":"rng0"}} {"error": {"class": "GenericError", "desc": "Hot-unplug failed: unsupported by the port device 'pcie_port_1'"}} {"execute":"device_del","arguments":{"id":"rng1"}} {"return": {}} 6. Check in the qmp and guest: {"execute":"query-pci"} # lspci
Test pass against virtio-keyboard, virtio-tablet, virtio-mouse packages used: 4.18.0-193.9.1.el8_2.x86_64(rhel vm) qemu-kvm-4.2.0-28.module+el8.2.1+7211+16dfe810.x86_64 Test against rhel 8.2 VM with qemu cli: ... -device pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2 \ -device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 \ -device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 \ -device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 \ -device pcie-root-port,port=0x14,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x4 \ -device pcie-root-port,port=0x15,chassis=6,id=pci.6,hotplug=off,bus=pcie.0,addr=0x2.0x5 \ -device pcie-root-port,port=0x16,chassis=7,id=pci.7,bus=pcie.0,addr=0x2.0x6 \ ... -device virtio-keyboard-pci,id=input3,bus=pci.6,addr=0x0 \ It will prompt hotplug/hotunplug failed when try to hotplug/hotunplug virtio keyboard, inside VM still able to see virtio-keyboard. Switch to virtio-tablet and virtio-mouse get same behaviors Change to hotplug=on, virtio keyboard/tablet/mouse can hotplug/unplug normally. I get same test results against win10 2004 x64 version. For virtio-vga, virtio-gpu-pci, still, whatever hotplug=on/off, these devices will always prompt devices are unhotplugable/unhotunplugable.
Verified with vsock device: PASS Host: kernel: 4.18.0-193.10.1.el8_2.x86_64 qemu-kvm: qemu-kvm-4.2.0-27.module+el8.2.1+7092+9d345e72.x86_64 Guest: rhel 8.3 qemu-kvm cmd for guest: -device pcie-root-port,port=0x15,chassis=6,id=pci.6,bus=pcie.0,addr=0x6,chassis=5,hotplug=off \ -device vhost-vsock-pci,id=vhost-vsock-pci0,bus=pci.6,guest-cid=3 \ Do hotunplug: {'execute':'qmp_capabilities'} {"return": {}} { "execute": "device_del", "arguments": { "id":"vhost-vsock-pci0"}} {"error": {"class": "GenericError", "desc": "Hot-unplug failed: unsupported by the port device 'pci.6'"}} qemu-kvm cmd for guest: -device pcie-root-port,port=0x15,chassis=6,id=pci.6,bus=pcie.0,addr=0x6,chassis=5,hotplug=off \ Do hotplug: {'execute':'qmp_capabilities'} {"return": {}} { "execute": "device_add", "arguments": { "driver": "vhost-vsock-pci","id":"vhost-vsock-pci0","bus":"pci.6","guest-cid":"3"}} {"error": {"class": "GenericError", "desc": "Hot-plug failed: unsupported by the port device 'pci.6'"}}
Test with vfio-vf/vfio-pf device: PASS Test env: host: 4.18.0-193.10.1.el8_2.x86_64 qemu-kvm-4.2.0-28.module+el8.2.1+7211+16dfe810.x86_64 guest: RHEL82+Win2019 Qemu cmd line: ... -device pcie-root-port,id=root.3,chassis=3,addr=0x2.2,hotplug=off \ -device pcie-root-port,id=root.4,chassis=4,addr=0x2.3,hotplug=off \ ... -device vfio-pci,host=0000:04:00.0,id=pf1,bus=root.3 \ or -device vfio-pci,host=0000:04:02.0,id=vf1,bus=root.3 \ (1)vfio-pf hotunplug_test: {"execute":"device_del","arguments":{"id":"pf1"}} {"error": {"class": "GenericError", "desc": "Hot-unplug failed: unsupported by the port device 'root.3'"}} hotplug_test: {"execute":"device_add","arguments":{"driver":"vfio-pci","host":"04:00.1","id":"pf2","bus":"root.4"}} {"error": {"class": "GenericError", "desc": "Hot-plug failed: unsupported by the port device 'root.4'"}} (2)vfio-vf hotunplug_test: {"execute":"device_del","arguments":{"id":"vf1"}} {"error": {"class": "GenericError", "desc": "Hot-unplug failed: unsupported by the port device 'root.3'"}} hotplug_test: {"execute":"device_add","arguments":{"driver":"vfio-pci","host":"04:02.1","id":"vf2","bus":"root.4"}} {"error": {"class": "GenericError", "desc": "Hot-plug failed: unsupported by the port device 'root.4'"}} Modify "hotplug=off" to "hotplug=on", vfio-pf and vfio-vf device can be hot-pluged and hot-unpluged normally.
Test pass against virtio-serial and balloon device. packages used: kernel-4.18.0-193.el8.x86_64 qemu-kvm-4.2.0-28.module+el8.2.1+7211+16dfe810.x86_64 vm: win2019 CLI: -device pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x3 \ -device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x3.0x1 \ -device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x3.0x2 \ -device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x3.0x3 \ -device pcie-root-port,port=0x14,chassis=5,id=pci.5,bus=pcie.0,addr=0x3.0x4,hotplug=off \ -device pcie-root-port,port=0x15,chassis=6,id=pci.6,bus=pcie.0,addr=0x3.0x5,hotplug=off \ -device virtio-serial-pci,id=virtio-serial1,max_ports=31,bus=pci.6 \ -chardev socket,id=channel2,path=/tmp/helloworld2,server,nowait -device virtserialport,bus=virtio-serial1.0,chardev=channel2,name=org.qemu.guest_agent.0 \ -device virtio-balloon-pci,id=balloon0,bus=pci.5 \ 1) boot up guest without balloon/serial device, and try to hotplug them to pci.5 and pci.6 via qmp cmd, it prompt reasonable error info. {'execute':'device_add','arguments':{'driver':'virtio-balloon-pci','id':'balloon1','bus':'pci.5'}} {"error": {"class": "GenericError", "desc": "Hot-plug failed: unsupported by the port device 'pci.5'"}} {'execute':'device_add','arguments':{'driver':'virtio-serial-pci','id':'serial1','bus':'pci.6'}} {"error": {"class": "GenericError", "desc": "Hot-plug failed: unsupported by the port device 'pci.6'"}} 2) boot up guest with balloon/serial device, and try to hotunplug them via qmp cmd, prompt reasonable error info. Also can't see them in eject device list in guest. {"execute":"device_del","arguments":{"id":"balloon0"}} {"error": {"class": "GenericError", "desc": "Hot-unplug failed: unsupported by the port device 'pci.5'"}} {"execute":"device_del","arguments":{"id":"virtio-serial1"}} {"error": {"class": "GenericError", "desc": "Hot-unplug failed: unsupported by the port device 'pci.6'"}}
Test pass against virtio-serial device. Test environments: Host: kernel-4.18.0-193.10.1.el8_2.x86_64 qemu-kvm-4.2.0-27.module+el8.2.1+7092+9d345e72.x86_64 Guest: RHEL.8.2.1, Win10 cmd: -chardev socket,id=chardev_vs1,nowait,server,path=/tmp/vs1 \ -device pcie-root-port,id=pcie-root-port-1,port=0x1,addr=0x1.0x1,bus=pcie.0,chassis=2,hotplug=on \ -device virtio-serial-pci,id=virtio_serial_pci0,bus=pcie-root-port-1,addr=0x0 \ -device virtserialport,id=vs1,name=vs1,chardev=chardev_vs1,bus=virtio_serial_pci0.0,nr=1 \ -chardev socket,id=chardev_vs2,nowait,server,path=/tmp/vs2 \ -chardev socket,id=chardev_vs3,nowait,server,path=/tmp/vs3 \ -device pcie-root-port,id=extra-pcie-root-port,multifunction=on,bus=pcie.0,addr=0x6,chassis=6 \ -device pcie-root-port,id=pcie-root-port-5,port=0x1,addr=0x6.0x1,bus=pcie.0,chassis=7,hotplug=off \ -device virtio-serial-pci,id=virtio_serial_pci1,bus=pcie-root-port-5,addr=0x0 \ -device virtserialport,id=vs2,name=vs2,chardev=chardev_vs2,bus=virtio_serial_pci1.0,nr=1 \ -device pcie-root-port,id=pcie-root-port-6,port=0x1,addr=0x6.0x2,bus=pcie.0,chassis=8 \ -device virtio-serial-pci,id=virtio_serial_pci2,bus=pcie-root-port-6,addr=0x0 \ -device virtserialport,id=vs3,name=vs3,chardev=chardev_vs3,bus=virtio_serial_pci2.0,nr=1 \ -device pcie-root-port,id=pcie-root-port-7,port=0x1,addr=0x6.0x3,bus=pcie.0,chassis=9,hotplug=off \ results: 1. can hotplug and hot-unplug virtserialport normally both three situations. {"execute":"device_del","arguments":{"id":"vs1"}} {"timestamp": {"seconds": 1593489256, "microseconds": 821657}, "event": "DEVICE_DELETED", "data": {"device": "vs1", "path": "/machine/peripheral/vs1"}} {"return": {}} {"execute":"device_del","arguments":{"id":"vs2"}} {"timestamp": {"seconds": 1593489314, "microseconds": 628424}, "event": "DEVICE_DELETED", "data": {"device": "vs2", "path": "/machine/peripheral/vs2"}} {"return": {}} {"execute":"device_del","arguments":{"id":"vs3"}} {"timestamp": {"seconds": 1593489361, "microseconds": 680021}, "event": "DEVICE_DELETED", "data": {"device": "vs3", "path": "/machine/peripheral/vs3"}} {"return": {}} {"execute":"device_add","arguments":{"driver":"virtserialport","bus":"virtio_serial_pci0.0","chardev":"chardev_vs1","name":"vs1","id":"vs1","nr":"1"}} {"return": {}} {"execute":"device_add","arguments":{"driver":"virtserialport","bus":"virtio_serial_pci1.0","chardev":"chardev_vs2","name":"vs2","id":"vs2","nr":"1"}} {"return": {}} {"execute":"device_add","arguments":{"driver":"virtserialport","bus":"virtio_serial_pci2.0","chardev":"chardev_vs3","name":"vs3","id":"vs3","nr":"1"}} {"return": {}} 2. can hotplug and hot-unplug virtio-serial device normally both "hotplug=on" and situation without parameter "hotplug". {"execute":"device_del","arguments":{"id":"virtio_serial_pci0"}} {"return": {}} {"timestamp": {"seconds": 1593489419, "microseconds": 678164}, "event": "DEVICE_DELETED", "data": {"path": "/machine/peripheral/virtio_serial_pci0/virtio-backend"}} {"timestamp": {"seconds": 1593489419, "microseconds": 731795}, "event": "DEVICE_DELETED", "data": {"device": "virtio_serial_pci0", "path": "/machine/peripheral/virtio_serial_pci0"}} {"execute":"device_del","arguments":{"id":"virtio_serial_pci2"}} {"return": {}} {"timestamp": {"seconds": 1593489516, "microseconds": 442311}, "event": "DEVICE_DELETED", "data": {"path": "/machine/peripheral/virtio_serial_pci2/virtio-backend"}} {"timestamp": {"seconds": 1593489516, "microseconds": 499252}, "event": "DEVICE_DELETED", "data": {"device": "virtio_serial_pci2", "path": "/machine/peripheral/virtio_serial_pci2"}} {"execute":"device_add", "arguments":{"driver":"virtio-serial-pci", "id":"virtio_serial_pci0","bus":"pcie-root-port-1","addr":"0x0"}} {"return": {}} "execute":"device_add", "arguments":{"driver":"virtio-serial-pci", "id":"virtio_serial_pci2","bus":"pcie-root-port-6","addr":"0x0"}} {"return": {}} 2. can't hotplug and hot-unplug virtio-serial device with "hotplug=off". {"execute":"device_del","arguments":{"id":"virtio_serial_pci1"}} {"error": {"class": "GenericError", "desc": "Hot-unplug failed: unsupported by the port device 'pcie-root-port-5'"}} {"execute":"device_add", "arguments":{"driver":"virtio-serial-pci", "id":"virtio_serial_pci3","bus":"pcie-root-port-7","addr":"0x0"}} {"error": {"class": "GenericError", "desc": "Hot-plug failed: unsupported by the port device 'pcie-root-port-7'"}}
Test pass against virtio-serial device according to the steps of Comment 31. Test environments: Host: kernel-4.18.0-193.10.1.el8_2.x86_64 qemu-kvm-4.2.0-28.module+el8.2.1+7211+16dfe810.x86_64 Guest: RHEL.8.2.1, Win10
Virtio balloon device test with rhel8.2 guest passed. qemu-kvm-4.2.0-28.module+el8.2.1+7211+16dfe810 libvirt-client-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64 Hotplug test: QEMU cli: -device pcie-root-port,id=pcie-root-port-4,port=0x4,addr=0x1.0x4,bus=pcie.0,chassis=5,hotplug=off \ -device virtio-balloon-pci,id=balloon0,bus=pcie-root-port-4,addr=0x0 {"execute":"device_del","arguments":{"id":"balloon0"}} {"error": {"class": "GenericError", "desc": "Hot-unplug failed: unsupported by the port device 'pcie-root-port-4'"}} Hot-unlug test: QEMU cli: -device pcie-root-port,id=pcie-root-port-4,port=0x4,addr=0x1.0x4,bus=pcie.0,chassis=5,hotplug=off {'execute':'device_add','arguments':{'driver':'virtio-balloon-pci','id':'balloon1','bus':'pcie-root-port-4'}} {"error": {"class": "GenericError", "desc": "Hot-plug failed: unsupported by the port device 'pcie-root-port-4'"}}
Test virtio-net-pci/e1000e device with rhel8.2 guest. Test result: passed Test Version: qemu-kvm-4.2.0-27.module+el8.2.1+7092+9d345e72.x86_64 kernel-4.18.0-193.11.1.el8_2.x86_64 Hot unplug test: virtio-net-pci: -device pcie-root-port,hotplug=off,id=pcie-root-port-3,port=0x3,addr=0x1.0x3,bus=pcie.0,chassis=4 \ -device virtio-net-pci,mac=9a:a1:ec:15:ae:1f,id=idGzgaba,netdev=idSbZeUj,bus=pcie-root-port-3,addr=0x0 \ -netdev tap,id=idSbZeUj,vhost=on \ {"execute":"device_del", "arguments": {"id": "idGzgaba"}} {"error": {"class": "GenericError", "desc": "Hot-unplug failed: unsupported by the port device 'pcie-root-port-3'"}} e1000e: -device pcie-root-port,hotplug=off,id=pcie-root-port-3,port=0x3,addr=0x1.0x3,bus=pcie.0,chassis=4 \ -device e1000e,mac=9a:a1:ec:15:ae:1f,id=idGzgaba,netdev=idSbZeUj,bus=pcie-root-port-3,addr=0x0 \ -netdev tap,id=idSbZeUj \ {"execute":"device_del", "arguments": {"id": "idGzgaba"}} {"error": {"class": "GenericError", "desc": "Hot-unplug failed: unsupported by the port device 'pcie-root-port-3'"}} Hot plug test： virtio-net-pci: -device pcie-root-port,hotplug=off,id=pcie-root-port-3,port=0x3,addr=0x1.0x3,bus=pcie.0,chassis=4 \ { "execute": "netdev_add", "arguments": { "type":"tap","id":"idb8GVnn","vhost":"on","script":"/etc/qemu-ifup"}} {"return": {}} {"execute": "device_add", "arguments": {"driver":"virtio-net-pci","netdev":"idb8GVnn","mac":"00:1a:4a:42:0b:01","id":"idVn2ph4","bus":"pcie-root-port-3"}} {"error": {"class": "GenericError", "desc": "Hot-plug failed: unsupported by the port device 'pcie-root-port-3'"} e1000e: -device pcie-root-port,hotplug=off,id=pcie-root-port-3,port=0x3,addr=0x1.0x3,bus=pcie.0,chassis=4 \ { "execute": "netdev_add", "arguments": { "type":"tap","id":"idb8GVnn","script":"/etc/qemu-ifup"}} {"return": {}} {"execute": "device_add", "arguments": {"driver":"e1000e","netdev":"idb8GVnn","mac":"00:1a:4a:42:0b:01","id":"idVn2ph4","bus":"pcie-root-port-3"}} {"error": {"class": "GenericError", "desc": "Hot-plug failed: unsupported by the port device 'pcie-root-port-3'"}}
Test pass on virtio-scsi, virtio-blk, virtio-input and virtio-rng device. build info: kernel-4.18.0-193.el8.x86_64 qemu-kvm-4.2.0-28.module+el8.2.1+7211+16dfe810.x86_64 guest: win2019 QEMU cli: -device pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x3 \ -device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x3.0x1 \ -device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x3.0x2 \ -device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x3.0x3,hotplug=off \ -device pcie-root-port,port=0x14,chassis=5,id=pci.5,bus=pcie.0,addr=0x3.0x4,hotplug=off \ -device pcie-root-port,port=0x15,chassis=6,id=pci.6,bus=pcie.0,addr=0x3.0x5,hotplug=off \ -device pcie-root-port,port=0x16,chassis=7,id=pci.7,bus=pcie.0,addr=0x3.0x6,hotplug=off \ -device pcie-root-port,port=0x17,chassis=8,id=pci.8,bus=pcie.0,addr=0x3.0x7 \ (1) boot up guest without blk/scsi/rng/input device, and hotplug them to pci.4 ,pci.5, pci.6,pci.7 via qmp cmd. return normal error message. {"execute": "device_add", "arguments": {"driver": "virtio-rng-pci","rng":"rng0","id":"rng0","bus":"pci.4"}} {"error": {"class": "GenericError", "desc": "Hot-plug failed: unsupported by the port device 'pci.4'"}} {"execute":"device_add","arguments":{"driver":"virtio-keyboard-pci","id":"kbd0","serial":"virtio-keyboard","bus":"pci.5"}} {"error": {"class": "GenericError", "desc": "Hot-plug failed: unsupported by the port device 'pci.5'"}} {"execute":"device_add","arguments":{"driver":"virtio-scsi-pci","id":"virtio_scsi_pci2","bus":"pci.6"}} {"error": {"class": "GenericError", "desc": "Hot-plug failed: unsupported by the port device 'pci.6'"}} {"execute":"device_add","arguments":{"driver":"virtio-blk-pci","drive":"disk2","id":"blk1","bus":"pci.7","addr":"0x0"}} {"error": {"class": "GenericError", "desc": "Hot-plug failed: unsupported by the port device 'pci.7'"}} (2) boot up guest with blk/scsi/rng/input device, and try to hotunplug them via qmp cmd, return noraml error message. and not see them in eject device list in guest. {"execute":"device_del","arguments":{"id":"rng0"}} {"error": {"class": "GenericError", "desc": "Hot-unplug failed: unsupported by the port device 'pci.4'"}} {"execute":"device_del","arguments":{"id":"kbd0"}} {"error": {"class": "GenericError", "desc": "Hot-unplug failed: unsupported by the port device 'pci.5'"}} {"execute":"device_del","arguments":{"id":"virtio_scsi_pci0"}} {"error": {"class": "GenericError", "desc": "Hot-unplug failed: unsupported by the port device 'pci.6'"}} {"execute":"device_del","arguments":{"id":"stg1"}} {"error": {"class": "GenericError", "desc": "Hot-unplug failed: unsupported by the port device 'pci.7'"}}
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:3172