Bug 2208473

Summary: [qemu-kvm] no response with QMP command device_add when repeatedly hotplug/unplug scsi disks
Product: Red Hat Enterprise Linux 9 Reporter: qing.wang <qinwang>
Component: qemu-kvmAssignee: Stefano Garzarella <sgarzare>
qemu-kvm sub component: virtio-blk,scsi QA Contact: qing.wang <qinwang>
Status: CLOSED CURRENTRELEASE Docs Contact:
Severity: high    
Priority: unspecified CC: aliang, coli, jinzhao, juzhang, kwolf, lijin, qizhu, vgoyal, virt-maint, xuwei, zhenyzha
Version: 9.3Keywords: CustomerScenariosInitiative, Regression, TestOnly, Triaged
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-07-31 07:52:42 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2176702    
Bug Blocks:    

Description qing.wang 2023-05-19 08:23:23 UTC 
Description of problem:

Repeatedly hotplug/unplug disks, sometimes 
there is no response with QMP command device_add.



Version-Release number of selected component (if applicable):
Red Hat Enterprise Linux release 9.3 Beta (Plow)
5.14.0-312.el9.x86_64
qemu-kvm-8.0.0-2.el9.x86_64
seabios-bin-1.16.1-1.el9.noarch
edk2-ovmf-20230301gitf80f052277c8-3.el9.noarch
libvirt-9.0.0-10.1.el9_2.x86_64


How reproducible:
>50%

Steps to Reproduce:
1.Create image files （40+）
/usr/bin/qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg0.qcow2 256M
...
/usr/bin/qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg40.qcow2 256M

2.Boot disk with multi disks
/usr/libexec/qemu-kvm \
     -S  \
     -name 'avocado-vt-vm1'  \
     -sandbox on  \
     -blockdev '{"node-name": "file_ovmf_code", "driver": "file", "filename": "/usr/share/OVMF/OVMF_CODE.secboot.fd", "auto-read-only": true, "discard": "unmap"}' \
     -blockdev '{"node-name": "drive_ovmf_code", "driver": "raw", "read-only": true, "file": "file_ovmf_code"}' \
     -blockdev '{"node-name": "file_ovmf_vars", "driver": "file", "filename": "/home/kar/workspace/root/avocado/data/avocado-vt/avocado-vt-vm1_rhel930-64-virtio-scsi-ovmf_qcow2_filesystem_VARS.fd", "auto-read-only": true, "discard": "unmap"}' \
     -blockdev '{"node-name": "drive_ovmf_vars", "driver": "raw", "read-only": false, "file": "file_ovmf_vars"}' \
     -machine q35,memory-backend=mem-machine_mem,pflash0=drive_ovmf_code,pflash1=drive_ovmf_vars \
     -device '{"id": "pcie-root-port-0", "driver": "pcie-root-port", "multifunction": true, "bus": "pcie.0", "addr": "0x1", "chassis": 1}' \
     -device '{"id": "pcie-pci-bridge-0", "driver": "pcie-pci-bridge", "addr": "0x0", "bus": "pcie-root-port-0"}'  \
     -nodefaults \
     -device '{"driver": "VGA", "bus": "pcie-pci-bridge-0", "addr": "0x1"}' \
     -m 12288 \
     -object '{"size": 12884901888, "id": "mem-machine_mem", "qom-type": "memory-backend-ram"}'  \
     -smp 10,maxcpus=10,cores=5,threads=1,dies=1,sockets=2  \
     -cpu 'Cascadelake-Server',ss=on,vmx=on,pdcm=on,hypervisor=on,tsc-adjust=on,umip=on,pku=on,md-clear=on,stibp=on,arch-capabilities=on,xsaves=on,ibpb=on,ibrs=on,amd-stibp=on,amd-ssbd=on,rdctl-no=on,ibrs-all=on,skip-l1dfl-vmentry=on,mds-no=on,pschange-mc-no=on,tsx-ctrl=on,hle=off,rtm=off,kvm_pv_unhalt=on \
     -chardev socket,wait=off,id=qmp_id_qmpmonitor1,path=/var/tmp/avocado_s7tr70rc/monitor-qmpmonitor1-20230517-040257-bB2Qrtk8,server=on  \
     -mon chardev=qmp_id_qmpmonitor1,mode=control \
     -chardev socket,wait=off,id=qmp_id_catch_monitor,path=/var/tmp/avocado_s7tr70rc/monitor-catch_monitor-20230517-040257-bB2Qrtk8,server=on  \
     -mon chardev=qmp_id_catch_monitor,mode=control \
     -device '{"ioport": 1285, "driver": "pvpanic", "id": "idSL7gp8"}' \
     -chardev socket,wait=off,id=chardev_serial0,path=/var/tmp/avocado_s7tr70rc/serial-serial0-20230517-040257-bB2Qrtk8,server=on \
     -device '{"id": "serial0", "driver": "isa-serial", "chardev": "chardev_serial0"}'  \
     -chardev socket,id=seabioslog_id_20230517-040257-bB2Qrtk8,path=/var/tmp/avocado_s7tr70rc/seabios-20230517-040257-bB2Qrtk8,server=on,wait=off \
     -device isa-debugcon,chardev=seabioslog_id_20230517-040257-bB2Qrtk8,iobase=0x402 \
     -device '{"id": "pcie-root-port-1", "port": 1, "driver": "pcie-root-port", "addr": "0x1.0x1", "bus": "pcie.0", "chassis": 2}' \
     -device '{"driver": "qemu-xhci", "id": "usb1", "bus": "pcie-root-port-1", "addr": "0x0"}' \
     -device '{"driver": "usb-tablet", "id": "usb-tablet1", "bus": "usb1.0", "port": "1"}' \
     -object '{"qom-type": "iothread", "id": "iothread0"}' \
     -object '{"qom-type": "iothread", "id": "iothread1"}' \
     -device '{"id": "pcie-root-port-2", "port": 2, "driver": "pcie-root-port", "addr": "0x1.0x2", "bus": "pcie.0", "chassis": 3}' \
     -device '{"id": "virtio_scsi_pci0", "driver": "virtio-scsi-pci", "bus": "pcie-root-port-2", "addr": "0x0", "iothread": "iothread0"}' \
     -blockdev '{"node-name": "file_image1", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/rhel930-64-virtio-scsi-ovmf.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_image1", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_image1"}' \
     -device '{"driver": "scsi-hd", "id": "image1", "drive": "drive_image1", "write-cache": "on"}' \
     -device '{"id": "pcie-root-port-3", "port": 3, "driver": "pcie-root-port", "addr": "0x1.0x3", "bus": "pcie.0", "chassis": 4}' \
     -device '{"id": "virtio_scsi_pci1", "driver": "virtio-scsi-pci", "bus": "pcie-root-port-3", "addr": "0x0", "iothread": "iothread1"}' \
     -blockdev '{"node-name": "file_stg0", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg0.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg0", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg0"}' \
     -device '{"driver": "scsi-hd", "id": "stg0", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg0", "write-cache": "on", "lun": 1}' \
     -blockdev '{"node-name": "file_stg1", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg1.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg1", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg1"}' \
     -device '{"driver": "scsi-hd", "id": "stg1", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg1", "write-cache": "on", "lun": 2}' \
     -blockdev '{"node-name": "file_stg2", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg2.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg2", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg2"}' \
     -device '{"driver": "scsi-hd", "id": "stg2", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg2", "write-cache": "on", "lun": 3}' \
     -blockdev '{"node-name": "file_stg3", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg3.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg3", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg3"}' \
     -device '{"driver": "scsi-hd", "id": "stg3", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg3", "write-cache": "on", "lun": 4}' \
     -blockdev '{"node-name": "file_stg4", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg4.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg4", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg4"}' \
     -device '{"driver": "scsi-hd", "id": "stg4", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg4", "write-cache": "on", "lun": 5}' \
     -blockdev '{"node-name": "file_stg5", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg5.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg5", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg5"}' \
     -device '{"driver": "scsi-hd", "id": "stg5", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg5", "write-cache": "on", "lun": 6}' \
     -blockdev '{"node-name": "file_stg6", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg6.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg6", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg6"}' \
     -device '{"driver": "scsi-hd", "id": "stg6", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg6", "write-cache": "on", "lun": 7}' \
     -blockdev '{"node-name": "file_stg7", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg7.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg7", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg7"}' \
     -device '{"driver": "scsi-hd", "id": "stg7", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg7", "write-cache": "on", "lun": 8}' \
     -blockdev '{"node-name": "file_stg8", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg8.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg8", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg8"}' \
     -device '{"driver": "scsi-hd", "id": "stg8", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg8", "write-cache": "on", "lun": 9}' \
     -blockdev '{"node-name": "file_stg9", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg9.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg9", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg9"}' \
     -device '{"driver": "scsi-hd", "id": "stg9", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg9", "write-cache": "on", "lun": 10}' \
     -blockdev '{"node-name": "file_stg10", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg10.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg10", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg10"}' \
     -device '{"driver": "scsi-hd", "id": "stg10", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg10", "write-cache": "on", "lun": 11}' \
     -blockdev '{"node-name": "file_stg11", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg11.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg11", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg11"}' \
     -device '{"driver": "scsi-hd", "id": "stg11", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg11", "write-cache": "on", "lun": 12}' \
     -blockdev '{"node-name": "file_stg12", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg12.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg12", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg12"}' \
     -device '{"driver": "scsi-hd", "id": "stg12", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg12", "write-cache": "on", "lun": 13}' \
     -blockdev '{"node-name": "file_stg13", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg13.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg13", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg13"}' \
     -device '{"driver": "scsi-hd", "id": "stg13", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg13", "write-cache": "on", "lun": 14}' \
     -blockdev '{"node-name": "file_stg14", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg14.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg14", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg14"}' \
     -device '{"driver": "scsi-hd", "id": "stg14", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg14", "write-cache": "on", "lun": 15}' \
     -blockdev '{"node-name": "file_stg15", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg15.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg15", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg15"}' \
     -device '{"driver": "scsi-hd", "id": "stg15", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg15", "write-cache": "on", "lun": 16}' \
     -blockdev '{"node-name": "file_stg16", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg16.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg16", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg16"}' \
     -device '{"driver": "scsi-hd", "id": "stg16", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg16", "write-cache": "on", "lun": 17}' \
     -blockdev '{"node-name": "file_stg17", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg17.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg17", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg17"}' \
     -device '{"driver": "scsi-hd", "id": "stg17", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg17", "write-cache": "on", "lun": 18}' \
     -blockdev '{"node-name": "file_stg18", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg18.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg18", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg18"}' \
     -device '{"driver": "scsi-hd", "id": "stg18", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg18", "write-cache": "on", "lun": 19}' \
     -blockdev '{"node-name": "file_stg19", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg19.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg19", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg19"}' \
     -device '{"driver": "scsi-hd", "id": "stg19", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg19", "write-cache": "on", "lun": 20}' \
     -blockdev '{"node-name": "file_stg20", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg20.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg20", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg20"}' \
     -device '{"driver": "scsi-hd", "id": "stg20", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg20", "write-cache": "on", "lun": 21}' \
     -blockdev '{"node-name": "file_stg21", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg21.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg21", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg21"}' \
     -device '{"driver": "scsi-hd", "id": "stg21", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg21", "write-cache": "on", "lun": 22}' \
     -blockdev '{"node-name": "file_stg22", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg22.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg22", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg22"}' \
     -device '{"driver": "scsi-hd", "id": "stg22", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg22", "write-cache": "on", "lun": 23}' \
     -blockdev '{"node-name": "file_stg23", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg23.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg23", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg23"}' \
     -device '{"driver": "scsi-hd", "id": "stg23", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg23", "write-cache": "on", "lun": 24}' \
     -blockdev '{"node-name": "file_stg24", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg24.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg24", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg24"}' \
     -device '{"driver": "scsi-hd", "id": "stg24", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg24", "write-cache": "on", "lun": 25}' \
     -blockdev '{"node-name": "file_stg25", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg25.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg25", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg25"}' \
     -device '{"driver": "scsi-hd", "id": "stg25", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg25", "write-cache": "on", "lun": 26}' \
     -blockdev '{"node-name": "file_stg26", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg26.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg26", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg26"}' \
     -device '{"driver": "scsi-hd", "id": "stg26", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg26", "write-cache": "on", "lun": 27}' \
     -blockdev '{"node-name": "file_stg27", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg27.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg27", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg27"}' \
     -device '{"driver": "scsi-hd", "id": "stg27", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg27", "write-cache": "on", "lun": 28}' \
     -blockdev '{"node-name": "file_stg28", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg28.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg28", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg28"}' \
     -device '{"driver": "scsi-hd", "id": "stg28", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg28", "write-cache": "on", "lun": 29}' \
     -blockdev '{"node-name": "file_stg29", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg29.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg29", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg29"}' \
     -device '{"driver": "scsi-hd", "id": "stg29", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg29", "write-cache": "on", "lun": 30}' \
     -blockdev '{"node-name": "file_stg30", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg30.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg30", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg30"}' \
     -device '{"driver": "scsi-hd", "id": "stg30", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg30", "write-cache": "on", "lun": 31}' \
     -blockdev '{"node-name": "file_stg31", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg31.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg31", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg31"}' \
     -device '{"driver": "scsi-hd", "id": "stg31", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg31", "write-cache": "on", "lun": 32}' \
     -blockdev '{"node-name": "file_stg32", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg32.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg32", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg32"}' \
     -device '{"driver": "scsi-hd", "id": "stg32", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg32", "write-cache": "on", "lun": 33}' \
     -blockdev '{"node-name": "file_stg33", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg33.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg33", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg33"}' \
     -device '{"driver": "scsi-hd", "id": "stg33", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg33", "write-cache": "on", "lun": 34}' \
     -blockdev '{"node-name": "file_stg34", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg34.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg34", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg34"}' \
     -device '{"driver": "scsi-hd", "id": "stg34", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg34", "write-cache": "on", "lun": 35}' \
     -blockdev '{"node-name": "file_stg35", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg35.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg35", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg35"}' \
     -device '{"driver": "scsi-hd", "id": "stg35", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg35", "write-cache": "on", "lun": 36}' \
     -blockdev '{"node-name": "file_stg36", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg36.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg36", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg36"}' \
     -device '{"driver": "scsi-hd", "id": "stg36", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg36", "write-cache": "on", "lun": 37}' \
     -blockdev '{"node-name": "file_stg37", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg37.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg37", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg37"}' \
     -device '{"driver": "scsi-hd", "id": "stg37", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg37", "write-cache": "on", "lun": 38}' \
     -blockdev '{"node-name": "file_stg38", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg38.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg38", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg38"}' \
     -device '{"driver": "scsi-hd", "id": "stg38", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg38", "write-cache": "on", "lun": 39}' \
     -blockdev '{"node-name": "file_stg39", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg39.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg39", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg39"}' \
     -device '{"driver": "scsi-hd", "id": "stg39", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg39", "write-cache": "on", "lun": 40}' \
     -blockdev '{"node-name": "file_stg40", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "native", "filename": "/home/kvm_autotest_root/images/stg40.qcow2", "cache": {"direct": true, "no-flush": false}}' \
     -blockdev '{"node-name": "drive_stg40", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_stg40"}' \
     -device '{"driver": "scsi-hd", "id": "stg40", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg40", "write-cache": "on", "lun": 41}' \
     -device '{"id": "pcie-root-port-4", "port": 4, "driver": "pcie-root-port", "addr": "0x1.0x4", "bus": "pcie.0", "chassis": 5}' \
     -device '{"driver": "virtio-net-pci", "mac": "9a:78:68:75:6e:d8", "id": "idohacqr", "netdev": "idPKD7AE", "bus": "pcie-root-port-4", "addr": "0x0"}'  \
     -netdev tap,id=idPKD7AE,vhost=on,vhostfd=16,fd=12  \
     -vnc :0  \
     -rtc base=utc,clock=host,driftfix=slew  \
     -boot menu=off,order=cdn,once=c,strict=off \
     -chardev socket,id=char_vtpm_avocado-vt-vm1_tpm0,path=/home/kar/workspace/root/avocado/data/avocado-vt/swtpm/avocado-vt-vm1_tpm0_swtpm.sock \
     -tpmdev emulator,chardev=char_vtpm_avocado-vt-vm1_tpm0,id=emulator_vtpm_avocado-vt-vm1_tpm0 \
     -device '{"id": "tpm-crb_vtpm_avocado-vt-vm1_tpm0", "tpmdev": "emulator_vtpm_avocado-vt-vm1_tpm0", "driver": "tpm-crb"}' \
     -enable-kvm \
     -device '{"id": "pcie_extra_root_port_0", "driver": "pcie-root-port", "multifunction": true, "bus": "pcie.0", "addr": "0x2", "chassis": 6}'

3.unplug the 40 disks
{"execute": "device_del", "arguments": {"id": "stg1"}, "id": "imSSHPYa"}
...
{"execute": "device_del", "arguments": {"id": "stg40"}, "id": "imSSHPYa"}


4. sleep 5 seconds

5. hotplug 40 disks

{"execute": "device_add", "arguments": {"driver": "scsi-hd", "id": "stg1", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg1", "write-cache": "on", "lun": 2}, "id": "Rvr8aVX7"}
...
{"execute": "device_add", "arguments": {"driver": "scsi-hd", "id": "stg40", "bus": "virtio_scsi_pci1.0", "drive": "drive_stg40", "write-cache": "on", "lun": 41}, "id": "Rvr8aVX7"}


6. sleep 5 seconds

7. repeat steps 3-6 500 times

Actual results:
Get no response on QMP command device_add
Expected results:
All operation succeed 


Additional info:
Not sure this issue has the same reason with:
Bug 2185688 - [qemu-kvm] no response with QMP command block_resize

The os will be broken sometimes due to 
Bug 2176702 - [RHEL9][virtio-scsi] scsi-hd cannot hot-plug successfully after hot-plug it repeatedly


Not hit issue on
Red Hat Enterprise Linux release 9.3 Beta (Plow)
5.14.0-312.el9.x86_64
qemu-kvm-7.1.0-4.el9.x86_64
seabios-bin-1.16.1-1.el9.noarch
edk2-ovmf-20230301gitf80f052277c8-3.el9.noarch
libvirt-9.0.0-10.1.el9_2.x86_64
virtio-win-prewhql-0.1-236.iso
Comment 9 qing.wang 2023-07-31 07:52:42 UTC 
Passed on

python ConfigTest.py --testcase=multi_disk_wild_hotplug.without_delay --iothread_scheme=roundrobin --nr_iothreads=2 --platform=x86_64 --guestname=RHEL.9.3.0 --driveformat=virtio_scsi --nicmodel=virtio_net --imageformat=qcow2 --machines=q35 --customsparams="vm_mem_limit = 12G" --firmware=ovmf --netdst=virbr0  --nrepeat=80 --clone=no

Red Hat Enterprise Linux release 9.3 Beta (Plow)
5.14.0-342.el9.x86_64
qemu-kvm-8.0.0-9.el9.x86_64
seabios-bin-1.16.1-1.el9.noarch
edk2-ovmf-20230524-2.el9.noarch
libvirt-9.3.0-2.el9.x86_64
virtio-win-prewhql-0.1-239.iso