Bug 1523017 - [virtio-win][vioscsi/viostor] cannot hot-unplug vioscsi/viostor controller with q35
Summary: [virtio-win][vioscsi/viostor] cannot hot-unplug vioscsi/viostor controller wi...
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: qemu-kvm
Version: 8.0
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: rc
: 8.1
Assignee: Vadim Rozenfeld
QA Contact: qing.wang
URL:
Whiteboard:
Depends On:
Blocks: 1558351 1744438
TreeView+ depends on / blocked
 
Reported: 2017-12-07 02:44 UTC by Yu Wang
Modified: 2021-02-22 06:18 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-02-15 07:33:48 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1517107 0 low CLOSED [q35] guest NIC interface still exists after hotunplug virtio-net-pci device 2021-02-22 00:41:40 UTC

Internal Links: 1517107

Description Yu Wang 2017-12-07 02:44:10 UTC
Description of problem:
cannot hot-unplug vioscsi controller with q35 

Version-Release number of selected component (if applicable):

kernel-3.10.0-798.el7.x86_64
qemu-kvm-rhev-2.10.0-10.el7.x86_64
seabios-bin-1.11.0-1.el7.noarch
OVMF-20171011-2.git92d07e48907f.el7.noarch
virtio-win-prewhql-145

How reproducible:
3/10

Steps to Reproduce:
1. boot with vioscsi device on q35 

/usr/libexec/qemu-kvm -name 143SCSW143SCSW10S64EOE -enable-kvm -m 4G -smp 4 -uuid 043fe384-92e0-4ed3-9500-7e031dfd2a9e -nodefconfig -nodefaults -cpu host,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff -chardev socket,id=charmonitor,path=/tmp/143SCSW10S64BZ3,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,driftfix=slew -boot order=cd,menu=on -device piix3-usb-uhci,id=usb -drive file=virtio-win-prewhql-0.1-143.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,script=/etc/qemu-ifup,downscript=no,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:52:3c:54:04:84 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=isa_serial0 -device usb-tablet,id=input0 -vnc 0.0.0.0:1 -vga std -M q35 -drive file=143SCSW10S64EOE_ovmf/OVMF_CODE.secboot.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=143SCSW10S64EOE_ovmf/OVMF_VARS.fd,if=pflash,format=raw,unit=1 -drive file=143SCSW10S64EOE_ovmf/UefiShell.iso,if=none,cache=none,snapshot=off,aio=native,media=cdrom,id=cdrom1 -device ide-cd,drive=cdrom1,id=ide-cd1 -object iothread,id=thread0 -device pcie-root-port,bus=pcie.0,id=root0,multifunction=on,chassis=1,addr=0xa.0 -device virtio-scsi-pci,iothread=thread0,id=scsi0,bus=root0,num_queues=2 -drive file=143SCSW10S64EOE,if=none,id=drive-scsi-disk0,format=raw,cache=none -device scsi-hd,bus=scsi0.0,drive=drive-scsi-disk0,id=scsi-disk0 -monitor stdio -qmp tcp:0:4445,server,nowait -device pcie-root-port,bus=pcie.0,id=root1,chassis=11,addr=0xa.1 -device virtio-scsi-pci,iothread=thread0,id=scsi1,bus=root1,num_queues=2 -drive file=data.raw,if=none,id=drive-scsi-disk1,format=raw,cache=none -device scsi-hd,bus=scsi1.0,drive=drive-scsi-disk1,id=scsi-disk1


2. { "execute": "device_del", "arguments": {"id": "scsi1" }}
3. check scsi device in device manager and disk in disk manager

Actual results:
1 virtio-scsi device still in device manager, but disk not in disk manager.
2 try to re-scan device in device manger, will always scanning not finish.
3 info qtree , cannot see vioscsi device in it.

Expected results:
virtio-scsi device can be deleted successfully 

Additional info:
1 both q35+seabios and q35+ovmf hit this issue.
2 on pc+seabios not hit this issue yet
3 7.4 release driver also hit this issue, so not a regression.

Comment 2 Yu Wang 2017-12-07 02:57:07 UTC
if directly delete controller will hit this issue.
if delete disk first ,then delete controller, will not hit this issue.

Thanks

Comment 3 Yu Wang 2017-12-18 03:15:54 UTC
hit the same issue when hot-unplug virtio-blk-pci controller with q35

kernel-3.10.0-823.el7.x86_64
qemu-kvm-rhev-2.10.0-14.el7.x86_64
seabios-bin-1.11.0-1.el7.noarch
OVMF-20171011-4.git92d07e48907f.el7.noarch
virtio-win-prewhql-145


Thanks
Yu Wang

Comment 4 Yu Wang 2017-12-18 07:18:54 UTC
(In reply to Yu Wang from comment #3)
> hit the same issue when hot-unplug virtio-blk-pci controller with q35
> 
> kernel-3.10.0-823.el7.x86_64
> qemu-kvm-rhev-2.10.0-14.el7.x86_64
> seabios-bin-1.11.0-1.el7.noarch
> OVMF-20171011-4.git92d07e48907f.el7.noarch
> virtio-win-prewhql-145
> 
> 
> Thanks
> Yu Wang

How reproducible with virio-blk-pci on win10/2016(q35+ovmf):
100%

1 virtio-blk device still in device manager, but disk not in disk manager.
2 try to re-scan device in device manger, will always scanning not finish.
3 info qtree , cannot see blk device in it.

win7 guests with seabios+q35 not hit this issue

Comment 5 Vadim Rozenfeld 2018-07-05 07:26:23 UTC
seems like qemu problem rather than viostor or vioscsi bug.
Moving to 7.7 for the future investigation.

Comment 7 Yu Wang 2018-09-27 08:30:44 UTC
Hit same issue when test virtio-blk-pci device on win7-32 with q35+seabios(RHEL8 host)

kernel-4.18.0-5.el8.x86_64
qemu-kvm-2.12.0-31.el8+1856+594f5537.x86_64

Thanks 
Yu Wang

Comment 8 Yu Wang 2018-09-27 09:22:55 UTC
(In reply to Yu Wang from comment #7)
> Hit same issue when test virtio-blk-pci device on win7-32 with
> q35+seabios(RHEL8 host)
> 
> kernel-4.18.0-5.el8.x86_64
> qemu-kvm-2.12.0-31.el8+1856+594f5537.x86_64
> 
> Thanks 
> Yu Wang

-drive file=/dev/vgtest1/srv-msql3-vol,if=none,id=drive-virtio-disk0,format=raw,serial=mike_cao,cache=none -device virtio-blk-pci,iothread=thread0,scsi=off,drive=drive-virtio-disk0,id=virtio-disk0 -drive file=data.raw,if=none,id=drive-virtio-disk1,format=raw,serial=mike_cao,cache=none -device virtio-blk-pci,iothread=thread0,scsi=off,drive=drive-virtio-disk1,id=virtio-disk1

1 It occurred on iscsi backend and q35+seabios
2 but pass this case on iscsi backend and pc+seabios

Comment 9 Xueqiang Wei 2019-02-18 12:08:19 UTC
Check virtio-scsi device in device manager until received "DEVICE_DELETED" event in qmp, not hit this issue.


Details:
Host:
kernel-4.18.0-67.el8.x86_64
qemu-kvm-2.12.0-61.module+el8+2786+5afd5ae3

Guest:
windows2019
virtio-win-prewhql-0.1-163.iso


cmd lines:

/usr/libexec/qemu-kvm \
-name 143SCSW143SCSW10S64EOE \
-enable-kvm \
-m 4G \
-smp 4 \
-uuid 043fe384-92e0-4ed3-9500-7e031dfd2a9e \
-nodefconfig \
-nodefaults \
-cpu host,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff \
-chardev socket,id=charmonitor,path=/tmp/143SCSW10S64BZ3,server,nowait \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=localtime,driftfix=slew \
-boot order=cd,menu=on \
-device piix3-usb-uhci,id=usb \
-drive file=/home/kvm_autotest_root/iso/windows/virtio-win-prewhql-0.1-163.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw \
-device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 \
-netdev tap,script=/etc/qemu-ifup,downscript=no,id=hostnet0 \
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:52:3c:54:04:84 \
-chardev pty,id=charserial0 \
-device isa-serial,chardev=charserial0,id=isa_serial0 \
-device usb-tablet,id=input0 \
-vnc :0 \
-vga std \
-M q35 \
-object iothread,id=thread0 \
-device pcie-root-port,bus=pcie.0,id=root0,multifunction=on,chassis=1,addr=0xa.0 \
-device virtio-scsi-pci,iothread=thread0,id=scsi0,bus=root0,num_queues=2 \
-drive file=/home/kvm_autotest_root/images/win2019-64-virtio-scsi.qcow2,if=none,id=drive-scsi-disk0,format=qcow2,cache=none \
-device scsi-hd,bus=scsi0.0,drive=drive-scsi-disk0,id=scsi-disk0 \
-monitor stdio \
-qmp tcp:0:4445,server,nowait \
-device pcie-root-port,bus=pcie.0,id=root1,chassis=11,addr=0xa.1 \
-device virtio-scsi-pci,iothread=thread0,id=scsi1,bus=root1,num_queues=2 \
-drive file=/home/kvm_autotest_root/images/data.raw,if=none,id=drive-scsi-disk1,format=raw,cache=none \
-device scsi-hd,bus=scsi1.0,drive=drive-scsi-disk1,id=scsi-disk1 \


{"execute":"qmp_capabilities"}
{"return": {}}

{ "execute": "device_del", "arguments": {"id": "scsi1" }}
{"return": {}}

{"timestamp": {"seconds": 1550487658, "microseconds": 502717}, "event": "DEVICE_DELETED", "data": {"device": "scsi-disk1", "path": "/machine/peripheral/scsi-disk1"}}
{"timestamp": {"seconds": 1550487658, "microseconds": 502893}, "event": "DEVICE_DELETED", "data": {"path": "/machine/peripheral/scsi1/virtio-backend"}}
{"timestamp": {"seconds": 1550487658, "microseconds": 510347}, "event": "DEVICE_DELETED", "data": {"device": "scsi1", "path": "/machine/peripheral/scsi1"}}



Yu, did you check it until received "DEVICE_DELETED" event? I try 30 times, not hit this issue.

Comment 10 Yu Wang 2019-02-22 07:35:22 UTC
I recieved the deleted event and cannot see it with "info qtree", but it can still see device in guest.
But it occurs when reported the bug. It used the qemu/virtio-win driver as comment#0.

Comment 11 Yu Wang 2019-03-01 07:28:26 UTC
I cannot reproduce this issue now, so close this bug. If hit it again, please feel free to re-open it.

Thanks
Yu Wang

Comment 12 lijin 2019-05-07 09:06:31 UTC
re-open this bug as I hit it again on win10-32 guest under q35;
pc+seabios does NOT hit this issue.

package info:
virtio-win-prewhql-171
qemu-kvm-3.1.0-24.module+el8.0.1+3117+9f83299e.x86_64
kernel-4.18.0-80.el8.x86_64
seabios-1.12.0-1.module+el8.0.1+2959+fecd1a40.x86_64

result is same with comment#0(qmp DEVICE_DELETED received)
full qemu cli:
/usr/libexec/qemu-kvm \
    -name 'avocado-vt-vm1' \
    -machine q35  \
    -nodefaults \
    -device VGA,bus=pcie.0,addr=0x1  \
    -device pcie-root-port,id=pcie.0-root-port-2,slot=2,chassis=2,addr=0x2,bus=pcie.0 \
    -device qemu-xhci,id=usb1,bus=pcie.0-root-port-2,addr=0x0 \
    -drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/win10-32-virtio.qcow2 \
    -device pcie-root-port,id=pcie.0-root-port-3,slot=3,chassis=3,addr=0x3,bus=pcie.0 \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pcie.0-root-port-3,addr=0x0 \
    -drive id=drive_disk1,if=none,snapshot=off,aio=threads,cache=none,format=qcow2,file=storage.qcow2 \
    -device pcie-root-port,id=pcie.0-root-port-4,slot=4,chassis=4,addr=0x4,bus=pcie.0 \
    -device virtio-blk-pci,id=disk1,drive=drive_disk1,bootindex=1,bus=pcie.0-root-port-4,addr=0x0,disable-legacy=off,disable-modern=true \
    -device pcie-root-port,id=pcie.0-root-port-5,slot=4,chassis=5,addr=0x5,bus=pcie.0 \
    -device virtio-net-pci,mac=9a:08:09:0a:0b:0c,id=idKpSzAy,vectors=4,netdev=idQq0m9U,bus=pcie.0-root-port-5,addr=0x0  \
    -netdev tap,id=idQq0m9U,vhost=on,script=/etc/qemu-ifup \
    -m 14336  \
    -smp 24,maxcpus=24,cores=12,threads=1,sockets=2  \
    -cpu 'Skylake-Server',hv_stimer,hv_synic,hv_vpindex,hv_reset,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv-tlbflush,+kvm_pv_unhalt \
    -drive id=drive_cd1,if=none,snapshot=off,aio=threads,cache=none,media=cdrom,file=/home/kvm_autotest_root/iso/windows/winutils.iso \
    -device ide-cd,id=cd1,drive=drive_cd1,bootindex=2,bus=ide.0,unit=0 \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -vnc :0  \
    -rtc base=localtime,clock=host,driftfix=slew  \
    -boot order=cdn,once=c,menu=off,strict=off \
    -monitor stdio \
    -qmp tcp:0:4445,server,nowait \
    -enable-kvm \

Comment 18 menli@redhat.com 2019-12-05 03:12:13 UTC
I hit an issue after hot-unplug virtio-scsi-pci controller with scsi-cd device  .Vioscsi controller still display  in device manager in guest after receive "DEVICE_DELETED" event in qmp. Then restart guest ,controller disappear(reboot guest very slow,about 3 mins.)

I hit the similar issue when hotplug a virtio-scsi-pci controller with scsi-cd device.

pkg:
kernel-4.18.0-151.el8.x86_64
Qemu-kvm-4.2.0-0.module+el8.2.0+4743+23ad88a2.x86_64
Seabios-1.12.0-5.module+el8.2.0+4673+ff4b3b61.x86_64
Virtio-win-prewhql-0.1-172.iso


Guest:
windows2012R2


Steps:
1.boot up guest with virtio-scsi-pci with scsi-cd device.
    -device virtio-scsi-pci,id=scsi1,num_queues=4,bus=pcie-root-port-6 \

    -drive file=/home/kvm_autotest_root/iso/windows/virtio-win-1.9.9-3.el8.iso,if=none,id=drive-scsi-disk1,format=raw,cache=none \

    -device scsi-cd,bus=scsi1.0,drive=drive-scsi-disk1,id=scsi-disk1 \

2.hotunplug virtio-scsi-pci controller directly via qmp
{"execute":"qmp_capabilities"}
{"return": {}}
{"execute":"device_del","arguments":{"id":"scsi1"}}
{"return": {}}
{"timestamp": {"seconds": 1575452381, "microseconds": 959878}, "event": "DEVICE_DELETED", "data": {"device": "scsi-disk1", "path": "/machine/peripheral/scsi-disk1"}}
{"timestamp": {"seconds": 1575452381, "microseconds": 959969}, "event": "DEVICE_DELETED", "data": {"path": "/machine/peripheral/scsi1/virtio-backend"}}
{"timestamp": {"seconds": 1575452382, "microseconds": 16649}, "event": "DEVICE_DELETED", "data": {"device": "scsi1", "path": "/machine/peripheral/scsi1"}}

3.check the device in guest.
Vioscsi controller still display  in device manager
Then restart guest ,controller disappear(reboot guest very slow,about 3 mins.)


Others:
Pc+seabios didn’t hit this issue.
If hotunplug the virtio-scsi-pci with scsi-hd,didn’t hit this issue.

cmd lines:
 /usr/libexec/qemu-kvm \
    -name 'avocado-vt-vm3' \
    -machine q35 \
    -nodefaults \
    -vga std  \
    -device pcie-root-port,id=pcie.0-root-port-3,slot=3,chassis=3,addr=0x3,bus=pcie.0 \
    -device pcie-root-port,id=pcie-root-port-5,slot=5,chassis=5,bus=pcie.0  \
    -device pcie-root-port,id=pcie-root-port-6,slot=6,chassis=6,bus=pcie.0  \
    -device pcie-root-port,id=pcie-root-port-7,slot=7,chassis=7,bus=pcie.0  \
    -device pcie-root-port,id=pcie-root-port-8,slot=8,chassis=8,bus=pcie.0  \
    -device pcie-root-port,id=pcie-root-port-9,slot=9,chassis=9,bus=pcie.0  \
    -device pcie-root-port,id=pcie-root-port-10,slot=10,chassis=10,bus=pcie.0  \
    -device pcie-root-port,id=pcie-root-port-11,slot=11,chassis=11,bus=pcie.0  \
    -device pcie-root-port,id=pcie-root-port-12,slot=12,chassis=12,bus=pcie.0  \
    -device pcie-root-port,id=pcie-root-port-13,slot=13,chassis=13,bus=pcie.0  \
    -device pcie-root-port,id=pcie-root-port-22,slot=22,chassis=22,bus=pcie.0  \
    -device virtio-scsi-pci,id=scsi0,num_queues=4,bus=pcie-root-port-5   \
    -drive file=win2012r2.qcow2,if=none,id=drive-scsi-disk0,format=qcow2,cache=none \
    -device scsi-hd,bus=scsi0.0,drive=drive-scsi-disk0,id=scsi-disk0 \
    -device virtio-scsi-pci,id=scsi1,num_queues=4,bus=pcie-root-port-6 \
    -drive file=/home/kvm_autotest_root/iso/windows/virtio-win-1.9.9-3.el8.iso,if=none,id=drive-scsi-disk1,format=raw,cache=none \
    -device scsi-cd,bus=scsi1.0,drive=drive-scsi-disk1,id=scsi-disk1 \
    -device virtio-net-pci,mac=9a:36:83:b6:3d:05,id=idJVpmsF,netdev=id23ZUK6,bus=pcie-root-port-7  \
    -netdev tap,id=id23ZUK6,vhost=on \
    -m 14336  \
    -smp 24,maxcpus=24,cores=12,threads=1,dies=1,sockets=2  \
    -cpu 'Skylake-Server',hv_stimer,hv_synic,hv_vpindex,hv_reset,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv-tlbflush,+kvm_pv_unhalt \
    -drive id=drive_cd2,if=none,snapshot=off,aio=threads,cache=none,media=cdrom,file=/home/kvm_autotest_root/iso/windows/virtio-win-prewhql-0.1-172.iso \
    -device ide-cd,id=cd2,drive=drive_cd2,bus=ide.0,unit=0 \
    -device piix3-usb-uhci,id=usb -device usb-tablet,id=input0 \
    -vnc :0  \
    -rtc base=localtime,clock=host,driftfix=slew  \
    -boot order=cdn,once=c,menu=off,strict=off \
    -enable-kvm \
    -qmp tcp:0:1234,server,nowait \
    -monitor stdio



.

Comment 20 Ademar Reis 2020-02-05 22:45:55 UTC
QEMU has been recently split into sub-components and as a one-time operation to avoid breakage of tools, we are setting the QEMU sub-component of this BZ to "General". Please review and change the sub-component if necessary the next time you review this BZ. Thanks

Comment 29 RHEL Program Management 2021-02-15 07:33:48 UTC
After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.

Comment 30 qing.wang 2021-02-22 06:18:54 UTC
Agree to close


Note You need to log in before you can comment on or make changes to this bug.