Bug 1708490

Summary: Hotplug/unplug a scsi-hd/scsi-cd via blockdev hits bsod in windows guest
Product: Red Hat Enterprise Linux 8 Reporter: yujie ma <yujma>
Component: virtio-winAssignee: Amnon Ilan <ailan>
virtio-win sub component: virtio-win-prewhql QA Contact: menli <menli>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: high CC: ailan, areis, coli, jinzhao, juzhang, kanderso, knoel, lijin, menli, phou, qinwang, virt-maint, vrozenfe, xuwei
Version: 8.2Keywords: Triaged
Target Milestone: rcFlags: pm-rhel: mirror+
Target Release: 8.0   
Hardware: Unspecified   
OS: Windows   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-07-21 15:32:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1718992, 1744438, 1758964    
Attachments:
Description Flags
Bsod.png none

Description yujie ma 2019-05-10 05:26:14 UTC
Created attachment 1566511 [details]
Bsod.png

Description of problem:
Hotplug a SISI disk hits bsod in windows guest.


Version-Release number of selected component (if applicable):
qemu-kvm-3.1.0-22.module+el8.0.1+3032+a09688b9
virtio-win-prewhql-0.1-171.iso
kernel-4.18.0-82.el8


How reproducible:
3/3


Steps to Reproduce:

1. Boot guest from a win2019 image

Full command lines:
/usr/libexec/qemu-kvm \
    -name 'guest-win2019' \
    -machine q35 \
    -nodefaults \
    -object   iothread,id=iothread0 \
    -vga qxl \
    -blockdev driver=file,cache.direct=on,cache.no-flush=off,filename=/home/test/win2019-64-virtio-scsi.qcow2,node-name=my_file1 \
    -blockdev driver=qcow2,node-name=my1,file=my_file1 \
    -device pcie-root-port,id=pcie.0-root-port-6,slot=6,chassis=6,addr=0x6,bus=pcie.0 \
    -device virtio-scsi-pci,bus=pcie.0-root-port-6,addr=0x0,id=scsi0,iothread=iothread0 \
    -device scsi-hd,drive=my1,id=image5,bootindex=0 \
    -device  pcie-root-port,id=pcie.0-root-port-5,slot=5,chassis=5,addr=0x5,bus=pcie.0 \
    -drive id=drive_virtio,if=none,snapshot=off,aio=threads,cache=none,media=cdrom,file=/home/nfs/windows/virtio-win-prewhql-0.1-171.iso \
    -device ide-cd,id=virtio,drive=drive_virtio,bus=ide.1,unit=0 \
    -vnc :0  \
    -monitor stdio \
    -m 8192 \
    -smp 8 \
    -device virtio-net-pci,mac=9a:b5:b6:b1:b2:b3,id=idMmq1jH,vectors=4,netdev=idxgXAlm,bus=pcie.0,addr=0x9  \
    -netdev tap,id=idxgXAlm \
    -chardev socket,id=qmp_id_qmpmonitor1,path=/home/yujma_win/monitor-qmpmonitor1-20180220-094308-h9I6hRsI,server,nowait \
    -qmp tcp:localhost:5902,server,nowait  \
    -device nec-usb-xhci,id=usb1,bus=pcie.0,addr=0x2 -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \

2.Create a data image
qemu-img create -f qcow2 /home/disk/data.qcow2 16T

3.Hotplug a scsi device in QMP: 
{'execute':'qmp_capabilities'}

{"execute":"blockdev-add","arguments":{"node-name":"file_image3","driver":"qcow2","file":{"driver":"file","filename":"/home/disk/data.qcow2"}}}

{'execute':'device_add','arguments':{'driver':'scsi-hd','drive':'file_image3','bus':'scsi0.0','id':'test'}}


Actual results:
Hit bsod in windows guest.


Expected results:
Hotplug disk successfully.


Additional info:
1. Using 2g size data image also has this error. And qemu info block can see that this disk has been added.
(qemu) info block
file_image3: /home/disk/data1.qcow2 (qcow2)
    Attached to:      test
    Cache mode:       writeback
2. Using blk device does not hit the issue.
3. Booting from Linux does not hit the issue.

Comment 1 yujie ma 2019-05-10 05:52:22 UTC
Memory dump info:

kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

CRITICAL_PROCESS_DIED (ef)
        A critical system process died
Arguments:
Arg1: ffff998d079640c0, Process object or thread object
Arg2: 0000000000000000, If this is 0, a process died. If this is 1, a thread died.
Arg3: 0000000000000000
Arg4: 0000000000000000

Debugging Details:
------------------


DUMP_CLASS: 1

DUMP_QUALIFIER: 401

BUILD_VERSION_STRING:  17763.1.amd64fre.rs5_release.180914-1434

SYSTEM_MANUFACTURER:  Red Hat

SYSTEM_PRODUCT_NAME:  KVM

SYSTEM_VERSION:  RHEL-8.0.0 PC (Q35 + ICH9, 2009)

BIOS_VENDOR:  SeaBIOS

BIOS_VERSION:  1.11.1-3.module+el8.1.0+2983+b2ae9c0a

BIOS_DATE:  04/01/2014

DUMP_TYPE:  1

BUGCHECK_P1: ffff998d079640c0

BUGCHECK_P2: 0

BUGCHECK_P3: 0

BUGCHECK_P4: 0

PROCESS_NAME:  services.exe

CRITICAL_PROCESS:  services.exe

EXCEPTION_RECORD:  0000000000001000 -- (.exr 0x1000)
Cannot read Exception record @ 0000000000001000

EXCEPTION_CODE: (Win32) 0x8611080 (140578944) - <Unable to get error code text>

ERROR_CODE: (NTSTATUS) 0x8611080 - <Unable to get error code text>

CPU_COUNT: 8

CPU_MHZ: d40

CPU_VENDOR:  GenuineIntel

CPU_FAMILY: 6

CPU_MODEL: d

CPU_STEPPING: 3

CPU_MICROCODE: 6,d,3,0 (F,M,S,R)  SIG: 1'00000000 (cache) 1'00000000 (init)

DEFAULT_BUCKET_ID:  WIN8_DRIVER_FAULT

BUGCHECK_STR:  0xEF

CURRENT_IRQL:  0

ANALYSIS_SESSION_HOST:  WIN-3IORRL4PE1F

ANALYSIS_SESSION_TIME:  05-09-2019 16:49:02.0870

ANALYSIS_VERSION: 10.0.16299.15 amd64fre

TRAP_FRAME:  ffff800000000000 -- (.trap 0xffff800000000000)
Unable to read trap frame at ffff8000`00000000

LAST_CONTROL_TRANSFER:  from fffff80438cc01dd to fffff80438663040

THREAD_SHA1_HASH_MOD_FUNC:  f6b73c86bdc1d41372e9d082a4b923d63e8e3404

THREAD_SHA1_HASH_MOD_FUNC_OFFSET:  3b482d8a300d2d7fed06562ce6f1311d64e91a58

THREAD_SHA1_HASH_MOD:  d12ec76460ff415570e040fd98385a1307dad670

FOLLOWUP_IP: 
ntdll!RtlDispatchException+f9
00007ffc`ff294799 482be1          sub     rsp,rcx

FAULT_INSTR_CODE:  4ce12b48

SYMBOL_STACK_INDEX:  b

SYMBOL_NAME:  ntdll!RtlDispatchException+f9

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: ntdll

IMAGE_NAME:  ntdll.dll

DEBUG_FLR_IMAGE_TIMESTAMP:  0

STACK_COMMAND:  .thread ; .cxr ; kb

BUCKET_ID_FUNC_OFFSET:  f9

FAILURE_BUCKET_ID:  0xEF_services.exe_BUGCHECK_CRITICAL_PROCESS_8611080_ntdll!RtlDispatchException

BUCKET_ID:  0xEF_services.exe_BUGCHECK_CRITICAL_PROCESS_8611080_ntdll!RtlDispatchException

PRIMARY_PROBLEM_CLASS:  0xEF_services.exe_BUGCHECK_CRITICAL_PROCESS_8611080_ntdll!RtlDispatchException

TARGET_TIME:  2019-05-09T15:23:46.000Z

OSBUILD:  17763

OSSERVICEPACK:  0

SERVICEPACK_NUMBER: 0

OS_REVISION: 0

SUITE_MASK:  400

PRODUCT_TYPE:  3

OSPLATFORM_TYPE:  x64

OSNAME:  Windows 10

OSEDITION:  Windows 10 Server TerminalServer DataCenter SingleUserTS

OS_LOCALE:  

USER_LCID:  0

OSBUILD_TIMESTAMP:  unknown_date

BUILDDATESTAMP_STR:  180914-1434

BUILDLAB_STR:  rs5_release

BUILDOSVER_STR:  10.0.17763.1.amd64fre.rs5_release.180914-1434

ANALYSIS_SESSION_ELAPSED_TIME:  c25

ANALYSIS_SOURCE:  KM

FAILURE_ID_HASH_STRING:  km:0xef_services.exe_bugcheck_critical_process_8611080_ntdll!rtldispatchexception

FAILURE_ID_HASH:  {44ab8179-9eac-fcad-999c-fed786e0018f}

Followup:     MachineOwner
---------

Comment 2 yujie ma 2019-05-10 06:37:23 UTC
(In reply to yujie ma from comment #0)
> 
> 1. Boot guest from a win2019 image
> 
> Full command lines:
> /usr/libexec/qemu-kvm \
>     -name 'guest-win2019' \
>     -machine q35 \
>     -nodefaults \
>     -object   iothread,id=iothread0 \
>     -vga qxl \
>     -blockdev
> driver=file,cache.direct=on,cache.no-flush=off,filename=/home/test/win2019-
> 64-virtio-scsi.qcow2,node-name=my_file1 \
>     -blockdev driver=qcow2,node-name=my1,file=my_file1 \
>     -device
> pcie-root-port,id=pcie.0-root-port-6,slot=6,chassis=6,addr=0x6,bus=pcie.0 \
>     -device
> virtio-scsi-pci,bus=pcie.0-root-port-6,addr=0x0,id=scsi0,iothread=iothread0 \
>     -device scsi-hd,drive=my1,id=image5,bootindex=0 \
>     -device 
> pcie-root-port,id=pcie.0-root-port-5,slot=5,chassis=5,addr=0x5,bus=pcie.0 \
>     -drive
> id=drive_virtio,if=none,snapshot=off,aio=threads,cache=none,media=cdrom,
> file=/home/nfs/windows/virtio-win-prewhql-0.1-171.iso \
>     -device ide-cd,id=virtio,drive=drive_virtio,bus=ide.1,unit=0 \
>     -vnc :0  \
>     -monitor stdio \
>     -m 8192 \
>     -smp 8 \
>     -device
> virtio-net-pci,mac=9a:b5:b6:b1:b2:b3,id=idMmq1jH,vectors=4,netdev=idxgXAlm,
> bus=pcie.0,addr=0x9  \
>     -netdev tap,id=idxgXAlm \
>     -chardev
> socket,id=qmp_id_qmpmonitor1,path=/home/yujma_win/monitor-qmpmonitor1-
> 20180220-094308-h9I6hRsI,server,nowait \
>     -qmp tcp:localhost:5902,server,nowait  \
>     -device nec-usb-xhci,id=usb1,bus=pcie.0,addr=0x2 -device
> usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
> 
> 2.Create a data image
> qemu-img create -f qcow2 /home/disk/data.qcow2 16T
> 
> 3.Hotplug a scsi device in QMP: 
> {'execute':'qmp_capabilities'}
> 
> {"execute":"blockdev-add","arguments":{"node-name":"file_image3","driver":
> "qcow2","file":{"driver":"file","filename":"/home/disk/data.qcow2"}}}
> 
> {'execute':'device_add','arguments':{'driver':'scsi-hd','drive':
> 'file_image3','bus':'scsi0.0','id':'test'}}

Correct steps:
1. Boot up guest 
/usr/libexec/qemu-kvm \
    -name 'guest-win2019' \
    -machine q35 \
    -nodefaults \
    -object   iothread,id=iothread0 \
    -vga qxl \
    -device pcie-root-port,id=pcie.0-root-port-7,slot=7,chassis=7,addr=0x7,bus=pcie.0 \
    -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie.0-root-port-7,addr=0x0 \
    -blockdev node-name=file_image1,driver=file,cache.direct=on,cache.no-flush=off,filename=/home/test/win2019-64-virtio-scsi.qcow2,aio=threads \
    -blockdev node-name=drive_image1,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_image1 \
    -device scsi-hd,id=image1,drive=drive_image1,bootindex=0,write-cache=on \
    -vnc :0  \
    -monitor stdio \
    -m 8192 \
    -smp 8 \
    -device pcie-root-port,id=pcie.0-root-port-5,slot=5,chassis=5,addr=0x5,bus=pcie.0 \
    -device virtio-net-pci,mac=9a:b5:b6:b1:b2:b3,id=idMmq1jH,vectors=4,netdev=idxgXAlm,bus=pcie.0-root-port-5,addr=0x0  \
    -netdev tap,id=idxgXAlm \
    -qmp tcp:localhost:5902,server,nowait  \
    -device nec-usb-xhci,id=usb1,bus=pcie.0,addr=0x2 -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \

2. Qmp hotplug a scsi disk to the existing virio-scsi-pci controller
{"execute":"blockdev-add","arguments":{"node-name":"file_image3","driver":"qcow2","file":{"driver":"file","filename":"/home/disk/data.qcow2"}}}
{"return": {}}
{'execute':'device_add','arguments':{'driver':'scsi-hd','drive':'file_image3','bus':'virtio_scsi_pci0.0','id':'test'}}
{"return": {}}

3.In qemu ,disk has been added successfully
(qemu) info block
drive_image1: /home/test/win2019-64-virtio-scsi.qcow2 (qcow2)
    Attached to:      image1
    Cache mode:       writeback, direct

file_image3: /home/disk/data.qcow2 (qcow2)
    Attached to:      test
    Cache mode:       writeback
(qemu) info status 
VM status: running


4. Win 2019 guest hang for a while then bsod.

~                                                                                                                                          
~

Comment 3 yujie ma 2019-05-10 06:41:36 UTC
It works well when hotplug a separate virtio-scsi-pci controller

{"execute":"blockdev-add","arguments":{"node-name":"file_image3","driver":"qcow2","file":{"driver":"file","filename":"/home/disk/data.qcow2"}}}
{"return": {}}
{"execute":"device_add","arguments":{"driver":"virtio-scsi-pci","id":"virtio_scsi_pci1","bus":"pcie.0-root-port-9","addr":"0x0"}}
{"return": {}}
{"execute":"device_add","arguments":{"driver":"scsi-hd","drive":"file_image3","id":"data_disk","bus":"virtio_scsi_pci1.0"}}
{"return": {}}
{"execute":"device_del","arguments":{"id":"data_disk"} }
{"timestamp": {"seconds": 1557470316, "microseconds": 226894}, "event": "DEVICE_DELETED", "data": {"device": "data_disk", "path": "/machine/peripheral/data_disk"}}
{"return": {}}

Comment 4 Xueqiang Wei 2019-05-10 09:08:12 UTC
Hit this issue on qemu 4.0.


Details:

Versions:
kernel-4.18.0-85.el8.x86_64
qemu-kvm-4.0.0-0.module+el8.1.0+3169+3c501422

Guest: windows2019 with virtio-win-prewhql-0.1-171.iso


1. create a 2G data disk
# # qemu-img create -f qcow2 /home/kvm_autotest_root/images/data.qcow2 2G

2. boot guest with below cmd lines.

/usr/libexec/qemu-kvm \
    -S  \
    -name 'avocado-vt-vm1'  \
    -sandbox off  \
    -machine q35  \
    -nodefaults \
    -device VGA,bus=pcie.0,addr=0x1 \
    -device pcie-root-port,id=pcie_root_port_0,slot=2,chassis=2,addr=0x2,bus=pcie.0 \
    -device pcie-root-port,id=pcie_root_port_1,slot=3,chassis=3,addr=0x3,bus=pcie.0 \
    -device pcie-root-port,id=pcie_root_port_2,slot=4,chassis=4,addr=0x4,bus=pcie.0  \
    -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/avocado_w2u90exl/monitor-qmpmonitor1-20181127-024837-wdAVx2FL,server,nowait \
    -mon chardev=qmp_id_qmpmonitor1,mode=control  \
    -chardev socket,id=qmp_id_catch_monitor,path=/var/tmp/avocado_w2u90exl/monitor-catch_monitor-20181127-024837-wdAVx2FL,server,nowait \
    -mon chardev=qmp_id_catch_monitor,mode=control \
    -device pvpanic,ioport=0x505,id=idulvcka  \
    -chardev socket,id=serial_id_serial0,path=/var/tmp/avocado_w2u90exl/serial-serial0-20181127-024837-wdAVx2FL,server,nowait \
    -device isa-serial,chardev=serial_id_serial0  \
    -chardev socket,id=seabioslog_id_20181127-024837-wdAVx2FL,path=/var/tmp/avocado_w2u90exl/seabios-20181127-024837-wdAVx2FL,server,nowait \
    -device isa-debugcon,chardev=seabioslog_id_20181127-024837-wdAVx2FL,iobase=0x402 \
    -device pcie-root-port,id=pcie.0-root-port-5,slot=5,chassis=5,addr=0x5,bus=pcie.0 \
    -device qemu-xhci,id=usb1,bus=pcie.0-root-port-5,addr=0x0 \
    -device pcie-root-port,id=pcie.0-root-port-6,slot=6,chassis=6,addr=0x6,bus=pcie.0 \
    -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie.0-root-port-6,addr=0x0 \
    -blockdev driver=file,cache.direct=on,cache.no-flush=off,filename=/home/kvm_autotest_root/images/win2019-64-virtio-scsi.qcow2,node-name=my_file \
    -blockdev driver=qcow2,node-name=my,file=my_file,cache.direct=on,cache.no-flush=off \
    -device scsi-hd,drive=my,bus=virtio_scsi_pci0.0,write-cache=on \
    -device pcie-root-port,id=pcie.0-root-port-7,slot=7,chassis=7,addr=0x7,bus=pcie.0 \
    -device virtio-net-pci,mac=9a:34:35:36:37:38,id=idyb3F88,vectors=4,netdev=idTAFS0s,bus=pcie.0-root-port-7,addr=0x0  \
    -netdev tap,id=idTAFS0s,vhost=on \
    -m 8G  \
    -smp 12,maxcpus=12,cores=6,threads=1,sockets=2  \
    -cpu 'Opteron_G5',hv_stimer,hv_synic,hv_vpindex,hv_reset,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv-tlbflush,+kvm_pv_unhalt \
    -device pcie-root-port,id=pcie.0-root-port-8,slot=8,chassis=8,addr=0x8,bus=pcie.0 \
    -device virtio-scsi-pci,id=virtio_scsi_pci2,bus=pcie.0-root-port-8,addr=0x0 \
    -blockdev driver=raw,file.driver=file,cache.direct=off,cache.no-flush=on,file.filename=/home/kvm_autotest_root/iso/windows/winutils.iso,node-name=drive2,read-only=on \
    -device scsi-cd,drive=drive2,id=data-disk1,bus=virtio_scsi_pci2.0 \
    -blockdev driver=raw,file.driver=file,cache.direct=off,cache.no-flush=on,file.filename=/home/kvm_autotest_root/iso/windows/virtio-win-prewhql-0.1-171.iso,node-name=drive3,read-only=on \
    -device scsi-cd,drive=drive3,id=data-disk2,bus=virtio_scsi_pci2.0 \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -vnc :0  \
    -rtc base=localtime,clock=host,driftfix=slew  \
    -boot order=cdn,once=c,menu=off,strict=off \
    -enable-kvm \
    -monitor stdio \
    -qmp tcp:0:4444,server,nowait \

3. hotplug the data disk to the same virtio-scsi-pci as system disk.

# telnet localhost 4444

{"execute":"qmp_capabilities"}
{"return": {}}

{"execute": "blockdev-add","arguments": {"node-name":"data_disk","driver":"file","filename":"/home/kvm_autotest_root/images/data.qcow2"}}
{"return": {}}

{"execute": "blockdev-add","arguments": {"node-name": "disk1","driver":"qcow2","file":"data_disk"}}
{"return": {}}

{"execute":"device_add","arguments":{"driver":"scsi-hd","drive":"disk1","id":"data_disk","bus":"virtio_scsi_pci0.0"}}
{"return": {}}

{"timestamp": {"seconds": 1557478522, "microseconds": 129741}, "event": "GUEST_PANICKED", "data": {"action": "pause"}}
{"timestamp": {"seconds": 1557478522, "microseconds": 130007}, "event": "GUEST_PANICKED", "data": {"action": "poweroff"}}
{"timestamp": {"seconds": 1557478522, "microseconds": 130135}, "event": "SHUTDOWN", "data": {"guest": true, "reason": "guest-panic"}}

Comment 5 Xueqiang Wei 2019-05-10 09:37:51 UTC
Tested with drive, not hit this issue.


1. create a 2G data disk
# # qemu-img create -f qcow2 /home/kvm_autotest_root/images/data.qcow2 2G

2. boot guest with drive instead of blockdev.
/usr/libexec/qemu-kvm \
    -S  \
    -name 'avocado-vt-vm1'  \
    -sandbox off  \
    -machine q35  \
    -nodefaults \
    -device VGA,bus=pcie.0,addr=0x1 \
    -device pcie-root-port,id=pcie_root_port_0,slot=2,chassis=2,addr=0x2,bus=pcie.0 \
    -device pcie-root-port,id=pcie_root_port_1,slot=3,chassis=3,addr=0x3,bus=pcie.0 \
    -device pcie-root-port,id=pcie_root_port_2,slot=4,chassis=4,addr=0x4,bus=pcie.0  \
    -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/avocado_w2u90exl/monitor-qmpmonitor1-20181127-024837-wdAVx2FL,server,nowait \
    -mon chardev=qmp_id_qmpmonitor1,mode=control  \
    -chardev socket,id=qmp_id_catch_monitor,path=/var/tmp/avocado_w2u90exl/monitor-catch_monitor-20181127-024837-wdAVx2FL,server,nowait \
    -mon chardev=qmp_id_catch_monitor,mode=control \
    -device pvpanic,ioport=0x505,id=idulvcka  \
    -chardev socket,id=serial_id_serial0,path=/var/tmp/avocado_w2u90exl/serial-serial0-20181127-024837-wdAVx2FL,server,nowait \
    -device isa-serial,chardev=serial_id_serial0  \
    -chardev socket,id=seabioslog_id_20181127-024837-wdAVx2FL,path=/var/tmp/avocado_w2u90exl/seabios-20181127-024837-wdAVx2FL,server,nowait \
    -device isa-debugcon,chardev=seabioslog_id_20181127-024837-wdAVx2FL,iobase=0x402 \
    -device pcie-root-port,id=pcie.0-root-port-5,slot=5,chassis=5,addr=0x5,bus=pcie.0 \
    -device qemu-xhci,id=usb1,bus=pcie.0-root-port-5,addr=0x0 \
    -device pcie-root-port,id=pcie.0-root-port-6,slot=6,chassis=6,addr=0x6,bus=pcie.0 \
    -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie.0-root-port-6,addr=0x0 \
    -drive file=/home/kvm_autotest_root/images/win2019-64-virtio-scsi.qcow2,if=none,format=qcow2,cache=none,werror=stop,rerror=stop,id=drive-virtio-disk0 \
    -device scsi-hd,bus=virtio_scsi_pci0.0,drive=drive-virtio-disk0,id=virtio-disk0 \
    -device pcie-root-port,id=pcie.0-root-port-7,slot=7,chassis=7,addr=0x7,bus=pcie.0 \
    -device virtio-net-pci,mac=9a:34:35:36:37:38,id=idyb3F88,vectors=4,netdev=idTAFS0s,bus=pcie.0-root-port-7,addr=0x0  \
    -netdev tap,id=idTAFS0s,vhost=on \
    -m 8G  \
    -smp 12,maxcpus=12,cores=6,threads=1,sockets=2  \
    -cpu 'Opteron_G5',hv_stimer,hv_synic,hv_vpindex,hv_reset,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv-tlbflush,+kvm_pv_unhalt \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -vnc :0  \
    -rtc base=localtime,clock=host,driftfix=slew  \
    -boot order=cdn,once=c,menu=off,strict=off \
    -enable-kvm \
    -monitor stdio \
    -qmp tcp:0:4444,server,nowait \

3. hotplug the data disk to the same virtio-scsi-pci as system disk.

(qemu) drive_add auto id=drive_stg0,if=none,snapshot=off,aio=threads,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/data.qcow2
OK
(qemu)


# telnet localhost 4444
{"execute":"qmp_capabilities"}
{"return": {}}
{"execute":"device_add","arguments":{"driver":"scsi-hd","drive":"drive_stg0","id":"stg0","bus":"virtio_scsi_pci0.0"}}
{"return": {}}

After step 3, guest works well and the added disk also works well.

Comment 6 Xueqiang Wei 2019-05-10 10:16:14 UTC
Also hit it on qemu-kvm-3.1.0-20.module+el8+2888+cdc893a8, it is not a regression bug.

Comment 9 Vadim Rozenfeld 2019-06-10 08:26:47 UTC
Will it be possible to upload the relevant crash dump file?

Thanks,
Vadim.

Comment 10 Xueqiang Wei 2019-06-10 08:37:38 UTC
(In reply to Vadim Rozenfeld from comment #9)
> Will it be possible to upload the relevant crash dump file?
> 
> Thanks,
> Vadim.

Memory dump info in Comment 1 is not useful?

Comment 11 Vadim Rozenfeld 2019-06-11 02:47:29 UTC
(In reply to Xueqiang Wei from comment #10)
> (In reply to Vadim Rozenfeld from comment #9)
> > Will it be possible to upload the relevant crash dump file?
> > 
> > Thanks,
> > Vadim.
> 
> Memory dump info in Comment 1 is not useful?

Dear  Xueqiang Wei,

For analysing and troubleshooting crashes like CRITICAL_PROCESS_DIED we usually need to see
the entire dump file mostly because the information reported by running "!analyze -v" is
is almost useless in this case. 
Even more. Taking into account that we need to see what happened long before  ntdll!RtlDispatchException
I can ask you to upload several different crash dumps (in case if one will not be enough).

Have a good day,
Vadim.

Comment 12 Xueqiang Wei 2019-06-19 07:04:23 UTC
(In reply to Vadim Rozenfeld from comment #11)
> (In reply to Xueqiang Wei from comment #10)
> > (In reply to Vadim Rozenfeld from comment #9)
> > > Will it be possible to upload the relevant crash dump file?
> > > 
> > > Thanks,
> > > Vadim.
> > 
> > Memory dump info in Comment 1 is not useful?
> 
> Dear  Xueqiang Wei,
> 
> For analysing and troubleshooting crashes like CRITICAL_PROCESS_DIED we
> usually need to see
> the entire dump file mostly because the information reported by running
> "!analyze -v" is
> is almost useless in this case. 
> Even more. Taking into account that we need to see what happened long before
> ntdll!RtlDispatchException
> I can ask you to upload several different crash dumps (in case if one will
> not be enough).
> 
> Have a good day,
> Vadim.


Retested on qemu-kvm-4.0.0-4.module+el8.1.0+3356+cda7f1ee, also hit this issue. 


For the Memory dump file, please refer to below link:

http://fileshare.englab.nay.redhat.com/pub/section2/images_backup/bug1708490/

Comment 18 Ademar Reis 2020-02-05 22:57:46 UTC
QEMU has been recently split into sub-components and as a one-time operation to avoid breakage of tools, we are setting the QEMU sub-component of this BZ to "General". Please review and change the sub-component if necessary the next time you review this BZ. Thanks

Comment 19 John Ferlan 2020-02-06 20:53:19 UTC
NB: Setting back to vrozenfe as it looks like Amnon only meant to change the subcomponent, but didn't notice that doing that causes defaults to reset.

Comment 23 Peixiu Hou 2020-05-09 08:47:22 UTC
Used versions for comment#22 tests:
kernel-4.18.0-193.el8.x86_64
qemu-kvm-4.2.0-19.module+el8.2.0+6296+6b821950.x86_64
seabios-1.13.0-1.module+el8.2.0+5520+4e5817f3.x86_64
virtio-win-prewhql-181

BR~
Peixiu

Comment 24 Vadim Rozenfeld 2020-05-10 13:02:27 UTC
(In reply to Peixiu Hou from comment #23)
> Used versions for comment#22 tests:
> kernel-4.18.0-193.el8.x86_64
> qemu-kvm-4.2.0-19.module+el8.2.0+6296+6b821950.x86_64
> seabios-1.13.0-1.module+el8.2.0+5520+4e5817f3.x86_64
> virtio-win-prewhql-181
> 
> BR~
> Peixiu

Thanks,
Can you please check if adding serial parameter (something like
-device scsi-hd,drive=my1,id=image5,bootindex=0,serial=XXXX)
solves the problem?

Best,
Vadim.

Comment 25 CongLi 2020-05-11 01:14:28 UTC
Hi Qing,

Please handle the needinfo in comment 24.

Thanks.

Comment 26 qing.wang 2020-05-12 08:11:23 UTC
same as https://bugzilla.redhat.com/show_bug.cgi?id=1722710 comment 26

Test on
qemu-kvm-core-4.2.0-20.module+el8.2.1+6467+49dc3278.x86_64
Guest:win2019

Test steps:
1.create images
qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg1.qcow2 1G
  qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg2.qcow2 2G
  qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg3.qcow2 3G
  qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg4.qcow2 4G


2.boot vm
/usr/libexec/qemu-kvm \
  -name 'avocado-vt-vm1' \
  -sandbox on \
  -machine q35 \
  -nodefaults \
  -device VGA,bus=pcie.0,addr=0x2 \
  -m 8096 \
  -smp 12,maxcpus=12,cores=6,threads=1,dies=1,sockets=2 \
  -device pvpanic,ioport=0x505,id=idWW4fRE \
  -device qemu-xhci,id=usb1,bus=pcie.0,addr=0x3 \
  \
  -device pcie-root-port,id=pcie_port_7,slot=7,chassis=7,addr=0x7,bus=pcie.0 \
  -device pcie-root-port,id=pcie_port_8,slot=8,chassis=8,addr=0x8,bus=pcie.0 \
  -object iothread,id=iothread0 \
  -device virtio-scsi-pci,id=scsi0,bus=pcie.0,addr=0x4,iothread=iothread0 \
  -blockdev node-name=file_image1,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/win2019-64-virtio-scsi.qcow2,cache.direct=on,cache.no-flush=off \
  -blockdev node-name=drive_image1,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_image1 \
  -device scsi-hd,id=image1,drive=drive_image1,bootindex=0,write-cache=on,serial=osdisk0 \
  \
  -blockdev node-name=data_image1,driver=file,cache.direct=on,cache.no-flush=off,filename=/home/kvm_autotest_root/images/stg1.qcow2,aio=threads \
  -blockdev node-name=data1,driver=qcow2,cache.direct=on,cache.no-flush=off,file=data_image1 \
  -device scsi-hd,id=disk1,drive=data1,write-cache=on,bus=scsi0.0  \
  \
  -blockdev node-name=data_image2,driver=file,cache.direct=on,cache.no-flush=off,filename=/home/kvm_autotest_root/images/stg2.qcow2,aio=threads \
  -blockdev node-name=data2,driver=qcow2,cache.direct=on,cache.no-flush=off,file=data_image2 \
  -device virtio-blk-pci,id=disk2,drive=data2,write-cache=on,bus=pcie_port_7  \
  \
  -device virtio-net-pci,mac=9a:7f:65:c9:ec:b8,id=idCBhCiy,netdev=id1uqNcV,bus=pcie.0,addr=0x5 \
  -netdev tap,id=id1uqNcV,vhost=on \
  -device virtio-scsi-pci,id=scsi1,bus=pcie.0,addr=0x6 \
  -blockdev node-name=file_cd1,driver=file,read-only=on,aio=threads,filename=/home/kvm_autotest_root/iso/windows/winutils.iso,cache.direct=on,cache.no-flush=off \
  -blockdev node-name=drive_cd1,driver=raw,read-only=on,cache.direct=on,cache.no-flush=off,file=file_cd1 \
  -device scsi-cd,id=cd1,drive=drive_cd1,write-cache=on,bus=scsi1.0 \
  -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
  -vnc :6 \
  -rtc base=localtime,clock=host,driftfix=slew \
  -boot menu=off,order=cdn,once=c,strict=off \
  -enable-kvm -monitor stdio \
  -qmp tcp:0:5956,server,nowait \



3.hotplug disks with serial
{'execute':'qmp_capabilities'}

{"execute":"blockdev-add","arguments":{"node-name":"data3","driver":"qcow2","file":{"driver":"file","filename":"/home/kvm_autotest_root/images/stg3.qcow2"}}}
{"execute":"blockdev-add","arguments":{"node-name":"data4","driver":"qcow2","file":{"driver":"file","filename":"/home/kvm_autotest_root/images/stg4.qcow2"}}}

{'execute':'device_add','arguments':{'driver':'scsi-hd','id':'disk3','drive':'data3','bus':'scsi0.0',"serial":"data3"}}
{"execute": "device_add", "arguments": {"driver": "virtio-blk-pci", "id": "disk4", "drive": "data4", "bus": "pcie_port_8","serial":"data4"}}

wmic diskdrive get index it liss 5 disks


5.hotunplug disks

{"execute":"device_del","arguments":{"id":"disk3"}}
{"execute":"device_del","arguments":{"id":"disk4"}}

wmic diskdrive get index it liss 3 disks

6.hotplug without serial

{'execute':'device_add','arguments':{'driver':'scsi-hd','id':'disk3','drive':'data3','bus':'scsi0.0'}}
{"execute": "device_add", "arguments": {"driver": "virtio-blk-pci", "id": "disk4", "drive": "data4", "bus": "pcie_port_8"}}

wmic diskdrive get index it liss 4 disks, 
It looks like the disk without serial can not be added. (disk1 already exist.)
and the diskmanager windows can not list disks steadily. 
But i did not hit BSOD issue.


So the question is it must specify the serias for the scsi disk? 
If it is not specified . it can be fixed using id as serail patentially. or it need to document it?

Comment 27 Vadim Rozenfeld 2020-05-12 08:50:47 UTC
(In reply to qing.wang from comment #26)
> same as https://bugzilla.redhat.com/show_bug.cgi?id=1722710 comment 26
> 
> Test on
> qemu-kvm-core-4.2.0-20.module+el8.2.1+6467+49dc3278.x86_64
> Guest:win2019
> 
> Test steps:
> 1.create images
> qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg1.qcow2 1G
>   qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg2.qcow2 2G
>   qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg3.qcow2 3G
>   qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg4.qcow2 4G
> 
> 
> 2.boot vm
> /usr/libexec/qemu-kvm \
>   -name 'avocado-vt-vm1' \
>   -sandbox on \
>   -machine q35 \
>   -nodefaults \
>   -device VGA,bus=pcie.0,addr=0x2 \
>   -m 8096 \
>   -smp 12,maxcpus=12,cores=6,threads=1,dies=1,sockets=2 \
>   -device pvpanic,ioport=0x505,id=idWW4fRE \
>   -device qemu-xhci,id=usb1,bus=pcie.0,addr=0x3 \
>   \
>   -device pcie-root-port,id=pcie_port_7,slot=7,chassis=7,addr=0x7,bus=pcie.0
> \
>   -device pcie-root-port,id=pcie_port_8,slot=8,chassis=8,addr=0x8,bus=pcie.0
> \
>   -object iothread,id=iothread0 \
>   -device virtio-scsi-pci,id=scsi0,bus=pcie.0,addr=0x4,iothread=iothread0 \
>   -blockdev
> node-name=file_image1,driver=file,aio=threads,filename=/home/
> kvm_autotest_root/images/win2019-64-virtio-scsi.qcow2,cache.direct=on,cache.
> no-flush=off \
>   -blockdev
> node-name=drive_image1,driver=qcow2,cache.direct=on,cache.no-flush=off,
> file=file_image1 \
>   -device
> scsi-hd,id=image1,drive=drive_image1,bootindex=0,write-cache=on,
> serial=osdisk0 \
>   \
>   -blockdev
> node-name=data_image1,driver=file,cache.direct=on,cache.no-flush=off,
> filename=/home/kvm_autotest_root/images/stg1.qcow2,aio=threads \
>   -blockdev
> node-name=data1,driver=qcow2,cache.direct=on,cache.no-flush=off,
> file=data_image1 \
>   -device scsi-hd,id=disk1,drive=data1,write-cache=on,bus=scsi0.0  \
>   \
>   -blockdev
> node-name=data_image2,driver=file,cache.direct=on,cache.no-flush=off,
> filename=/home/kvm_autotest_root/images/stg2.qcow2,aio=threads \
>   -blockdev
> node-name=data2,driver=qcow2,cache.direct=on,cache.no-flush=off,
> file=data_image2 \
>   -device virtio-blk-pci,id=disk2,drive=data2,write-cache=on,bus=pcie_port_7
> \
>   \
>   -device
> virtio-net-pci,mac=9a:7f:65:c9:ec:b8,id=idCBhCiy,netdev=id1uqNcV,bus=pcie.0,
> addr=0x5 \
>   -netdev tap,id=id1uqNcV,vhost=on \
>   -device virtio-scsi-pci,id=scsi1,bus=pcie.0,addr=0x6 \
>   -blockdev
> node-name=file_cd1,driver=file,read-only=on,aio=threads,filename=/home/
> kvm_autotest_root/iso/windows/winutils.iso,cache.direct=on,cache.no-
> flush=off \
>   -blockdev
> node-name=drive_cd1,driver=raw,read-only=on,cache.direct=on,cache.no-
> flush=off,file=file_cd1 \
>   -device scsi-cd,id=cd1,drive=drive_cd1,write-cache=on,bus=scsi1.0 \
>   -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
>   -vnc :6 \
>   -rtc base=localtime,clock=host,driftfix=slew \
>   -boot menu=off,order=cdn,once=c,strict=off \
>   -enable-kvm -monitor stdio \
>   -qmp tcp:0:5956,server,nowait \
> 
> 
> 
> 3.hotplug disks with serial
> {'execute':'qmp_capabilities'}
> 
> {"execute":"blockdev-add","arguments":{"node-name":"data3","driver":"qcow2",
> "file":{"driver":"file","filename":"/home/kvm_autotest_root/images/stg3.
> qcow2"}}}
> {"execute":"blockdev-add","arguments":{"node-name":"data4","driver":"qcow2",
> "file":{"driver":"file","filename":"/home/kvm_autotest_root/images/stg4.
> qcow2"}}}
> 
> {'execute':'device_add','arguments':{'driver':'scsi-hd','id':'disk3','drive':
> 'data3','bus':'scsi0.0',"serial":"data3"}}
> {"execute": "device_add", "arguments": {"driver": "virtio-blk-pci", "id":
> "disk4", "drive": "data4", "bus": "pcie_port_8","serial":"data4"}}
> 
> wmic diskdrive get index it liss 5 disks
> 
> 
> 5.hotunplug disks
> 
> {"execute":"device_del","arguments":{"id":"disk3"}}
> {"execute":"device_del","arguments":{"id":"disk4"}}
> 
> wmic diskdrive get index it liss 3 disks
> 
> 6.hotplug without serial
> 
> {'execute':'device_add','arguments':{'driver':'scsi-hd','id':'disk3','drive':
> 'data3','bus':'scsi0.0'}}
> {"execute": "device_add", "arguments": {"driver": "virtio-blk-pci", "id":
> "disk4", "drive": "data4", "bus": "pcie_port_8"}}
> 
> wmic diskdrive get index it liss 4 disks, 
> It looks like the disk without serial can not be added. (disk1 already
> exist.)
> and the diskmanager windows can not list disks steadily. 
> But i did not hit BSOD issue.
> 
> 
> So the question is it must specify the serias for the scsi disk? 
> If it is not specified . it can be fixed using id as serail patentially. or
> it need to document it?


The problem is that Windows 10 rely on information revealed by reporting
VPD 0x83 proper information. serial helps to solve this problem. From the
other side I tried to create some sort of workaround when even without 
serial we still will be reporting a valid unique identification.  

https://github.com/virtio-win/kvm-guest-drivers-windows/commit/b57548c769ed9f431c34f6449ce432dd077cb02e

Comment 28 Vadim Rozenfeld 2020-05-12 22:16:17 UTC
can you please re-check this bug with the latest virtio-win build (b183)
https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=1191255

Thanks,
Vadim.

Comment 29 qing.wang 2020-05-13 10:51:01 UTC
Test steps:
1.create images
qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg1.qcow2 1G
  qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg2.qcow2 2G
  qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg3.qcow2 3G
  qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg4.qcow2 4G


2.boot vm
/usr/libexec/qemu-kvm \
  -name 'avocado-vt-vm1' \
  -sandbox on \
  -machine q35 \
  -nodefaults \
  -device VGA,bus=pcie.0,addr=0x2 \
  -m 8096 \
  -smp 12,maxcpus=12,cores=6,threads=1,dies=1,sockets=2 \
  -device pvpanic,ioport=0x505,id=idWW4fRE \
  -device qemu-xhci,id=usb1,bus=pcie.0,addr=0x3 \
  \
  -device pcie-root-port,id=pcie_port_7,slot=7,chassis=7,addr=0x7,bus=pcie.0 \
  -device pcie-root-port,id=pcie_port_8,slot=8,chassis=8,addr=0x8,bus=pcie.0 \
  -object iothread,id=iothread0 \
  -device virtio-scsi-pci,id=scsi0,bus=pcie.0,addr=0x4,iothread=iothread0 \
  -blockdev node-name=file_image1,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/win2019-64-virtio-scsi.qcow2,cache.direct=on,cache.no-flush=off \
  -blockdev node-name=drive_image1,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_image1 \
  -device scsi-hd,id=image1,drive=drive_image1,bootindex=0,write-cache=on  \
  \
  -blockdev node-name=data_image1,driver=file,cache.direct=on,cache.no-flush=off,filename=/home/kvm_autotest_root/images/stg1.qcow2,aio=threads \
  -blockdev node-name=data1,driver=qcow2,cache.direct=on,cache.no-flush=off,file=data_image1 \
  -device scsi-hd,id=disk1,drive=data1,write-cache=on,bus=scsi0.0  \
  \
  -blockdev node-name=data_image2,driver=file,cache.direct=on,cache.no-flush=off,filename=/home/kvm_autotest_root/images/stg2.qcow2,aio=threads \
  -blockdev node-name=data2,driver=qcow2,cache.direct=on,cache.no-flush=off,file=data_image2 \
  -device virtio-blk-pci,id=disk2,drive=data2,write-cache=on,bus=pcie_port_7  \
  \
  -device virtio-net-pci,mac=9a:7f:65:c9:ec:b8,id=idCBhCiy,netdev=id1uqNcV,bus=pcie.0,addr=0x5 \
  -netdev tap,id=id1uqNcV,vhost=on \
  -device virtio-scsi-pci,id=scsi1,bus=pcie.0,addr=0x6 \
  -blockdev node-name=file_cd1,driver=file,read-only=on,aio=threads,filename=/home/kvm_autotest_root/iso/windows/winutils.iso,cache.direct=on,cache.no-flush=off \
  -blockdev node-name=drive_cd1,driver=raw,read-only=on,cache.direct=on,cache.no-flush=off,file=file_cd1 \
  -device scsi-cd,id=cd1,drive=drive_cd1,write-cache=on,bus=scsi1.0 \
  -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
  -vnc :6 \
  -rtc base=localtime,clock=host,driftfix=slew \
  -boot menu=off,order=cdn,once=c,strict=off \
  -enable-kvm -monitor stdio \
  -qmp tcp:0:5956,server,nowait \



3.hotplug disks without serial
{'execute':'qmp_capabilities'}

{"execute":"blockdev-add","arguments":{"node-name":"data3","driver":"qcow2","file":{"driver":"file","filename":"/home/kvm_autotest_root/images/stg3.qcow2"}}}
{"execute":"blockdev-add","arguments":{"node-name":"data4","driver":"qcow2","file":{"driver":"file","filename":"/home/kvm_autotest_root/images/stg4.qcow2"}}}

{'execute':'device_add','arguments':{'driver':'scsi-hd','id':'disk3','drive':'data3','bus':'scsi0.0' }}
{"execute": "device_add", "arguments": {"driver": "virtio-blk-pci", "id": "disk4", "drive": "data4", "bus": "pcie_port_8" }}

wmic diskdrive get index it liss 5 disks


5.hotunplug disks

{"execute":"device_del","arguments":{"id":"disk3"}}
{"execute":"device_del","arguments":{"id":"disk4"}}

wmic diskdrive get index it liss 3 disks

6.hotplug without serial

{'execute':'device_add','arguments':{'driver':'scsi-hd','id':'disk3','drive':'data3','bus':'scsi0.0'}}
{"execute": "device_add", "arguments": {"driver": "virtio-blk-pci", "id": "disk4", "drive": "data4", "bus": "pcie_port_8"}}


wmic diskdrive get index it liss 5 disks

It looks good for me.

Comment 30 qing.wang 2020-05-14 06:14:14 UTC
Tested on virtio-win-prewhql-0.1-184

Comment 31 menli@redhat.com 2020-05-14 06:29:54 UTC
move it to virtio-win due to fix in driver

Comment 33 Amnon Ilan 2020-05-21 08:31:58 UTC
*** Bug 1722710 has been marked as a duplicate of this bug. ***

Comment 36 errata-xmlrpc 2020-07-21 15:32:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2020:3055