Bug 1312289

Summary: "qemu-kvm: /builddir/build/BUILD/qemu-1.5.3/hw/scsi/virtio-scsi.c:533: virtio_scsi_push_event: Assertion `event == 0' failed" after hotplug 20 virtio-scsi disks then hotunplug them
Product: Red Hat Enterprise Linux 7 Reporter: Gu Nini <ngu>
Component: qemu-kvmAssignee: Fam Zheng <famz>
Status: CLOSED ERRATA QA Contact: FuXiangChun <xfu>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.3CC: chayang, famz, huding, jinzhao, juzhang, knoel, meyang, ngu, pbonzini, pingl, rbalakri, shuang, virt-maint, xfu, xiagao, xutian, xuwei
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: qemu-kvm-1.5.3-112.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-11-03 20:10:23 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
The gdb debug info none

Description Gu Nini 2016-02-26 10:38:16 UTC
Created attachment 1130813 [details]
The gdb debug info

Description of problem:
After hot plug 20 virtio-scsi disks then hot unplug them, the guest crashes for core dump.

Version-Release number of selected component (if applicable):
Host kernel: 3.10.0-351.el7.x86_64
Guest kernel: 3.10.0-351.el7.x86_64
Qemu-kvm: qemu-kvm-1.5.3-108.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Start a RHEL7.3 guest with following cmd:

/usr/libexec/qemu-kvm \
     -name 'virt-tests-vm1'  \
     -sandbox off  \
     -machine pc  \
     -nodefaults  \
     -vga cirrus  \
     -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor-qmp,server,nowait \
     -mon chardev=qmp_id_qmpmonitor1,mode=readline  \
     -chardev socket,id=qmp_id_catch_monitor,path=/tmp/monitor-catch_monitor-20160226-143126-yPq2dhAK,server,nowait \
     -mon chardev=qmp_id_catch_monitor,mode=control \
     -device pvpanic,ioport=0x505,id=idt1gEue  \
     -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20160226-143126-yPq2dhAK,server,nowait \
     -device isa-serial,chardev=serial_id_serial0  \
     -chardev socket,id=seabioslog_id_20160226-143126-yPq2dhAK,path=/tmp/seabios-20160226-143126-yPq2dhAK,server,nowait \
     -device isa-debugcon,chardev=seabioslog_id_20160226-143126-yPq2dhAK,iobase=0x402 \
     -device ich9-usb-ehci1,id=usb1,addr=1d.7,multifunction=on,bus=pci.0 \
     -device ich9-usb-uhci1,id=usb1.0,multifunction=on,masterbus=usb1.0,addr=1d.0,firstport=0,bus=pci.0 \
     -device ich9-usb-uhci2,id=usb1.1,multifunction=on,masterbus=usb1.0,addr=1d.2,firstport=2,bus=pci.0 \
     -device ich9-usb-uhci3,id=usb1.2,multifunction=on,masterbus=usb1.0,addr=1d.4,firstport=4,bus=pci.0 \
     -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=03 \
     -drive id=drive_image1,if=none,cache=none,snapshot=off,aio=native,format=qcow2,file=/home/devel/autotest/client/tests/virt/shared/data/images/RHEL-Server-7.3-64-virtio-scsi.qcow2 \
     -device scsi-hd,id=image1,drive=drive_image1,bootindex=0 \
     -device virtio-net-pci,mac=9a:0d:0e:0f:10:11,id=idtH17bb,vectors=4,netdev=idZ9uD3D,bus=pci.0,addr=04  \
     -netdev tap,id=idZ9uD3D,vhost=on  \
    -m 32768  \
     -smp 64,maxcpus=64,cores=32,threads=1,sockets=2  \
     -cpu 'SandyBridge',+kvm_pv_unhalt \
     -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
     -vnc :0  \
     -rtc base=utc,clock=host,driftfix=slew  \
     -boot order=cdn,once=c,menu=off,strict=off \
     -enable-kvm \
     -monitor stdio

2. After the guest boots up, try to hot plug 20 virtio-scsi disks with following script:

#!/bin/bash
i=0
for i in {0..19}
qemu-img create -f qcow2 /home/devel/autotest/client/tests/virt/shared/data/images/stg$i 20G
do
echo "__com.redhat_drive_add file=/home/devel/autotest/client/tests/virt/shared/data/images/stg$i,format=qcow2,id=drive_stg$i" | nc -U /tmp/monitor-qmp
echo "device_add scsi-hd,bus=virtio_scsi_pci0.0,drive=drive_stg$i,id=stg$i" | nc -U /tmp/monitor-qmp
i=$(($i+1))
done

3. Check the hot plugged disks inside the guest with cmd 'll /dev/sd*'

4. Try to hot unplug the disks with following script:

#!/bin/bash
i=0
for i in {0..19}
do
echo "device_del stg$i" | nc -U /tmp/monitor-qmp
i=$(($i+1))
done


Actual results:
After step4, the guest crashed with following error info:

(qemu) qemu-kvm: /builddir/build/BUILD/qemu-1.5.3/hw/scsi/virtio-scsi.c:533: virtio_scsi_push_event: Assertion `event == 0' failed.
Aborted (core dumped)


Expected results:
After step4, the guest could be hot unplugged without any problem


Additional info:
Failed to reproduce the bug on current RHEL7.2z host with qemu-kvm-rhev:
Host kernel: 3.10.0-327.12.1.el7.x86_64
Guest kernel: 3.10.0-327.el7.x86_64 & 3.10.0-351.el7.x86_64
Qemu-kvm: qemu-kvm-rhev-2.3.0-31.el7_2.8.x86_64

While could reproduce it on the RHEL7.2  released host with qemu-kvm:
Host kernel: 3.10.0-327.el7.x86_64
Guest kernel: 3.10.0-327.el7.x86_64
Qemu-kvm: qemu-kvm-1.5.3.105.el7.x86_64

Comment 2 Gu Nini 2016-02-26 10:58:59 UTC
The core file is core.11560 under nfs dir 10.73.194.27:/vol/s2coredump/test_result

Comment 3 Paolo Bonzini 2016-02-26 13:59:22 UTC
Fixed by upstream commit 49fb65c7f985baa56d2964e0a85c1f098e3e2a9d.

Comment 4 Ademar Reis 2016-04-14 15:18:50 UTC
(In reply to Paolo Bonzini from comment #3)
> Fixed by upstream commit 49fb65c7f985baa56d2964e0a85c1f098e3e2a9d.

Hopefully a simple backport.

Comment 9 Miroslav Rezanina 2016-05-16 08:06:43 UTC
Fix included in qemu-kvm-1.5.3-112.el7

Comment 11 FuXiangChun 2016-09-05 05:12:00 UTC
According to comment0.

1.reproduced bug with qemu-kvm-1.5.3-105. 

qemu-kvm: /builddir/build/BUILD/qemu-1.5.3/hw/scsi/virtio-scsi.c:533: virtio_scsi_push_event: Assertion `event == 0' failed.
cli: line 33: 26063 Aborted 


2.verified this bug with qemu-kvm-1.5.3-122.el7.x86_64 & 3.10.0-493.el7.x86_64
guest works

Comment 13 errata-xmlrpc 2016-11-03 20:10:23 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-2585.html