Bug 1363938

Summary: qemu aborted after enter "q" to hmp:virtio-scsi.c:543: virtio_scsi_handle_cmd_req_prepare: Assertion `blk_get_aio_context(d->conf.blk) == s->ctx' failed
Product: Red Hat Enterprise Linux 7 Reporter: Zhengtong <zhengtli>
Component: qemu-kvm-rhevAssignee: Fam Zheng <famz>
Status: CLOSED ERRATA QA Contact: Xueqiang Wei <xuwei>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.3CC: aliang, chayang, coli, hachen, juzhang, knoel, meyang, michen, mrezanin, ngu, pingl, virt-maint, xfu, xuwei, zhengtli
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Unspecified   
Whiteboard:
Fixed In Version: QEMU 2.8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-08-01 23:32:13 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Zhengtong 2016-08-04 04:29:47 UTC
Description of problem:
After guest boot up, input "q" to hmp of the guest. qemu will be aborted.
 

Version-Release number of selected component (if applicable):

qemu-kvm-rhev-2.6.0-17.el7
3.10.0-483.el7.x86_64

How reproducible:
~ 1/7
I have reproduced 4~5 times in my test.

Steps to Reproduce:
1.Boot up guest
2.After guest boot up. enter "q" to hmp
3.Observe the result

Actual results:
qemu aborted
(qemu) q
qemu-kvm: /builddir/build/BUILD/qemu-2.6.0/hw/scsi/virtio-scsi.c:543: virtio_scsi_handle_cmd_req_prepare: Assertion `blk_get_aio_context(d->conf.blk) == s->ctx' failed.

Program received signal SIGABRT, Aborted.


Expected results:
qemu quit normally

Additional info:
guest boot cmd:
/usr/libexec/qemu-kvm \
    -name 'avocado-vt-vm1'  \
    -sandbox off  \
    -nodefaults  \
    -vga std  \
    -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20160517-065527-hNEWQqpC,server,nowait \
    -mon chardev=qmp_id_qmpmonitor1,mode=control  \
    -chardev socket,id=qmp_id_catch_monitor,path=/tmp/monitor-catch_monitor-20160517-065527-hNEWQqpC,server,nowait \
    -mon chardev=qmp_id_catch_monitor,mode=control  \
    -device ich9-usb-uhci1,id=usb1,bus=pci.0,addr=03 \
    -object iothread,id=iothread0 \
    -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=04,disable-legacy=off,disable-modern=on,iothread=iothread0 \
    -drive id=drive_image1,if=none,cache=none,snapshot=on,format=qcow2,file=/home/test/RHEL-Server-7.3-64-virtio-scsi.qcow2 \
    -device scsi-hd,id=image1,drive=drive_image1 \
    -device virtio-net-pci,mac=9a:3b:3c:3d:3e:3f,id=idzvFbfi,netdev=idTuGYjU,bus=pci.0,addr=05,disable-legacy=off,disable-modern=on  \
    -netdev tap,id=idTuGYjU,vhost=on,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown \
    -m 512  \
    -smp 16 \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -vnc :0  \
    -rtc base=utc,clock=host  \
    -boot order=cdn,once=c,menu=off,strict=off \
    -no-shutdown \
    -device usb-kbd \
    -monitor stdio \
    -enable-kvm \

Comment 3 Fam Zheng 2017-02-07 08:58:40 UTC
I expect this is fixed in QEMU 2.8:

commit dce8921b2baaf95974af8176406881872067adfa
Author: Fam Zheng <famz>
Date:   Thu Sep 8 17:28:51 2016 +0800

    iothread: Stop threads before main() quits
    
    Right after main_loop ends, we release various things but keep iothread
    alive. The latter is not prepared to the sudden change of resources.
    
    Specifically, after bdrv_close_all(), virtio-scsi dataplane get a
    surprise at the empty BlockBackend:
    
    (gdb) bt
        at /usr/src/debug/qemu-2.6.0/hw/scsi/virtio-scsi.c:543
        at /usr/src/debug/qemu-2.6.0/hw/scsi/virtio-scsi.c:577
    
    It is because the d->conf.blk->root is set to NULL, then
    blk_get_aio_context() returns qemu_aio_context, whereas s->ctx is still
    pointing to the iothread:
    
        hw/scsi/virtio-scsi.c:543:
    
        if (s->dataplane_started) {
            assert(blk_get_aio_context(d->conf.blk) == s->ctx);
        }
    
    To fix this, let's stop iothreads before doing bdrv_close_all().
    
    Cc: qemu-stable
    Signed-off-by: Fam Zheng <famz>
    Reviewed-by: Paolo Bonzini <pbonzini>
    Message-id: 1473326931-9699-1-git-send-email-famz
    Signed-off-by: Stefan Hajnoczi <stefanha>

Comment 4 Zhengtong 2017-02-08 02:39:56 UTC
Yes , right.
I tried with package from qemu-kvm-rhev-2.6.0-27 to qemu-kvm-rhev-2.8.0-3 which include the commit above. the issue disappear. I think this fixed the problem.

Comment 7 Xueqiang Wei 2017-02-14 06:33:04 UTC
Reproduce it on: 
kernel-3.10.0-514.2.2.el7.x86_64
qemu-kvm-rhev-2.6.0-22.el7

(qemu) q
qemu-kvm: /builddir/build/BUILD/qemu-2.6.0/hw/scsi/virtio-scsi.c:543: virtio_scsi_handle_cmd_req_prepare: Assertion `blk_get_aio_context(d->conf.blk) == s->ctx' failed.


Tested on: 
kernel-3.10.0-514.2.2.el7.x86_64
qemu-kvm-rhev-2.8.0-4.el7, the result is passed. So verify it.

# sh test.sh 
QEMU 2.8.0 monitor - type 'help' for more information
(qemu) q

# sh test.sh 
QEMU 2.8.0 monitor - type 'help' for more information
(qemu) q

Comment 9 errata-xmlrpc 2017-08-01 23:32:13 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392

Comment 10 errata-xmlrpc 2017-08-02 01:09:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392

Comment 11 errata-xmlrpc 2017-08-02 02:01:51 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392

Comment 12 errata-xmlrpc 2017-08-02 02:42:37 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392

Comment 13 errata-xmlrpc 2017-08-02 03:07:20 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392

Comment 14 errata-xmlrpc 2017-08-02 03:27:29 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392