Bug 1139562

Summary: qemu-kvm with vhost=off and sndbuf=100 crashed when stop it during pktgen test from guest to host
Product: Red Hat Enterprise Linux 7 Reporter: Qian Guo <qiguo>
Component: qemu-kvmAssignee: Amnon Ilan <ailan>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.1CC: chayang, hhuang, huding, jasowang, jen, juzhang, knoel, lmiksik, michen, mkenneth, qiguo, rbalakri, rpacheco, virt-bugs, virt-maint
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: qemu-kvm-1.5.3-87.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1138228
: 1139564 (view as bug list) Environment:
Last Closed: 2015-11-19 04:56:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1138228    
Bug Blocks: 1139564    

Comment 1 Qian Guo 2014-09-09 08:26:24 UTC
Can reproduce this bug with qemu-kvm-1.5.3-69.el7.x86_64 and qemu-kvm-rhev-2.1.0-3.el7.x86_64, so I cloned this bug to RHEL7.1 components, and please refer to bug 1138228 comment 4 for more detail infos.

Comment 3 Ronen Hod 2014-10-27 09:46:55 UTC
Deferring to 7.2
Not a real-life use case. Guests should not be stopped (like a stuck hardware).
See also bug 1100656

Comment 4 jason wang 2014-10-29 03:35:20 UTC
Note: Upstream have fixed this issue.

Comment 8 Miroslav Rezanina 2015-03-18 11:24:06 UTC
Fix included in qemu-kvm-1.5.3-87.el7

Comment 9 huiqingding 2015-03-25 05:57:25 UTC
Reproduce this bug using the following version:
kernel-3.10.0-232.el7.x86_64
qemu-kvm-1.5.3-86.el7.x86_64

Steps to Reproduce:
1.Boot guest with sndbuf=100 and vhost=off
# /usr/libexec/qemu-kvm -cpu Penryn -enable-kvm -m 4G -smp 4,sockets=1,cores=4,threads=1 -name test -rtc base=localtime,clock=host,driftfix=slew  -k en-us  -boot menu=on -vnc :3 -vga cirrus -usb -device usb-tablet -monitor stdio -drive file=/home/rhel7.1.qcow2,if=none,id=drive-system-disk,media=disk,format=qcow2,aio=native,werror=stop,rerror=stop -device virtio-blk-pci,drive=drive-system-disk,id=system-disk,addr=0x3 -qmp unix:/tmp/q1,server,nowait  -netdev tap,id=hostnet0,vhost=off,sndbuf=100,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,id=vnet0,mac=52:54:00:1a:2b:01

2.In guest, start pktgen test to host
# sh pktgen.sh eth0 1

3.stop guest under hmp
(qemu) stop

Actual results:
qemu crash:
(qemu) qemu-kvm: /builddir/build/BUILD/qemu-1.5.3/hw/net/virtio-net.c:1028: virtio_net_flush_tx: Assertion `vdev->vm_running' failed.

(gdb) bt
#0  0x00007ffff18135d7 in raise () from /lib64/libc.so.6
#1  0x00007ffff1814cc8 in abort () from /lib64/libc.so.6
#2  0x00007ffff180c546 in __assert_fail_base () from /lib64/libc.so.6
#3  0x00007ffff180c5f2 in __assert_fail () from /lib64/libc.so.6
#4  0x000055555574f0e6 in virtio_net_flush_tx (q=0x55555671ccc0) at /usr/src/debug/qemu-1.5.3/hw/net/virtio-net.c:1028
#5  0x00005555556af044 in qemu_net_queue_flush (queue=0x5555564c3fd0) at net/queue.c:263
#6  0x00005555556ad845 in qemu_flush_queued_packets (nc=<optimized out>) at net/net.c:446
#7  0x000055555569f1ee in qemu_iohandler_poll (pollfds=0x5555564c0200, ret=20164, ret@entry=3) at iohandler.c:147
#8  0x00005555556a3768 in main_loop_wait (nonblocking=<optimized out>) at main-loop.c:465
#9  0x00005555555c7340 in main_loop () at vl.c:1987
#10 main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4378

Comment 10 huiqingding 2015-03-25 06:02:32 UTC
Test this bug using the following version:
kernel-3.10.0-232.el7.x86_64
qemu-kvm-1.5.3-86.el7.x86_64

The test steps are same as comment 9. The test result is pass, qemu does not crash, guest and host work well.

Comment 13 Qian Guo 2015-06-15 08:05:03 UTC
Reproduced this bug with qemu-kvm-1.5.3-86.el7.x86_64

steps 
1. boot guest 
# /usr/libexec/qemu-kvm -cpu Penryn -enable-kvm -m 4G -smp 4,sockets=1,cores=4,threads=1 -name test -rtc base=localtime,clock=host,driftfix=slew  -k en-us  -boot menu=on -vnc :3 -vga cirrus -usb -device usb-tablet -monitor stdio -drive file=/home/qiguo/RHEL-Server-7.2-64-2.qcow2,if=none,id=drive-system-disk,media=disk,format=qcow2,aio=native,werror=stop,rerror=stop -device virtio-blk-pci,drive=drive-system-disk,id=system-disk,addr=0x3 -qmp unix:/tmp/q1,server,nowait  -netdev tap,id=hostnet0,vhost=off,sndbuf=100,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,id=vnet0,mac=52:54:00:1a:2b:01


2.In guest, do pktgen test
# cat pktgen.sh
...
    pgset "dst $host_ip"
    pgset "dst_mac $different_mac from host"
...

# ./pktgen.sh eth0

3. stop guest via hmp:
QEMU 1.5.3 monitor - type 'help' for more information
(qemu) 
(qemu) stop

Result, qemu coredumped

(qemu) qemu-kvm: /builddir/build/BUILD/qemu-1.5.3/hw/net/virtio-net.c:1028: virtio_net_flush_tx: Assertion `vdev->vm_running' failed.
Aborted (core dumped)


Verify this bug with qemu-kvm-1.5.3-90.el7.x86_64

steps as above,

Result: after stopped, qemu did not hit coredumped, repeated times, works fine.

So according to above, this bug is fixed.

steps as above

Result: g

Comment 15 errata-xmlrpc 2015-11-19 04:56:27 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2213.html