Red Hat Bugzilla – Bug 843797
qemu-kvm core dumps when virtio-net(w/ tx=timer and vhost=on) RHEL.6(w/ msi-x enabled) guest shutting down
Last modified: 2015-05-24 20:06:54 EDT
Description of problem: qemu-kvm core dumps when virtio-net(w/ tx=timer and vhost=on) RHEL.6(w/ msi-x enabled) guest shutting down Version-Release number of selected component (if applicable): qemu-kvm-rhev-0.12.1.2-2.298.el6_3.x86_64 2.6.32-279.el6.x86_64 How reproducible: 100% Steps to Reproduce: 1. Boot a virtio-net (-device virtio-net-pci,tx=timer -netdev tap,vhost=on) with RHEL guest with MSI-X enabled(eg: RHEL.6.3) qemu-kvm -monitor stdio -nodefaults -chardev socket,id=serial_id_20120726-200643-igwg,path=/tmp/serial-20120726-200643-igwg,server,nowait -device isa-serial,chardev=serial_id_20120726-200643-igwg -device ich9-usb-uhci1,id=usb1,bus=pci.0,addr=0x4 -drive file='/home/staf-kvm-devel/autotest-devel/client/tests/kvm/images/RHEL-Server-6.3-64-virtio.qcow2',if=none,id=drive-virtio-disk1,media=disk,cache=none,boot=off,snapshot=off,format=qcow2,aio=native -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk1,id=virtio-disk1 -m 4096 -smp 2,cores=1,threads=1,sockets=2 -cpu 'Penryn' -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 -vnc :0 -vga std -rtc base=utc,clock=host,driftfix=slew -M rhel6.3.0 -boot order=cdn,once=c,menu=off -no-kvm-pit-reinjection -bios /usr/share/seabios/bios-pm.bin -enable-kvm -S \ \ \ \ -device virtio-net-pci,netdev=idBO9VqI,mac=9a:7a:eb:5b:bd:17,id=ndev00idBO9VqI,bus=pci.0,addr=0x3,tx=timer \ \ -netdev tap,id=idBO9VqI,vhost=on 2. shutdown the guest by typing 'poweroff' 3. Actual results: qemu-kvm core dumps Expected results: guest shutdown successfully, not qemu-kvm core dump Additional info: 1) booting && shutting down a MSI-X ENABLED (vhost=on ) W2k8r2 guest Works well 2) booting && shutting down a MSI-X ENABLED (vhost=off) RHEL.6 guest Works well 3) booting && shutting down a MSI-X DISABLED(vhost=on ) RHEL.6 guest Works well
Created attachment 600751 [details] guest serial output
Created attachment 601135 [details] gdb detail output detail gdb info attached, but for whom'd like to have quick glance (gdb) bt #0 0x00007f1e44caff67 in qemu_mod_timer (ts=0x7f1e45e0ed70, expire_time=71917049194) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:1284 #1 0x00007f1e44cc5e55 in virtio_net_handle_tx_timer (vdev=<value optimized out>, vq=0x7f1e4721e740) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/virtio-net.c:756 #2 0x00007f1e44cc62e0 in virtio_pci_set_host_notifier_internal (proxy=0x7f1e45e13010, n=1, assign=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/virtio-pci.c:224 #3 0x00007f1e44cca821 in vhost_dev_disable_notifiers (hdev=0x7f1e45c52f40, vdev=0x7f1e472125c0) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/vhost.c:677 #4 0x00007f1e44cc9fac in vhost_net_stop (net=0x7f1e45c52f40, dev=0x7f1e472125c0) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/vhost_net.c:202 #5 0x00007f1e44cc44eb in virtio_net_set_status (vdev=0x7f1e472125c0, status=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/virtio-net.c:133 #6 0x00007f1e44d0f65d in qemu_del_vlan_client (vc=0x7f1e45c41880) at net.c:329 #7 0x00007f1e44d0f6d9 in net_cleanup () at net.c:1358 #8 0x00007f1e44cb2297 in main (argc=20, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:6432
Michael, is the combination of timer and vhost=on is relevant? IIRC we should just fail it.
Talked with mst, reassign this to me.
re-test this issue with fixed qemu-kvm-0.12.1.2-2.376.el6.x86_64 still get the same result with comment 2. (gdb) bt #0 0x00007ffff7ddf9a7 in qemu_mod_timer (ts=0x7ffff9c99fa0, expire_time=59258125675) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:1286 #1 0x00007ffff7df60a5 in virtio_net_handle_tx_timer (vdev=0x7ffff9caf860, vq=0x7ffff9cbb9e0) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/virtio-net.c:752 #2 0x00007ffff7df6520 in virtio_pci_set_host_notifier_internal (proxy=0x7ffff88af5e0, n=1, assign=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/virtio-pci.c:224 #3 0x00007ffff7dface1 in vhost_dev_disable_notifiers (hdev=0x7ffff86ef0e0, vdev=0x7ffff9caf860) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/vhost.c:677 #4 0x00007ffff7dfa46c in vhost_net_stop (net=0x7ffff86ef0e0, dev=0x7ffff9caf860) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/vhost_net.c:202 #5 0x00007ffff7df472b in virtio_net_set_status (vdev=0x7ffff9caf860, status=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/virtio-net.c:133 #6 0x00007ffff7e448ad in qemu_del_vlan_client (vc=0x7ffff86dda20) at /usr/src/debug/qemu-kvm-0.12.1.2/net.c:329 #7 0x00007ffff7e44929 in net_cleanup () at /usr/src/debug/qemu-kvm-0.12.1.2/net.c:1363 #8 0x00007ffff7de1e7b in main (argc=36, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:6527 (gdb) q So reopen this bug.
Verify this bug with qemu-kvm-0.12.1.2-2.382.el6.x86_64 steps: 1./usr/libexec/qemu-kvm -monitor stdio -nodefaults -chardev socket,id=serial_id_20120726-200643-igwg,path=/tmp/serial-20120726-200643-igwg,server,nowait -device isa-serial,chardev=serial_id_20120726-200643-igwg -device ich9-usb-uhci1,id=usb1,bus=pci.0,addr=0x4 -drive file=/home/RHEL-Server-6.3-64-virtio.qcow2,if=none,id=drive-virtio-disk1,media=disk,cache=none,boot=off,snapshot=off,format=qcow2,aio=native -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk1,id=virtio-disk1 -m 4096 -smp 2,cores=1,threads=1,sockets=2 -cpu 'SandyBridge' -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 -vnc :0 -vga std -rtc base=utc,clock=host,driftfix=slew -M rhel6.3.0 -boot order=cdn,once=c,menu=off -no-kvm-pit-reinjection -enable-kvm -device virtio-net-pci,netdev=idBO9VqI,mac=9a:7a:eb:5b:bd:17,id=ndev00idBO9VqI,bus=pci.0,addr=0x3,tx=timer -netdev tap,id=idBO9VqI,vhost=on 2. poweroff in guest result: guest shutdown successfully, not qemu-kvm core dump so I think this bug is fixed
reproduce with 2.6.32-358.el6.x86_64 kernel and qemu-kvm-0.12.1.2-2.355.el6.x86_64 Steps to Reproduce: 1. boot a guest with gdb tools: gdb /usr/libexec/qemu-kvm 2. (gdb) run -M pc -cpu SandyBridge -enable-kvm -m 4G -smp 4,sockets=1,cores=2,threads=2 -name scalability-test -rtc base=localtime,clock=host,driftfix=slew -k en-us -boot menu=on -spice disable-ticketing,port=5931 -vga qxl -monitor stdio -device virtio-balloon-pci,id=ballooning -qmp tcp:0:7777,server,nowait -serial unix:/tmp/ttyS0,server,nowait -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -drive file=/home/RHEL-Server-6.5-64.qcow2,if=none,id=drive-system-disk,media=disk,format=qcow2,aio=native,werror=stop,rerror=stop -device virtio-blk-pci,scsi=off,bus=pci.0,drive=drive-system-disk,id=system-disk,bootindex=1,addr=0x5 -netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,id=virtio-net-pci0,mac=00:22:15:27:54:8d,bus=pci.0,addr=0x9,tx=timer 3. remote spice://$host_ip:5931 4. in guest: shutdown -h now Actual results: qemu-kvm core dump Program received signal SIGSEGV, Segmentation fault. 0x00007ffff7ddf9a7 in ?? () (gdb) bt #0 0x00007ffff7ddf9a7 in ?? () #1 0x00007ffff7df60a5 in ?? () #2 0x00007ffff7df6520 in ?? () #3 0x00007ffff7dface1 in ?? () #4 0x00007ffff7dfa46c in ?? () #5 0x00007ffff7df472b in ?? () #6 0x00007ffff7e448bd in ?? () #7 0x00007ffff7e44939 in ?? () #8 0x00007ffff7de1e7b in main () Verify with 2.6.32-414.el6.x86_64 kernel and qemu-kvm-0.12.1.2-2.398.el6.x86_64 steps as above Actual results: qemu-kvm quit normally. Program exited normally. About comment #22 gdb /usr/libexec/qemu-kvm (gdb) run -chardev pipe Starting program: /usr/libexec/qemu-kvm -chardev pipe [Thread debugging using libthread_db enabled] qemu-kvm: -chardev pipe: chardev: no id specified Program exited with code 01. Based above information, so I think the bug has been fixed.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHSA-2013-1553.html