Bug 1018537
| Summary: | qemu core dump after run read/randwr fio in guest with usb or scsi disk | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 6 | Reporter: | langfang <flang> |
| Component: | qemu-kvm | Assignee: | Fam Zheng <famz> |
| Status: | CLOSED ERRATA | QA Contact: | Virtualization Bugs <virt-bugs> |
| Severity: | high | Docs Contact: | |
| Priority: | medium | ||
| Version: | 6.5 | CC: | bsarathy, famz, flang, juli, juzhang, mazhang, mkenneth, qzhang, rbalakri, rmainz, sluo, tlavigne, virt-maint |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | qemu-kvm-0.12.1.2-2.441.el6 | Doc Type: | Bug Fix |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2014-10-14 06:53:14 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Hi flang,
IIRC, i did not meet it when i run this testing, could you help check if is a regression issue or guest speciecied issue ?
BTW, i did not meet this issue in my intel host with qemu-kvm-rhev-0.12.1.2-2.412.el6.x86_64.
host info:
# uname -r && rpm -q qemu-kvm-rhev
2.6.32-422.el6.x86_64
qemu-kvm-rhev-0.12.1.2-2.412.el6.x86_64
guest info:
kernel-2.6.32-422.el6.x86_64
Best Regards,
sluo
(In reply to Sibiao Luo from comment #2) > Hi flang, > > IIRC, i did not meet it when i run this testing, could you help check if > is a regression issue or guest speciecied issue ? > BTW, i did not meet this issue in my intel host with > qemu-kvm-rhev-0.12.1.2-2.412.el6.x86_64. > host info: > # uname -r && rpm -q qemu-kvm-rhev > 2.6.32-422.el6.x86_64 > qemu-kvm-rhev-0.12.1.2-2.412.el6.x86_64 > guest info: > kernel-2.6.32-422.el6.x86_64 > > Best Regards, > sluo Reproduce this bug as follow version: host: # uname -r 2.6.32-422.el6.x86_64 # rpm -q qemu-kvm-rhev qemu-kvm-rhev-0.12.1.2-2.412.el6.x86_64 # rpm -q seabios seabios-0.6.1.2-28.el6.x86_64 Guest kernel-2.6.32-358.24.1.el6.i686.rpm Steps: 1.boot guest 2.(qemu)block_set_io_throttle drive-usb-2-0 1000 0 0 0 0 0 3.In guest #fio --filename=/dev/sdc --direct=1 --rw=read --bs=1M --size=10M --name=test --iodepth=1 #fio --filename=/dev/sdc --direct=1 --rw=randrw --bs=1M --size=10M --name=test --iodepth=1 4.If can't reproduce ,please do step3 many times Results:Guest ... qemu-kvm: /builddir/build/BUILD/qemu-kvm-0.12.1.2/hw/usb-msd.c:356: usb_msd_cancel_io: Assertion `s->packet == p' failed. Program received signal SIGABRT, Aborted. 0x00007ffff4c93925 in raise () from /lib64/libc.so.6 (gdb) bt #0 0x00007ffff4c93925 in raise () from /lib64/libc.so.6 #1 0x00007ffff4c95105 in abort () from /lib64/libc.so.6 #2 0x00007ffff4c8ca4e in __assert_fail_base () from /lib64/libc.so.6 #3 0x00007ffff4c8cb10 in __assert_fail () from /lib64/libc.so.6 #4 0x00007ffff7e43b74 in usb_msd_cancel_io (dev=<value optimized out>, p=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-msd.c:356 #5 0x00007ffff7e3cc0a in usb_cancel_packet (p=0x7ffff9769918) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb.c:356 #6 0x00007ffff7f237f9 in ehci_free_queue (q=0x7ffff97698a0, async=1) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-ehci.c:681 #7 0x00007ffff7f2552d in ehci_queues_rip_unseen (ehci=0x7ffff9713430) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-ehci.c:735 #8 ehci_advance_async_state (ehci=0x7ffff9713430) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-ehci.c:2075 #9 0x00007ffff7df8fc1 in qemu_bh_poll () at /usr/src/debug/qemu-kvm-0.12.1.2/async.c:70 #10 0x00007ffff7e01466 in qemu_aio_wait () at /usr/src/debug/qemu-kvm-0.12.1.2/aio.c:145 #11 0x00007ffff7e016f5 in qemu_aio_flush () at /usr/src/debug/qemu-kvm-0.12.1.2/aio.c:113 #12 0x00007ffff7e46fd2 in scsi_cancel_io (req=0x7ffff977d6d0) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/scsi-disk.c:105 #13 0x00007ffff7e44ee2 in scsi_req_cancel (req=0x7ffff977d6d0) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/scsi-bus.c:1424 #14 0x00007ffff7e3cc0a in usb_cancel_packet (p=0x7ffff9769918) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb.c:356 #15 0x00007ffff7f237f9 in ehci_free_queue (q=0x7ffff97698a0, async=1) ---Type <return> to continue, or q <return> to quit--- at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-ehci.c:681 #16 0x00007ffff7f2552d in ehci_queues_rip_unseen (ehci=0x7ffff9713430) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-ehci.c:735 #17 ehci_advance_async_state (ehci=0x7ffff9713430) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-ehci.c:2075 #18 0x00007ffff7f25812 in ehci_frame_timer (opaque=0x7ffff9713430) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-ehci.c:2213 #19 0x00007ffff7dc16ba in qemu_run_timers (timeout=1000) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:1339 #20 main_loop_wait (timeout=1000) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4083 #21 0x00007ffff7de440a in kvm_main_loop () at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:2245 #22 0x00007ffff7dc42a9 in main_loop (argc=63, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4266 #23 main (argc=63, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:6644 CLI as same as comment0 Same steps as comment 3 use scsi disk also hit qemu coredump Steps: 1.BOOt guest with scsi disk ...-drive file=/home/test3.qcow2,if=none,id=drive-scsi-disk-1,format=qcow2,cache=none,werror=stop,rerror=stop -device virtio-scsi-pci,id=scsi0,addr=0x5 -device scsi-disk,drive=drive-scsi-disk-1,bus=scsi0.0,scsi-id=0,id=scsi-disk-1 2.(qemu)block_set_io_throttle drive-scsi-disk-1 10000 0 0 0 0 0 3.In guest /dev/sdb--->scsi disk #fio --filename=/dev/sdb --direct=1 --rw=write --bs=1M --size=10M --name=test -- Resutls: Wait about 6 min,qemu coredump ... [New Thread 0x7fffef4c5700 (LWP 16014)] qemu-kvm: /builddir/build/BUILD/qemu-kvm-0.12.1.2/hw/scsi-disk.c:239: scsi_dma_complete: Assertion `r->req.aiocb != ((void *)0)' failed. Program received signal SIGABRT, Aborted. 0x00007ffff4c93925 in raise () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install alsa-lib-1.0.22-3.el6.x86_64 celt051-0.5.1.3-0.el6.x86_64 cyrus-sasl-gssapi-2.1.23-13.el6_3.1.x86_64 cyrus-sasl-lib-2.1.23-13.el6_3.1.x86_64 cyrus-sasl-md5-2.1.23-13.el6_3.1.x86_64 cyrus-sasl-plain-2.1.23-13.el6_3.1.x86_64 db4-4.7.25-18.el6_4.x86_64 dbus-libs-1.2.24-7.el6_3.x86_64 flac-1.2.1-6.1.el6.x86_64 glib2-2.26.1-3.el6.x86_64 glibc-2.12-1.130.el6.x86_64 glusterfs-api-3.4.0.34rhs-1.el6.x86_64 glusterfs-libs-3.4.0.34rhs-1.el6.x86_64 gnutls-2.8.5-10.el6_4.2.x86_64 keyutils-libs-1.4-4.el6.x86_64 krb5-libs-1.10.3-10.el6_4.6.x86_64 libICE-1.0.6-1.el6.x86_64 libSM-1.2.1-2.el6.x86_64 libX11-1.5.0-4.el6.x86_64 libXau-1.0.6-4.el6.x86_64 libXext-1.3.1-2.el6.x86_64 libXi-1.6.1-3.el6.x86_64 libXtst-1.2.1-2.el6.x86_64 libaio-0.3.107-10.el6.x86_64 libasyncns-0.8-1.1.el6.x86_64 libcom_err-1.41.12-18.el6.x86_64 libgcrypt-1.4.5-9.el6_2.2.x86_64 libgpg-error-1.7-4.el6.x86_64 libjpeg-turbo-1.2.1-1.el6.x86_64 libogg-1.1.4-2.1.el6.x86_64 libselinux-2.0.94-5.3.el6_4.1.x86_64 libsndfile-1.0.20-5.el6.x86_64 libtasn1-2.3-3.el6_2.1.x86_64 libuuid-2.17.2-12.14.el6.x86_64 libvorbis-1.2.3-4.el6_2.1.x86_64 libxcb-1.8.1-1.el6.x86_64 nss-softokn-freebl-3.14.3-9.el6.x86_64 openssl-1.0.1e-15.el6.x86_64 pixman-0.26.2-5.el6_4.x86_64 pulseaudio-libs-0.9.21-14.el6_3.x86_64 spice-server-0.12.4-4.el6.x86_64 tcp_wrappers-libs-7.6-57.el6.x86_64 usbredir-0.5.1-1.el6.x86_64 zlib-1.2.3-29.el6.x86_64 (gdb) bt #0 0x00007ffff4c93925 in raise () from /lib64/libc.so.6 #1 0x00007ffff4c95105 in abort () from /lib64/libc.so.6 #2 0x00007ffff4c8ca4e in __assert_fail_base () from /lib64/libc.so.6 #3 0x00007ffff4c8cb10 in __assert_fail () from /lib64/libc.so.6 #4 0x00007ffff7e475b1 in scsi_dma_complete (opaque=0x7fffd8000910, ret=0) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/scsi-disk.c:239 #5 0x00007ffff7f26e81 in dma_complete (dbs=0x7fffdc000db0, ret=0) at /usr/src/debug/qemu-kvm-0.12.1.2/dma-helpers.c:88 #6 0x00007ffff7f27052 in dma_bdrv_cb (opaque=0x7fffdc000db0, ret=0) at /usr/src/debug/qemu-kvm-0.12.1.2/dma-helpers.c:114 #7 0x00007ffff7dfc6ce in bdrv_co_em_bh (opaque=0x7fffdc000e50) at /usr/src/debug/qemu-kvm-0.12.1.2/block.c:4009 #8 0x00007ffff7df8fc1 in qemu_bh_poll () at /usr/src/debug/qemu-kvm-0.12.1.2/async.c:70 #9 0x00007ffff7dc1629 in main_loop_wait (timeout=0) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4096 #10 0x00007ffff7de440a in kvm_main_loop () at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:2245 #11 0x00007ffff7dc42a9 in main_loop (argc=45, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4266 #12 main (argc=45, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:6644 (gdb) I can reproduce the case in comment #4, but it takes a while like an hour or so to repeat the fio workload as above, with io throttled (bps=10000). The crashing code is in scsi io completion/cancellation code, don't have any conclusion looking at the backtrace and context code, yet but it seems like an unexpected second run of completion code path on a request timeout, which triggers assertion. Resetting assignee to get it triaged again. Fam Sorry, I should have provided you the RHEV build, because IO throttling is disabled in RHEL: http://brewweb.devel.redhat.com/brew/taskinfo?taskID=7831919 Please try again, Fam (In reply to Fam Zheng from comment #15) > Sorry, I should have provided you the RHEV build, because IO throttling is > disabled in RHEL: > > http://brewweb.devel.redhat.com/brew/taskinfo?taskID=7831919 > > Please try again, > Fam Test above build Version: Host: # uname -r 2.6.32-431.29.2.el6.x86_64 # rpm -q qemu-kvm-rhev qemu-kvm-rhev-0.12.1.2-2.436.el6.test.x86_64 guest: 2.6.32-431.el6.x86_64 Steps: 1.Boot guest with usb storage #qemu-img create -f qcow2 usb.qcow2 800M ... -drive file=/home/usb.qcow2,if=none,id=drive-usb-2-0,media=disk,format=qcow2,cache=none, -device usb-storage,drive=drive-usb-2-0,id=usb-0-0,removable=on,bus=ehci.0,port=1 2.(qemu)block_set_io_throttle drive-scsi-disk-1 10000 0 0 0 0 0 3.In guest /dev/sdb--->usb disk #fio --filename=/dev/sdb --direct=1 --rw=write --bs=1M --size=10M --name=test -- Resutls:guest run fio about 2 hours,work well,qemu not core dump Thanks for the update! Fam Fix included in qemu-kvm-0.12.1.2-2.441.el6 Reproduced this bug.
Host:
qemu-kvm-rhev-tools-0.12.1.2-2.438.el6.x86_64
qemu-kvm-rhev-0.12.1.2-2.438.el6.x86_64
gpxe-roms-qemu-0.9.7-6.12.el6.noarch
qemu-img-rhev-0.12.1.2-2.438.el6.x86_64
qemu-kvm-rhev-debuginfo-0.12.1.2-2.438.el6.x86_64
kernel-2.6.32-497.el6.x86_64
Guest:
kernel-2.6.32-497.el6.x86_64
Steps:
1. boot vm:
gdb --args /usr/libexec/qemu-kvm \
-machine rhel6.6.0,dump-guest-core=off \
-cpu SandyBridge \
-m 2G \
-smp 4,sockets=2,cores=2,threads=1,maxcpus=160 \
-enable-kvm \
-name rhel6.6 \
-uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \
-smbios type=1,manufacturer='Red Hat',product='RHEV Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-e63fcdb67170 \
-k en-us \
-rtc base=localtime,clock=host,driftfix=slew \
-nodefaults \
-monitor stdio \
-qmp tcp:0:5555,server,nowait \
-boot menu=on \
-bios /usr/share/seabios/bios.bin \
-monitor unix:/tmp/monitor2,server,nowait \
-vga qxl \
-spice port=5900,disable-ticketing \
-usb \
-device usb-tablet,id=tablet0 \
-device virtio-scsi-pci,id=si0 \
-drive file=/home/RHEL-Server-6.6-64-1.qcow2,if=none,media=disk,id=drive-scsi-disk,format=qcow2,cache=none,werror=stop,rerror=stop,aio=native \
-device scsi-hd,drive=drive-scsi-disk,bus=si0.0,id=scsi-disk0,bootindex=0 \
-device usb-ehci,id=ehci \
-drive file=/home/storage.qcow2,if=none,id=drive-usb-2-0,media=disk,format=qcow2,cache=none \
-device usb-storage,drive=drive-usb-2-0,id=usb-0-0,removable=on,bus=ehci.0,port=1 \
-netdev tap,id=hostnet0,vhost=on \
-device e1000,netdev=hostnet0,id=net0,mac=00:01:02:B6:40:23 \
2. (qemu)block_set_io_throttle drive-scsi-disk-1 10000 0 0 0 0 0
3. Fio test in guest.
#fio --filename=/dev/sdb --direct=1 --rw=read --bs=1M --size=10M --name=test
Result:
qemu-kvm core dumped.
qemu-kvm: /builddir/build/BUILD/qemu-kvm-0.12.1.2/hw/usb-msd.c:356: usb_msd_cancel_io: Assertion `s->packet == p' failed.
Program received signal SIGABRT, Aborted.
0x00007ffff4836915 in raise () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install alsa-lib-1.0.22-3.el6.x86_64 celt051-0.5.1.3-0.el6.x86_64 cyrus-sasl-gssapi-2.1.23-15.el6.x86_64 cyrus-sasl-lib-2.1.23-15.el6.x86_64 cyrus-sasl-md5-2.1.23-15.el6.x86_64 cyrus-sasl-plain-2.1.23-15.el6.x86_64 db4-4.7.25-18.el6_4.x86_64 dbus-libs-1.2.24-7.el6_3.x86_64 flac-1.2.1-6.1.el6.x86_64 glib2-2.28.8-1.el6.x86_64 glibc-2.12-1.148.el6.x86_64 glusterfs-api-3.6.0.27-1.el6.x86_64 glusterfs-libs-3.6.0.27-1.el6.x86_64 gnutls-2.8.5-14.el6_5.x86_64 keyutils-libs-1.4-5.el6.x86_64 krb5-libs-1.10.3-31.el6.x86_64 libICE-1.0.6-1.el6.x86_64 libSM-1.2.1-2.el6.x86_64 libX11-1.6.0-2.2.el6.x86_64 libXau-1.0.6-4.el6.x86_64 libXext-1.3.2-2.1.el6.x86_64 libXi-1.7.2-2.2.el6.x86_64 libXtst-1.2.2-2.1.el6.x86_64 libaio-0.3.107-10.el6.x86_64 libasyncns-0.8-1.1.el6.x86_64 libcom_err-1.41.12-20.el6.x86_64 libgcc-4.4.7-10.el6.x86_64 libgcrypt-1.4.5-11.el6_4.x86_64 libgpg-error-1.7-4.el6.x86_64 libjpeg-turbo-1.2.1-3.el6_5.x86_64 libogg-1.1.4-2.1.el6.x86_64 libselinux-2.0.94-5.8.el6.x86_64 libsndfile-1.0.20-5.el6.x86_64 libstdc++-4.4.7-10.el6.x86_64 libtasn1-2.3-6.el6_5.x86_64 libuuid-2.17.2-12.18.el6.x86_64 libvorbis-1.2.3-4.el6_2.1.x86_64 libxcb-1.9.1-2.el6.x86_64 lzo-2.03-3.1.el6_5.1.x86_64 nss-softokn-freebl-3.14.3-15.el6.x86_64 openssl-1.0.1e-28.el6.x86_64 pixman-0.32.4-4.el6.x86_64 pulseaudio-libs-0.9.21-17.el6.x86_64 snappy-1.1.0-1.el6.x86_64 spice-server-0.12.4-11.el6.x86_64 tcp_wrappers-libs-7.6-57.el6.x86_64 usbredir-0.5.1-1.el6.x86_64 zlib-1.2.3-29.el6.x86_64
(gdb) bt
#0 0x00007ffff4836915 in raise () from /lib64/libc.so.6
#1 0x00007ffff48380f5 in abort () from /lib64/libc.so.6
#2 0x00007ffff482fa3e in __assert_fail_base () from /lib64/libc.so.6
#3 0x00007ffff482fb00 in __assert_fail () from /lib64/libc.so.6
#4 0x00007ffff7e36fd4 in usb_msd_cancel_io (dev=<value optimized out>, p=<value optimized out>)
at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-msd.c:356
#5 0x00007ffff7e2fcaa in usb_cancel_packet (p=0x7fffffdd2be8) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb.c:356
#6 0x00007ffff7f1d6d9 in ehci_free_queue (q=0x7fffffdd2b70, async=1)
at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-ehci.c:681
#7 0x00007ffff7f1f40d in ehci_queues_rip_unseen (ehci=0x7ffff9473920)
at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-ehci.c:735
#8 ehci_advance_async_state (ehci=0x7ffff9473920) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-ehci.c:2075
#9 0x00007ffff7de7101 in qemu_bh_poll () at /usr/src/debug/qemu-kvm-0.12.1.2/async.c:70
#10 0x00007ffff7def756 in qemu_aio_wait () at /usr/src/debug/qemu-kvm-0.12.1.2/aio.c:145
#11 0x00007ffff7def9e5 in qemu_aio_flush () at /usr/src/debug/qemu-kvm-0.12.1.2/aio.c:113
#12 0x00007ffff7e3a6c2 in scsi_cancel_io (req=<value optimized out>)
at /usr/src/debug/qemu-kvm-0.12.1.2/hw/scsi-disk.c:105
#13 0x00007ffff7e38702 in scsi_req_cancel (req=0x7ffff91bed70) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/scsi-bus.c:1488
#14 0x00007ffff7e2fcaa in usb_cancel_packet (p=0x7fffffdd2be8) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb.c:356
#15 0x00007ffff7f1d6d9 in ehci_free_queue (q=0x7fffffdd2b70, async=1)
at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-ehci.c:681
#16 0x00007ffff7f1f40d in ehci_queues_rip_unseen (ehci=0x7ffff9473920)
at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-ehci.c:735
#17 ehci_advance_async_state (ehci=0x7ffff9473920) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-ehci.c:2075
#18 0x00007ffff7f1f6f2 in ehci_frame_timer (opaque=0x7ffff9473920)
at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-ehci.c:2213
#19 0x00007ffff7daed4a in qemu_run_timers (timeout=1000) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:1341
#20 main_loop_wait (timeout=1000) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4085
#21 0x00007ffff7dd24ea in kvm_main_loop () at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:2258
#22 0x00007ffff7db3767 in main_loop (argc=<value optimized out>, argv=<value optimized out>,
envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4268
#23 main (argc=<value optimized out>, argv=<value optimized out>, envp=<value optimized out>)
at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:6725
scsi disk also hit this problem.
Verify this bug on qemu-kvm-rhev-0.12.1.2-2.441.el6.x86_64. Host: qemu-img-rhev-0.12.1.2-2.441.el6.x86_64 qemu-kvm-rhev-debuginfo-0.12.1.2-2.441.el6.x86_64 qemu-kvm-rhev-tools-0.12.1.2-2.441.el6.x86_64 qemu-kvm-rhev-0.12.1.2-2.441.el6.x86_64 gpxe-roms-qemu-0.9.7-6.12.el6.noarch kernel-2.6.32-497.el6.x86_64 Guest: kernel-2.6.32-497.el6.x86_64 Result: Both usb-storage and scsi-hd works well, the problem has gone. So this bug has been fixed. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2014-1490.html |
Description of problem: Guest core dump after run read/randwr usb disk use fio Version-Release number of selected component (if applicable): Host # uname -r 2.6.32-423.el6.x86_64 # rpm -q qemu-kvm-rhev qemu-kvm-rhev-0.12.1.2-2.412.el6.x86_64 # rpm -q seabios seabios-0.6.1.2-28.el6.x86_64 Guest:6.4.z-32 kernel-2.6.32-358.24.1.el6.i686.rpm How reproducible: 60% Steps to Reproduce: 1.Boot guest with usb storage #qemu-img create -f qcow2 usb.qcow2 800M ...-drive file=/home/RHEL6.4-20130130.0-Server-i386-DVD1.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=1,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=1 -drive file=/home/test2.qcow2,if=none,id=drive-scsi-disk,format=qcow2,cache=none,werror=stop,rerror=stop -device virtio-scsi-pci,id=scsi0,addr=0x5 -device scsi-disk,drive=drive-scsi-disk,bus=scsi0.0,scsi-id=0,id=scsi-disk -drive file=/home/floopy.qcow2,if=none,id=drive-fdc0-0-0,format=qcow2,cache=none -global isa-fdc.driveA=drive-fdc0-0-0 -drive file=/home/cdrom_scsi.qcow2,if=none,media=cdrom,readonly=on,format=qcow2,id=cdrom1 -device scsi-cd,bus=scsi0.0,drive=cdrom1,id=scsi0-0 -device usb-ehci,id=ehci -drive file=/home/usb.qcow2,if=none,id=drive-usb-2-0,media=disk,format=qcow2,cache=none, -device usb-storage,drive=drive-usb-2-0,id=usb-0-0,removable=on,bus=ehci.0,port=1... 2.IN guest Install fio #fio --filename=/dev/sdc --direct=1 --rw=read --bs=1M --size=10M --name=test --iodepth=1 #fio --filename=/dev/sdc --direct=1 --rw=randrw --bs=1M --size=10M --name=test --iodepth=1 Actual results: Guest core dump ... qemu-kvm: /builddir/build/BUILD/qemu-kvm-0.12.1.2/hw/usb-msd.c:356: usb_msd_cancel_io: Assertion `s->packet == p' failed. Program received signal SIGABRT, Aborted. 0x00007ffff4c93925 in raise () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install alsa-lib-1.0.22-3.el6.x86_64 celt051-0.5.1.3-0.el6.x86_64 cyrus-sasl-gssapi-2.1.23-13.el6_3.1.x86_64 cyrus-sasl-lib-2.1.23-13.el6_3.1.x86_64 cyrus-sasl-md5-2.1.23-13.el6_3.1.x86_64 cyrus-sasl-plain-2.1.23-13.el6_3.1.x86_64 db4-4.7.25-18.el6_4.x86_64 dbus-libs-1.2.24-7.el6_3.x86_64 flac-1.2.1-6.1.el6.x86_64 glib2-2.26.1-3.el6.x86_64 glibc-2.12-1.130.el6.x86_64 glusterfs-api-3.4.0.34rhs-1.el6.x86_64 glusterfs-libs-3.4.0.34rhs-1.el6.x86_64 gnutls-2.8.5-10.el6_4.2.x86_64 keyutils-libs-1.4-4.el6.x86_64 krb5-libs-1.10.3-10.el6_4.6.x86_64 libICE-1.0.6-1.el6.x86_64 libSM-1.2.1-2.el6.x86_64 libX11-1.5.0-4.el6.x86_64 libXau-1.0.6-4.el6.x86_64 libXext-1.3.1-2.el6.x86_64 libXi-1.6.1-3.el6.x86_64 libXtst-1.2.1-2.el6.x86_64 libaio-0.3.107-10.el6.x86_64 libasyncns-0.8-1.1.el6.x86_64 libcom_err-1.41.12-18.el6.x86_64 libgcrypt-1.4.5-9.el6_2.2.x86_64 libgpg-error-1.7-4.el6.x86_64 libjpeg-turbo-1.2.1-1.el6.x86_64 libogg-1.1.4-2.1.el6.x86_64 libselinux-2.0.94-5.3.el6_4.1.x86_64 libsndfile-1.0.20-5.el6.x86_64 libtasn1-2.3-3.el6_2.1.x86_64 libuuid-2.17.2-12.14.el6.x86_64 libvorbis-1.2.3-4.el6_2.1.x86_64 libxcb-1.8.1-1.el6.x86_64 nss-softokn-freebl-3.14.3-9.el6.x86_64 openssl-1.0.1e-15.el6.x86_64 pixman-0.26.2-5.el6_4.x86_64 pulseaudio-libs-0.9.21-14.el6_3.x86_64 spice-server-0.12.4-4.el6.x86_64 tcp_wrappers-libs-7.6-57.el6.x86_64 usbredir-0.5.1-1.el6.x86_64 zlib-1.2.3-29.el6.x86_64 (gdb) bt #0 0x00007ffff4c93925 in raise () from /lib64/libc.so.6 #1 0x00007ffff4c95105 in abort () from /lib64/libc.so.6 #2 0x00007ffff4c8ca4e in __assert_fail_base () from /lib64/libc.so.6 #3 0x00007ffff4c8cb10 in __assert_fail () from /lib64/libc.so.6 #4 0x00007ffff7e43b74 in usb_msd_cancel_io (dev=<value optimized out>, p=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-msd.c:356 #5 0x00007ffff7e3cc0a in usb_cancel_packet (p=0x7ffff99cc9a8) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb.c:356 #6 0x00007ffff7f237f9 in ehci_free_queue (q=0x7ffff99cc930, async=1) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-ehci.c:681 #7 0x00007ffff7f2552d in ehci_queues_rip_unseen (ehci=0x7ffff9713430) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-ehci.c:735 #8 ehci_advance_async_state (ehci=0x7ffff9713430) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-ehci.c:2075 #9 0x00007ffff7df8fc1 in qemu_bh_poll () at /usr/src/debug/qemu-kvm-0.12.1.2/async.c:70 #10 0x00007ffff7e01466 in qemu_aio_wait () at /usr/src/debug/qemu-kvm-0.12.1.2/aio.c:145 #11 0x00007ffff7e016f5 in qemu_aio_flush () at /usr/src/debug/qemu-kvm-0.12.1.2/aio.c:113 #12 0x00007ffff7e46fd2 in scsi_cancel_io (req=0x7ffff8da3190) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/scsi-disk.c:105 #13 0x00007ffff7e44ee2 in scsi_req_cancel (req=0x7ffff8da3190) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/scsi-bus.c:1424 #14 0x00007ffff7e3cc0a in usb_cancel_packet (p=0x7ffff99cc9a8) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb.c:356 #15 0x00007ffff7f237f9 in ehci_free_queue (q=0x7ffff99cc930, async=1) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-ehci.c:681 #16 0x00007ffff7f2552d in ehci_queues_rip_unseen (ehci=0x7ffff9713430) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-ehci.c:735 #17 ehci_advance_async_state (ehci=0x7ffff9713430) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-ehci.c:2075 #18 0x00007ffff7f25812 in ehci_frame_timer (opaque=0x7ffff9713430) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-ehci.c:2213 #19 0x00007ffff7dc16ba in qemu_run_timers (timeout=1000) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:1339 #20 main_loop_wait (timeout=1000) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4083 #21 0x00007ffff7de440a in kvm_main_loop () at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:2245 ---Type <return> to continue, or q <return> to quit--- #22 0x00007ffff7dc42a9 in main_loop (argc=63, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4266 #23 main (argc=63, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:6644 Expected results: Guest work well Additional info: 1) MY CLI: (gdb) r -M rhel6.5.0 -cpu Opteron_G3 -m 2G -smp 4,sockets=2,cores=2,threads=1 -enable-kvm -usb -device usb-tablet,id=input0 -name rhel6.4-z-32 -uuid 0dc2ab15-843a-4b40-844e-615fd9219236 -rtc base=localtime,clock=host,driftfix=slew -drive file=/dev/vg-flang/lv-flang,format=raw,if=none,id=ide0 -device ide-drive,drive=ide0,bus=ide.0,unit=0,id=ide0-0-0,bootindex=0 -vnc :1 -monitor stdio -boot menu=on -netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup -device rtl8139,netdev=hostnet0,id=virtio-net-pci0,mac=92:31:61:E0:31:26,bus=pci.0,addr=0x6 -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -qmp tcp:0:4444,server,nowait -drive file=/home/RHEL6.4-20130130.0-Server-i386-DVD1.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=1,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=1 -drive file=/home/test2.qcow2,if=none,id=drive-scsi-disk,format=qcow2,cache=none,werror=stop,rerror=stop -device virtio-scsi-pci,id=scsi0,addr=0x5 -device scsi-disk,drive=drive-scsi-disk,bus=scsi0.0,scsi-id=0,id=scsi-disk -drive file=/home/floopy.qcow2,if=none,id=drive-fdc0-0-0,format=qcow2,cache=none -global isa-fdc.driveA=drive-fdc0-0-0 -drive file=/home/cdrom_scsi.qcow2,if=none,media=cdrom,readonly=on,format=qcow2,id=cdrom1 -device scsi-cd,bus=scsi0.0,drive=cdrom1,id=scsi0-0 -device usb-ehci,id=ehci -drive file=/home/usb.qcow2,if=none,id=drive-usb-2-0,media=disk,format=qcow2,cache=none, -device usb-storage,drive=drive-usb-2-0,id=usb-0-0,removable=on,bus=ehci.0,port=1 Starting program: /usr/libexec/qemu-kvm -M rhel6.5.0 -cpu Opteron_G3 -m 2G -smp 4,sockets=2,cores=2,threads=1 -enable-kvm -usb -device usb-tablet,id=input0 -name rhel6.4-z-32 -uuid 0dc2ab15-843a-4b40-844e-615fd9219236 -rtc base=localtime,clock=host,driftfix=slew -drive file=/dev/vg-flang/lv-flang,format=raw,if=none,id=ide0 -device ide-drive,drive=ide0,bus=ide.0,unit=0,id=ide0-0-0,bootindex=0 -vnc :1 -monitor stdio -boot menu=on -netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup -device rtl8139,netdev=hostnet0,id=virtio-net-pci0,mac=92:31:61:E0:31:26,bus=pci.0,addr=0x6 -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -qmp tcp:0:4444,server,nowait -drive file=/home/RHEL6.4-20130130.0-Server-i386-DVD1.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=1,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=1 -drive file=/home/test2.qcow2,if=none,id=drive-scsi-disk,format=qcow2,cache=none,werror=stop,rerror=stop -device virtio-scsi-pci,id=scsi0,addr=0x5 -device scsi-disk,drive=drive-scsi-disk,bus=scsi0.0,scsi-id=0,id=scsi-disk -drive file=/home/floopy.qcow2,if=none,id=drive-fdc0-0-0,format=qcow2,cache=none -global isa-fdc.driveA=drive-fdc0-0-0 -drive file=/home/cdrom_scsi.qcow2,if=none,media=cdrom,readonly=on,format=qcow2,id=cdrom1 -device scsi-cd,bus=scsi0.0,drive=cdrom1,id=scsi0-0 -device usb-ehci,id=ehci -drive file=/home/usb.qcow2,if=none,id=drive-usb-2-0,media=disk,format=qcow2,cache=none, -device usb-storage,drive=drive-usb-2-0,id=usb-0-0,removable=on,bus=ehci.0,port=1 2)Host #cat /proc/cpuinfo .. processor : 3 vendor_id : AuthenticAMD cpu family : 21 model : 16 model name : AMD A10-5800K APU with Radeon(tm) HD Graphics stepping : 1 cpu MHz : 1400.000 cache size : 2048 KB physical id : 0 siblings : 4 core id : 3 cpu cores : 2 apicid : 19 initial apicid : 3 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nonstop_tsc extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core cpb npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold bmi1 bogomips : 7585.93 TLB size : 1536 4K pages clflush size : 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro