Bug 807313 - qemu-kvm core dumped while booting guest with usb-storage running on uhci
qemu-kvm core dumped while booting guest with usb-storage running on uhci
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: qemu-kvm (Show other bugs)
6.3
Unspecified Unspecified
high Severity high
: rc
: ---
Assigned To: Gerd Hoffmann
Virtualization Bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-03-27 09:39 EDT by Chao Yang
Modified: 2012-06-20 07:45 EDT (History)
14 users (show)

See Also:
Fixed In Version: qemu-kvm-0.12.1.2-2.282.el6
Doc Type: Bug Fix
Doc Text:
Cause: In some cases usb-storage emulation fails to update state correctly on io request cancelation. Consequence: usb-storage state machine is confused and triggers a assert() in the usb core code, making qemu dump core. Fix: Handle status update correctly. Consequence: qemu core dumps are gone.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-06-20 07:45:28 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2012:0746 normal SHIPPED_LIVE qemu-kvm bug fix and enhancement update 2012-06-19 15:31:48 EDT

  None (edit)
Description Chao Yang 2012-03-27 09:39:39 EDT
Description of problem:
Boot a guest with usb-storage attached on usb-hub, qemu-kvm core dumped.

Version-Release number of selected component (if applicable):
qemu-kvm-0.12.1.2-2.253.el6.x86_64

How reproducible:
100%

Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:
(qemu) qemu-kvm: /builddir/build/BUILD/qemu-kvm-0.12.1.2/hw/usb.c:345: usb_packet_complete: Assertion `p->owner != ((void *)0)' failed.

Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffee648700 (LWP 24572)]
0x00007ffff5a668a5 in raise () from /lib64/libc.so.6
(gdb) bt
#0  0x00007ffff5a668a5 in raise () from /lib64/libc.so.6
#1  0x00007ffff5a68085 in abort () from /lib64/libc.so.6
#2  0x00007ffff5a5fa1e in __assert_fail_base () from /lib64/libc.so.6
#3  0x00007ffff5a5fae0 in __assert_fail () from /lib64/libc.so.6
#4  0x00007ffff7e5b204 in usb_packet_complete (dev=<value optimized out>, p=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb.c:345
#5  0x00007ffff7e61ca5 in usb_msd_handle_reset (dev=0x7ffff95dee60) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-msd.c:299
#6  0x00007ffff7e5b3d1 in usb_generic_handle_packet (s=0x7ffff95dee60, p=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb.c:233
#7  0x00007ffff7e5b2b2 in usb_handle_packet (dev=0x7ffff95dee60, p=0x7fffee647a70) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb.c:325
#8  0x00007ffff7e5b7a4 in usb_send_msg (dev=<value optimized out>, msg=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb.c:312
#9  0x00007ffff7f713bd in uhci_ioport_writew (opaque=0x7ffff8b91010, addr=<value optimized out>, val=<value optimized out>)
    at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-uhci.c:485
#10 0x00007ffff7e16dd5 in kvm_handle_io (env=0x7ffff8b38fc0) at /usr/src/debug/qemu-kvm-0.12.1.2/kvm-all.c:587
#11 kvm_run (env=0x7ffff8b38fc0) at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:1048
#12 0x00007ffff7e16e89 in kvm_cpu_exec (env=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:1743
#13 0x00007ffff7e17d6d in kvm_main_loop_cpu (_env=0x7ffff8b38fc0) at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:2004
#14 ap_main_loop (_env=0x7ffff8b38fc0) at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:2060
#15 0x00007ffff7752851 in start_thread () from /lib64/libpthread.so.0
#16 0x00007ffff5b1b5ad in clone () from /lib64/libc.so.6


CLI:
# /usr/libexec/qemu-kvm -M rhel6.3.0 -cpu SandyBridge -enable-kvm -m 2048 -smp 2,sockets=1,cores=2,threads=1 -name usb-device -uuid 0a94ee47-f42a-4f3f-a8b3-6939740989c1 -boot menu=on -monitor stdio -rtc base=utc,clock=host,driftfix=slew -device virtio-serial-pci,id=virtio-serial0,max_ports=16,bus=pci.0 -device piix3-usb-uhci,id=usb,bus=pci.0 -device virtio-scsi-pci,id=scsi-pci -drive file=/dev/chayang-ag/usb-device_virtio-scsi,if=none,id=drive-virtio-disk0,format=qcow2,serial=69-a2ad-85f195fee049,cache=none,werror=stop,rerror=stop,aio=native -device scsi-disk,bus=scsi-pci.0,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=0,lun=0 -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,id=hostnet0,vhost=on -device virtio-net-pci,netdev=hostnet0,id=net0,mac=44:37:E6:5D:42:68,bus=pci.0 -chardev socket,id=charchannel0,path=/opt/usbdevice.com.redhat.rhevm.vdsm,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.rhevm.vdsm -chardev spicevmc,id=charchannel1,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=com.redhat.spice.0 -spice port=9000,disable-ticketing -k en-us -vga qxl -global qxl-vga.vram_size=67108864 -device virtio-balloon-pci,id=balloon -device usb-ehci,id=ehci -drive file=/dev/chayang-ag/usb-storage,if=none,id=drive-virtio-disk1,format=qcow2,cache=none,werror=stop,rerror=stop,aio=native,serial=e4977805-a8f6-4854-98a0-edbd1ee9ed58 -device usb-storage,bus=ehci.0,drive=drive-virtio-disk1,id=virtio-disk1,port=1,removable=on -drive file=/dev/chayang-ag/usb-storage-1,if=none,id=drive-virtio-disk2,format=qcow2,cache=none,werror=stop,rerror=stop,aio=native,serial=4071bf55-4710-4652-9af5-206a9ece1644 -usb -device usb-storage,bus=usb.0,drive=drive-virtio-disk2,id=virtio-disk2,port=2,removable=off
Comment 2 Gerd Hoffmann 2012-03-27 09:47:24 EDT
Which guest OS?
Comment 3 Chao Yang 2012-03-27 10:03:38 EDT
(In reply to comment #2)
> Which guest OS?

RHEL 6.3 one, 2.6.32-251.el6.x86_64
Comment 4 Gerd Hoffmann 2012-03-29 03:56:32 EDT
Hmm, doesn't reproduce easily.

Where is the usb-hub mentioned in the title?  The CLI doesn't add one, and as port= is specified for the usb-storage device qemu shouldn't add one on its own.

If you remove the ehci controller, does it still reproduce?
If you switch from virtio-scsi to virtio-blk, does it still reproduce?
Comment 5 Chao Yang 2012-03-29 05:17:20 EDT
(In reply to comment #4)
> Hmm, doesn't reproduce easily.
> 
Try system_reset in monitor if it succeeds to boot up, a second initialization will make it easier. 

> Where is the usb-hub mentioned in the title?  The CLI doesn't add one, and as
> port= is specified for the usb-storage device qemu shouldn't add one on its
> own.
> 
Indeed, no usb-hub attached. Sorry. The summary is expressing wrong information. Will change it to the correct one. 

> If you remove the ehci controller, does it still reproduce?
Yes. 
(qemu) system_reset 
(qemu) qemu-kvm: /builddir/build/BUILD/qemu-kvm-0.12.1.2/hw/usb.c:345: usb_packet_complete: Assertion `p->owner != ((void *)0)' failed.

#0  0x00007f8f5133e8a5 in raise () from /lib64/libc.so.6
#1  0x00007f8f51340085 in abort () from /lib64/libc.so.6
#2  0x00007f8f51337a1e in __assert_fail_base () from /lib64/libc.so.6
#3  0x00007f8f51337ae0 in __assert_fail () from /lib64/libc.so.6
#4  0x00007f8f53733204 in usb_packet_complete (dev=<value optimized out>, p=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb.c:345
#5  0x00007f8f53739ca5 in usb_msd_handle_reset (dev=0x7f8f55ff0710) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-msd.c:299
#6  0x00007f8f537333d1 in usb_generic_handle_packet (s=0x7f8f55ff0710, p=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb.c:233
#7  0x00007f8f537332b2 in usb_handle_packet (dev=0x7f8f55ff0710, p=0x7fff032d39e0) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb.c:325
#8  0x00007f8f537337a4 in usb_send_msg (dev=<value optimized out>, msg=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb.c:312
#9  0x00007f8f5384917c in uhci_reset (opaque=0x7f8f555aa010) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-uhci.c:339
#10 0x00007f8f536c963a in qemu_system_reset () at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:3422
#11 0x00007f8f536ed03c in qemu_kvm_system_reset () at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:1978
#12 kvm_main_loop () at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:2258
#13 0x00007f8f536ce41c in main_loop (argc=20, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4331
#14 main (argc=20, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:6591

CLI:
... -usb -drive file=/dev/chayang-ag/usb-storage,if=none,id=drive-virtio-disk1,format=qcow2,cache=none,werror=stop,rerror=stop,aio=native,serial=e4977805-a8f6-4854-98a0-edbd1ee9ed58 -device usb-storage,bus=usb.0,port=1,drive=drive-virtio-disk1,id=virtio-disk1,port=1,removable=on -drive file=/dev/chayang-ag/usb-storage-1,if=none,id=drive-virtio-disk2,format=qcow2,cache=none,werror=stop,rerror=stop,aio=native,serial=4071bf55-4710-4652-9af5-206a9ece1644 -device usb-storage,bus=usb.0,drive=drive-virtio-disk2,id=virtio-disk2,port=2,removable=off

> If you switch from virtio-scsi to virtio-blk, does it still reproduce?
Yes. Reproducible if switch from virtio-scsi to virtio-blk and assign two usb-storage(use bus=usb.0,port=1 and bus=usb.0,port=2) onto uhci
(qemu) system_reset 
(qemu) qemu-kvm: /builddir/build/BUILD/qemu-kvm-0.12.1.2/hw/usb.c:345: usb_packet_complete: Assertion `p->owner != ((void *)0)' failed.
Aborted (core dumped)
#0  0x00007f3cd123f8a5 in raise () from /lib64/libc.so.6
#1  0x00007f3cd1241085 in abort () from /lib64/libc.so.6
#2  0x00007f3cd1238a1e in __assert_fail_base () from /lib64/libc.so.6
#3  0x00007f3cd1238ae0 in __assert_fail () from /lib64/libc.so.6
#4  0x00007f3cd3634204 in usb_packet_complete (dev=<value optimized out>, p=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb.c:345
#5  0x00007f3cd363aca5 in usb_msd_handle_reset (dev=0x7f3cd68848e0) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-msd.c:299
#6  0x00007f3cd36343d1 in usb_generic_handle_packet (s=0x7f3cd68848e0, p=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb.c:233
#7  0x00007f3cd36342b2 in usb_handle_packet (dev=0x7f3cd68848e0, p=0x7f3cc9e20a70) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb.c:325
#8  0x00007f3cd36347a4 in usb_send_msg (dev=<value optimized out>, msg=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb.c:312
#9  0x00007f3cd374a3bd in uhci_ioport_writew (opaque=0x7f3cd5e3d010, addr=<value optimized out>, val=<value optimized out>)
    at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb-uhci.c:485
#10 0x00007f3cd35efdd5 in kvm_handle_io (env=0x7f3cd5de4dc0) at /usr/src/debug/qemu-kvm-0.12.1.2/kvm-all.c:587
#11 kvm_run (env=0x7f3cd5de4dc0) at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:1048
#12 0x00007f3cd35efe89 in kvm_cpu_exec (env=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:1743
#13 0x00007f3cd35f0d6d in kvm_main_loop_cpu (_env=0x7f3cd5de4dc0) at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:2004
#14 ap_main_loop (_env=0x7f3cd5de4dc0) at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:2060
#15 0x00007f3cd2f2b851 in start_thread () from /lib64/libpthread.so.0
#16 0x00007f3cd12f45ad in clone () from /lib64/libc.so.6
Comment 13 langfang 2012-04-25 05:49:35 EDT
reporduce this issue with steps and  environment as follows:
version:
#uname -r 
2.6.32-262.el6.x86_64
#rpm -q qemu-kvm
qemu-kvm-0.12.1.2-2.204.el6.x86_64

step:
1)boot guest 

r -m 2G -smp 2 -cpu Penryn,+x2apic -drive file=/home/tracing-run-rhel6.3-copy1.qcow2,format=qcow2,if=none,id=virtio-drive-disk0,werror=stop,rerror=stop,cache=none -device virtio-blk-pci,drive=virtio-drive-disk0,id=ide0-0-0,bootindex=1 -netdev tap,id=hostnet0,script=/etc/qemu-ifup -device e1000,netdev=hostnet0,mac=28:19:2e:29:37:58,bus=pci.0,addr=0x4,id=net0 -boot order=dcn,menu=on -uuid e85e6987-c012-4025-878a-d4a5f521f8a5 -rtc base=utc,clock=host,driftfix=slew -no-kvm-pit-reinjection -monitor stdio -name rhel6.3 -spice port=5840,disable-ticketing -vga qxl -device virtio-balloon-pci,bus=pci.0,id=balloon0 -device intel-hda,id=sound0,bus=pci.0 -drive file=/dev/chayang-ag/usb-storage-1,if=none,id=drive-virtio-disk2,format=qcow2,cache=none,werror=stop,rerror=stop,aio=native -usb -device usb-storage,bus=usb.0,drive=drive-virtio-disk2,id=virtio-disk2,port=2,removable=off  -drive file=/dev/chayang-ag/usb-storage,if=none,id=drive-virtio-disk1,format=qcow2,cache=none,werror=stop,rerror=stop,aio=native -device usb-storage,bus=usb.0,port=1,drive=drive-virtio-disk1,id=virtio-disk1,removable=on
Starting program: /usr/libexec/qemu-kvm -m 2G -smp 2 -cpu Penryn,+x2apic -drive file=/home/tracing-run-rhel6.3-copy1.qcow2,format=qcow2,if=none,id=virtio-drive-disk0,werror=stop,rerror=stop,cache=none -device virtio-blk-pci,drive=virtio-drive-disk0,id=ide0-0-0,bootindex=1 -netdev tap,id=hostnet0,script=/etc/qemu-ifup -device e1000,netdev=hostnet0,mac=28:19:2e:29:37:58,bus=pci.0,addr=0x4,id=net0 -boot order=dcn,menu=on -uuid e85e6987-c012-4025-878a-d4a5f521f8a5 -rtc base=utc,clock=host,driftfix=slew -no-kvm-pit-reinjection -monitor stdio -name rhel6.3 -spice port=5840,disable-ticketing -vga qxl -device virtio-balloon-pci,bus=pci.0,id=balloon0 -device intel-hda,id=sound0,bus=pci.0 -drive file=/dev/chayang-ag/usb-storage-1,if=none,id=drive-virtio-disk2,format=qcow2,cache=none,werror=stop,rerror=stop,aio=native -usb -device usb-storage,bus=usb.0,drive=drive-virtio-disk2,id=virtio-disk2,port=2,removable=off  -drive file=/dev/chayang-ag/usb-storage,if=none,id=drive-virtio-disk1,format=qcow2,cache=none,werror=stop,rerror=stop,aio=native -device usb-storage,bus=usb.0,port=1,drive=drive-virtio-disk1,id=virtio-disk1,removable=on

results:
error:qemu-kvm: /builddir/build/BUILD/qemu-kvm-0.12.1.2/hw/usb.c:345: usb_packet_complete: Assertion `p->owner != ((void *)0)' failed.

Program received signal SIGABRT, Aborted.
0x0000003560c328a5 in raise () from /lib64/libc.so.6

(gdb) bt
#0  0x0000003560c328a5 in raise () from /lib64/libc.so.6
#1  0x0000003560c34085 in abort () from /lib64/libc.so.6
#2  0x0000003560c2ba1e in __assert_fail_base () from /lib64/libc.so.6
#3  0x0000003560c2bae0 in __assert_fail () from /lib64/libc.so.6
#4  0x00000000004b0739 in usb_packet_complete (dev=<value optimized out>, p=0x1275260) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/usb.c:345
#5  0x00000000004b9eed in scsi_read_complete (opaque=0x134cb10, ret=0) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/scsi-disk.c:151
#6  0x0000000000495287 in qcow2_aio_read_cb (opaque=0x1043b20, ret=<value optimized out>) at block/qcow2.c:558
#7  0x0000000000485bda in qemu_laio_process_completion (s=<value optimized out>, laiocb=0xd5b710) at linux-aio.c:68
#8  0x0000000000485def in qemu_laio_enqueue_completed (opaque=0xd0ac60) at linux-aio.c:107
#9  qemu_laio_completion_cb (opaque=0xd0ac60) at linux-aio.c:144
#10 0x000000000040c3ef in main_loop_wait (timeout=1000) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4024
#11 0x000000000042aeaa in kvm_main_loop () at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:2225
#12 0x000000000040de35 in main_loop (argc=<value optimized out>, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4234
#13 main (argc=<value optimized out>, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:6470

verify this issue with the same CLI and steps.
results:
guest work well,no core dump
Comment 16 Michal Novotny 2012-05-04 09:15:34 EDT
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
NEEDINFO
Comment 17 Gerd Hoffmann 2012-05-04 09:34:39 EDT
    Technical note updated. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    Diffed Contents:
@@ -1 +1,7 @@
-NEEDINFO+Cause: In some cases usb-storage emulation fails to update state correctly on io request cancelation.
+
+Consequence: usb-storage state machine is confused and triggers a assert() in the usb core code, making qemu dump core.
+
+Fix: Handle status update correctly.
+
+Consequence: qemu core dumps are gone.
Comment 18 errata-xmlrpc 2012-06-20 07:45:28 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2012-0746.html

Note You need to log in before you can comment on or make changes to this bug.