Bug 1381630

Summary: QEMU segfaults when using a lot of pci bridges and USB devices
Product: Red Hat Enterprise Linux 7 Reporter: Thomas Huth <thuth>
Component: qemu-kvm-rhevAssignee: Gerd Hoffmann <kraxel>
Status: CLOSED ERRATA QA Contact: Xujun Ma <xuma>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.3CC: chayang, jinzhao, juzhang, kraxel, mdeng, michen, mrezanin, qzhang, thuth, virt-maint, xuma, yduan
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: qemu-2.8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-08-01 23:37:14 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Thomas Huth 2016-10-04 15:24:44 UTC
Description of problem:
When I start QEMU with a lot of pci-bridge and USB devices, the program crashes with a segfault.

Version-Release number of selected component (if applicable):
 qemu-kvm-rhev-2.6.0-27.el7.x86_64
 kernel-3.10.0-512.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
Start QEMU in a shell like this:
/usr/libexec/qemu-kvm -nographic -m 1G \
  -device pci-bridge,chassis_nr=31,id=pci0 \
  `y=0; for ((x=1;x<20;x++)) ; do \
    echo " -device pci-bridge,chassis_nr=$x,id=pci$x,bus=pci$y,addr=$x" \
      ; y=$x ; done ` \
  -device usb-ehci,id=controller0,bus=pci19,addr=1 \
  `for ((x=0;x<29;x++)); do echo -n " -device usb-mouse" ; done` \
  -device usb-tablet,id=usbtablet,bus=controller0.0

Actual results:
QEMU crashes with a segfault.

Expected results:
No crash.

Additional info:
Backtrace with GDB indicates that the problem might be somewhere in the USB code:
Program received signal SIGSEGV, Segmentation fault.
0x00005555558f0036 in usb_device_cancel_packet (dev=0x0, p=0x55555a6176b0) at hw/usb/bus.c:149
149	    if (klass->cancel_packet) {
(gdb) bt
#0  0x00005555558f0036 in usb_device_cancel_packet (dev=0x0, p=0x55555a6176b0) at hw/usb/bus.c:149
#1  0x000055555a616000 in ?? ()
#2  0x00007fffffffcf70 in ?? ()
#3  0x00005555558f0dd6 in usb_device_realize (errp=0x7fffffffcf20, dev=0x55555a616000)
    at hw/usb/bus.c:125
#4  usb_qdev_realize (qdev=0x55555a616000, errp=0x55555a616000) at hw/usb/bus.c:262
#5  0x000055555588a766 in device_set_realized (obj=<optimized out>, value=<optimized out>, 
    errp=0x7fffffffd060) at hw/core/qdev.c:1076
#6  0x000055555595f07e in property_set_bool (obj=0x55555a616000, v=<optimized out>, 
    name=<optimized out>, opaque=0x55555a4eee10, errp=0x7fffffffd060) at qom/object.c:1861
#7  0x0000555555962d47 in object_property_set_qobject (obj=obj@entry=0x55555a616000, 
    value=value@entry=0x55555a26e340, name=name@entry=0x555555a5c66d "realized", 
    errp=errp@entry=0x7fffffffd060) at qom/qom-qobject.c:26
#8  0x0000555555960bc0 in object_property_set_bool (obj=0x55555a616000, value=<optimized out>, 
    name=0x555555a5c66d "realized", errp=0x7fffffffd060) at qom/object.c:1158
#9  0x00005555558efcfc in usb_try_create_simple (bus=<optimized out>, name=0x555555a276b1 "usb-hub", 
    errp=0x0) at hw/usb/bus.c:333
#10 0x00005555558f0945 in usb_claim_port (dev=dev@entry=0x55555a613000, errp=errp@entry=0x7fffffffd0e0)
    at hw/usb/bus.c:441
#11 0x00005555558f0d61 in usb_qdev_realize (qdev=0x55555a613000, errp=0x7fffffffd130)
    at hw/usb/bus.c:256
#12 0x000055555588a766 in device_set_realized (obj=<optimized out>, value=<optimized out>, 
    errp=0x7fffffffd268) at hw/core/qdev.c:1076
#13 0x000055555595f07e in property_set_bool (obj=0x55555a613000, v=<optimized out>, 
    name=<optimized out>, opaque=0x55555a4eed20, errp=0x7fffffffd268) at qom/object.c:1861
#14 0x0000555555962d47 in object_property_set_qobject (obj=obj@entry=0x55555a613000, 
    value=value@entry=0x55555a26e2a0, name=name@entry=0x555555a5c66d "realized", 
    errp=errp@entry=0x7fffffffd268) at qom/qom-qobject.c:26
#15 0x0000555555960bc0 in object_property_set_bool (obj=0x55555a613000, value=<optimized out>, 
    name=0x555555a5c66d "realized", errp=0x7fffffffd268) at qom/object.c:1158
#16 0x0000555555838f9c in qdev_device_add (opts=0x555556c3c870, errp=errp@entry=0x7fffffffd340)
    at qdev-monitor.c:617
#17 0x0000555555842fd7 in device_init_func (opaque=<optimized out>, opts=<optimized out>, 
    errp=<optimized out>) at vl.c:2365
#18 0x0000555555a0b0da in qemu_opts_foreach (list=<optimized out>, 
    func=func@entry=0x555555842fb0 <device_init_func>, opaque=opaque@entry=0x0, errp=errp@entry=0x0)
    at util/qemu-option.c:1116
#19 0x0000555555736ba5 in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>)
    at vl.c:4578

Comment 1 Thomas Huth 2016-10-04 15:36:45 UTC
The problem also occurs with the latest upstream QEMU master (commit bbc4c3f4f3c624e2de64fdcb79f4dd8c1a508e9d)

Comment 3 Gerd Hoffmann 2016-10-05 08:00:58 UTC
> Steps to Reproduce:
> Start QEMU in a shell like this:
> /usr/libexec/qemu-kvm -nographic -m 1G \
>   -device pci-bridge,chassis_nr=31,id=pci0 \
>   `y=0; for ((x=1;x<20;x++)) ; do \
>     echo " -device pci-bridge,chassis_nr=$x,id=pci$x,bus=pci$y,addr=$x" \
>       ; y=$x ; done ` \
>   -device usb-ehci,id=controller0,bus=pci19,addr=1 \
>   `for ((x=0;x<29;x++)); do echo -n " -device usb-mouse" ; done` \
>   -device usb-tablet,id=usbtablet,bus=controller0.0

That command line is incomplete.  You can't plug more than 6 devices into ehci (without companions) as ehci is usb2 and the usb-hub is usb1.  Any chance you have a '-usb' somewhere so all the usb mice end up on the default uhci controller?

Comment 4 Thomas Huth 2016-10-05 09:08:47 UTC
The command line is complete. QEMU seems to crash here before it reaches the sanity check with the free USB ports - if I decrease the number of pci-bridges, I can finally see the "-device usb-mouse: tried to attach usb device QEMU USB Mouse to a bus with no free ports" error message instead.

Anyway, I can also replace the "usb-ehci" with "piix3-usb-uhci" in the above example, and I get a segfault with that, too.

Comment 5 Gerd Hoffmann 2016-10-05 09:36:13 UTC
https://patchwork.ozlabs.org/patch/678377/

Comment 6 Thomas Huth 2016-11-04 09:27:50 UTC
Patch has been merged into upstream master branch:
http://git.qemu.org/?p=qemu.git;a=commitdiff;h=0136464d10f1fd9393a8

Comment 9 Min Deng 2017-05-11 07:31:11 UTC
Reproduced the bug on the following build
qemu-kvm-rhev-2.6.0-27.el7.x86_64
Steps,
1.boot up a guest with the following cli
/usr/libexec/qemu-kvm -nographic -m 1G \
  -device pci-bridge,chassis_nr=31,id=pci0 \
  `y=0; for ((x=1;x<20;x++)) ; do \
    echo " -device pci-bridge,chassis_nr=$x,id=pci$x,bus=pci$y,addr=$x" \
      ; y=$x ; done ` \
  -device usb-ehci,id=controller0,bus=pci19,addr=1 \
  `for ((x=0;x<29;x++)); do echo -n " -device usb-mouse" ; done` \
  -device usb-tablet,id=usbtablet,bus=controller0.0

Actual results,
The qemu-kvm got coredump

Expected results,
There should be a message for sanity tests of free ports
"-device usb-mouse: tried to attach usb device QEMU USB Mouse to a bus with no free ports"

Verified bug on the following builds
qemu-kvm-rhev-2.9.0-3.el7.x86_64
kernel-3.10.0-664.el7.x86_64

Steps,
boot up a guest with the following cli
/usr/libexec/qemu-kvm -nographic -m 1G \
  -device pci-bridge,chassis_nr=31,id=pci0 \
  `y=0; for ((x=1;x<20;x++)) ; do \
    echo " -device pci-bridge,chassis_nr=$x,id=pci$x,bus=pci$y,addr=$x" \
      ; y=$x ; done ` \
  -device usb-ehci,id=controller0,bus=pci19,addr=1 \
  `for ((x=0;x<29;x++)); do echo -n " -device usb-mouse" ; done` \
  -device usb-tablet,id=usbtablet,bus=controller0.0

Actual results,
There was not any coredump but an message was like
"qemu-kvm: -device usb-mouse: tried to attach usb device QEMU USB Mouse to a bus with no free ports"

Expected results,
There should be a message for sanity tests of free ports
"-device usb-mouse: tried to attach usb device QEMU USB Mouse to a bus with no free ports"

 In a summary,the original issue has been fixed already,so move it to verified

Comment 11 errata-xmlrpc 2017-08-01 23:37:14 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392

Comment 12 errata-xmlrpc 2017-08-02 01:14:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392

Comment 13 errata-xmlrpc 2017-08-02 02:06:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392

Comment 14 errata-xmlrpc 2017-08-02 02:47:39 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392

Comment 15 errata-xmlrpc 2017-08-02 03:12:20 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392

Comment 16 errata-xmlrpc 2017-08-02 03:32:30 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392