Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 924149

Summary: guest soft lockup when booting rhel7.0 with more than 5 usb-storage in a xhci controller
Product: Red Hat Enterprise Linux 7 Reporter: FuXiangChun <xfu>
Component: qemu-kvmAssignee: Gerd Hoffmann <kraxel>
Status: CLOSED CURRENTRELEASE QA Contact: Virtualization Bugs <virt-bugs>
Severity: high Docs Contact:
Priority: high    
Version: 7.0CC: acathrow, hhuang, juzhang, michen, mrezanin, rhod, sluo, virt-maint
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: qemu-kvm-1.5.0-1.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-06-13 09:28:22 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 920108    

Description FuXiangChun 2013-03-21 09:55:08 UTC
Description of problem:
First time I boot rhel7.0 guest with multiple usb-storage(more than 5). qemu will take several minutes to boot guest because soft lockup occurred. Boot usb-storage the more then will spent time longer. Too many usb-storage maybe lead to qemu core dump.

Version-Release number of selected component (if applicable):
host and guest kernel version:
# uname -r 
3.8.0-0.40.el7.x86_64

qemu version:
qemu-kvm-1.4.0-1.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1./usr/libexec/qemu-kvm -M q35 -enable-kvm -m 4096 -smp 4,sockets=2,cores=2,threads=1 -no-kvm-pit-reinjection -uuid 350e716b-5f98-4bf0-9a2a-c8e423295244 -rtc base=localtime,clock=host,driftfix=slew -drive file=/home/rhel7-new-bak.qcow2,if=none,id=drive-system-disk,format=qcow2,aio=native,werror=stop,rerror=stop -device ide-drive,drive=drive-system-disk,id=system-disk,bootindex=1 -netdev tap,id=hostnet0,vhost=on -device rtl8139,netdev=hostnet0,id=virtio-net-pci0,mac=08:2E:5F:0A:1D:B2 -vnc :1 -monitor stdio -serial unix:/tmp/tty0,server,nowait \
-device nec-usb-xhci,id=xhci0 \
-drive file=storage/usb-storage1.qcow2,if=none,id=drive-usb-0-1,media=disk,format=qcow2,cache=none,aio=native -device usb-storage,drive=drive-usb-0-1,id=usb-0-1,removable=on  \
-drive file=storage/usb-storage2.qcow2,if=none,id=drive-usb-0-2,media=disk,format=qcow2,cache=none,aio=native -device usb-storage,drive=drive-usb-0-2,id=usb-0-2,removable=on \
-drive file=storage/usb-storage3.qcow2,if=none,id=drive-usb-0-3,media=disk,format=qcow2,cache=none,aio=native -device usb-storage,drive=drive-usb-0-3,id=usb-0-3,removable=on \
-drive file=storage/usb-storage4.qcow2,if=none,id=drive-usb-0-4,media=disk,format=qcow2,cache=none,aio=native -device usb-storage,drive=drive-usb-0-4,id=usb-0-4,removable=on \
-drive file=storage/usb-storage5.qcow2,if=none,id=drive-usb-0-5,media=disk,format=qcow2,cache=none,aio=native -device usb-storage,drive=drive-usb-0-5,id=usb-0-5,removable=on \
-drive file=storage/usb-storage6.qcow2,if=none,id=drive-usb-0-6,media=disk,format=qcow2,cache=none,aio=native -device usb-storage,drive=drive-usb-0-6,id=usb-0-6,removable=on \
-drive file=storage/usb-storage7.qcow2,if=none,id=drive-usb-0-7,media=disk,format=qcow2,cache=none,aio=native -device usb-storage,drive=drive-usb-0-7,id=usb-0-7,removable=on \
2.
3.
  
Actual results:
]# nc -U /tmp/tty0
[    0.024000] Failed to access perfctr msr (MSR c0010001 is ffffffffffffffff)
[   28.050008] BUG: soft lockup - CPU#2 stuck for 23s! [systemd-udevd:162]
[   56.050010] BUG: soft lockup - CPU#2 stuck for 23s! [systemd-udevd:162]
[   84.050006] BUG: soft lockup - CPU#2 stuck for 23s! [systemd-udevd:162]
[  112.050007] BUG: soft lockup - CPU#2 stuck for 22s! [systemd-udevd:162]
[  140.050006] BUG: soft lockup - CPU#2 stuck for 22s! [systemd-udevd:162]
[  168.050009] BUG: soft lockup - CPU#2 stuck for 22s! [systemd-udevd:162]
[  196.050008] BUG: soft lockup - CPU#2 stuck for 22s! [systemd-udevd:162]
[  224.050006] BUG: soft lockup - CPU#2 stuck for 22s! [systemd-udevd:162]
[  252.050014] BUG: soft lockup - CPU#2 stuck for 22s! [systemd-udevd:162]
[  280.050006] BUG: soft lockup - CPU#2 stuck for 22s! [systemd-udevd:162]
[  308.050006] BUG: soft lockup - CPU#2 stuck for 22s! [systemd-udevd:162]
[  336.050006] BUG: soft lockup - CPU#2 stuck for 22s! [systemd-udevd:162]
[  364.050009] BUG: soft lockup - CPU#2 stuck for 23s! [systemd-udevd:162]
[  392.050009] BUG: soft lockup - CPU#2 stuck for 23s! [systemd-udevd:162]
[  420.050006] BUG: soft lockup - CPU#2 stuck for 23s! [systemd-udevd:162]
[  448.050009] BUG: soft lockup - CPU#2 stuck for 23s! [systemd-udevd:162]
[  476.050006] BUG: soft lockup - CPU#2 stuck for 22s! [systemd-udevd:162]
[  504.050010] BUG: soft lockup - CPU#2 stuck for 22s! [systemd-udevd:162]
[    1.694596] INFO: rcu_sched self-detected stall on CPU[  506.606956] INFO: rcu_sched detected stalls on CPUs/tasks:
[  OK  ] Started Show Plymouth Boot Screen.

........

Expected results:
without soft lockup

Additional info:

Comment 2 FuXiangChun 2013-04-01 06:19:15 UTC
As this issue cause guest will take long time to boot. so set bug's priority and severity as high high.

Comment 3 Gerd Hoffmann 2013-04-23 10:22:30 UTC
*** Bug 955571 has been marked as a duplicate of this bug. ***

Comment 4 Gerd Hoffmann 2013-05-14 13:37:58 UTC
Fixed by upstream commit a309ee6e0a256f690760abfba44fceaa52a7c2f3.

Comment 5 Miroslav Rezanina 2013-05-23 12:05:52 UTC
Build in qemu-kvm-1.5.0-1.el7

Comment 6 Sibiao Luo 2013-07-05 10:02:55 UTC
Reproduce this issue qemu-kvm-1.4.0-1.el7.x86_64 with the same step as comment #0. the guest kernel dmesg will output 'BUG: soft lockup - CPU#2 stuck for 23s! [systemd-udevd:162]', and qemu core dump.

Verify this issue on qemu-kvm-1.5.1-2.el7.x86_64 with 9 usb storage.
host info:
3.10.0-0.rc7.64.el7.x86_64
qemu-kvm-1.5.1-2.el7.x86_64
guest info:
3.10.0-0.rc7.64.el7.x86_64

# /usr/libexec/qemu-kvm -M q35 -cpu SandyBridge...-device nec-usb-xhci,id=xhci,bus=bridge1,addr=0x8 -drive file=/home/my-usb-storage1.qcow3,if=none,id=storage1,media=disk,cache=none,format=qcow2 -device usb-storage,bus=xhci.0,drive=storage1,id=usb-storage1 -drive file=/home/my-usb-storage2.qcow3,if=none,id=storage2,media=disk,cache=none,format=qcow2 -device usb-storage,bus=xhci.0,drive=storage2,id=usb-storage2 -drive file=/home/my-usb-storage3.qcow3,if=none,id=storage3,media=disk,cache=none,format=qcow2 -device usb-storage,bus=xhci.0,drive=storage3,id=usb-storage3 -drive file=/home/my-usb-storage4.qcow3,if=none,id=storage4,media=disk,cache=none,format=qcow2 -device usb-storage,bus=xhci.0,drive=storage4,id=usb-storage4 -drive file=/home/my-usb-storage5.qcow3,if=none,id=storage5,media=disk,cache=none,format=qcow2 -device usb-storage,bus=xhci.0,drive=storage5,id=usb-storage5 -drive file=/home/my-usb-storage6.qcow3,if=none,id=storage6,media=disk,cache=none,format=qcow2 -device usb-storage,bus=xhci.0,drive=storage6,id=usb-storage6 -drive file=/home/my-usb-storage7.qcow3,if=none,id=storage7,media=disk,cache=none,format=qcow2 -device usb-storage,bus=xhci.0,drive=storage7,id=usb-storage7 -drive file=/home/my-usb-storage8.qcow3,if=none,id=storage8,media=disk,cache=none,format=qcow2 -device usb-storage,bus=xhci.0,drive=storage8,id=usb-storage8 -drive file=/home/my-usb-storage9.qcow3,if=none,id=storage9,media=disk,cache=none,format=qcow2 -device usb-storage,bus=xhci.0,drive=storage9,id=usb-storage9
(qemu) info block
drive-system-disk: removable=0 io-status=ok file=/home/RHEL-7.0-20130628.0-Server-x86_64.qcow3 ro=0 drv=qcow2 encrypted=0 bps=0 bps_rd=0 bps_wr=0 iops=0 iops_rd=0 iops_wr=0
storage1: removable=0 io-status=ok file=/home/my-usb-storage1.qcow3 ro=0 drv=qcow2 encrypted=0 bps=0 bps_rd=0 bps_wr=0 iops=0 iops_rd=0 iops_wr=0
storage2: removable=0 io-status=ok file=/home/my-usb-storage2.qcow3 ro=0 drv=qcow2 encrypted=0 bps=0 bps_rd=0 bps_wr=0 iops=0 iops_rd=0 iops_wr=0
storage3: removable=0 io-status=ok file=/home/my-usb-storage3.qcow3 ro=0 drv=qcow2 encrypted=0 bps=0 bps_rd=0 bps_wr=0 iops=0 iops_rd=0 iops_wr=0
storage4: removable=0 io-status=ok file=/home/my-usb-storage4.qcow3 ro=0 drv=qcow2 encrypted=0 bps=0 bps_rd=0 bps_wr=0 iops=0 iops_rd=0 iops_wr=0
storage5: removable=0 io-status=ok file=/home/my-usb-storage5.qcow3 ro=0 drv=qcow2 encrypted=0 bps=0 bps_rd=0 bps_wr=0 iops=0 iops_rd=0 iops_wr=0
storage6: removable=0 io-status=ok file=/home/my-usb-storage6.qcow3 ro=0 drv=qcow2 encrypted=0 bps=0 bps_rd=0 bps_wr=0 iops=0 iops_rd=0 iops_wr=0
storage7: removable=0 io-status=ok file=/home/my-usb-storage7.qcow3 ro=0 drv=qcow2 encrypted=0 bps=0 bps_rd=0 bps_wr=0 iops=0 iops_rd=0 iops_wr=0
storage8: removable=0 io-status=ok file=/home/my-usb-storage8.qcow3 ro=0 drv=qcow2 encrypted=0 bps=0 bps_rd=0 bps_wr=0 iops=0 iops_rd=0 iops_wr=0
storage9: removable=0 io-status=ok file=/home/my-usb-storage9.qcow3 ro=0 drv=qcow2 encrypted=0 bps=0 bps_rd=0 bps_wr=0 iops=0 iops_rd=0 iops_wr=0

The guest dmesg did not have any the error, and all the usb storage in guest work well.

Base on above, this issue has been fixed correctly, set to verified status.

Best Regards,
sluo

Comment 7 Ludek Smid 2014-06-13 09:28:22 UTC
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.