Bug 230322 - Soft lockup on SMP machine on insertion of USB memory sticks
Summary: Soft lockup on SMP machine on insertion of USB memory sticks
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: 6
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Kernel Maintainer List
QA Contact: Brian Brock
URL:
Whiteboard:
Depends On:
Blocks: 230628
TreeView+ depends on / blocked
 
Reported: 2007-02-28 10:37 UTC by Jonathan Underwood
Modified: 2007-11-30 22:11 UTC (History)
1 user (show)

Fixed In Version: 2.6.19-1.2911.6.4.fc6
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2007-03-05 10:59:21 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
Output of lspci -vv (14.22 KB, text/plain)
2007-02-28 10:41 UTC, Jonathan Underwood
no flags Details
output of lsusb -vv with usb stick inserted (23.14 KB, text/plain)
2007-02-28 10:42 UTC, Jonathan Underwood
no flags Details

Description Jonathan Underwood 2007-02-28 10:37:33 UTC
Description of problem:
When I insert a USB memory stick, after a few seconds (and before the stick is
properly mounted and appearing on the desktop) the machine totally freezes for
3-4 seconds. dmesg reveals a soft lockup occuring:

usb-storage: device found at 14
usb-storage: waiting for device to settle before scanning
scsi 5:0:0:0: Direct-Access              USB DRIVE        2.00 PQ: 0 ANSI: 2
SCSI device sdb: 2023424 512-byte hdwr sectors (1036 MB)
sdb: Write Protect is off
sdb: Mode Sense: 03 00 00 00
sdb: assuming drive cache: write through
SCSI device sdb: 2023424 512-byte hdwr sectors (1036 MB)
sdb: Write Protect is off
sdb: Mode Sense: 03 00 00 00
sdb: assuming drive cache: write through
 sdb:<3>BUG: soft lockup detected on CPU#1!

Call Trace:
 [<ffffffff8026999a>] show_trace+0x34/0x47
 [<ffffffff802699bf>] dump_stack+0x12/0x17
 [<ffffffff802b6ced>] softlockup_tick+0xdb/0xf6
 [<ffffffff80293c2f>] update_process_times+0x42/0x68
 [<ffffffff802749d9>] smp_local_timer_interrupt+0x34/0x55
 [<ffffffff8027508d>] smp_apic_timer_interrupt+0x51/0x69
 [<ffffffff8025ccf6>] apic_timer_interrupt+0x66/0x70
 [<ffffffff802690f2>] mwait_idle_with_hints+0x44/0x45
 [<ffffffff80255543>] mwait_idle+0xc/0x20
 [<ffffffff802476d0>] cpu_idle+0x8b/0xae
 [<ffffffff802747e6>] start_secondary+0x462/0x471

BUG: soft lockup detected on CPU#0!

Call Trace:
 [<ffffffff8026999a>] show_trace+0x34/0x47
 [<ffffffff802699bf>] dump_stack+0x12/0x17
 [<ffffffff802b6ced>] softlockup_tick+0xdb/0xf6
 [<ffffffff80293c2f>] update_process_times+0x42/0x68
 [<ffffffff802749d9>] smp_local_timer_interrupt+0x34/0x55
 [<ffffffff8027508d>] smp_apic_timer_interrupt+0x51/0x69
 [<ffffffff8025ccf6>] apic_timer_interrupt+0x66/0x70
 [<ffffffff8042f098>] rt_check_expire+0x78/0x159
 [<ffffffff80293662>] run_timer_softirq+0x13b/0x1b5
 [<ffffffff80211ee5>] __do_softirq+0x55/0xc4
 [<ffffffff8025d24c>] call_softirq+0x1c/0x30
 [<ffffffff8026aa2f>] do_softirq+0x2c/0x97
 [<ffffffff80275092>] smp_apic_timer_interrupt+0x56/0x69
 [<ffffffff8025ccf6>] apic_timer_interrupt+0x66/0x70
 [<ffffffff80262c03>] _spin_unlock_irq+0xb/0xc
 [<ffff810037e6c180>]
DWARF2 unwinder stuck at 0xffff810037e6c180

Leftover inexact backtrace:

 [<ffffffff8029c362>] autoremove_wake_function+0x0/0x2e
 [<ffffffff802fce27>] kmsg_read+0x3a/0x44
 [<ffffffff8020b226>] vfs_read+0xcb/0x170
 [<ffffffff80211731>] sys_read+0x45/0x6e
 [<ffffffff8025c11e>] system_call+0x7e/0x83

 sdb1
sd 5:0:0:0: Attached scsi removable disk sdb
sd 5:0:0:0: Attached scsi generic sg2 type 0
usb-storage: device scan complete
SELinux: initialized (dev sdb1, type vfat), uses genfs_contexts


Version-Release number of selected component (if applicable):
2.6.19-1.2911.fc6 #1 SMP

How reproducible:
Everytime

Steps to Reproduce:
1.Insert memory stick
2.Sit back, enjoy the show.
3.
  
Actual results:
Machine temporarily hangs, soft lockup occurs

Expected results:
No lockup :)

Additional info:
I'll attach the output of lspci in a sec.

Comment 1 Jonathan Underwood 2007-02-28 10:41:15 UTC
Created attachment 148903 [details]
Output of lspci -vv

Comment 2 Jonathan Underwood 2007-02-28 10:42:08 UTC
Created attachment 148904 [details]
output of lsusb -vv with usb stick inserted

Comment 3 Jonathan Underwood 2007-02-28 10:45:15 UTC
I should also add that I've seen this bug with kernels back as far as
kernel.x86_64 2.6.18-1.2869.fc6

Comment 4 Jonathan Underwood 2007-02-28 11:02:55 UTC
I should also add that booting with noapic doesn't fix the problem, and the
machine doesn't manage to boot with the nolapic option.

Comment 5 Pete Zaitcev 2007-02-28 18:00:00 UTC
I officially hate the unwinder. What a useless pile of garbage.
So now there's no telling where the either of CPUs got stuck...
I wanted to take this bug, but I'm not even sure that usb-storage
is actually a culprit.

BTW, what does happen if you boot with libusual.bias="ub" in grub.conf?


Comment 6 Jonathan Underwood 2007-03-01 11:07:15 UTC
Hi Pete, thanks for your response. Adding libusual.bias="ub" fixes the problem,
once I had disabled SElinux.  I'm not sure that the problem is specific to the
usb-storeage layer though, as I am also seeing soft lockups when vmware tries to
create its virtual ethernet interfaces.  These also disappear with
libusual.bias="ub"

[Just to put your mind at rest though - the problem originally reported in this
bug is present with an untainted kernel (i.e. without the vmware module loaded).]


As an aside, if there are any plans to enable libusual.bias="ub" out of the box,
then I guess the SElinux issue will need fixing up. The SElinux messages
displayed are:
 audit(1172746861.823:7): avc:  denied  { read } for  pid=5340
comm="hald-probe-volu" name="uba1" dev=tmpfs ino=20335
scontext=system_u:system_r:hald_t:s0 tcontext=system_u:object_r:device_t:s0
tclass=blk_file
audit(1172746861.824:8): avc:  denied  { ioctl } for  pid=5340
comm="hald-probe-volu" name="uba1" dev=tmpfs ino=20335
scontext=system_u:system_r:hald_t:s0 tcontext=system_u:object_r:device_t:s0
tclass=blk_file
SELinux: initialized (dev uba1, type vfat), uses genfs_contexts
audit(1172746862.109:9): avc:  denied  { getattr } for  pid=4470 comm="hald"
name="uba1" dev=tmpfs ino=20335 scontext=system_u:system_r:hald_t:s0
tcontext=system_u:object_r:device_t:s0 tclass=blk_file

Comment 7 Pete Zaitcev 2007-03-01 21:15:21 UTC
Thanks for the testing, Jonathan. I'll clone this bug for Dan Walsh regarding
the SElinux issue.

But ub is a workaround. The usb-storage is the official driver, so I'll need
to fix that. There's also a bug 226742 where the requestor managed to capture
a useable trace without unwinder. I'll work under assumption that these
two are the same, and we'll take it from there.


Comment 8 Jonathan Underwood 2007-03-05 10:59:21 UTC
Hi Pete,

I'm happy to report that this bug has vanished with the latest kernel
(2.6.19-1.2911.6.4.fc6), and so I'll close the bug accordingly.


Note You need to log in before you can comment on or make changes to this bug.