Bug 439653

Summary:	Rawhide has a broken rt2x00 driver
Product:	[Fedora] Fedora	Reporter:	Arjan van de Ven <arjan>
Component:	kernel	Assignee:	John W. Linville <linville>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	high	Docs Contact:
Priority:	low
Version:	9	CC:	colin_sh, kernel-maint
Target Milestone:	---
Target Release:	---
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2008-08-19 18:35:03 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Arjan van de Ven 2008-03-30 01:12:16 UTC

Description of problem:

The rt2x00 driver set in rawhide is rather broken and is responsible for 2 out 
of the top 10 kernel oopses in the one-week kenreloops.org rollup.

The most specific problem is that rt2x00usb_vendor_request_buff() is a 
sleeping function, but it gets called from rt73usb_config_filter (and 
rt2x00mac_configure_filter)... which are called with various locks held.
There are so many reports that it's well possible there are more problems than 
these.

This bug is very obvious and is a 100% hit, which makes me wonder if the 
developers of the driver have actually tested this code.

The code that does this is not present in the kernel.org kernel.

http://www.kerneloops.org/search.php?search=rt2x00usb_vendor_request_buff
http://www.kerneloops.org/search.php?search=rtnetlink_rcv

Comment 1 Arjan van de Ven 2008-03-30 01:25:59 UTC

http://www.kerneloops.org/searchweek.php?search=_spin_unlock_irqrestore

this one is not top 10 (only top 13) but still

Comment 2 Arjan van de Ven 2008-03-31 15:30:10 UTC

http://www.kerneloops.org/searchweek.php?search=__ieee80211_rx

another one

Comment 3 John W. Linville 2008-04-07 13:54:44 UTC

I think the original report and comment 1 are resolved, still investigating 
comment 2...

Comment 4 Bug Zapper 2008-05-14 08:25:33 UTC

Changing version to '9' as part of upcoming Fedora 9 GA.
More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 5 Colin Lai 2008-06-19 03:23:22 UTC

Kernel(2.6.25.6-55.fc9.i686.debug) dump when I want to shutdown my rt73 wireless
card (ifconfig wlan3 down), card was in mangled mode, connected to a public ssid.
command list:
iwconfig wlan3 essid any
ifconfig wlan3 up
(connected to ssid)
ifconfig wlan3 down
(kernel dump)


Workaround: use 2.6.24.7-92.fc8debug kernel.

Dump
2.6.25.6-55.fc9.i686.debug
2.6.25.6-55.fc9.i686

============kernel dump 1 begin===============
usbcore: registered new interface driver rt73usb
udev: renamed network interface wlan0 to wlan4
ADDRCONF(NETDEV_UP): wlan4: link is not ready
wlan4: Initial auth_alg=0
wlan4: authenticate with AP 00:02:8a:42:1e:70
wlan4: RX authentication from 00:02:8a:42:1e:70 (alg=0 transaction=2 status=0)
wlan4: authenticated
wlan4: associate with AP 00:02:8a:42:1e:70
wlan4: RX AssocResp from 00:02:8a:42:1e:70 (capab=0x421 status=0 aid=232)
wlan4: associated
wlan4: switched to short barker preamble (BSSID=00:02:8a:42:1e:70)
ADDRCONF(NETDEV_CHANGE): wlan4: link becomes ready

=======================================================
[ INFO: possible circular locking dependency detected ]
2.6.25.6-55.fc9.i686.debug #1
-------------------------------------------------------
ifconfig/2370 is trying to acquire lock:
 ((name)){--..}, at: [<c0437442>] flush_workqueue+0x0/0x85

but task is already holding lock:
 (rtnl_mutex){--..}, at: [<c05d81a1>] rtnl_lock+0xf/0x11

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #2 (rtnl_mutex){--..}:
       [<c0445f2c>] __lock_acquire+0xa99/0xc11
       [<c044610e>] lock_acquire+0x6a/0x90
       [<c06431e9>] mutex_lock_nested+0xdb/0x271
       [<c05d81a1>] rtnl_lock+0xf/0x11
       [<e0b29997>] ieee80211_iterate_active_interfaces+0x1c/0x79 [mac80211]
       [<e0aa105e>] rt2x00lib_intf_scheduled+0x16/0x18 [rt2x00lib]
       [<c0436c04>] run_workqueue+0xd3/0x1a1
       [<c0436d88>] worker_thread+0xb6/0xc2
       [<c04399b2>] kthread+0x3b/0x61
       [<c040690b>] kernel_thread_helper+0x7/0x10
       [<ffffffff>] 0xffffffff

-> #1 (&rt2x00dev->intf_work){--..}:
       [<c0445f2c>] __lock_acquire+0xa99/0xc11
       [<c044610e>] lock_acquire+0x6a/0x90
       [<c0436bfe>] run_workqueue+0xcd/0x1a1
       [<c0436d88>] worker_thread+0xb6/0xc2
       [<c04399b2>] kthread+0x3b/0x61
       [<c040690b>] kernel_thread_helper+0x7/0x10
       [<ffffffff>] 0xffffffff

-> #0 ((name)){--..}:
       [<c0445e4b>] __lock_acquire+0x9b8/0xc11
       [<c044610e>] lock_acquire+0x6a/0x90
       [<c0437486>] flush_workqueue+0x44/0x85
       [<e0b1649d>] ieee80211_stop+0x279/0x334 [mac80211]
       [<c05d09fb>] dev_close+0x52/0x6f
       [<c05d0752>] dev_change_flags+0x9a/0x14d
       [<c060e28c>] devinet_ioctl+0x218/0x52c
       [<c060f0ec>] inet_ioctl+0x8c/0xa8
       [<c05c4c81>] sock_ioctl+0x1b8/0x1da
       [<c04958b6>] vfs_ioctl+0x22/0x69
       [<c0495b36>] do_vfs_ioctl+0x239/0x24c
       [<c0495b89>] sys_ioctl+0x40/0x5b
       [<c0405c2e>] syscall_call+0x7/0xb
       [<ffffffff>] 0xffffffff

other info that might help us debug this:

1 lock held by ifconfig/2370:
 #0:  (rtnl_mutex){--..}, at: [<c05d81a1>] rtnl_lock+0xf/0x11

stack backtrace:
Pid: 2370, comm: ifconfig Not tainted 2.6.25.6-55.fc9.i686.debug #1
 [<c044527d>] print_circular_bug_tail+0x5b/0x66
 [<c04450f0>] ? print_circular_bug_entry+0x39/0x43
 [<c0445e4b>] __lock_acquire+0x9b8/0xc11
 [<c0444deb>] ? mark_held_locks+0x4e/0x66
 [<c044610e>] lock_acquire+0x6a/0x90
 [<c0437442>] ? flush_workqueue+0x0/0x85
 [<c0437486>] flush_workqueue+0x44/0x85
 [<c0437442>] ? flush_workqueue+0x0/0x85
 [<e0b1649d>] ieee80211_stop+0x279/0x334 [mac80211]
 [<c0644816>] ? _spin_unlock_bh+0x25/0x28
 [<c05d09fb>] dev_close+0x52/0x6f
 [<c05d0752>] dev_change_flags+0x9a/0x14d
 [<c060e28c>] devinet_ioctl+0x218/0x52c
 [<c060f0ec>] inet_ioctl+0x8c/0xa8
 [<c05c4c81>] sock_ioctl+0x1b8/0x1da
 [<c05c4ac9>] ? sock_ioctl+0x0/0x1da
 [<c04958b6>] vfs_ioctl+0x22/0x69
 [<c0495b36>] do_vfs_ioctl+0x239/0x24c
 [<c0495b89>] sys_ioctl+0x40/0x5b
 [<c0405c2e>] syscall_call+0x7/0xb
 =======================
BUG: unable to handle kernel paging request at 09263040
IP: [<c05898ca>] usb_hcd_unlink_urb+0xf/0x5a
*pde = 00000000
Oops: 0000 [#1] SMP
Modules linked in: arc4 ecb crypto_blkcipher rt73usb crc_itu_t rt2x00usb
rt2x00lib rfkill input_polldev mac80211 cfg80211 ipv6 ipt_REJECT xt_tcpudp
nf_conntrack_ipv4 xt_state nf_conntrack iptable_filter ip_tables x_tables
dm_mirror dm_mod ppdev snd_intel8x0 snd_usb_audio snd_usb_lib snd_rawmidi
snd_hwdep dcdbas parport_pc parport snd_seq_dummy floppy snd_intel8x0m
snd_ac97_codec ac97_bus snd_seq_oss snd_seq_midi_event snd_seq pcspkr
snd_seq_device snd_pcm_oss 3c59x snd_mixer_oss mii joydev snd_pcm snd_timer
i2c_i801 snd soundcore i2c_core snd_page_alloc iTCO_wdt iTCO_vendor_support sg
battery video ac output button sr_mod cdrom ata_piix pata_acpi ata_generic
libata sd_mod scsi_mod ext3 jbd mbcache uhci_hcd ohci_hcd ehci_hcd [last
unloaded: scsi_wait_scan]

Pid: 2370, comm: ifconfig Not tainted (2.6.25.6-55.fc9.i686.debug #1)
EIP: 0060:[<c05898ca>] EFLAGS: 00010246 CPU: 0
EIP is at usb_hcd_unlink_urb+0xf/0x5a
EAX: 09263010 EBX: cccccccc ECX: fffffffe EDX: cccccccc
ESI: d9229040 EDI: cccccccc EBP: df158dd4 ESP: df158dc8
 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process ifconfig (pid: 2370, ti=df158000 task=d9680000 task.ti=df158000)
Stack: cccccccc d9229040 d9229040 df158dfc c058ab7a 00000000 d9680000 c0439c03
       df158de8 df158de8 00000002 d9229040 d9229040 df158e0c e0a9ba55 df158e50
       d92282c0 df158e60 e0ab689c df158e24 00000246 e0b0fe08 fffffff4 df158e30
Call Trace:
 [<c058ab7a>] ? usb_kill_urb+0x60/0xdc
 [<c0439c03>] ? autoremove_wake_function+0x0/0x33
 [<e0a9ba55>] ? rt2x00usb_disable_radio+0x64/0x7a [rt2x00usb]
 [<e0ab689c>] ? rt73usb_set_device_state+0xc2d/0xd35 [rt73usb]
 [<c0644816>] ? _spin_unlock_bh+0x25/0x28
 [<e0ae9c59>] ? fib6_run_gc+0xbc/0xbf [ipv6]
 [<e0aa1800>] ? rt2x00lib_disable_radio+0x62/0x77 [rt2x00lib]
 [<e0aa1bd2>] ? rt2x00lib_stop+0x14/0x43 [rt2x00lib]
 [<e0aa22ed>] ? rt2x00mac_stop+0x14/0x16 [rt2x00lib]
 [<e0b16517>] ? ieee80211_stop+0x2f3/0x334 [mac80211]
 [<c05d09fb>] ? dev_close+0x52/0x6f
 [<c05d0752>] ? dev_change_flags+0x9a/0x14d
 [<c060e28c>] ? devinet_ioctl+0x218/0x52c
 [<c060f0ec>] ? inet_ioctl+0x8c/0xa8
 [<c05c4c81>] ? sock_ioctl+0x1b8/0x1da
 [<c05c4ac9>] ? sock_ioctl+0x0/0x1da
 [<c04958b6>] ? vfs_ioctl+0x22/0x69
 [<c0495b36>] ? do_vfs_ioctl+0x239/0x24c
 [<c0495b89>] ? sys_ioctl+0x40/0x5b
 [<c0405c2e>] ? syscall_call+0x7/0xb
 =======================
Code: 00 00 89 d8 e8 61 16 00 00 8b 45 e0 8b 55 e8 39 50 0c 75 b2 8d 65 f4 5b 5e
5f 5d c3 55 89 d1 89 e5 57 89 c7 56 89 fa 53 8b 40 28 <8b> 40 30 e8 ab fd ff ff
85 c0 89 c6 75 07 be 8d ff ff ff eb 2c
EIP: [<c05898ca>] usb_hcd_unlink_urb+0xf/0x5a SS:ESP 0068:df158dc8
---[ end trace cd4e858566a406f8 ]---
wlan4: no IPv6 routers present

============kernel dump 1 end===============


============kernel dump 2 begin===============
ADDRCONF(NETDEV_UP): wlan3: link is not ready
wlan3: Initial auth_alg=0
wlan3: authenticate with AP 00:02:8a:42:1e:70
wlan3: RX authentication from 00:02:8a:42:1e:70 (alg=0 transaction=2 status=0)
wlan3: authenticated
wlan3: associate with AP 00:02:8a:42:1e:70
wlan3: associate with AP 00:02:8a:42:1e:70
wlan3: RX AssocResp from 00:02:8a:42:1e:70 (capab=0x421 status=0 aid=231)
wlan3: associated
wlan3: switched to short barker preamble (BSSID=00:02:8a:42:1e:70)
ADDRCONF(NETDEV_CHANGE): wlan3: link becomes ready
BUG: unable to handle kernel paging request at 4589ec45
IP: [<4589ec45>]
*pde = 00000000
Oops: 0000 [#1] SMP
Modules linked in: ipv6 ipt_REJECT xt_tcpudp nf_conntrack_ipv4 xt_state
nf_conntrack iptable_filter ip_tables x_tables dm_mirror dm_mod ppdev
snd_intel8x0m snd_usb_audio snd_usb_lib snd_rawmidi snd_hwdep dcdbas parport_pc
parport floppy rt2500usb arc4 pcspkr joydev ecb crypto_blkcipher battery
snd_seq_dummy ac video 3c59x snd_intel8x0 output snd_ac97_codec mii ac97_bus
snd_seq_oss snd_seq_midi_event rt73usb snd_seq crc_itu_t button snd_seq_device
rt2x00usb snd_pcm_oss rt2x00lib snd_mixer_oss snd_pcm rfkill input_polldev
snd_timer snd mac80211 i2c_i801 soundcore snd_page_alloc cfg80211 iTCO_wdt
i2c_core iTCO_vendor_support sr_mod cdrom sg ata_piix pata_acpi ata_generic
libata sd_mod scsi_mod ext3 jbd mbcache uhci_hcd ohci_hcd ehci_hcd [last
unloaded: scsi_wait_scan]

Pid: 2277, comm: ifconfig Not tainted (2.6.25.6-55.fc9.i686 #1)
EIP: 0060:[<4589ec45>] EFLAGS: 00010202 CPU: 0
EIP is at 0x4589ec45
EAX: dec2d054 EBX: e0851476 ECX: fffffffe EDX: dec2d000
ESI: dec2d054 EDI: dec2d000 EBP: df14edc8 ESP: df14edb0
 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process ifconfig (pid: 2277, ti=df14e000 task=df148e70 task.ti=df14e000)
Stack: c0574a72 fffffffe 00000246 dec2d000 dfb9ee60 dfb9ee60 df14edd4 c0574b7c
       dec2d000 df14edfc c0575c6c 00000000 df148e70 c0437b0f df14ede8 df14ede8
       00000002 dfb9ee60 dfb9ee60 df14ee0c e091ea55 df14ee50 dfb9e1a0 df14ee60
Call Trace:
 [<c0574a72>] ? unlink1+0xab/0xb6
 [<c0574b7c>] ? usb_hcd_unlink_urb+0x15/0x22
 [<c0575c6c>] ? usb_kill_urb+0x5e/0xd8
 [<c0437b0f>] ? autoremove_wake_function+0x0/0x33
 [<e091ea55>] ? rt2x00usb_disable_radio+0x64/0x7a [rt2x00usb]
 [<e09a589b>] ? rt73usb_set_device_state+0xc2d/0xd35 [rt73usb]
 [<c042f764>] ? mod_timer+0x55/0x59
 [<c062b092>] ? _spin_unlock_bh+0xe/0x10
 [<e0b4659c>] ? fib6_run_gc+0xbc/0xbf [ipv6]
 [<e09767f7>] ? rt2x00lib_disable_radio+0x62/0x77 [rt2x00lib]
 [<e0976b81>] ? rt2x00lib_stop+0x14/0x43 [rt2x00lib]
 [<e0977285>] ? rt2x00mac_stop+0x14/0x16 [rt2x00lib]
 [<e0938420>] ? ieee80211_stop+0x2c5/0x306 [mac80211]
 [<c05ba115>] ? dev_close+0x52/0x6f
 [<c05b9e6c>] ? dev_change_flags+0x9a/0x14d
 [<c05f652a>] ? devinet_ioctl+0x218/0x52a
 [<c05f7316>] ? inet_ioctl+0x8c/0xa8
 [<c05aeb43>] ? sock_ioctl+0x1af/0x1d1
 [<c05ae994>] ? sock_ioctl+0x0/0x1d1
 [<c048c546>] ? vfs_ioctl+0x22/0x69
 [<c048c7c6>] ? do_vfs_ioctl+0x239/0x24c
 [<c048c819>] ? sys_ioctl+0x40/0x5b
 [<c0405bf2>] ? syscall_call+0x7/0xb
 [<c0620000>] ? agp_amd64_probe+0x2e4/0x3ee
 =======================
Code:  Bad EIP value.
EIP: [<4589ec45>] 0x4589ec45 SS:ESP 0068:df14edb0
---[ end trace c277f34575aba786 ]---

============kernel dump 2 end ===============

Comment 6 Colin Lai 2008-06-19 03:26:21 UTC

Kernel(2.6.25.6-55.fc9.i686.debug) dump when I want to shutdown my rt73 wireless
card (ifconfig wlan3 down), card was in monitor mode.

command list:
iwconfig wlan3 mode monitor
ifconfig wlan3 up
ifconfig wlan3 down
(kernel dump)


Workaround: use 2.6.24.7-92.fc8debug kernel.


============kernel dump 3 begin ===============
BUG: unable to handle kernel paging request at 09263040
IP: [<c05898ca>] usb_hcd_unlink_urb+0xf/0x5a
*pde = 00000000
Oops: 0000 [#1] SMP
Modules linked in: ipv6 ipt_REJECT xt_tcpudp nf_conntrack_ipv4 xt_state
nf_conntrack iptable_filter ip_tables x_tables dm_mirror dm_mod ppdev
snd_intel8x0 snd_intel8x0m snd_ac97_codec ac97_bus snd_usb_audio snd_usb_lib
snd_rawmidi snd_hwdep dcdbas snd_seq_dummy parport_pc parport snd_seq_oss floppy
snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss joydev
snd_pcm 3c59x pcspkr i2c_i801 snd_timer mii snd i2c_core soundcore
snd_page_alloc iTCO_wdt iTCO_vendor_support sg arc4 ecb battery crypto_blkcipher
video ac output button rt73usb crc_itu_t rt2x00usb rt2x00lib rfkill
input_polldev mac80211 cfg80211 sr_mod cdrom ata_piix pata_acpi ata_generic
libata sd_mod scsi_mod ext3 jbd mbcache uhci_hcd ohci_hcd ehci_hcd [last
unloaded: scsi_wait_scan]

Pid: 2384, comm: ifconfig Not tainted (2.6.25.6-55.fc9.i686.debug #1)
EIP: 0060:[<c05898ca>] EFLAGS: 00010246 CPU: 0
EIP is at usb_hcd_unlink_urb+0xf/0x5a
EAX: 09263010 EBX: cccccccc ECX: fffffffe EDX: cccccccc
ESI: df391040 EDI: cccccccc EBP: db092dd4 ESP: db092dc8
 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process ifconfig (pid: 2384, ti=db092000 task=db086000 task.ti=db092000)
Stack: cccccccc df391040 df391040 db092dfc c058ab7a 00000000 db086000 c0439c03
       db092de8 db092de8 00000002 df391040 df391040 db092e0c e08f8a55 db092e50
       df3902c0 db092e60 e090889c db092e24 00000246 e0b77e08 fffffff2 db092e30
Call Trace:
 [<c058ab7a>] ? usb_kill_urb+0x60/0xdc
 [<c0439c03>] ? autoremove_wake_function+0x0/0x33
 [<e08f8a55>] ? rt2x00usb_disable_radio+0x64/0x7a [rt2x00usb]
 [<e090889c>] ? rt73usb_set_device_state+0xc2d/0xd35 [rt73usb]
 [<c0644816>] ? _spin_unlock_bh+0x25/0x28
 [<e0b51c59>] ? fib6_run_gc+0xbc/0xbf [ipv6]
 [<e08cd800>] ? rt2x00lib_disable_radio+0x62/0x77 [rt2x00lib]
 [<e08cdbd2>] ? rt2x00lib_stop+0x14/0x43 [rt2x00lib]
 [<e08ce2ed>] ? rt2x00mac_stop+0x14/0x16 [rt2x00lib]
 [<e0932517>] ? ieee80211_stop+0x2f3/0x334 [mac80211]
 [<c0644816>] ? _spin_unlock_bh+0x25/0x28
 [<c05d09fb>] ? dev_close+0x52/0x6f
 [<c05d0752>] ? dev_change_flags+0x9a/0x14d
 [<c060e28c>] ? devinet_ioctl+0x218/0x52c
 [<c060f0ec>] ? inet_ioctl+0x8c/0xa8
 [<c05c4c81>] ? sock_ioctl+0x1b8/0x1da
 [<c05c4ac9>] ? sock_ioctl+0x0/0x1da
 [<c04958b6>] ? vfs_ioctl+0x22/0x69
 [<c0495b36>] ? do_vfs_ioctl+0x239/0x24c
 [<c0495b89>] ? sys_ioctl+0x40/0x5b
 [<c0405c2e>] ? syscall_call+0x7/0xb
 =======================
Code: 00 00 89 d8 e8 61 16 00 00 8b 45 e0 8b 55 e8 39 50 0c 75 b2 8d 65 f4 5b 5e
5f 5d c3 55 89 d1 89 e5 57 89 c7 56 89 fa 53 8b 40 28 <8b> 40 30 e8 ab fd ff ff
85 c0 89 c6 75 07 be 8d ff ff ff eb 2c
EIP: [<c05898ca>] usb_hcd_unlink_urb+0xf/0x5a SS:ESP 0068:db092dc8
---[ end trace fffe0a5f0dafce17 ]---
============kernel dump 3 end ===============

Comment 7 John W. Linville 2008-08-19 18:35:03 UTC

I think this is resolved now...