Bug 210062
| Summary: | possible circular locking dependency detected (kdm and 2.6.18-1.2189.fc5) | ||
|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Orion Poplawski <orion> |
| Component: | kernel | Assignee: | Dave Jones <davej> |
| Status: | CLOSED NEXTRELEASE | QA Contact: | Brian Brock <bbrock> |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 6 | CC: | pfrields, wtogami |
| Target Milestone: | --- | Keywords: | Reopened |
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2007-10-17 20:29:47 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
*** Bug 210063 has been marked as a duplicate of this bug. *** A new kernel update has been released (Version: 2.6.18-1.2200.fc5) based upon a new upstream kernel release. Please retest against this new kernel, as a large number of patches go into each upstream release, possibly including changes that may address this problem. This bug has been placed in NEEDINFO state. Due to the large volume of inactive bugs in bugzilla, if this bug is still in this state in two weeks time, it will be closed. Should this bug still be relevant after this period, the reporter can reopen the bug at any time. Any other users on the Cc: list of this bug can request that the bug be reopened by adding a comment to the bug. In the last few updates, some users upgrading from FC4->FC5 have reported that installing a kernel update has left their systems unbootable. If you have been affected by this problem please check you only have one version of device-mapper & lvm2 installed. See bug 207474 for further details. If this bug is a problem preventing you from installing the release this version is filed against, please see bug 169613. If this bug has been fixed, but you are now experiencing a different problem, please file a separate bug for the new problem. Thank you. Now, it appears that 1.2200.fc5 does not contain the lock dependency validator, as the following lines are missing from dmesg under 1.2200.fc5. I'm assuming that this is what generated the above output and that without it I will not be able to reproduce the above issue. Is that true? < Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar < ... MAX_LOCKDEP_SUBCLASSES: 8 < ... MAX_LOCK_DEPTH: 30 < ... MAX_LOCKDEP_KEYS: 2048 < ... CLASSHASH_SIZE: 1024 < ... MAX_LOCKDEP_ENTRIES: 8192 < ... MAX_LOCKDEP_CHAINS: 8192 < ... CHAINHASH_SIZE: 4096 < memory used by lock dependency info: 696 kB < per task-struct memory footprint: 1200 bytes correct, the validator is disabled in the production kernel. This bug should be fixed regardless, so shouldn't reoccur if it's turned back on at any point. This looks a lot like the same problem again:
Feb 22 12:35:38 apapane kernel: [ INFO: possible circular locking dependency
detected ]
Feb 22 12:35:38 apapane kernel: 2.6.19-1.2911.fc6debug #1
Feb 22 12:35:38 apapane kernel:
-------------------------------------------------------
Feb 22 12:35:38 apapane kernel: sh/4424 is trying to acquire lock:
Feb 22 12:35:38 apapane kernel: (sk_lock-AF_INET){--..}, at:
[<ffffffff8022762d>] tcp_se
ndmsg+0x1c/0xb63
Feb 22 12:35:38 apapane kernel:
Feb 22 12:35:38 apapane kernel: but task is already holding lock:
Feb 22 12:35:38 apapane kernel: (&inode->i_alloc_sem){--..}, at:
[<ffffffff8022e43b>] no
tify_change+0xfd/0x2e1
Feb 22 12:35:38 apapane kernel:
Feb 22 12:35:38 apapane kernel: which lock already depends on the new lock.
Feb 22 12:35:38 apapane kernel:
Feb 22 12:35:38 apapane kernel:
Feb 22 12:35:38 apapane kernel: the existing dependency chain (in reverse order) is:
Feb 22 12:35:38 apapane kernel:
Feb 22 12:35:38 apapane kernel: -> #3 (&inode->i_alloc_sem){--..}:
Feb 22 12:35:38 apapane kernel: [<ffffffff802a5bd4>]
__lock_acquire+0x8bf/0x9d4
Feb 22 12:35:38 apapane kernel: [<ffffffff802a626e>] lock_acquire+0x48/0x63
Feb 22 12:35:38 apapane kernel: [<ffffffff802a20af>] down_write+0x34/0x3d
Feb 22 12:35:38 apapane kernel: [<ffffffff8022e43b>] notify_change+0xfd/0x2e1
Feb 22 12:35:38 apapane kernel: [<ffffffff802dddaf>] do_truncate+0x4e/0x69
Feb 22 12:35:38 apapane kernel: [<ffffffff80212dd0>] may_open+0x1e0/0x252
Feb 22 12:35:38 apapane kernel: [<ffffffff8021beac>] open_namei+0x2cb/0x698
Feb 22 12:35:38 apapane kernel: [<ffffffff80228ea8>] do_filp_open+0x1c/0x38
Feb 22 12:35:38 apapane kernel: [<ffffffff8021a90a>] do_sys_open+0x44/0xbe
Feb 22 12:35:38 apapane kernel: [<ffffffff8025e11e>] system_call+0x7e/0x83
Feb 22 12:35:38 apapane kernel: [<000000000055f5a0>] 0x55f5a0
Feb 22 12:35:38 apapane kernel: [<ffffffffffffffff>] 0xffffffffffffffff
Feb 22 12:35:38 apapane kernel:
Feb 22 12:35:38 apapane kernel: -> #2 (&sysfs_inode_imutex_key){--..}:
Feb 22 12:35:38 apapane kernel: [<ffffffff802a5bd4>]
__lock_acquire+0x8bf/0x9d4
Feb 22 12:35:38 apapane kernel: [<ffffffff802a626e>] lock_acquire+0x48/0x63
Feb 22 12:35:38 apapane kernel: [<ffffffff8026414a>]
__mutex_lock_slowpath+0xed/0x
26f
Feb 22 12:35:38 apapane kernel: [<ffffffff8030b262>] create_dir+0x24/0x1d5
Feb 22 12:35:38 apapane kernel: [<ffffffff8030b823>]
sysfs_create_dir+0x6d/0x8b
Feb 22 12:35:39 apapane kernel: [<ffffffff8034dde2>] kobject_add+0xe2/0x19f
Feb 22 12:35:39 apapane kernel: [<ffffffff803b9d8a>]
class_device_add+0xb2/0x464
Feb 22 12:35:39 apapane kernel: [<ffffffff80421806>]
register_netdevice+0x29a/0x36
8
Feb 22 12:35:39 apapane kernel: [<ffffffff8042192d>]
register_netdev+0x59/0x66
Feb 22 12:35:39 apapane kernel: [<ffffffff8072bdac>]
net_olddevs_init+0xb/0xac
Feb 22 12:35:39 apapane kernel: [<ffffffff8026b71d>] init+0x1e6/0x3ac
Feb 22 12:35:39 apapane kernel: [<ffffffff8025ef48>] child_rip+0xa/0x12
Feb 22 12:35:39 apapane kernel: [<ffffffffffffffff>] 0xffffffffffffffff
Feb 22 12:35:39 apapane kernel:
Feb 22 12:35:39 apapane kernel: -> #1 (rtnl_mutex){--..}:
Feb 22 12:35:39 apapane kernel: [<ffffffff802a5bd4>]
__lock_acquire+0x8bf/0x9d4
Feb 22 12:35:39 apapane kernel: [<ffffffff802a626e>] lock_acquire+0x48/0x63
Feb 22 12:35:39 apapane kernel: [<ffffffff8026414a>]
__mutex_lock_slowpath+0xed/0x
26f
Feb 22 12:35:39 apapane kernel: [<ffffffff80457635>]
ip_mc_leave_group+0x23/0xb8
Feb 22 12:35:39 apapane kernel: [<ffffffff8044193f>]
do_ip_setsockopt+0x6df/0x9cc
Feb 22 12:35:39 apapane kernel: [<ffffffff80441cc6>] ip_setsockopt+0x22/0x78
Feb 22 12:35:39 apapane kernel: [<ffffffff80419039>] sys_setsockopt+0x91/0xb7
Feb 22 12:35:39 apapane kernel: [<ffffffff8025e11e>] system_call+0x7e/0x83
Feb 22 12:35:39 apapane kernel: [<00002aaaabc3a32a>] 0x2aaaabc3a32a
Feb 22 12:35:39 apapane kernel: [<ffffffffffffffff>] 0xffffffffffffffff
Feb 22 12:35:39 apapane kernel:
Feb 22 12:35:39 apapane kernel: -> #0 (sk_lock-AF_INET){--..}:
Feb 22 12:35:39 apapane kernel: [<ffffffff802a5ae8>]
__lock_acquire+0x7d3/0x9d4
Feb 22 12:35:39 apapane kernel: [<ffffffff802a626e>] lock_acquire+0x48/0x63
Feb 22 12:35:39 apapane kernel: [<ffffffff802350ef>] lock_sock+0xd1/0xe1
Feb 22 12:35:39 apapane kernel: [<ffffffff8022762d>] tcp_sendmsg+0x1c/0xb63
Feb 22 12:35:39 apapane kernel: [<ffffffff8025532a>] sock_sendmsg+0xea/0x107
Feb 22 12:35:39 apapane kernel: [<ffffffff80419f28>] kernel_sendmsg+0x2c/0x3e
Feb 22 12:35:39 apapane kernel: [<ffffffff881b76f7>]
xs_tcp_send_request+0x100/0x2
c5 [sunrpc]
Feb 22 12:35:39 apapane kernel: [<ffffffff881b658a>]
xprt_transmit+0xc4/0x1ab [sun
rpc]
Feb 22 12:35:39 apapane kernel: [<ffffffff881b4436>]
call_transmit+0x1f9/0x233 [su
nrpc]
Feb 22 12:35:39 apapane kernel: [<ffffffff881b9bb3>]
__rpc_execute+0xa7/0x1ed [sun
rpc]
Feb 22 12:35:39 apapane kernel: [<ffffffff881b4b0d>]
rpc_call_sync+0x9d/0xca [sunr
pc]
Feb 22 12:35:39 apapane kernel: [<ffffffff88241b39>]
nfs3_rpc_wrapper+0x29/0x6d [n
fs]
Feb 22 12:35:39 apapane kernel: [<ffffffff88241e0a>]
nfs3_proc_setattr+0x97/0xcc [
nfs]
Feb 22 12:35:39 apapane kernel: [<ffffffff88239904>]
nfs_setattr+0xd9/0x10b [nfs]
Feb 22 12:35:39 apapane kernel: [<ffffffff8022e489>]
notify_change+0x14b/0x2e1
Feb 22 12:35:39 apapane kernel: [<ffffffff802dddaf>] do_truncate+0x4e/0x69
Feb 22 12:35:39 apapane kernel: [<ffffffff80212dd0>] may_open+0x1e0/0x252
Feb 22 12:35:39 apapane kernel: [<ffffffff8021beac>] open_namei+0x2cb/0x698
Feb 22 12:35:39 apapane kernel: [<ffffffff80228ea8>] do_filp_open+0x1c/0x38
Feb 22 12:35:39 apapane kernel: [<ffffffff8021a90a>] do_sys_open+0x44/0xbe
Feb 22 12:35:39 apapane kernel: [<ffffffff8025e11e>] system_call+0x7e/0x83
Feb 22 12:35:39 apapane kernel: [<00002aaaab18c480>] 0x2aaaab18c480
Feb 22 12:35:39 apapane kernel: [<ffffffffffffffff>] 0xffffffffffffffff
Feb 22 12:35:39 apapane kernel:
Feb 22 12:35:39 apapane kernel: other info that might help us debug this:
Feb 22 12:35:39 apapane kernel:
Feb 22 12:35:39 apapane kernel: 2 locks held by sh/4424:
Feb 22 12:35:39 apapane kernel: #0: (&inode->i_mutex){--..}, at:
[<ffffffff802ddda4>] d
o_truncate+0x43/0x69
Feb 22 12:35:39 apapane kernel: #1: (&inode->i_alloc_sem){--..}, at:
[<ffffffff8022e43b
>] notify_change+0xfd/0x2e1
Feb 22 12:35:39 apapane kernel:
Feb 22 12:35:39 apapane kernel: stack backtrace:
Feb 22 12:35:39 apapane kernel:
Feb 22 12:35:39 apapane kernel: Call Trace:
Feb 22 12:35:39 apapane kernel: [<ffffffff8026c97e>] show_trace+0x34/0x47
Feb 22 12:35:39 apapane kernel: [<ffffffff8026c9a3>] dump_stack+0x12/0x17
Feb 22 12:35:39 apapane kernel: [<ffffffff802a459f>]
print_circular_bug_tail+0x65/0x6e
Feb 22 12:35:39 apapane kernel: [<ffffffff802a5ae8>] __lock_acquire+0x7d3/0x9d4
Feb 22 12:35:39 apapane kernel: [<ffffffff802a626e>] lock_acquire+0x48/0x63
Feb 22 12:35:39 apapane kernel: [<ffffffff802350ef>] lock_sock+0xd1/0xe1
Feb 22 12:35:39 apapane kernel: [<ffffffff8022762d>] tcp_sendmsg+0x1c/0xb63
Feb 22 12:35:39 apapane kernel: [<ffffffff8025532a>] sock_sendmsg+0xea/0x107
Feb 22 12:35:39 apapane kernel: [<ffffffff80419f28>] kernel_sendmsg+0x2c/0x3e
Feb 22 12:35:39 apapane kernel: [<ffffffff881b76f7>]
:sunrpc:xs_tcp_send_request+0x100/0
x2c5
Feb 22 12:35:39 apapane kernel: [<ffffffff881b658a>]
:sunrpc:xprt_transmit+0xc4/0x1ab
Feb 22 12:35:39 apapane kernel: [<ffffffff881b4436>]
:sunrpc:call_transmit+0x1f9/0x233
Feb 22 12:35:39 apapane kernel: [<ffffffff881b9bb3>]
:sunrpc:__rpc_execute+0xa7/0x1ed
Feb 22 12:35:39 apapane kernel: [<ffffffff881b4b0d>]
:sunrpc:rpc_call_sync+0x9d/0xca
Feb 22 12:35:39 apapane kernel: [<ffffffff88241b39>]
:nfs:nfs3_rpc_wrapper+0x29/0x6d
Feb 22 12:35:39 apapane kernel: [<ffffffff88241e0a>]
:nfs:nfs3_proc_setattr+0x97/0xcc
Feb 22 12:35:39 apapane kernel: [<ffffffff88239904>] :nfs:nfs_setattr+0xd9/0x10b
Feb 22 12:35:39 apapane kernel: [<ffffffff8022e489>] notify_change+0x14b/0x2e1
Feb 22 12:35:39 apapane kernel: [<ffffffff802dddaf>] do_truncate+0x4e/0x69
Feb 22 12:35:39 apapane kernel: [<ffffffff80212dd0>] may_open+0x1e0/0x252
Feb 22 12:35:39 apapane kernel: [<ffffffff8021beac>] open_namei+0x2cb/0x698
Feb 22 12:35:39 apapane kernel: [<ffffffff80228ea8>] do_filp_open+0x1c/0x38
Feb 22 12:35:39 apapane kernel: [<ffffffff8021a90a>] do_sys_open+0x44/0xbe
Feb 22 12:35:39 apapane kernel: [<ffffffff8025e11e>] system_call+0x7e/0x83
Feb 22 12:35:39 apapane kernel: [<00002aaaab18c480>]
Feb 22 12:35:39 apapane kernel:
|
[ INFO: possible circular locking dependency detected ] 2.6.18-1.2189.fc5 #1 ------------------------------------------------------- kdm/3790 is trying to acquire lock: (sk_lock-AF_INET){--..}, at: [<ffffffff802277d2>] tcp_sendmsg+0x1f/0xb1a but task is already holding lock: (&inode->i_alloc_sem){--..}, at: [<ffffffff8022eddd>] notify_change+0x105/0x2f7 which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #3 (&inode->i_alloc_sem){--..}: [<ffffffff802a94c5>] lock_acquire+0x4a/0x69 [<ffffffff802a608d>] down_write+0x3a/0x47 [<ffffffff8022eddc>] notify_change+0x104/0x2f7 [<ffffffff802dfef2>] do_truncate+0x52/0x72 [<ffffffff80212c5b>] may_open+0x1d5/0x231 [<ffffffff8021beae>] open_namei+0x2c7/0x6e8 [<ffffffff802290e8>] do_filp_open+0x27/0x46 [<ffffffff8021a8be>] do_sys_open+0x4e/0xcd [<ffffffff80233d5b>] sys_open+0x1a/0x1d [<ffffffff8026068d>] system_call+0x7d/0x83 -> #2 (&sysfs_inode_imutex_key){--..}: [<ffffffff802a94c5>] lock_acquire+0x4a/0x69 [<ffffffff80266384>] __mutex_lock_slowpath+0xe4/0x261 [<ffffffff8026652a>] mutex_lock+0x29/0x2e [<ffffffff8030d2dd>] create_dir+0x2c/0x1e2 [<ffffffff8030d898>] sysfs_create_dir+0x59/0x78 [<ffffffff80350b08>] kobject_add+0xe5/0x1a9 [<ffffffff803be998>] class_device_add+0xb5/0x471 [<ffffffff8042f8f3>] netdev_register_sysfs+0x98/0xa2 [<ffffffff8042676d>] register_netdevice+0x27a/0x35a [<ffffffff804268a7>] register_netdev+0x5a/0x69 [<ffffffff806cf8c2>] loopback_init+0x4e/0x53 [<ffffffff806cf7c8>] net_olddevs_init+0xb/0xb7 [<ffffffff8026d994>] init+0x1fc/0x3cd [<ffffffff802615df>] child_rip+0x9/0x12 -> #1 (rtnl_mutex){--..}: [<ffffffff802a94c5>] lock_acquire+0x4a/0x69 [<ffffffff80266384>] __mutex_lock_slowpath+0xe4/0x261 [<ffffffff8026652a>] mutex_lock+0x29/0x2e [<ffffffff8042d902>] rtnl_lock+0xf/0x12 [<ffffffff8045b500>] ip_mc_leave_group+0x27/0xc3 [<ffffffff80445404>] do_ip_setsockopt+0x6ad/0x9b2 [<ffffffff804457b7>] ip_setsockopt+0x2a/0x84 [<ffffffff80453714>] udp_setsockopt+0xd/0x1c [<ffffffff8041f0d1>] sock_common_setsockopt+0xe/0x11 [<ffffffff8041e21c>] sys_setsockopt+0x8e/0xb4 [<ffffffff8026068d>] system_call+0x7d/0x83 -> #0 (sk_lock-AF_INET){--..}: [<ffffffff802a94c5>] lock_acquire+0x4a/0x69 [<ffffffff80236254>] lock_sock+0xd4/0xe7 [<ffffffff802277d1>] tcp_sendmsg+0x1e/0xb1a [<ffffffff8024743c>] inet_sendmsg+0x45/0x53 [<ffffffff80257b90>] sock_sendmsg+0x110/0x130 [<ffffffff8041ed49>] kernel_sendmsg+0x3c/0x52 [<ffffffff88413869>] xs_tcp_send_request+0x117/0x320 [sunrpc] [<ffffffff88412895>] xprt_transmit+0xc8/0x1a8 [sunrpc] [<ffffffff88410463>] call_transmit+0x1f7/0x22e [sunrpc] [<ffffffff88415cd6>] __rpc_execute+0x9b/0x1e6 [sunrpc] [<ffffffff88415e46>] rpc_execute+0x1a/0x1d [sunrpc] [<ffffffff88410600>] rpc_call_sync+0x87/0xb9 [sunrpc] [<ffffffff8857eeab>] nfs3_rpc_wrapper+0x2e/0x74 [nfs] [<ffffffff8857f1a1>] nfs3_proc_setattr+0x9c/0xd4 [nfs] [<ffffffff88576702>] nfs_setattr+0xec/0x121 [nfs] [<ffffffff8022ee2c>] notify_change+0x154/0x2f7 [<ffffffff802dfef2>] do_truncate+0x52/0x72 [<ffffffff80212c5b>] may_open+0x1d5/0x231 [<ffffffff8021beae>] open_namei+0x2c7/0x6e8 [<ffffffff802290e8>] do_filp_open+0x27/0x46 [<ffffffff8021a8be>] do_sys_open+0x4e/0xcd [<ffffffff80233d5b>] sys_open+0x1a/0x1d [<ffffffff802dfe2a>] sys_creat+0xf/0x12 [<ffffffff8026068d>] system_call+0x7d/0x83 other info that might help us debug this: 2 locks held by kdm/3790: #0: (&inode->i_mutex){--..}, at: [<ffffffff8026652b>] mutex_lock+0x2a/0x2e #1: (&inode->i_alloc_sem){--..}, at: [<ffffffff8022eddd>] notify_change+0x105/0x2f7 stack backtrace: Call Trace: [<ffffffff8026e990>] show_trace+0xae/0x32b [<ffffffff8026ec22>] dump_stack+0x15/0x17 [<ffffffff802a7710>] print_circular_bug_tail+0x6c/0x77 [<ffffffff802a8d24>] __lock_acquire+0x858/0xa59 [<ffffffff802a94c6>] lock_acquire+0x4b/0x69 [<ffffffff80236255>] lock_sock+0xd5/0xe7 [<ffffffff802277d2>] tcp_sendmsg+0x1f/0xb1a [<ffffffff8024743d>] inet_sendmsg+0x46/0x53 [<ffffffff80257b91>] sock_sendmsg+0x111/0x130 [<ffffffff8041ed4a>] kernel_sendmsg+0x3d/0x52 [<ffffffff8841386a>] :sunrpc:xs_tcp_send_request+0x118/0x320 [<ffffffff88412896>] :sunrpc:xprt_transmit+0xc9/0x1a8 [<ffffffff88410464>] :sunrpc:call_transmit+0x1f8/0x22e [<ffffffff88415cd7>] :sunrpc:__rpc_execute+0x9c/0x1e6 [<ffffffff88415e47>] :sunrpc:rpc_execute+0x1b/0x1d [<ffffffff88410601>] :sunrpc:rpc_call_sync+0x88/0xb9 [<ffffffff8857eeac>] :nfs:nfs3_rpc_wrapper+0x2f/0x74 [<ffffffff8857f1a2>] :nfs:nfs3_proc_setattr+0x9d/0xd4 [<ffffffff88576703>] :nfs:nfs_setattr+0xed/0x121 [<ffffffff8022ee2d>] notify_change+0x155/0x2f7 [<ffffffff802dfef3>] do_truncate+0x53/0x72 [<ffffffff80212c5c>] may_open+0x1d6/0x231 [<ffffffff8021beaf>] open_namei+0x2c8/0x6e8 [<ffffffff802290e9>] do_filp_open+0x28/0x46 [<ffffffff8021a8bf>] do_sys_open+0x4f/0xcd [<ffffffff80233d5c>] sys_open+0x1b/0x1d [<ffffffff802dfe2b>] sys_creat+0x10/0x12 [<ffffffff8026068e>] system_call+0x7e/0x83 DWARF2 unwinder stuck at system_call+0x7e/0x83 Leftover inexact backtrace: