Bug 210062 - possible circular locking dependency detected (kdm and 2.6.18-1.2189.fc5)
possible circular locking dependency detected (kdm and 2.6.18-1.2189.fc5)
Status: CLOSED NEXTRELEASE
Product: Fedora
Classification: Fedora
Component: kernel (Show other bugs)
6
x86_64 Linux
medium Severity medium
: ---
: ---
Assigned To: Dave Jones
Brian Brock
: Reopened
: 210063 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2006-10-09 15:54 EDT by Orion Poplawski
Modified: 2015-01-04 17:28 EST (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2007-10-17 16:29:47 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Orion Poplawski 2006-10-09 15:54:29 EDT
[ INFO: possible circular locking dependency detected ]
2.6.18-1.2189.fc5 #1
-------------------------------------------------------
kdm/3790 is trying to acquire lock:
 (sk_lock-AF_INET){--..}, at: [<ffffffff802277d2>] tcp_sendmsg+0x1f/0xb1a

but task is already holding lock:
 (&inode->i_alloc_sem){--..}, at: [<ffffffff8022eddd>] notify_change+0x105/0x2f7

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #3 (&inode->i_alloc_sem){--..}:
       [<ffffffff802a94c5>] lock_acquire+0x4a/0x69
       [<ffffffff802a608d>] down_write+0x3a/0x47
       [<ffffffff8022eddc>] notify_change+0x104/0x2f7
       [<ffffffff802dfef2>] do_truncate+0x52/0x72
       [<ffffffff80212c5b>] may_open+0x1d5/0x231
       [<ffffffff8021beae>] open_namei+0x2c7/0x6e8
       [<ffffffff802290e8>] do_filp_open+0x27/0x46
       [<ffffffff8021a8be>] do_sys_open+0x4e/0xcd
       [<ffffffff80233d5b>] sys_open+0x1a/0x1d
       [<ffffffff8026068d>] system_call+0x7d/0x83

-> #2 (&sysfs_inode_imutex_key){--..}:
       [<ffffffff802a94c5>] lock_acquire+0x4a/0x69
       [<ffffffff80266384>] __mutex_lock_slowpath+0xe4/0x261
       [<ffffffff8026652a>] mutex_lock+0x29/0x2e
       [<ffffffff8030d2dd>] create_dir+0x2c/0x1e2
       [<ffffffff8030d898>] sysfs_create_dir+0x59/0x78
       [<ffffffff80350b08>] kobject_add+0xe5/0x1a9
       [<ffffffff803be998>] class_device_add+0xb5/0x471
       [<ffffffff8042f8f3>] netdev_register_sysfs+0x98/0xa2
       [<ffffffff8042676d>] register_netdevice+0x27a/0x35a
       [<ffffffff804268a7>] register_netdev+0x5a/0x69
       [<ffffffff806cf8c2>] loopback_init+0x4e/0x53
       [<ffffffff806cf7c8>] net_olddevs_init+0xb/0xb7
       [<ffffffff8026d994>] init+0x1fc/0x3cd
       [<ffffffff802615df>] child_rip+0x9/0x12

-> #1 (rtnl_mutex){--..}:
       [<ffffffff802a94c5>] lock_acquire+0x4a/0x69
       [<ffffffff80266384>] __mutex_lock_slowpath+0xe4/0x261
       [<ffffffff8026652a>] mutex_lock+0x29/0x2e
       [<ffffffff8042d902>] rtnl_lock+0xf/0x12
       [<ffffffff8045b500>] ip_mc_leave_group+0x27/0xc3
       [<ffffffff80445404>] do_ip_setsockopt+0x6ad/0x9b2
       [<ffffffff804457b7>] ip_setsockopt+0x2a/0x84
       [<ffffffff80453714>] udp_setsockopt+0xd/0x1c
       [<ffffffff8041f0d1>] sock_common_setsockopt+0xe/0x11
       [<ffffffff8041e21c>] sys_setsockopt+0x8e/0xb4
       [<ffffffff8026068d>] system_call+0x7d/0x83

-> #0 (sk_lock-AF_INET){--..}:
       [<ffffffff802a94c5>] lock_acquire+0x4a/0x69
       [<ffffffff80236254>] lock_sock+0xd4/0xe7
       [<ffffffff802277d1>] tcp_sendmsg+0x1e/0xb1a
       [<ffffffff8024743c>] inet_sendmsg+0x45/0x53
       [<ffffffff80257b90>] sock_sendmsg+0x110/0x130
       [<ffffffff8041ed49>] kernel_sendmsg+0x3c/0x52
       [<ffffffff88413869>] xs_tcp_send_request+0x117/0x320 [sunrpc]
       [<ffffffff88412895>] xprt_transmit+0xc8/0x1a8 [sunrpc]
       [<ffffffff88410463>] call_transmit+0x1f7/0x22e [sunrpc]
       [<ffffffff88415cd6>] __rpc_execute+0x9b/0x1e6 [sunrpc]
       [<ffffffff88415e46>] rpc_execute+0x1a/0x1d [sunrpc]
       [<ffffffff88410600>] rpc_call_sync+0x87/0xb9 [sunrpc]
       [<ffffffff8857eeab>] nfs3_rpc_wrapper+0x2e/0x74 [nfs]
       [<ffffffff8857f1a1>] nfs3_proc_setattr+0x9c/0xd4 [nfs]
       [<ffffffff88576702>] nfs_setattr+0xec/0x121 [nfs]
       [<ffffffff8022ee2c>] notify_change+0x154/0x2f7
       [<ffffffff802dfef2>] do_truncate+0x52/0x72
       [<ffffffff80212c5b>] may_open+0x1d5/0x231
       [<ffffffff8021beae>] open_namei+0x2c7/0x6e8
       [<ffffffff802290e8>] do_filp_open+0x27/0x46
       [<ffffffff8021a8be>] do_sys_open+0x4e/0xcd
       [<ffffffff80233d5b>] sys_open+0x1a/0x1d
       [<ffffffff802dfe2a>] sys_creat+0xf/0x12
       [<ffffffff8026068d>] system_call+0x7d/0x83

other info that might help us debug this:

2 locks held by kdm/3790:
 #0:  (&inode->i_mutex){--..}, at: [<ffffffff8026652b>] mutex_lock+0x2a/0x2e
 #1:  (&inode->i_alloc_sem){--..}, at: [<ffffffff8022eddd>]
notify_change+0x105/0x2f7

stack backtrace:

Call Trace:
 [<ffffffff8026e990>] show_trace+0xae/0x32b
 [<ffffffff8026ec22>] dump_stack+0x15/0x17
 [<ffffffff802a7710>] print_circular_bug_tail+0x6c/0x77
 [<ffffffff802a8d24>] __lock_acquire+0x858/0xa59
 [<ffffffff802a94c6>] lock_acquire+0x4b/0x69
 [<ffffffff80236255>] lock_sock+0xd5/0xe7
 [<ffffffff802277d2>] tcp_sendmsg+0x1f/0xb1a
 [<ffffffff8024743d>] inet_sendmsg+0x46/0x53
 [<ffffffff80257b91>] sock_sendmsg+0x111/0x130
 [<ffffffff8041ed4a>] kernel_sendmsg+0x3d/0x52
 [<ffffffff8841386a>] :sunrpc:xs_tcp_send_request+0x118/0x320
 [<ffffffff88412896>] :sunrpc:xprt_transmit+0xc9/0x1a8
 [<ffffffff88410464>] :sunrpc:call_transmit+0x1f8/0x22e
 [<ffffffff88415cd7>] :sunrpc:__rpc_execute+0x9c/0x1e6
 [<ffffffff88415e47>] :sunrpc:rpc_execute+0x1b/0x1d
 [<ffffffff88410601>] :sunrpc:rpc_call_sync+0x88/0xb9
 [<ffffffff8857eeac>] :nfs:nfs3_rpc_wrapper+0x2f/0x74
 [<ffffffff8857f1a2>] :nfs:nfs3_proc_setattr+0x9d/0xd4
 [<ffffffff88576703>] :nfs:nfs_setattr+0xed/0x121
 [<ffffffff8022ee2d>] notify_change+0x155/0x2f7
 [<ffffffff802dfef3>] do_truncate+0x53/0x72
 [<ffffffff80212c5c>] may_open+0x1d6/0x231
 [<ffffffff8021beaf>] open_namei+0x2c8/0x6e8
 [<ffffffff802290e9>] do_filp_open+0x28/0x46
 [<ffffffff8021a8bf>] do_sys_open+0x4f/0xcd
 [<ffffffff80233d5c>] sys_open+0x1b/0x1d
 [<ffffffff802dfe2b>] sys_creat+0x10/0x12
 [<ffffffff8026068e>] system_call+0x7e/0x83
DWARF2 unwinder stuck at system_call+0x7e/0x83
Leftover inexact backtrace:
Comment 1 Jeffrey Moyer 2006-10-12 11:00:44 EDT
*** Bug 210063 has been marked as a duplicate of this bug. ***
Comment 2 Dave Jones 2006-10-16 17:41:20 EDT
A new kernel update has been released (Version: 2.6.18-1.2200.fc5)
based upon a new upstream kernel release.

Please retest against this new kernel, as a large number of patches
go into each upstream release, possibly including changes that
may address this problem.

This bug has been placed in NEEDINFO state.
Due to the large volume of inactive bugs in bugzilla, if this bug is
still in this state in two weeks time, it will be closed.

Should this bug still be relevant after this period, the reporter
can reopen the bug at any time. Any other users on the Cc: list
of this bug can request that the bug be reopened by adding a
comment to the bug.

In the last few updates, some users upgrading from FC4->FC5
have reported that installing a kernel update has left their
systems unbootable. If you have been affected by this problem
please check you only have one version of device-mapper & lvm2
installed.  See bug 207474 for further details.

If this bug is a problem preventing you from installing the
release this version is filed against, please see bug 169613.

If this bug has been fixed, but you are now experiencing a different
problem, please file a separate bug for the new problem.

Thank you.
Comment 3 Orion Poplawski 2006-10-17 17:05:25 EDT
Now, it appears that 1.2200.fc5 does not contain the lock dependency validator,
as the following lines are missing from dmesg under 1.2200.fc5.  I'm assuming
that this is what generated the above output and that without it I will not be
able to reproduce the above issue.  Is that true?

< Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
< ... MAX_LOCKDEP_SUBCLASSES:    8
< ... MAX_LOCK_DEPTH:          30
< ... MAX_LOCKDEP_KEYS:        2048
< ... CLASSHASH_SIZE:           1024
< ... MAX_LOCKDEP_ENTRIES:     8192
< ... MAX_LOCKDEP_CHAINS:      8192
< ... CHAINHASH_SIZE:          4096
<  memory used by lock dependency info: 696 kB
<  per task-struct memory footprint: 1200 bytes
Comment 4 Dave Jones 2006-10-20 21:55:14 EDT
correct, the validator is disabled in the production kernel.
This bug should be fixed regardless, so shouldn't reoccur if it's turned back on
at any point.
Comment 5 Orion Poplawski 2007-02-22 16:10:49 EST
This looks a lot like the same problem again:

Feb 22 12:35:38 apapane kernel: [ INFO: possible circular locking dependency
detected ]
Feb 22 12:35:38 apapane kernel: 2.6.19-1.2911.fc6debug #1
Feb 22 12:35:38 apapane kernel:
-------------------------------------------------------
Feb 22 12:35:38 apapane kernel: sh/4424 is trying to acquire lock:
Feb 22 12:35:38 apapane kernel:  (sk_lock-AF_INET){--..}, at:
[<ffffffff8022762d>] tcp_se
ndmsg+0x1c/0xb63
Feb 22 12:35:38 apapane kernel:
Feb 22 12:35:38 apapane kernel: but task is already holding lock:
Feb 22 12:35:38 apapane kernel:  (&inode->i_alloc_sem){--..}, at:
[<ffffffff8022e43b>] no
tify_change+0xfd/0x2e1
Feb 22 12:35:38 apapane kernel:
Feb 22 12:35:38 apapane kernel: which lock already depends on the new lock.
Feb 22 12:35:38 apapane kernel:
Feb 22 12:35:38 apapane kernel:
Feb 22 12:35:38 apapane kernel: the existing dependency chain (in reverse order) is:
Feb 22 12:35:38 apapane kernel:
Feb 22 12:35:38 apapane kernel: -> #3 (&inode->i_alloc_sem){--..}:
Feb 22 12:35:38 apapane kernel:        [<ffffffff802a5bd4>]
__lock_acquire+0x8bf/0x9d4
Feb 22 12:35:38 apapane kernel:        [<ffffffff802a626e>] lock_acquire+0x48/0x63
Feb 22 12:35:38 apapane kernel:        [<ffffffff802a20af>] down_write+0x34/0x3d
Feb 22 12:35:38 apapane kernel:        [<ffffffff8022e43b>] notify_change+0xfd/0x2e1
Feb 22 12:35:38 apapane kernel:        [<ffffffff802dddaf>] do_truncate+0x4e/0x69
Feb 22 12:35:38 apapane kernel:        [<ffffffff80212dd0>] may_open+0x1e0/0x252
Feb 22 12:35:38 apapane kernel:        [<ffffffff8021beac>] open_namei+0x2cb/0x698
Feb 22 12:35:38 apapane kernel:        [<ffffffff80228ea8>] do_filp_open+0x1c/0x38
Feb 22 12:35:38 apapane kernel:        [<ffffffff8021a90a>] do_sys_open+0x44/0xbe
Feb 22 12:35:38 apapane kernel:        [<ffffffff8025e11e>] system_call+0x7e/0x83
Feb 22 12:35:38 apapane kernel:        [<000000000055f5a0>] 0x55f5a0
Feb 22 12:35:38 apapane kernel:        [<ffffffffffffffff>] 0xffffffffffffffff
Feb 22 12:35:38 apapane kernel:
Feb 22 12:35:38 apapane kernel: -> #2 (&sysfs_inode_imutex_key){--..}:
Feb 22 12:35:38 apapane kernel:        [<ffffffff802a5bd4>]
__lock_acquire+0x8bf/0x9d4
Feb 22 12:35:38 apapane kernel:        [<ffffffff802a626e>] lock_acquire+0x48/0x63
Feb 22 12:35:38 apapane kernel:        [<ffffffff8026414a>]
__mutex_lock_slowpath+0xed/0x
26f
Feb 22 12:35:38 apapane kernel:        [<ffffffff8030b262>] create_dir+0x24/0x1d5
Feb 22 12:35:38 apapane kernel:        [<ffffffff8030b823>]
sysfs_create_dir+0x6d/0x8b
Feb 22 12:35:39 apapane kernel:        [<ffffffff8034dde2>] kobject_add+0xe2/0x19f
Feb 22 12:35:39 apapane kernel:        [<ffffffff803b9d8a>]
class_device_add+0xb2/0x464
Feb 22 12:35:39 apapane kernel:        [<ffffffff80421806>]
register_netdevice+0x29a/0x36
8
Feb 22 12:35:39 apapane kernel:        [<ffffffff8042192d>]
register_netdev+0x59/0x66
Feb 22 12:35:39 apapane kernel:        [<ffffffff8072bdac>]
net_olddevs_init+0xb/0xac
Feb 22 12:35:39 apapane kernel:        [<ffffffff8026b71d>] init+0x1e6/0x3ac
Feb 22 12:35:39 apapane kernel:        [<ffffffff8025ef48>] child_rip+0xa/0x12
Feb 22 12:35:39 apapane kernel:        [<ffffffffffffffff>] 0xffffffffffffffff
Feb 22 12:35:39 apapane kernel:
Feb 22 12:35:39 apapane kernel: -> #1 (rtnl_mutex){--..}:
Feb 22 12:35:39 apapane kernel:        [<ffffffff802a5bd4>]
__lock_acquire+0x8bf/0x9d4
Feb 22 12:35:39 apapane kernel:        [<ffffffff802a626e>] lock_acquire+0x48/0x63
Feb 22 12:35:39 apapane kernel:        [<ffffffff8026414a>]
__mutex_lock_slowpath+0xed/0x
26f
Feb 22 12:35:39 apapane kernel:        [<ffffffff80457635>]
ip_mc_leave_group+0x23/0xb8
Feb 22 12:35:39 apapane kernel:        [<ffffffff8044193f>]
do_ip_setsockopt+0x6df/0x9cc
Feb 22 12:35:39 apapane kernel:        [<ffffffff80441cc6>] ip_setsockopt+0x22/0x78
Feb 22 12:35:39 apapane kernel:        [<ffffffff80419039>] sys_setsockopt+0x91/0xb7
Feb 22 12:35:39 apapane kernel:        [<ffffffff8025e11e>] system_call+0x7e/0x83
Feb 22 12:35:39 apapane kernel:        [<00002aaaabc3a32a>] 0x2aaaabc3a32a
Feb 22 12:35:39 apapane kernel:        [<ffffffffffffffff>] 0xffffffffffffffff
Feb 22 12:35:39 apapane kernel:
Feb 22 12:35:39 apapane kernel: -> #0 (sk_lock-AF_INET){--..}:
Feb 22 12:35:39 apapane kernel:        [<ffffffff802a5ae8>]
__lock_acquire+0x7d3/0x9d4
Feb 22 12:35:39 apapane kernel:        [<ffffffff802a626e>] lock_acquire+0x48/0x63
Feb 22 12:35:39 apapane kernel:        [<ffffffff802350ef>] lock_sock+0xd1/0xe1
Feb 22 12:35:39 apapane kernel:        [<ffffffff8022762d>] tcp_sendmsg+0x1c/0xb63
Feb 22 12:35:39 apapane kernel:        [<ffffffff8025532a>] sock_sendmsg+0xea/0x107
Feb 22 12:35:39 apapane kernel:        [<ffffffff80419f28>] kernel_sendmsg+0x2c/0x3e
Feb 22 12:35:39 apapane kernel:        [<ffffffff881b76f7>]
xs_tcp_send_request+0x100/0x2
c5 [sunrpc]
Feb 22 12:35:39 apapane kernel:        [<ffffffff881b658a>]
xprt_transmit+0xc4/0x1ab [sun
rpc]
Feb 22 12:35:39 apapane kernel:        [<ffffffff881b4436>]
call_transmit+0x1f9/0x233 [su
nrpc]
Feb 22 12:35:39 apapane kernel:        [<ffffffff881b9bb3>]
__rpc_execute+0xa7/0x1ed [sun
rpc]
Feb 22 12:35:39 apapane kernel:        [<ffffffff881b4b0d>]
rpc_call_sync+0x9d/0xca [sunr
pc]
Feb 22 12:35:39 apapane kernel:        [<ffffffff88241b39>]
nfs3_rpc_wrapper+0x29/0x6d [n
fs]
Feb 22 12:35:39 apapane kernel:        [<ffffffff88241e0a>]
nfs3_proc_setattr+0x97/0xcc [
nfs]
Feb 22 12:35:39 apapane kernel:        [<ffffffff88239904>]
nfs_setattr+0xd9/0x10b [nfs]
Feb 22 12:35:39 apapane kernel:        [<ffffffff8022e489>]
notify_change+0x14b/0x2e1
Feb 22 12:35:39 apapane kernel:        [<ffffffff802dddaf>] do_truncate+0x4e/0x69
Feb 22 12:35:39 apapane kernel:        [<ffffffff80212dd0>] may_open+0x1e0/0x252
Feb 22 12:35:39 apapane kernel:        [<ffffffff8021beac>] open_namei+0x2cb/0x698
Feb 22 12:35:39 apapane kernel:        [<ffffffff80228ea8>] do_filp_open+0x1c/0x38
Feb 22 12:35:39 apapane kernel:        [<ffffffff8021a90a>] do_sys_open+0x44/0xbe
Feb 22 12:35:39 apapane kernel:        [<ffffffff8025e11e>] system_call+0x7e/0x83
Feb 22 12:35:39 apapane kernel:        [<00002aaaab18c480>] 0x2aaaab18c480
Feb 22 12:35:39 apapane kernel:        [<ffffffffffffffff>] 0xffffffffffffffff
Feb 22 12:35:39 apapane kernel:
Feb 22 12:35:39 apapane kernel: other info that might help us debug this:
Feb 22 12:35:39 apapane kernel:
Feb 22 12:35:39 apapane kernel: 2 locks held by sh/4424:
Feb 22 12:35:39 apapane kernel:  #0:  (&inode->i_mutex){--..}, at:
[<ffffffff802ddda4>] d
o_truncate+0x43/0x69
Feb 22 12:35:39 apapane kernel:  #1:  (&inode->i_alloc_sem){--..}, at:
[<ffffffff8022e43b
>] notify_change+0xfd/0x2e1
Feb 22 12:35:39 apapane kernel:
Feb 22 12:35:39 apapane kernel: stack backtrace:
Feb 22 12:35:39 apapane kernel:
Feb 22 12:35:39 apapane kernel: Call Trace:
Feb 22 12:35:39 apapane kernel:  [<ffffffff8026c97e>] show_trace+0x34/0x47
Feb 22 12:35:39 apapane kernel:  [<ffffffff8026c9a3>] dump_stack+0x12/0x17
Feb 22 12:35:39 apapane kernel:  [<ffffffff802a459f>]
print_circular_bug_tail+0x65/0x6e
Feb 22 12:35:39 apapane kernel:  [<ffffffff802a5ae8>] __lock_acquire+0x7d3/0x9d4
Feb 22 12:35:39 apapane kernel:  [<ffffffff802a626e>] lock_acquire+0x48/0x63
Feb 22 12:35:39 apapane kernel:  [<ffffffff802350ef>] lock_sock+0xd1/0xe1
Feb 22 12:35:39 apapane kernel:  [<ffffffff8022762d>] tcp_sendmsg+0x1c/0xb63
Feb 22 12:35:39 apapane kernel:  [<ffffffff8025532a>] sock_sendmsg+0xea/0x107
Feb 22 12:35:39 apapane kernel:  [<ffffffff80419f28>] kernel_sendmsg+0x2c/0x3e
Feb 22 12:35:39 apapane kernel:  [<ffffffff881b76f7>]
:sunrpc:xs_tcp_send_request+0x100/0
x2c5
Feb 22 12:35:39 apapane kernel:  [<ffffffff881b658a>]
:sunrpc:xprt_transmit+0xc4/0x1ab
Feb 22 12:35:39 apapane kernel:  [<ffffffff881b4436>]
:sunrpc:call_transmit+0x1f9/0x233
Feb 22 12:35:39 apapane kernel:  [<ffffffff881b9bb3>]
:sunrpc:__rpc_execute+0xa7/0x1ed
Feb 22 12:35:39 apapane kernel:  [<ffffffff881b4b0d>]
:sunrpc:rpc_call_sync+0x9d/0xca
Feb 22 12:35:39 apapane kernel:  [<ffffffff88241b39>]
:nfs:nfs3_rpc_wrapper+0x29/0x6d
Feb 22 12:35:39 apapane kernel:  [<ffffffff88241e0a>]
:nfs:nfs3_proc_setattr+0x97/0xcc
Feb 22 12:35:39 apapane kernel:  [<ffffffff88239904>] :nfs:nfs_setattr+0xd9/0x10b
Feb 22 12:35:39 apapane kernel:  [<ffffffff8022e489>] notify_change+0x14b/0x2e1
Feb 22 12:35:39 apapane kernel:  [<ffffffff802dddaf>] do_truncate+0x4e/0x69
Feb 22 12:35:39 apapane kernel:  [<ffffffff80212dd0>] may_open+0x1e0/0x252
Feb 22 12:35:39 apapane kernel:  [<ffffffff8021beac>] open_namei+0x2cb/0x698
Feb 22 12:35:39 apapane kernel:  [<ffffffff80228ea8>] do_filp_open+0x1c/0x38
Feb 22 12:35:39 apapane kernel:  [<ffffffff8021a90a>] do_sys_open+0x44/0xbe
Feb 22 12:35:39 apapane kernel:  [<ffffffff8025e11e>] system_call+0x7e/0x83
Feb 22 12:35:39 apapane kernel:  [<00002aaaab18c480>]
Feb 22 12:35:39 apapane kernel:

Note You need to log in before you can comment on or make changes to this bug.