Bug 238813

Summary: drop_caches lockdep warning
Product: [Fedora] Fedora Reporter: Jeremy Katz <katzj>
Component: kernelAssignee: Jerome Marchand <jmarchan>
Status: CLOSED CANTFIX QA Contact: Brian Brock <bbrock>
Severity: medium Docs Contact:
Priority: medium    
Version: rawhideCC: harshula, triage
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard: bzcl34nup
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2008-04-27 04:39:50 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 202141    

Description Jeremy Katz 2007-05-03 03:15:17 UTC
=======================================================
[ INFO: possible circular locking dependency detected ]
2.6.21-1.3116.fc7 #1
-------------------------------------------------------
bash/29976 is trying to acquire lock:
 (&journal->j_list_lock){--..}, at: [<f88c2fa0>]
journal_try_to_free_buffers+0xa2/0x138 [jbd]

but task is already holding lock:
 (inode_lock){--..}, at: [<c0497b3f>] drop_pagecache+0x45/0xcd

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (inode_lock){--..}:
       [<c044299f>] __lock_acquire+0xa19/0xba4
       [<c0442f1c>] lock_acquire+0x56/0x6f
       [<c0616189>] _spin_lock+0x2b/0x38
       [<c049765d>] __mark_inode_dirty+0xe1/0x15c
       [<c04654c3>] __set_page_dirty_nobuffers+0xc4/0xcf
       [<c049aa98>] mark_buffer_dirty+0x1e/0x20
       [<f88c1179>] __journal_temp_unlink_buffer+0x174/0x17b [jbd]
       [<f88c139e>] __journal_unfile_buffer+0xb/0x15 [jbd]
       [<f88c1412>] __journal_refile_buffer+0x6a/0xe3 [jbd]
       [<f88c3ede>] journal_commit_transaction+0xdd2/0x1020 [jbd]
       [<f88c7385>] kjournald+0xab/0x1e8 [jbd]
       [<c043832b>] kthread+0xb3/0xdc
       [<c0405cd3>] kernel_thread_helper+0x7/0x10
       [<ffffffff>] 0xffffffff

-> #0 (&journal->j_list_lock){--..}:
       [<c0442883>] __lock_acquire+0x8fd/0xba4
       [<c0442f1c>] lock_acquire+0x56/0x6f
       [<c0616189>] _spin_lock+0x2b/0x38
       [<f88c2fa0>] journal_try_to_free_buffers+0xa2/0x138 [jbd]
       [<f89864a9>] ext3_releasepage+0x71/0x7d [ext3]
       [<c0462734>] try_to_release_page+0x33/0x44
       [<c046748e>] invalidate_mapping_pages+0x6b/0xcd
       [<c0497b60>] drop_pagecache+0x66/0xcd
       [<c0497bfd>] drop_caches_sysctl_handler+0x36/0x4d
       [<c04b2eac>] proc_sys_write+0x6b/0x85
       [<c047e9ef>] vfs_write+0xaf/0x163
       [<c047f03d>] sys_write+0x3d/0x61
       [<c0405078>] syscall_call+0x7/0xb
       [<ffffffff>] 0xffffffff

other info that might help us debug this:

2 locks held by bash/29976:
 #0:  (&type->s_umount_key#18){----}, at: [<c0497b2f>] drop_pagecache+0x35/0xcd
 #1:  (inode_lock){--..}, at: [<c0497b3f>] drop_pagecache+0x45/0xcd

stack backtrace:
 [<c04061e9>] show_trace_log_lvl+0x1a/0x2f
 [<c04067ad>] show_trace+0x12/0x14
 [<c0406831>] dump_stack+0x16/0x18
 [<c0441096>] print_circular_bug_tail+0x5f/0x68
 [<c0442883>] __lock_acquire+0x8fd/0xba4
 [<c0442f1c>] lock_acquire+0x56/0x6f
 [<c0616189>] _spin_lock+0x2b/0x38
 [<f88c2fa0>] journal_try_to_free_buffers+0xa2/0x138 [jbd]
 [<f89864a9>] ext3_releasepage+0x71/0x7d [ext3]
 [<c0462734>] try_to_release_page+0x33/0x44
 [<c046748e>] invalidate_mapping_pages+0x6b/0xcd
 [<c0497b60>] drop_pagecache+0x66/0xcd
 [<c0497bfd>] drop_caches_sysctl_handler+0x36/0x4d
 [<c04b2eac>] proc_sys_write+0x6b/0x85
 [<c047e9ef>] vfs_write+0xaf/0x163
 [<c047f03d>] sys_write+0x3d/0x61
 [<c0405078>] syscall_call+0x7/0xb
 =======================

Comment 1 Linda Wang 2007-05-03 16:58:17 UTC
upstream discussion about this issue: http://lkml.org/lkml/2007/2/23/46

Comment 2 Jeremy Katz 2007-05-03 17:27:54 UTC
Yeah, I was actually doing cache dropping to get the yum-updatesd startup time
down, but then noticed this only a while later.

Seems much less worrisome if that's the trigger

Comment 3 Bug Zapper 2008-04-04 00:27:06 UTC
Based on the date this bug was created, it appears to have been reported
against rawhide during the development of a Fedora release that is no
longer maintained. In order to refocus our efforts as a project we are
flagging all of the open bugs for releases which are no longer
maintained. If this bug remains in NEEDINFO thirty (30) days from now,
we will automatically close it.

If you can reproduce this bug in a maintained Fedora version (7, 8, or
rawhide), please change this bug to the respective version and change
the status to ASSIGNED. (If you're unable to change the bug's version
or status, add a comment to the bug and someone will change it for you.)

Thanks for your help, and we apologize again that we haven't handled
these issues to this point.

The process we're following is outlined here:
http://fedoraproject.org/wiki/BugZappers/F9CleanUp

We will be following the process here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping to ensure this
doesn't happen again.

Comment 4 Harshula Jayasuriya 2009-04-14 08:09:51 UTC
Hi,

Looks like this has been fixed in TOT:
-------------------------------------------------
commit eccb95cee4f0d56faa46ef22fb94dd4a3578d3eb
Author: Jan Kara <jack>
Date:   Tue Apr 29 00:59:37 2008 -0700

    vfs: fix lock inversion in drop_pagecache_sb()
    
    Fix longstanding lock inversion in drop_pagecache_sb by dropping inode_lock
    before calling __invalidate_mapping_pages().  We just have to make sure inode
    won't go away from under us by keeping reference to it and putting the
    reference only after we have safely resumed the scan of the inode list.  A bit
    tricky but not too bad...
    
    Signed-off-by: Jan Kara <jack>
    Cc: Fengguang Wu <wfg.edu.cn>
    Cc: David Chinner <dgc>
    Signed-off-by: Andrew Morton <akpm>
    Signed-off-by: Linus Torvalds <torvalds>

diff --git a/fs/drop_caches.c b/fs/drop_caches.c
index e2c6b65..50f9087 100644
--- a/fs/drop_caches.c
+++ b/fs/drop_caches.c
@@ -14,15 +14,21 @@ int sysctl_drop_caches;
 
 static void drop_pagecache_sb(struct super_block *sb)
 {
-	struct inode *inode;
+	struct inode *inode, *toput_inode = NULL;
 
 	spin_lock(&inode_lock);
 	list_for_each_entry(inode, &sb->s_inodes, i_sb_list) {
 		if (inode->i_state & (I_FREEING|I_WILL_FREE))
 			continue;
+		__iget(inode);
+		spin_unlock(&inode_lock);
 		__invalidate_mapping_pages(inode->i_mapping, 0, -1, true);
+		iput(toput_inode);
+		toput_inode = inode;
+		spin_lock(&inode_lock);
 	}
 	spin_unlock(&inode_lock);
+	iput(toput_inode);
 }
 
 static void drop_pagecache(void)
-------------------------------------------------

Regards,
Harshula