Bug 769917

I reproduced this deadlock with multipath when using queue_if_no_path.

block layer was modified to increase the window for DM devices to hit the flush on close code path (allowing me to fail the remaining path(s) in the multipath device before the block layer proceeds to flush outstanding buffered IO on the last close):

diff --git a/fs/block_dev.c b/fs/block_dev.c
index b07f1da..d055ed4 100644
--- a/fs/block_dev.c
+++ b/fs/block_dev.c
@@ -26,6 +26,7 @@
 #include <linux/log2.h>
 #include <linux/kmemleak.h>
 #include <asm/uaccess.h>
+#include <linux/delay.h>
 #include "internal.h"
 
 struct bdev_inode {
@@ -1429,6 +1430,10 @@ static int __blkdev_put(struct block_device *bdev, fmode_t mode, int for_part)
 
        if (!--bdev->bd_openers) {
                WARN_ON_ONCE(bdev->bd_holders);
+               if (MAJOR(bdev->bd_dev) == 253) {
+                       printk("%s: sleeping for 60 seconds\n", __func__);
+                       msleep(60000);
+               }
                sync_blockdev(bdev);
                kill_bdev(bdev);
                /* ->release can cause the old bdi to disappear,

1) stop multipathd
2) fail 3 of the 4 paths:
   # dmsetup message mpathb 0 fail_path /dev/sda
   # dmsetup message mpathb 0 fail_path /dev/sdb
   # dmsetup message mpathb 0 fail_path /dev/sdc
3) start buffered IO:
   # dd if=/dev/zero of=/dev/mapper/mpathb bs=1024k count=100
   (you'll see: "__blkdev_put: sleeping for 60 seconds" in the kernel log)
4) fail the last path in the mpath device:
   # dmsetup message mpathb 0 fail_path /dev/sdd

Results in the following backtrace (once the msleep completes):
dd              D ffff8801174e5750  5104  1555   1272 0x00000000
 ffff880118eb7bf8 0000000000000046 0000000000036afa ffffffffb0fcef3b
 ffff880118eb6010 ffff8801174e53c0 00000000001d25c0 ffff880118eb7fd8
 ffff880118eb7fd8 00000000001d25c0 ffff880118468600 ffff8801174e53c0
Call Trace:
 [<ffffffff810c94d4>] ? __lock_page+0x6d/0x6d
 [<ffffffff8138c9ce>] schedule+0x5a/0x5c
 [<ffffffff8138ca5c>] io_schedule+0x8c/0xcf
 [<ffffffff810c94e2>] sleep_on_page+0xe/0x12
 [<ffffffff8138cee6>] __wait_on_bit+0x48/0x7b
 [<ffffffff810c96d8>] wait_on_page_bit+0x72/0x79
 [<ffffffff8105fb87>] ? autoremove_wake_function+0x3d/0x3d
 [<ffffffff810d2869>] ? pagevec_lookup_tag+0x25/0x2e
 [<ffffffff810c9a26>] filemap_fdatawait_range+0xa4/0x171
 [<ffffffff810c9b17>] filemap_fdatawait+0x24/0x26
 [<ffffffff810ca0e7>] filemap_write_and_wait+0x2f/0x3b
 [<ffffffff81132da2>] __sync_blockdev+0x2c/0x2e
 [<ffffffff81132db7>] sync_blockdev+0x13/0x15
 [<ffffffff81132e65>] __blkdev_put+0xac/0x198
 [<ffffffff8113307d>] blkdev_put+0x12c/0x135
 [<ffffffff811330ad>] blkdev_close+0x27/0x29
 [<ffffffff81107d76>] fput+0x114/0x1e4
 [<ffffffff8110472d>] filp_close+0x75/0x80
 [<ffffffff811047e3>] sys_close+0xab/0xe5
 [<ffffffff81395b02>] system_call_fastpath+0x16/0x1b

Any other mpathb open attempt (e.g. dmsetup info mpathb) will hang trying to get bdev->bd_mutex

(reinstating a path allows IO to continue, eliminating the bdev->bd_mutex deadlock)

This bug appears to have been reported against 'rawhide' during the Fedora 19 development cycle.
Changing version to '19'.

(As we did not run this process for some time, it could affect also pre-Fedora 19 development
cycle bugs. We are very sorry. It will help us with cleanup during Fedora 19 End Of Life. Thank you.)

More information and reason for this action is here:
https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora19

Is this still a problem with 3.9 based F19 kernels?

This bug is being closed with INSUFFICIENT_DATA as there has not been a
response in 2 weeks.  If you are still experiencing this issue,
please reopen and attach the relevant data from the latest kernel you are
running and any data that might have been requested previously.

This shouldn't have been closed.  Is it still a problem or if not, which change fixed it?

*********** MASS BUG UPDATE **************

We apologize for the inconvenience.  There is a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 19 kernel bugs.

Fedora 19 has now been rebased to 3.12.6-200.fc19.  Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel.

If you have moved on to Fedora 20, and are still experiencing this issue, please change the version to Fedora 20.

If you experience different issues, please open a new bug report for those.

*********** MASS BUG UPDATE **************

This bug has been in a needinfo state for more than 1 month and is being closed with insufficient data due to inactivity. If this is still an issue with Fedora 19, please feel free to reopen the bug and provide the additional information requested.