Description of problem: dm-multipath sometimes stalls on resume if noflush is used Version-Release number of selected component: kernel-2.6.18-1.2839.el5 (should occur with 2.6.18-1.2725.el5 or later) How reproducible: Always Steps to Reproduce: 1. Create a multipath device by dmsetup 2. Fail all paths during mkfs 3. Do dmsetup suspend --noflush and load new map with healthy paths 4. Do dmsetup resume (Attached test script will do the above steps.) Actual results: dmsetup resume stalls Expected results: dmsetup resume shouldn't stall Additional info: How to run the script: # modprobe dm-mod # modprobe dm-multipath # modprobe dm-round-robin # tar zxf noflush-stall-test.tgz # cd noflush-stall-test # ./run.sh The backtrace of dmsetup resume will look like: PID: 3940 TASK: f75dd000 CPU: 1 COMMAND: "dmsetup" #0 [f58a9d3c] schedule at c060f021 #1 [f58a9da4] __mutex_lock_slowpath at c060fa95 #2 [f58a9dc0] .text.lock.mutex (via mutex_lock) at c060fad3 #3 [f58a9dcc] dm_swap_table at f88b833b #4 [f58a9de8] dev_suspend at f88bad8e #5 [f58a9e00] ctl_ioctl at f88bb5b3 #6 [f58a9f6c] do_ioctl at c047e57d #7 [f58a9f84] vfs_ioctl at c047e7db #8 [f58a9fa0] sys_ioctl at c047e839 #9 [f58a9fb8] system_call at c0403fc4 EAX: ffffffda EBX: 00000003 ECX: c134fd06 EDX: 0825df18 DS: 007b ESI: 4ed41e40 ES: 007b EDI: 4ed3dc8b SS: 007b ESP: bfd3bb14 EBP: bfd3bc78 CS: 0073 EIP: 0030f402 ERR: 00000036 EFLAGS: 00000246 Sometimes, the stall occurs in dmsetup suspend in a similar way. What happens is that: 1 Someone does fsync() on the block dev holding inode->i_sem and possibly I_LOCK is set 2 The fsync write is blocked by the log failure 3 dm_suspend() tries to bdget() and waits forever on the inode being unlocked. Or, in the middle of dm_resume(), __bind() tries to get inode->i_sem to do __set_size() and waits forever The bug is in the common code of noflush feature. So it could occur on other targets if they start supporting noflush feature.
Created attachment 144740 [details] Reproduction test script
Sorry, the scenario description was for the case of mirror, the feature which is not there. For multipath, the scenario is similar. That is: 1 Someone does fsync() on the block dev holding inode->i_sem and possibly I_LOCK is set 2 The fsync write is blocked by all-paths-down and queue_if_no_path 3 dm_suspend() tries to bdget() and waits forever on the inode being unlocked. Or, in the middle of dm_resume(), __bind() tries to get inode->i_sem to do __set_size() and waits forever The test script should be able to cover both cases.
This request was evaluated by Red Hat Product Management for inclusion in a Red Hat Enterprise Linux maintenance release. Product Management has requested further review of this request by Red Hat Engineering, for potential inclusion in a Red Hat Enterprise Linux Update release for currently deployed products. This request is not yet committed for inclusion in an Update release.
in 2.6.18-16.el5
A fix for this issue should have been included in the packages contained in the most recent snapshot (partners.redhat.com) for RHEL5.1. Requested action: Please verify that your issue is fixed as soon as possible to ensure that it is included in this update release. After you (Red Hat Partner) have verified that this issue has been addressed, please perform the following: 1) Change the *status* of this bug to VERIFIED. 2) Add *keyword* of PartnerVerified (leaving the existing keywords unmodified) If this issue is not fixed, please add a comment describing the most recent symptoms of the problem you are having and change the status of the bug to FAILS_QA. More assistance: If you cannot access bugzilla, please reply with a message to Issue Tracker and I will change the status for you. If you need assistance accessing ftp://partners.redhat.com, please contact your Partner Manager.
Confirmed the testcase in comment#1 doesn't reproduce the problem.
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on the solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHBA-2007-0959.html