Bug 1089004
Summary: | Cluster FS resource force unmount and linked shared objects. | ||||||
---|---|---|---|---|---|---|---|
Product: | Red Hat Enterprise Linux 6 | Reporter: | John Boero <jboero> | ||||
Component: | resource-agents | Assignee: | David Vossel <dvossel> | ||||
Status: | CLOSED ERRATA | QA Contact: | Cluster QE <mspqa-list> | ||||
Severity: | high | Docs Contact: | |||||
Priority: | high | ||||||
Version: | 6.5 | CC: | agk, cluster-maint, djansa, fdinitto, jruemker, mnovacek, nobody, pzimek | ||||
Target Milestone: | rc | Keywords: | Regression | ||||
Target Release: | --- | ||||||
Hardware: | Unspecified | ||||||
OS: | Unspecified | ||||||
Whiteboard: | |||||||
Fixed In Version: | resource-agents-3.9.5-9.el6 | Doc Type: | Bug Fix | ||||
Doc Text: | Story Points: | --- | |||||
Clone Of: | Environment: | ||||||
Last Closed: | 2014-10-14 05:00:52 UTC | Type: | Bug | ||||
Regression: | --- | Mount Type: | --- | ||||
Documentation: | --- | CRM: | |||||
Verified Versions: | Category: | --- | |||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
Cloudforms Team: | --- | Target Upstream Version: | |||||
Embargoed: | |||||||
Attachments: |
|
Description
John Boero
2014-04-17 16:05:55 UTC
Note that this problem seems to have been introduced in RHEL 6.5. RHEL 6.4 appears to use fuser instead of find /proc. This logic has gone through several iterations for various reasons. We'll have to go back through the history why we ended up scanning the /proc/ directory to determine if 'lsof' usage is safe or not. -- Vossel We can't use lsof or fuser because of this issue, https://bugzilla.redhat.com/show_bug.cgi?id=1014298 Those tools use functions that can potentially block the file system resource-agents when nfs is in use somewhere on the system. There is a lsof '-b' option that apparently tries to reduce the potential for blocking to occur, but after doing an strace I'm not convinced blocking will not occur. As an alternative we can continue to use our custom shell logic and expand it to handle the use case where the process is using shared libraries that exist on the mount point being unmounted. This is the logic I've introduced. grep " ${mp}" /proc/[0-9]*/maps | awk -F/ '{print $3}' | uniq The full patch can be found in this upstream pull request. https://github.com/ClusterLabs/resource-agents/pull/422 -- Vossel Created attachment 910804 [details] unit test I've attached a unit test. This test requires the bind-mount agent that was introduced in this bugzilla. https://bugzilla.redhat.com/show_bug.cgi?id=1094789 Make sure you are using a build that has that new agent in it. This test is going to copy shared libraries over to a bind mount then execute commands that use those shared libraries. #./bind-test_v3.sh . . . . PASSED *** Bug 1118358 has been marked as a duplicate of this bug. *** I have verified with unittest from comment 7 that processes blocking umounting of the system are correctly identified and killed with resource-agents-3.9.5-11.el6.x86_64 ----- # rpm -q resource-agents resource-agents-3.9.5-11.el6.x86_64 # ./bind-test_v3.sh umount: /root/testsrc: not mounted umount: /root/target: not mounted <debug> Checking fs "testrsc", Level [bind-mount.sh] Checking fs "testrsc", Level <err> default: /root/testsrc is not mounted on /root/target [bind-mount.sh] default: /root/testsrc is not mounted on /root/target <info> /root/testsrc is not mounted [bind-mount.sh] /root/testsrc is not mounted <info> mounting /root/testsrc on /root/target [bind-mount.sh] mounting /root/testsrc on /root/target <debug> mount -o bind /root/testsrc /root/target [bind-mount.sh] mount -o bind /root/testsrc /root/target <debug> Checking fs "testrsc", Level [bind-mount.sh] Checking fs "testrsc", Level <debug> /root/testsrc already mounted [bind-mount.sh] /root/testsrc already mounted <info> unmounting /root/target [bind-mount.sh] unmounting /root/target <info> /root/testsrc is not mounted [bind-mount.sh] /root/testsrc is not mounted <debug> Checking fs "testrsc", Level [bind-mount.sh] Checking fs "testrsc", Level <err> default: /root/testsrc is not mounted on /root/target [bind-mount.sh] default: /root/testsrc is not mounted on /root/target <info> mounting /root/testsrc on /root/target [bind-mount.sh] mounting /root/testsrc on /root/target <debug> mount -o bind /root/testsrc /root/target [bind-mount.sh] mount -o bind /root/testsrc /root/target starting bg process to hold target file open this is test file starting bg process to hold target file open this is test file <debug> Checking fs "testrsc", Level [bind-mount.sh] Checking fs "testrsc", Level <info> unmounting /root/target [bind-mount.sh] unmounting /root/target umount: /root/target: device is busy. (In some cases useful info about processes that use the device is found by lsof(8) or fuser(1)) <debug> umount failed: 1 [bind-mount.sh] umount failed: 1 <warning>Sending SIGTERM to processes on /root/target [bind-mount.sh] Sending SIGTERM to processes on /root/target <info> unmounting /root/target [bind-mount.sh] unmounting /root/target ./bind-test_v3.sh: line 31: 17793 Terminated tail -f "$target/testfile" ./bind-test_v3.sh: line 31: 17798 Terminated tail -f "$tmpfile" ./bind-test_v3.sh: line 31: 17799 Terminated tail -f "$tmpfile" ./bind-test_v3.sh: line 31: 17800 Terminated tail -f "$tmpfile" ./bind-test_v3.sh: line 31: 17801 Terminated tail -f "$target/testfile" <debug> Checking fs "testrsc", Level [bind-mount.sh] Checking fs "testrsc", Level <err> default: /root/testsrc is not mounted on /root/target [bind-mount.sh] default: /root/testsrc is not mounted on /root/target PASSED Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2014-1428.html |