Bug 1245065
Summary: | "rm -rf *" from multiple mount points fails to remove directories on all the subvolumes | |||
---|---|---|---|---|
Product: | [Community] GlusterFS | Reporter: | Sakshi <sabansal> | |
Component: | distribute | Assignee: | Sakshi <sabansal> | |
Status: | CLOSED CURRENTRELEASE | QA Contact: | ||
Severity: | high | Docs Contact: | ||
Priority: | unspecified | |||
Version: | mainline | CC: | bugs, olim, rgowdapp, sabansal, sankarshan, smohan, spalai, spandura | |
Target Milestone: | --- | Keywords: | Triaged | |
Target Release: | --- | |||
Hardware: | Unspecified | |||
OS: | Unspecified | |||
Whiteboard: | dht-rm-rf | |||
Fixed In Version: | glusterfs-3.8rc2 | Doc Type: | Bug Fix | |
Doc Text: | Story Points: | --- | ||
Clone Of: | 1115367 | |||
: | 1257894 (view as bug list) | Environment: | ||
Last Closed: | 2016-06-16 13:25:33 UTC | Type: | Bug | |
Regression: | --- | Mount Type: | --- | |
Documentation: | --- | CRM: | ||
Verified Versions: | Category: | --- | ||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
Cloudforms Team: | --- | Target Upstream Version: | ||
Embargoed: | ||||
Bug Depends On: | 1115367 | |||
Bug Blocks: | 960910, 960938, 1128737, 1235633, 1257894 |
Description
Sakshi
2015-07-21 07:10:23 UTC
REVIEW: http://review.gluster.org/11725 (dht : lock on hashed subvol to prevent lookup vs rmdir race) posted (#1) for review on master by Sakshi Bansal (sabansal) Please provide a public facing description of the issue. REVIEW: http://review.gluster.org/11725 (dht:lock on hashed subvol to prevent lookup vs rmdir race) posted (#2) for review on master by Sakshi Bansal (sabansal) REVIEW: http://review.gluster.org/11725 (dht: lock on subvols to prevent lookup vs rmdir race) posted (#5) for review on master by Sakshi Bansal (sabansal) REVIEW: http://review.gluster.org/11725 (dht: lock on subvols to prevent lookup vs rmdir race) posted (#6) for review on master by Sakshi Bansal (sabansal) REVIEW: http://review.gluster.org/11725 (dht : lock on subvols to prevent lookup vs rmdir race) posted (#7) for review on master by Sakshi Bansal (sabansal) REVIEW: http://review.gluster.org/11916 (lock : check if inode exists before granting blocked locks) posted (#1) for review on master by Sakshi Bansal (sabansal) REVIEW: http://review.gluster.org/11725 (dht: lock on subvols to prevent lookup vs rmdir race) posted (#8) for review on master by Sakshi Bansal (sabansal) REVIEW: http://review.gluster.org/11725 (dht: lock on subvols to prevent lookup vs rmdir race) posted (#9) for review on master by Raghavendra G (rgowdapp) REVIEW: http://review.gluster.org/11725 (dht : lock on subvols to prevent lookup vs rmdir race) posted (#10) for review on master by Sakshi Bansal (sabansal) REVIEW: http://review.gluster.org/11916 (lock : check if inode exists before granting blocked locks) posted (#3) for review on master by Sakshi Bansal (sabansal) REVIEW: http://review.gluster.org/11725 (dht: lock on subvols to prevent lookup vs rmdir race) posted (#11) for review on master by Sakshi Bansal (sabansal) REVIEW: http://review.gluster.org/11725 (dht : lock on subvols to prevent lookup vs rmdir race) posted (#12) for review on master by Sakshi Bansal (sabansal) REVIEW: http://review.gluster.org/12035 (dht: lookup after selfheal acquires lock in the mkdir phase) posted (#1) for review on master by Sakshi Bansal (sabansal) REVIEW: http://review.gluster.org/12035 (dht: lookup after selfheal acquires lock in the mkdir phase) posted (#2) for review on master by Sakshi Bansal (sabansal) REVIEW: http://review.gluster.org/12125 (dht : lock on all subvols to prevent rmdir vs lookup selfheal race) posted (#1) for review on master by Sakshi Bansal (sabansal) REVIEW: http://review.gluster.org/12125 (dht : reverting changes that takes lock on all subvols to prevent rmdir vs lookup selfheal race) posted (#2) for review on master by Sakshi Bansal (sabansal) REVIEW: http://review.gluster.org/12125 (dht : reverting changes that takes lock on all subvols to prevent rmdir vs lookup selfheal race) posted (#3) for review on master by Dan Lambright (dlambrig) REVIEW: http://review.gluster.org/12125 (dht: reverting changes that takes lock on all subvols to prevent rmdir vs lookup selfheal race) posted (#4) for review on master by Sakshi Bansal (sabansal) COMMIT: http://review.gluster.org/12125 committed in master by Raghavendra G (rgowdapp) ------ commit 7b9135045685125d7c94d75f06d762fa1c5ba4b9 Author: Sakshi <sabansal> Date: Mon Aug 31 16:06:35 2015 +0530 dht: reverting changes that takes lock on all subvols to prevent rmdir vs lookup selfheal race Locking on all subvols before an rmdir is unable to remove all directory entries. Hence reverting the patch for now. Change-Id: I31baf2b2fa2f62c57429cd44f3f229c35eff1939 BUG: 1245065 Signed-off-by: Sakshi <sabansal> Reviewed-on: http://review.gluster.org/12125 Tested-by: Gluster Build System <jenkins.com> Tested-by: NetBSD Build System <jenkins.org> Reviewed-by: Raghavendra G <rgowdapp> REVIEW: http://review.gluster.org/13528 (dht: lock on subvols to prevent lookup vs rmdir race) posted (#1) for review on master by Sakshi Bansal REVIEW: http://review.gluster.org/13528 (dht: lock on subvols to prevent lookup vs rmdir race) posted (#2) for review on master by Sakshi Bansal REVIEW: http://review.gluster.org/13528 (dht: lock on subvols to prevent lookup vs rmdir race) posted (#3) for review on master by Sakshi Bansal REVIEW: http://review.gluster.org/13816 (mount/fuse: report ESTALE as ENOENT) posted (#1) for review on master by Raghavendra G (rgowdapp) REVIEW: http://review.gluster.org/13818 (gluster-NFS: For remove fop(), report ENOENT for ESTALE) posted (#2) for review on master by Raghavendra G (rgowdapp) REVIEW: http://review.gluster.org/13816 (mount/fuse: report ESTALE as ENOENT) posted (#2) for review on master by Raghavendra G (rgowdapp) REVIEW: http://review.gluster.org/13816 (mount/fuse: report ESTALE as ENOENT) posted (#3) for review on master by Raghavendra G (rgowdapp) This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune with any questions REVIEW: http://review.gluster.org/13528 (dht: lock on subvols to prevent lookup vs rmdir race) posted (#4) for review on master by Raghavendra G (rgowdapp) REVIEW: http://review.gluster.org/13528 (dht: lock on subvols to prevent lookup vs rmdir race) posted (#5) for review on master by Raghavendra G (rgowdapp) REVIEW: http://review.gluster.org/13528 (dht: lock on subvols to prevent lookup vs rmdir race) posted (#6) for review on master by Raghavendra G (rgowdapp) COMMIT: http://review.gluster.org/13528 committed in master by Raghavendra G (rgowdapp) ------ commit c25f88c953215b1bfc135aeafc43dc00a663206d Author: Sakshi <sabansal> Date: Thu Jul 16 14:31:03 2015 +0530 dht: lock on subvols to prevent lookup vs rmdir race There is a possibility that while an rmdir is completed on some non-hashed subvol and proceeding to others, a lookup selfheal can recreate the same directory on those subvols for which the rmdir had succeeded. Now the deletion of the parent directory will fail with an ENOTEMPTY. To fix this take blocking inodelk on the subvols before starting rmdir. Selfheal must also take blocking inodelk before creating the entry. Change-Id: I168a195c35ac1230ba7124d3b0ca157755b3df96 BUG: 1245065 Signed-off-by: Sakshi <sabansal> Reviewed-on: http://review.gluster.org/13528 CentOS-regression: Gluster Build System <jenkins.com> NetBSD-regression: NetBSD Build System <jenkins.org> Smoke: Gluster Build System <jenkins.com> Reviewed-by: Raghavendra G <rgowdapp> Tested-by: Raghavendra G <rgowdapp> COMMIT: http://review.gluster.org/13816 committed in master by Raghavendra G (rgowdapp) ------ commit 26d16b90ec7f8acbe07e56e8fe1baf9c9fa1519e Author: Raghavendra G <rgowdapp> Date: Wed Mar 23 13:47:27 2016 +0530 mount/fuse: report ESTALE as ENOENT When the inode/gfid is missing, brick report back as an ESTALE error. However, most of the applications don't accept ESTALE as an error for a file-system object missing, changing their behaviour. For eg., rm -rf ignores ENOENT errors during unlink of files/directories. But with ESTALE error it doesn't send rmdir on a directory if unlink had failed with ESTALE for any of the files or directories within it. Thanks to Ravishankar N <ravishankar>, here is a link as to why we split up ENOENT into ESTALE and ENOENT. http://review.gluster.org/#/c/6318/ Change-Id: I467df0fdf22734a8ef20c79ac52606410fad04d1 BUG: 1245065 Signed-off-by: Raghavendra G <rgowdapp> Reviewed-on: http://review.gluster.org/13816 Smoke: Gluster Build System <jenkins.com> Reviewed-by: Jeff Darcy <jdarcy> Reviewed-by: N Balachandran <nbalacha> Reviewed-by: Niels de Vos <ndevos> Tested-by: N Balachandran <nbalacha> CentOS-regression: Gluster Build System <jenkins.com> NetBSD-regression: NetBSD Build System <jenkins.org> This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.0, please open a new bug report. glusterfs-3.8.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://blog.gluster.org/2016/06/glusterfs-3-8-released/ [2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user |