Description of problem: In a replicated setup, with the granular entry self-heal feature, when a directory tree is created and deleted while a replica was offline, index translator would need to remove the directory index (or move it into landfill so the janitor thread can pick it up). Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
REVIEW: http://review.gluster.org/14742 (features/index: Do deletion of name-index in unwind path) posted (#1) for review on master by Krutika Dhananjay (kdhananj)
REVIEW: http://review.gluster.org/14742 (features/index: Do deletion of name-index in unwind path) posted (#2) for review on master by Krutika Dhananjay (kdhananj)
REVIEW: http://review.gluster.org/14781 (features/index: Delete parent dir indices when heal on it is complete) posted (#1) for review on master by Krutika Dhananjay (kdhananj)
COMMIT: http://review.gluster.org/14742 committed in master by Pranith Kumar Karampuri (pkarampu) ------ commit 59523a782d7ed00b31c8cdf8da74310384272355 Author: Krutika Dhananjay <kdhananj> Date: Wed Jun 15 16:02:44 2016 +0530 features/index: Do deletion of name-index in unwind path Previously the deletion of name index was being done before winding the xattrop to posix. This order needs to be changed to ensure we don't lose the index in the event the xattrop fails in posix xl. Change-Id: I4aa892b102cbf5269cede041b871064563cb7348 BUG: 1331323 Signed-off-by: Krutika Dhananjay <kdhananj> Reviewed-on: http://review.gluster.org/14742 Smoke: Gluster Build System <jenkins.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu> NetBSD-regression: NetBSD Build System <jenkins.org> Reviewed-by: Ravishankar N <ravishankar> CentOS-regression: Gluster Build System <jenkins.org>
REVIEW: http://review.gluster.org/14781 (features/index: Delete parent dir indices when heal on it is complete) posted (#2) for review on master by Krutika Dhananjay (kdhananj)
REVIEW: http://review.gluster.org/14781 (features/index: Delete parent dir indices when heal on it is complete) posted (#3) for review on master by Krutika Dhananjay (kdhananj)
REVIEW: http://review.gluster.org/14781 (features/index: Delete parent dir indices when heal on it is complete) posted (#4) for review on master by Krutika Dhananjay (kdhananj)
COMMIT: http://review.gluster.org/14781 committed in master by Pranith Kumar Karampuri (pkarampu) ------ commit 2ca5388692e70e079c99771d8c66a06bd90d5268 Author: Krutika Dhananjay <kdhananj> Date: Wed Jun 22 14:52:58 2016 +0530 features/index: Delete parent dir indices when heal on it is complete In this patch, the state information about whether a directory gfid index is present or not is stored in the inode ctx with values IN and NOTIN. This saves index xl the need to perform stat() everytime an index_entry_create() is called. When a brick is restarted these in-memory inode ctx records will be gone. So when granular entry heal happens after a brick is restarted, and a post-op is done on the parent, if the state gotten from inode ctx is UNKNOWN, then index xl does a stat to initialize the state as IN or NOTIN. Note that this is a one-time operation for the lifetime of the brick. Such a change also helps avoid calling index_del() in xattrop_index_action() periodically even when granular self-heal is disabled or when the volume type is disperse. Change-Id: Ib92c17350e6531b91ab81477410fe1e7a5856207 BUG: 1331323 Signed-off-by: Krutika Dhananjay <kdhananj> Reviewed-on: http://review.gluster.org/14781 Smoke: Gluster Build System <jenkins.org> CentOS-regression: Gluster Build System <jenkins.org> NetBSD-regression: NetBSD Build System <jenkins.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu>
REVIEW: http://review.gluster.org/14832 (afr, index: Clean up stale directory and file indices in granular entry sh) posted (#1) for review on master by Krutika Dhananjay (kdhananj)
REVIEW: http://review.gluster.org/14832 (afr, index: Clean up stale directory and file indices in granular entry sh) posted (#2) for review on master by Krutika Dhananjay (kdhananj)
REVIEW: http://review.gluster.org/14870 (features/index: Ignore regular files when examining state of the inode wrt ENTRY_CHANGES) posted (#1) for review on master by Krutika Dhananjay (kdhananj)
REVIEW: http://review.gluster.org/14832 (afr, index: Clean up stale directory and file indices in granular entry sh) posted (#3) for review on master by Krutika Dhananjay (kdhananj)
REVIEW: http://review.gluster.org/14832 (afr, index: Clean up stale directory and file indices in granular entry sh) posted (#4) for review on master by Krutika Dhananjay (kdhananj)
COMMIT: http://review.gluster.org/14832 committed in master by Pranith Kumar Karampuri (pkarampu) ------ commit ceac3b4172e7c689941ec3e734eca6c537e69368 Author: Krutika Dhananjay <kdhananj> Date: Mon Jun 27 15:21:26 2016 +0530 afr, index: Clean up stale directory and file indices in granular entry sh Specifically when a directory tree is removed (rm -rf) while a brick is down, both the directory index and the name indices of the files and subdirs under it will remain. Self-heal will need to pick up these and remove them. Towards this, afr sh will now also crawl indices/entry-changes and call an rmdir on the dir if the directory index is stale. On the brick side, rmdir fop has been implemented for index xl, which would delete the directory index and its contents if present in a synctask. Change-Id: I8b527331c2547e6c141db6c57c14055ad1198a7e BUG: 1331323 Signed-off-by: Krutika Dhananjay <kdhananj> Reviewed-on: http://review.gluster.org/14832 Reviewed-by: Ravishankar N <ravishankar> Smoke: Gluster Build System <jenkins.org> NetBSD-regression: NetBSD Build System <jenkins.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu> CentOS-regression: Gluster Build System <jenkins.org>
COMMIT: http://review.gluster.org/14870 committed in master by Pranith Kumar Karampuri (pkarampu) ------ commit 633d88ea7ef621bcce9a408188bc57858f6ca942 Author: Krutika Dhananjay <kdhananj> Date: Thu Jul 7 14:33:20 2016 +0530 features/index: Ignore regular files when examining state of the inode wrt ENTRY_CHANGES This prevents unnecessary sys_lstat() syscall when xattrop is operating on regular files. Also, to avoid (even) the one-time execution of sys_lstat() syscalls on disperse volumes or replicate volumes when granular entry self-heal is disabled, we pass a key in afr_selfheal_undo_pending() indicating to index that this xattrop is on a granular directory index. Index will accordingly decide whether or not to attempt the index state initialization. Change-Id: I616d9a4b3704066f4a2770d3ca307f7dca9122e0 BUG: 1331323 Signed-off-by: Krutika Dhananjay <kdhananj> Reviewed-on: http://review.gluster.org/14870 Smoke: Gluster Build System <jenkins.org> NetBSD-regression: NetBSD Build System <jenkins.org> CentOS-regression: Gluster Build System <jenkins.org> Reviewed-by: Anuradha Talur <atalur> Reviewed-by: Pranith Kumar Karampuri <pkarampu>
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.9.0, please open a new bug report. glusterfs-3.9.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://lists.gluster.org/pipermail/gluster-users/2016-November/029281.html [2] https://www.gluster.org/pipermail/gluster-users/