+++ This bug was initially created as a clone of Bug #1378547 +++ Description of problem: The unsplit-brain mechanism is triggered along the self-healing mechanism. Since the self-healing mechanism is asynchronous, so is the unsplit-brain mechanism. Therefore, even tough the split-brain is resolved eventually, all system calls made before this happens causes an IOE to occur. This pushes the responsibility back to the client application, which needs to retry the system call, which in turn cause a waste of resources. The self-heal mechanism should still be asynchronous, but the right version of the favorite child policy should be resolved synchronously to prevent the Input/Output exception to occur. Version-Release number of selected component (if applicable): 3.8.4-1 How reproducible: Create a split-brained file and assert that the first read still always causes an Input/Output Error. Steps to Reproduce: 1. Set cluster.entry-self-heal to on, cluster.data-self-heal to on, cluster.metadata-self-heal to on and cluster.favorite-child-policy to mtime 2. Create a split-brained file 3. Cat the split-brained file -> Ensure that an Input/Output Error is raised 4. Cat the file again ~1sec later -> Ensure that the file was healed Actual results: [root@host vol]# cat test cat: test: Input/output error [root@host vol]# cat test [root@host vol]# Expected results: [root@host vol]# cat test [root@host vol]# Additional info:
REVIEW: http://review.gluster.org/15673 (afr: allow I/O when favorite-child-policy is enabled) posted (#1) for review on master by Ravishankar N (ravishankar)
REVIEW: http://review.gluster.org/15673 (afr: allow I/O when favorite-child-policy is enabled) posted (#2) for review on master by Ravishankar N (ravishankar)
REVIEW: http://review.gluster.org/15673 (afr: allow I/O when favorite-child-policy is enabled) posted (#3) for review on master by Ravishankar N (ravishankar)
REVIEW: http://review.gluster.org/15673 (afr: allow I/O when favorite-child-policy is enabled) posted (#4) for review on master by Ravishankar N (ravishankar)
REVIEW: http://review.gluster.org/15673 (afr: allow I/O when favorite-child-policy is enabled) posted (#5) for review on master by Ravishankar N (ravishankar)
REVIEW: http://review.gluster.org/15673 (afr: allow I/O when favorite-child-policy is enabled) posted (#6) for review on master by Ravishankar N (ravishankar)
REVIEW: http://review.gluster.org/15673 (afr: allow I/O when favorite-child-policy is enabled) posted (#7) for review on master by Ravishankar N (ravishankar)
REVIEW: http://review.gluster.org/15673 (afr: allow I/O when favorite-child-policy is enabled) posted (#8) for review on master by Ravishankar N (ravishankar)
REVIEW: http://review.gluster.org/15673 (afr: allow I/O when favorite-child-policy is enabled) posted (#9) for review on master by Pranith Kumar Karampuri (pkarampu)
COMMIT: http://review.gluster.org/15673 committed in master by Pranith Kumar Karampuri (pkarampu) ------ commit a07ddd8fcc8dcdcf7ccfa61211d258f13b9f9229 Author: Ravishankar N <ravishankar> Date: Sat Nov 26 21:24:01 2016 +0530 afr: allow I/O when favorite-child-policy is enabled Problem: Currently, I/O on a split-brained file fails even when the favorite-child-policy is set until the self-heal is complete. Fix: If a valid 'source' is found using the set favorite-child-policy, inspect and reset the afr pending xattrs on the 'sinks' (inside appropriate locks), refresh the inode and then proceed with the read or write transaction. The resetting itself happens in the self-heal code and hence can also happen in the client side background-heal or by the shd's index-heal in addition to the txn code path explained above. When it happens in via heal, we also add checks in undo-pending to not reset the sink xattrs again. Change-Id: Ic8c1317720cb26bd114b6fe6af4e58c73b864626 BUG: 1386188 Signed-off-by: Ravishankar N <ravishankar> Reported-by: Simon Turcotte-Langevin <simon.turcotte-langevin> Reviewed-on: http://review.gluster.org/15673 Tested-by: Pranith Kumar Karampuri <pkarampu> Smoke: Gluster Build System <jenkins.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu> NetBSD-regression: NetBSD Build System <jenkins.org> CentOS-regression: Gluster Build System <jenkins.org>
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.10.0, please open a new bug report. glusterfs-3.10.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://lists.gluster.org/pipermail/gluster-users/2017-February/030119.html [2] https://www.gluster.org/pipermail/gluster-users/