Description of problem: glusterfs user Mat Clayton told me on IRC that sometimes `ls -l of /mount/filename` on an arbiter volume reported zero bytes despite all bricks being up and the actual data file on the non-arbiter bricks being 120 Meg. Version-Release number of selected component (if applicable): v3.7.8 How reproducible: Sometimes the value is right and other times it is zero.
RCA:I was able to re-create the issue intermittently. Basically if afr_stat() gets called on the file whose gfid hashes to the arbiter brick, afr_read_subvol_select_by_policy() picks up the arbiter brick to wind the call and hence we end up serving zero as the iatt size.
REVIEW: http://review.gluster.org/13539 (afr: do not set arbiter as a readable subvol in inode context) posted (#1) for review on master by Ravishankar N (ravishankar)
REVIEW: http://review.gluster.org/13539 (afr: do not set arbiter as a readable subvol in inode context) posted (#2) for review on master by Ravishankar N (ravishankar)
REVIEW: http://review.gluster.org/13539 (afr: do not set arbiter as a readable subvol in inode context) posted (#3) for review on master by Ravishankar N (ravishankar)
REVIEW: http://review.gluster.org/13539 (afr: do not set arbiter as a readable subvol in inode context) posted (#4) for review on master by Ravishankar N (ravishankar)
REVIEW: http://review.gluster.org/13539 (afr: do not set arbiter as a readable subvol in inode context) posted (#5) for review on master by Ravishankar N (ravishankar)
REVIEW: http://review.gluster.org/13539 (afr: do not set arbiter as a readable subvol in inode context) posted (#6) for review on master by Ravishankar N (ravishankar)
REVIEW: http://review.gluster.org/13539 (afr: do not set arbiter as a readable subvol in inode context) posted (#7) for review on master by Ravishankar N (ravishankar)
REVIEW: http://review.gluster.org/13539 (afr: do not set arbiter as a readable subvol in inode context) posted (#8) for review on master by Ravishankar N (ravishankar)
COMMIT: http://review.gluster.org/13539 committed in master by Jeff Darcy (jdarcy) ------ commit 8ab87137c855c9a0551a9100c146e4ca76dbbda2 Author: Ravishankar N <ravishankar> Date: Mon Feb 29 05:16:50 2016 +0000 afr: do not set arbiter as a readable subvol in inode context Problem: If afr_lookup_done() or afr_read_subvol_select_by_policy() chooses the arbiter brick to serve the stat() data, file size will be reported as zero from the mount, despite other data bricks being available. This can break programs like tar which use the stat info to decide how much to read. Fix: In the inode-context, mark arbiter as a non-readable subvol for both data and metadata. It it to be noted that by making this fix, we are *not* going to serve metadata FOPS anymore from the arbiter brick despite the brick storing the metadata. It makes sense to do this because the ever increasing over-loaded FOPs (getxattr returning stat data etc.) and compound FOPS in gluster will otherwise make it difficult to add checks in code to handle corner cases. Change-Id: Ic60b25d77fd05e0897481b7fcb3716d4f2101001 BUG: 1310171 Signed-off-by: Ravishankar N <ravishankar> Reported-by: Mat Clayton <mat> Reviewed-on: http://review.gluster.org/13539 Reviewed-by: Anuradha Talur <atalur> Reviewed-by: Krutika Dhananjay <kdhananj> Smoke: Gluster Build System <jenkins.com> NetBSD-regression: NetBSD Build System <jenkins.org> CentOS-regression: Gluster Build System <jenkins.com> Reviewed-by: Jeff Darcy <jdarcy>
REVIEW: http://review.gluster.org/13611 (tests: Use fource_umount instead of umount) posted (#1) for review on master by Pranith Kumar Karampuri (pkarampu)
REVIEW: http://review.gluster.org/13611 (tests: Use force_umount instead of umount) posted (#2) for review on master by Pranith Kumar Karampuri (pkarampu)
REVIEW: http://review.gluster.org/13611 (tests: Use force_umount instead of umount) posted (#3) for review on master by Pranith Kumar Karampuri (pkarampu)
REVIEW: http://review.gluster.org/13611 (tests: Use force_umount instead of umount) posted (#4) for review on master by Pranith Kumar Karampuri (pkarampu)
COMMIT: http://review.gluster.org/13611 committed in master by Pranith Kumar Karampuri (pkarampu) ------ commit 805a9c7a40e22f811019f4a854636c30c164525b Author: Pranith Kumar K <pkarampu> Date: Sat Mar 5 21:41:33 2016 +0530 tests: Use force_umount instead of umount umount leads to spurious failures with "mount is busy" kind of errors at the time of umount. Use 'force_umount' instead. BUG: 1310171 Change-Id: I5a5579288f002de14effc00b793143fef86eb828 Signed-off-by: Pranith Kumar K <pkarampu> Reviewed-on: http://review.gluster.org/13611 NetBSD-regression: NetBSD Build System <jenkins.org> Smoke: Gluster Build System <jenkins.com> CentOS-regression: Gluster Build System <jenkins.com> Reviewed-by: Ravishankar N <ravishankar> Reviewed-by: Krutika Dhananjay <kdhananj>
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.0, please open a new bug report. glusterfs-3.8.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://blog.gluster.org/2016/06/glusterfs-3-8-released/ [2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user