Bug 1305031 - AFR winds a few reads of a file in metadata split-brain.
AFR winds a few reads of a file in metadata split-brain.
Status: CLOSED CURRENTRELEASE
Product: GlusterFS
Classification: Community
Component: replicate (Show other bugs)
mainline
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Ravishankar N
: Triaged
Depends On:
Blocks: 1349879 1349881
  Show dependency treegraph
 
Reported: 2016-02-05 06:34 EST by Ravishankar N
Modified: 2017-03-27 14:13 EDT (History)
1 user (show)

See Also:
Fixed In Version: glusterfs-3.9.0
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1349879 1349881 (view as bug list)
Environment:
Last Closed: 2017-03-27 14:13:07 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Ravishankar N 2016-02-05 06:34:57 EST
Description of problem:

AFR winds a few reads of a file in metadata split-brain.

How reproducible:
Always

Steps to Reproduce:
1. cat a big file that is in metadata split-brain
2. It spews out a lot of data before eventually failing it with EIO.



Actual results:
readvs are sent to the brick

Expected results:
No readv should be wound to the brick.

Additional info:
Comment 1 Vijay Bellur 2016-02-05 06:37:25 EST
REVIEW: http://review.gluster.org/13389 (afr:Don't wind reads for files in metadata split-brain) posted (#1) for review on master by Ravishankar N (ravishankar@redhat.com)
Comment 2 Vijay Bellur 2016-03-01 08:03:45 EST
REVIEW: http://review.gluster.org/13389 (afr:Don't wind reads for files in metadata split-brain) posted (#2) for review on master by Ravishankar N (ravishankar@redhat.com)
Comment 3 Vijay Bellur 2016-03-01 08:07:49 EST
REVIEW: http://review.gluster.org/13389 (afr:Don't wind reads for files in metadata split-brain) posted (#3) for review on master by Ravishankar N (ravishankar@redhat.com)
Comment 4 Vijay Bellur 2016-03-01 23:07:13 EST
REVIEW: http://review.gluster.org/13389 (afr:Don't wind reads for files in metadata split-brain) posted (#4) for review on master by Ravishankar N (ravishankar@redhat.com)
Comment 5 Mike McCune 2016-03-28 19:24:30 EDT
This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune@redhat.com with any questions
Comment 6 Vijay Bellur 2016-06-23 05:20:56 EDT
REVIEW: http://review.gluster.org/13389 (afr:Don't wind reads for files in metadata split-brain) posted (#5) for review on master by Ravishankar N (ravishankar@redhat.com)
Comment 7 Vijay Bellur 2016-06-24 07:26:05 EDT
COMMIT: http://review.gluster.org/13389 committed in master by Pranith Kumar Karampuri (pkarampu@redhat.com) 
------
commit 7a1c1e2904701496968ed14b6d7479fb706c3188
Author: Ravishankar N <ravishankar@redhat.com>
Date:   Fri Feb 5 15:10:06 2016 +0530

    afr:Don't wind reads for files in metadata split-brain
    
    Problem: For a read on  a file in metadata split-brain:
    1.lookup_done resets event_generation to zero.
    2. readv is issued, goes to inode refresh due to mismatching event_gen.
    3. After refresh is successful, we update event_generation, data and
    metdata readable.
    3. We then call afr_read_txn_refresh_done() which in turn calls
    afr_inode_get_readable() but doesn't check for EIO. So afr_readv_wind
    is called with local->readable (which is populated with data_readable),
    thus winding the read to a brick.
    4. Also, further parallel reads that come directly go to the wind path
    because there is no inode_refresh needed.
    
    Fix:
    1.For any afr_read_txn(), readable must be an intersection of data and metadata
    readable.
    2.Check for EIO in afr_read_txn_refresh_done().
    
    Change-Id: I22dd221fdfaf96d7aced2f474e28ed1337d69f0e
    BUG: 1305031
    Signed-off-by: Ravishankar N <ravishankar@redhat.com>
    Reviewed-on: http://review.gluster.org/13389
    Reviewed-by: Ashish Pandey <aspandey@redhat.com>
    Smoke: Gluster Build System <jenkins@build.gluster.org>
    NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
    CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Comment 8 Shyamsundar 2017-03-27 14:13:07 EDT
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.9.0, please open a new bug report.

glusterfs-3.9.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://lists.gluster.org/pipermail/gluster-users/2016-November/029281.html
[2] https://www.gluster.org/pipermail/gluster-users/

Note You need to log in before you can comment on or make changes to this bug.