Bug 1266876

Summary: cluster/afr: AFR2 returns empty readdir results to clients if brick is added back into cluster after re-imaging/formatting
Product: [Community] GlusterFS Reporter: Anuradha <atalur>
Component: replicateAssignee: Pranith Kumar K <pkarampu>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: urgent Docs Contact:
Priority: unspecified    
Version: mainlineCC: amukherj, bugs, pkarampu, pportant, ravishankar, rwareing, smohan, sshreyas, storage-qa-internal
Target Milestone: ---Keywords: ZStream
Target Release: ---   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: glusterfs-3.9.0 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1256524 Environment:
Last Closed: 2017-03-27 18:27:08 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1256524    
Bug Blocks:    

Comment 1 Anuradha 2015-09-29 12:14:09 UTC
Patch posted for review :
http://review.gluster.org/#/c/12250/

Comment 2 Mike McCune 2016-03-28 22:17:27 UTC
This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune with any questions

Comment 3 Vijay Bellur 2016-06-21 07:58:37 UTC
REVIEW: http://review.gluster.org/12250 (glusterd : Introduce reset brick) posted (#5) for review on master by Anuradha Talur (atalur)

Comment 4 Anuradha 2016-06-22 08:56:04 UTC
*** Bug 1316577 has been marked as a duplicate of this bug. ***

Comment 5 Vijay Bellur 2016-06-24 09:30:47 UTC
REVIEW: http://review.gluster.org/12250 (glusterd : Introduce reset brick) posted (#6) for review on master by Anuradha Talur (atalur)

Comment 6 Vijay Bellur 2016-06-24 10:43:34 UTC
REVIEW: http://review.gluster.org/12250 (glusterd : Introduce reset brick) posted (#7) for review on master by Anuradha Talur (atalur)

Comment 7 Vijay Bellur 2016-08-16 08:22:17 UTC
REVIEW: http://review.gluster.org/12250 (glusterd : Introduce reset brick) posted (#8) for review on master by Anuradha Talur (atalur)

Comment 8 Worker Ant 2016-08-23 09:34:59 UTC
REVIEW: http://review.gluster.org/12250 (glusterd : Introduce reset brick) posted (#9) for review on master by Anuradha Talur (atalur)

Comment 9 Worker Ant 2016-08-24 11:13:33 UTC
REVIEW: http://review.gluster.org/12250 (glusterd : Introduce reset brick) posted (#10) for review on master by Anuradha Talur (atalur)

Comment 10 Worker Ant 2016-08-24 12:26:51 UTC
REVIEW: http://review.gluster.org/12250 (glusterd : Introduce reset brick) posted (#11) for review on master by Anuradha Talur (atalur)

Comment 11 Worker Ant 2016-08-25 06:35:57 UTC
REVIEW: http://review.gluster.org/12250 (glusterd : Introduce reset brick) posted (#12) for review on master by Anuradha Talur (atalur)

Comment 12 Worker Ant 2016-08-26 10:17:43 UTC
REVIEW: http://review.gluster.org/12250 (    glusterd : Introduce reset brick) posted (#13) for review on master by Anuradha Talur (atalur)

Comment 13 Worker Ant 2016-08-29 05:10:29 UTC
REVIEW: http://review.gluster.org/12250 (glusterd : Introduce reset brick) posted (#14) for review on master by Anuradha Talur (atalur)

Comment 14 Worker Ant 2016-08-29 17:08:42 UTC
REVIEW: http://review.gluster.org/12250 (glusterd : Introduce reset brick) posted (#15) for review on master by Anuradha Talur (atalur)

Comment 15 Worker Ant 2016-08-30 02:55:58 UTC
COMMIT: http://review.gluster.org/12250 committed in master by Pranith Kumar Karampuri (pkarampu) 
------
commit 936f8aeac3252951e7fa0cdaa5d260fad3bd5ea0
Author: Anuradha Talur <atalur>
Date:   Mon Aug 22 13:22:03 2016 -0400

    glusterd : Introduce reset brick
    
    The command basically allows replace brick with src and
    dst bricks as same.
    
    Usage:
    gluster v reset-brick <volname> <hostname:brick-path> start
    This command kills the brick to be reset. Once this command is run,
    admin can do other manual operations that they need to do,
    like configuring some options for the brick. Once this is done,
    resetting the brick can be continued with the following options.
    
    gluster v reset-brick <vname> <hostname:brick> <hostname:brick> commit {force}
    
    Does the job of resetting the brick. 'force' option should be used
    when the brick already contains volinfo id.
    
    Problem: On doing a disk-replacement of a brick in a replicate volume
    the following 2 scenarios may occur :
    
    a) there is a chance that reads are served from this replaced-disk brick,
    which leads to empty reads. b) potential data loss if next writes succeed
    only on replaced brick, and heal is done to other bricks from this one.
    
    Solution: After disk-replacement, make sure that reset-brick command is
    run for that brick so that pending markers are set for the brick and it
    is not chosen as source for reads and heal. But, as of now replace-brick
    for the same brick-path is not allowed. In order to fix the above
    mentioned problem, same brick-path replace-brick is needed.
    With this patch reset-brick commit {force} will be allowed even when
    source and destination <hostname:brickpath> are identical as long as
    1) destination brick is not alive
    2) source and destination brick have the same brick uuid and path.
    Also, the destination brick after replace-brick will use the same port
    as the source brick.
    
    Change-Id: I440b9e892ffb781ea4b8563688c3f85c7a7c89de
    BUG: 1266876
    Signed-off-by: Anuradha Talur <atalur>
    Reviewed-on: http://review.gluster.org/12250
    Smoke: Gluster Build System <jenkins.org>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>
    Reviewed-by: Ashish Pandey <aspandey>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu>

Comment 16 Worker Ant 2016-08-31 10:06:42 UTC
REVIEW: http://review.gluster.org/15370 (event: fix gf_event messages for replace/reset brick op) posted (#1) for review on master by Anuradha Talur (atalur)

Comment 17 Worker Ant 2016-08-31 10:08:23 UTC
REVIEW: http://review.gluster.org/15370 (event: fix gf_event messages for replace/reset brick op) posted (#2) for review on master by Anuradha Talur (atalur)

Comment 18 Worker Ant 2016-08-31 11:14:20 UTC
REVIEW: http://review.gluster.org/15370 (event: fix gf_event messages for replace/reset brick op) posted (#3) for review on master by Anuradha Talur (atalur)

Comment 19 Worker Ant 2016-08-31 11:15:34 UTC
REVIEW: http://review.gluster.org/15370 (event: fix gf_event messages for replace/reset brick op) posted (#4) for review on master by Anuradha Talur (atalur)

Comment 20 Worker Ant 2016-08-31 22:56:35 UTC
COMMIT: http://review.gluster.org/15370 committed in master by Pranith Kumar Karampuri (pkarampu) 
------
commit 054b09eb5f040ccc0adbb86759da403b49544083
Author: Anuradha Talur <atalur>
Date:   Wed Aug 31 15:27:31 2016 +0530

    event: fix gf_event messages for replace/reset brick op
    
    Change-Id: I80ebeeaffd2b228d7d0796c8d08bc2a051c4ccac
    BUG: 1266876
    Signed-off-by: Anuradha Talur <atalur>
    Reviewed-on: http://review.gluster.org/15370
    Smoke: Gluster Build System <jenkins.org>
    Reviewed-by: Atin Mukherjee <amukherj>
    CentOS-regression: Gluster Build System <jenkins.org>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu>

Comment 23 Shyamsundar 2017-03-27 18:27:08 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.9.0, please open a new bug report.

glusterfs-3.9.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://lists.gluster.org/pipermail/gluster-users/2016-November/029281.html
[2] https://www.gluster.org/pipermail/gluster-users/