Bug 1405577

Summary: [GANESHA] failed to create directory of hostname of new node in var/lib/nfs/ganesha/ in already existing cluster nodes
Product: [Community] GlusterFS Reporter: Kaleb KEITHLEY <kkeithle>
Component: common-haAssignee: Kaleb KEITHLEY <kkeithle>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: unspecified Docs Contact:
Priority: medium    
Version: 3.8CC: bugs, msaini, rhs-bugs, storage-qa-internal
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.8.8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1405576 Environment:
Last Closed: 2017-01-16 12:27:19 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1400599, 1400613, 1405576    
Bug Blocks:    

Comment 1 Worker Ant 2016-12-16 18:26:51 UTC
REVIEW: http://review.gluster.org/16171 (common-ha: add node create new node dirs in shared storage) posted (#1) for review on release-3.8 by Kaleb KEITHLEY (kkeithle)

Comment 2 Worker Ant 2016-12-18 12:44:16 UTC
COMMIT: http://review.gluster.org/16171 committed in release-3.8 by Kaleb KEITHLEY (kkeithle) 
------
commit bcff9a195c3d7ab8ce87a4add448643aa5ee1a4e
Author: Kaleb S. KEITHLEY <kkeithle>
Date:   Fri Dec 16 13:24:57 2016 -0500

    common-ha: add node create new node dirs in shared storage
    
    When adding a node to the ganesha HA cluster, create the directory
    tree in shared storage for the added node and create sets of symlinks
    to match what is/was created for the other nodes.  I.e. in a four
    node cluster the new node needs a set of links to the four existing
    nodes:
     /run/gluster/shared/nfs-ganesha/$new/nfs/{ganesha,statd}/$e1 -> e1
     /run/gluster/shared/nfs-ganesha/$new/nfs/{ganesha,statd}/$e2 -> e2
     /run/gluster/shared/nfs-ganesha/$new/nfs/{ganesha,statd}/$e3 -> e3
     /run/gluster/shared/nfs-ganesha/$new/nfs/{ganesha,statd}/$e4 -> e4
    and all the existing nodes need links added for the new node:
     /run/gluster/shared/nfs-ganesha/$e1/nfs/{ganesha,statd}/$new -> new
     /run/gluster/shared/nfs-ganesha/$e2/nfs/{ganesha,statd}/$new -> new
     /run/gluster/shared/nfs-ganesha/$e3/nfs/{ganesha,statd}/$new -> new
     /run/gluster/shared/nfs-ganesha/$e5/nfs/{ganesha,statd}/$new -> new
    
    Likewise when deleting, remove the dir and symlinks.
    
    master BZ: 1400613
    master http://review.gluster.org/#/c/16036/
    release-3.9 BZ: 1405576
    release-3.9 http://review.gluster.org/16170
    
    Change-Id: Id2f78f70946f29c3503e1e6db141b66cb431e0ea
    BUG: 1405577
    Signed-off-by: Kaleb S. KEITHLEY <kkeithle>
    Reviewed-on: http://review.gluster.org/16171
    Smoke: Gluster Build System <jenkins.org>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>

Comment 3 Worker Ant 2016-12-20 16:09:29 UTC
REVIEW: http://review.gluster.org/16218 (common-ha: add node create new node dirs in shared storage) posted (#1) for review on release-3.8 by Kaleb KEITHLEY (kkeithle)

Comment 4 Worker Ant 2016-12-21 23:22:22 UTC
REVIEW: http://review.gluster.org/16262 (common-ha: add node create new node dirs in shared storage) posted (#1) for review on release-3.8-fb by Kevin Vigor (kvigor)

Comment 5 Worker Ant 2016-12-22 14:20:33 UTC
COMMIT: http://review.gluster.org/16218 committed in release-3.8 by Kaleb KEITHLEY (kkeithle) 
------
commit 12190c0338dea6bbfafb7bf7693ce4ee547b5954
Author: Kaleb S. KEITHLEY <kkeithle>
Date:   Tue Dec 20 11:06:34 2016 -0500

    common-ha: add node create new node dirs in shared storage
    
    When adding a node to the ganesha HA cluster, create the directory
    tree in shared storage for the added node and create sets of symlinks
    to match what is/was created for the other nodes.  I.e. in a four
    node cluster the new node needs a set of links to the four existing
    nodes:
     /run/gluster/shared/nfs-ganesha/$new/nfs/{ganesha,statd}/$e1 -> e1
     /run/gluster/shared/nfs-ganesha/$new/nfs/{ganesha,statd}/$e2 -> e2
     /run/gluster/shared/nfs-ganesha/$new/nfs/{ganesha,statd}/$e3 -> e3
     /run/gluster/shared/nfs-ganesha/$new/nfs/{ganesha,statd}/$e4 -> e4
    and all the existing nodes need links added for the new node:
     /run/gluster/shared/nfs-ganesha/$e1/nfs/{ganesha,statd}/$new -> new
     /run/gluster/shared/nfs-ganesha/$e2/nfs/{ganesha,statd}/$new -> new
     /run/gluster/shared/nfs-ganesha/$e3/nfs/{ganesha,statd}/$new -> new
     /run/gluster/shared/nfs-ganesha/$e5/nfs/{ganesha,statd}/$new -> new
    
    Likewise when deleting, remove the dir and symlinks.
    
    original change http://review.gluster.org/16036
    original change release-3.9 http://review.gluster.org/16170
    original change release-3.8 http://review.gluster.org/16171
    master change http://review.gluster.org/16216
    master BZ 1400613
    release-3.9 change http://review.gluster.org/16217
    release-3.9 BZ 1405576
    
    Change-Id: I52839046745728d06ab5a07f38081c032093bff6
    BUG: 1405577
    Signed-off-by: Kaleb S. KEITHLEY <kkeithle>
    Reviewed-on: http://review.gluster.org/16218
    Smoke: Gluster Build System <jenkins.org>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>

Comment 6 Niels de Vos 2017-01-16 12:27:19 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.8, please open a new bug report.

glusterfs-3.8.8 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] https://lists.gluster.org/pipermail/announce/2017-January/000064.html
[2] https://www.gluster.org/pipermail/gluster-users/