Bug 1400816

Summary: [GANESHA] Symlinks from /etc/ganesha/ganesha.conf to shared_storage are created on the non-ganesha nodes in 8 node gluster having 4 node ganesha cluster
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Manisha Saini <msaini>
Component: glusterdAssignee: Jiffin <jthottan>
Status: CLOSED ERRATA QA Contact: Manisha Saini <msaini>
Severity: unspecified Docs Contact:
Priority: low    
Version: rhgs-3.2CC: aloganat, amukherj, asrivast, bmohanra, jthottan, kkeithle, ndevos, olim, rcyriac, rhinduja, rhs-bugs, skoduri, storage-qa-internal, vbellur
Target Milestone: ---Keywords: ZStream
Target Release: RHGS 3.3.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.8.4-25 Doc Type: Bug Fix
Doc Text:
glusterd tries to create symlink "ganesh.conf" on every node of trusted storage pool. Symlink creation fails if the nfs-ganesha package is missing. Workaround: Install nfs-ganesha package on all the nodes
Story Points: ---
Clone Of:
: 1401877 (view as bug list) Environment:
Last Closed: 2017-09-21 04:28:23 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1401877, 1417147    

Description Manisha Saini 2016-12-02 07:23:07 UTC
Description of problem:
In case of 8 Node gluster having 4 node Ganesha cluster on it , symlinks are created on non-ganesha nodes from /etc/ganesha/ganesha.conf to /var/run/gluster/shared_storage/

Version-Release number of selected component (if applicable):

glusterfs-ganesha-3.8.4-5.el7rhgs.x86_64
nfs-ganesha-gluster-2.4.1-1.el7rhgs.x86_64

How reproducible:


Steps to Reproduce:
1.Create 8 node gluster (Do peer probe to all 8 nodes having gluster packages)
2.Create 4 node cluster and enable ganesha on it

#gluster nfs-ganesha enable

Actual results:
Symlinks from /etc/ganesha/ganesha.conf -> /var/run/gluster/shared_storage/nfs-ganesha/ganesha.conf  are created on non-ganesha nodes which are part of 8 node gluster

Expected results:

Nodes which do not have ganesha enable,On those nodes symlinks should not be created from /etc/ganesha/ganesha.conf -> /var/run/gluster/shared_storage/nfs-ganesha/ganesha.conf

Additional info:

Comment 2 Manisha Saini 2016-12-05 13:54:48 UTC

While trying creating ganesha cluster on 4 nodes out of 8 nodes in gluster,then if any of the node does not have ganesha packages,then it fails to enable nfs-ganesha cluster 


[root@dhcp47-3 ganesha]# gluster nfs-ganesha enable
Enabling NFS-Ganesha requires Gluster-NFS to be disabled across the trusted pool. Do you still want to continue?
 (y/n) y
This will take a few minutes to complete. Please wait ..
nfs-ganesha: failed: Staging failed on dhcp46-232.lab.eng.blr.redhat.com. Error: creation of symlink ganesha.conf in /etc/ganesha failed


dhcp46-232.lab.eng.blr.redhat.com -Node which do not have ganesha packages

Comment 5 Arthy Loganathan 2017-01-04 11:24:19 UTC
Hitting similar issue, while disabling nfs-ganesha.

Steps:

1. Create a storage pool of 4 nodes.
2. Create 4 node nfs-ganesha cluster in the storage pool.
3. Add two new nodes to the storage pool (nfs-ganesha package is not installed on the 2 new nodes)
4. Try to disable 4 node nfs-ganesha cluster.

If nfs-ganesha package is not installed on the 2 new nodes, then the user can not disable the existing 4 node nfs-ganesha cluster.

[root@dhcp37-145 ~]# gluster nfs-ganesha disable
Disabling NFS-Ganesha will tear down entire ganesha cluster across the trusted pool. Do you still want to continue?
 (y/n) y
This will take a few minutes to complete. Please wait ..
nfs-ganesha: failed: Staging failed on dhcp37-136.lab.eng.blr.redhat.com. Error: removal of symlink ganesha.conf in /etc/ganesha failed
Staging failed on dhcp37-115.lab.eng.blr.redhat.com. Error: removal of symlink ganesha.conf in /etc/ganesha failed


This has to be documented as a requirement that the nfs-ganesha packages need to be installed on all the nodes of the Gluster trusted storage pool even though only few of them shall be part of nfs-ganesha cluster.

Comment 6 Jiffin 2017-02-15 07:20:57 UTC
Patch got merged upstream : https://review.gluster.org/#/c/16041/

Comment 9 Atin Mukherjee 2017-03-24 09:15:08 UTC
downstream patch : https://code.engineering.redhat.com/gerrit/#/c/101295/

Comment 11 Manisha Saini 2017-04-28 09:39:49 UTC
Moving this bug to Assigned state.

There were 2 issues reported in this bug.

1.Enabling ganesha on 4 node out of 8 node gluster cluster should not create symlinks to shared-storage on non-ganesha nodes

2.Disabling the 4 node ganesha cluster out of 8 node gluster cluster.


Case 1 is fixed and working as expected.
Case 2 is still failing for nodes which do not have ganesha packages.


While disabling 4 node ganesha cluster on 8 node gluster cluster 

# gluster nfs-ganesha disable
Disabling NFS-Ganesha will tear down entire ganesha cluster across the trusted pool. Do you still want to continue?
 (y/n) y
This will take a few minutes to complete. Please wait ..
nfs-ganesha: failed: Staging failed on dhcp46-56.lab.eng.blr.redhat.com. Error: removal of symlink ganesha.conf in /etc/ganesha failed
Staging failed on dhcp46-109.lab.eng.blr.redhat.com. Error: removal of symlink ganesha.conf in /etc/ganesha failed



Out of the other 4 node on which ganesha cluster is not there, 2 node does not have ganesha packages.So while disabling ganesha,its throwing staging failed messages for those node not having ganesha packages.

Comment 12 Jiffin 2017-04-28 12:07:29 UTC
(In reply to Manisha Saini from comment #11)
> Moving this bug to Assigned state.
> 
> There were 2 issues reported in this bug.
> 
> 1.Enabling ganesha on 4 node out of 8 node gluster cluster should not create
> symlinks to shared-storage on non-ganesha nodes
> 
> 2.Disabling the 4 node ganesha cluster out of 8 node gluster cluster.
> 
> 
> Case 1 is fixed and working as expected.
> Case 2 is still failing for nodes which do not have ganesha packages.
> 
> 
> While disabling 4 node ganesha cluster on 8 node gluster cluster 
> 
> # gluster nfs-ganesha disable
> Disabling NFS-Ganesha will tear down entire ganesha cluster across the
> trusted pool. Do you still want to continue?
>  (y/n) y
> This will take a few minutes to complete. Please wait ..
> nfs-ganesha: failed: Staging failed on dhcp46-56.lab.eng.blr.redhat.com.
> Error: removal of symlink ganesha.conf in /etc/ganesha failed
> Staging failed on dhcp46-109.lab.eng.blr.redhat.com. Error: removal of
> symlink ganesha.conf in /etc/ganesha failed
> 
> 
> 
> Out of the other 4 node on which ganesha cluster is not there, 2 node does
> not have ganesha packages.So while disabling ganesha,its throwing staging
> failed messages for those node not having ganesha packages.

Patch posted upstream to fix this issue https://review.gluster.org/#/c/17138/

Comment 13 Jiffin 2017-05-10 07:01:58 UTC
Downstream patch link https://code.engineering.redhat.com/gerrit/#/c/105741/

Comment 15 Manisha Saini 2017-06-02 05:39:18 UTC
Verified this bug on

# rpm -qa | grep ganesha
glusterfs-ganesha-3.8.4-26.el7rhgs.x86_64
nfs-ganesha-2.4.4-6.el7rhgs.x86_64
nfs-ganesha-gluster-2.4.4-6.el7rhgs.x86_64

Enabling and disabling 4 node ganesha cluster on 8 node gluster cluster is successfull even when the ganesha packages are not present on non-ganesha nodes.

Moving this bug to verified state.

Comment 17 errata-xmlrpc 2017-09-21 04:28:23 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2774

Comment 18 errata-xmlrpc 2017-09-21 04:54:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2774