+++ This bug was initially created as a clone of Bug #1570541 +++
Description of problem:
Ganesha enable command errors out while setting up ganesha on 4 node out of 5 node gluster cluster
# gluster nfs-ganesha enable
Enabling NFS-Ganesha requires Gluster-NFS to be disabled across the trusted pool. Do you still want to continue?
(y/n) y
This will take a few minutes to complete. Please wait ..
nfs-ganesha: failed: Commit failed on dhcp37-121.lab.eng.blr.redhat.com. Please check log file for details.
glusterd.log
----------
[2018-04-22 19:01:38.994769] I [MSGID: 106474] [glusterd-ganesha.c:433:check_host_list] 0-management: ganesha host found Hostname is dhcp46-116.lab.eng.blr.redhat.com
[2018-04-22 19:03:34.675747] E [MSGID: 106153] [glusterd-syncop.c:113:gd_collate_errors] 0-glusterd: Commit failed on dhcp37-121.lab.eng.blr.redhat.com. Please check log file for details.
--------
Version-Release number of selected component (if applicable):
# rpm -qa | grep ganesha
nfs-ganesha-gluster-2.5.5-5.el7rhgs.x86_64
glusterfs-ganesha-3.12.2-8.el7rhgs.x86_64
nfs-ganesha-debuginfo-2.5.5-4.el7rhgs.x86_64
nfs-ganesha-2.5.5-5.el7rhgs.x86_64
How reproducible:
2/2
Steps to Reproduce:
1.Create 4 node ganesha cluster out of 5 node gluster cluster
Actual results:
"gluster nfs-ganesha" command errors out.But if check on backend,pcs status output shows cluster is up and running on 4 nodes
Expected results:
"gluster nfs-ganesha" should not error out
Additional info:
# pcs status
Cluster name: ganesha-ha-360
Stack: corosync
Current DC: dhcp47-193.lab.eng.blr.redhat.com (version 1.1.18-11.el7-2b07d5c5a9) - partition with quorum
Last updated: Mon Apr 23 00:05:48 2018
Last change: Sun Apr 22 23:53:58 2018 by root via cibadmin on dhcp47-193.lab.eng.blr.redhat.com
4 nodes configured
24 resources configured
Online: [ dhcp46-116.lab.eng.blr.redhat.com dhcp46-184.lab.eng.blr.redhat.com dhcp47-193.lab.eng.blr.redhat.com dhcp47-2.lab.eng.blr.redhat.com ]
Full list of resources:
Clone Set: nfs_setup-clone [nfs_setup]
Started: [ dhcp46-116.lab.eng.blr.redhat.com dhcp46-184.lab.eng.blr.redhat.com dhcp47-193.lab.eng.blr.redhat.com dhcp47-2.lab.eng.blr.redhat.com ]
Clone Set: nfs-mon-clone [nfs-mon]
Started: [ dhcp46-116.lab.eng.blr.redhat.com dhcp46-184.lab.eng.blr.redhat.com dhcp47-193.lab.eng.blr.redhat.com dhcp47-2.lab.eng.blr.redhat.com ]
Clone Set: nfs-grace-clone [nfs-grace]
Started: [ dhcp46-116.lab.eng.blr.redhat.com dhcp46-184.lab.eng.blr.redhat.com dhcp47-193.lab.eng.blr.redhat.com dhcp47-2.lab.eng.blr.redhat.com ]
Resource Group: dhcp47-193.lab.eng.blr.redhat.com-group
dhcp47-193.lab.eng.blr.redhat.com-nfs_block (ocf::heartbeat:portblock): Started dhcp47-193.lab.eng.blr.redhat.com
dhcp47-193.lab.eng.blr.redhat.com-cluster_ip-1 (ocf::heartbeat:IPaddr): Started dhcp47-193.lab.eng.blr.redhat.com
dhcp47-193.lab.eng.blr.redhat.com-nfs_unblock (ocf::heartbeat:portblock): Started dhcp47-193.lab.eng.blr.redhat.com
Resource Group: dhcp46-116.lab.eng.blr.redhat.com-group
dhcp46-116.lab.eng.blr.redhat.com-nfs_block (ocf::heartbeat:portblock): Started dhcp46-116.lab.eng.blr.redhat.com
dhcp46-116.lab.eng.blr.redhat.com-cluster_ip-1 (ocf::heartbeat:IPaddr): Started dhcp46-116.lab.eng.blr.redhat.com
dhcp46-116.lab.eng.blr.redhat.com-nfs_unblock (ocf::heartbeat:portblock): Started dhcp46-116.lab.eng.blr.redhat.com
Resource Group: dhcp46-184.lab.eng.blr.redhat.com-group
dhcp46-184.lab.eng.blr.redhat.com-nfs_block (ocf::heartbeat:portblock): Started dhcp46-184.lab.eng.blr.redhat.com
dhcp46-184.lab.eng.blr.redhat.com-cluster_ip-1 (ocf::heartbeat:IPaddr): Started dhcp46-184.lab.eng.blr.redhat.com
dhcp46-184.lab.eng.blr.redhat.com-nfs_unblock (ocf::heartbeat:portblock): Started dhcp46-184.lab.eng.blr.redhat.com
Resource Group: dhcp47-2.lab.eng.blr.redhat.com-group
dhcp47-2.lab.eng.blr.redhat.com-nfs_block (ocf::heartbeat:portblock): Started dhcp47-2.lab.eng.blr.redhat.com
dhcp47-2.lab.eng.blr.redhat.com-cluster_ip-1 (ocf::heartbeat:IPaddr): Started dhcp47-2.lab.eng.blr.redhat.com
dhcp47-2.lab.eng.blr.redhat.com-nfs_unblock (ocf::heartbeat:portblock): Started dhcp47-2.lab.eng.blr.redhat.com
Daemon Status:
corosync: active/disabled
pacemaker: active/disabled
pcsd: active/enabled
-------------------------
Node on which ganesha was not set up---
[2018-04-22 19:03:33.723111] E [MSGID: 106470] [glusterd-ganesha.c:357:glusterd_op_set_ganesha] 0-management: Initial NFS-Ganesha set up failed
[2018-04-22 19:03:33.723136] E [MSGID: 106123] [glusterd-op-sm.c:5959:glusterd_op_ac_commit_op] 0-management: Commit of operation 'Volume (null)' failed: -1
[2018-04-22 19:03:33.821219] E [MSGID: 106062] [glusterd-op-sm.c:4241:glusterd_op_ac_unlock] 0-management: Unable to acquire volname
# cd /etc/ganesha/
[root@dhcp37-121 ganesha]# ll
total 12
-rw-r--r--. 1 root root 1576 Apr 22 07:48 ganesha.conf
-rw-r--r--. 1 root root 2345 Apr 19 10:32 ganesha.conf.example
-rw-r--r--. 1 root root 776 Apr 4 05:32 ganesha-ha.conf.sample
[root@dhcp37-121 ganesha]# cat /var/lib/glusterd/options
global-option-version=90
cluster.enable-shared-storage=enable
nfs-ganesha=disable
# pcs status
Error: cluster is not currently running on this node
[root@dhcp37-121 ganesha]# gluster peer status
Number of Peers: 4
Hostname: dhcp46-116.lab.eng.blr.redhat.com
Uuid: 856cb4ed-c1d1-4da0-b753-b7adaaa46a4f
State: Peer in Cluster (Connected)
Hostname: dhcp47-193.lab.eng.blr.redhat.com
Uuid: 97de04a8-5040-45e0-be11-0b891473d022
State: Peer in Cluster (Connected)
Hostname: dhcp46-184.lab.eng.blr.redhat.com
Uuid: f8c8ba49-c161-4502-a524-54e3b6f995e3
State: Peer in Cluster (Connected)
Hostname: dhcp47-2.lab.eng.blr.redhat.com
Uuid: afd776a4-3ad2-4aab-beec-f4697994be74
State: Peer in Cluster (Connected)
--- Additional comment from Jiffin on 2018-04-24 00:22:04 EDT ---
RCA : regression caused by https://review.gluster.org/16733.
The pre_setup() function returns -1 if the node is not mentioned in the ganesha-ha.conf
REVIEW: https://review.gluster.org/19949 (glusterd/ganesha : Skip non-ganesha nodes properly for ganesha HA set up) posted (#1) for review on release-3.10 by jiffin tony Thottan
COMMIT: https://review.gluster.org/19949 committed in release-3.10 by "jiffin tony Thottan" <jthottan> with a commit message- glusterd/ganesha : Skip non-ganesha nodes properly for ganesha HA set up
Change-Id: Iff7bc3ead43e97847219c5a5cc8b967bf0967903
BUG: 1573078
Signed-off-by: Jiffin Tony Thottan <jthottan>
This bug reported is against a version of Gluster that is no longer maintained (or has been EOL'd). See https://www.gluster.org/release-schedule/ for the versions currently maintained.
As a result this bug is being closed.
If the bug persists on a maintained version of gluster or against the mainline gluster repository, request that it be reopened and the Version field be marked appropriately.