+++ This bug was initially created as a clone of Bug #1570541 +++ Description of problem: Ganesha enable command errors out while setting up ganesha on 4 node out of 5 node gluster cluster # gluster nfs-ganesha enable Enabling NFS-Ganesha requires Gluster-NFS to be disabled across the trusted pool. Do you still want to continue? (y/n) y This will take a few minutes to complete. Please wait .. nfs-ganesha: failed: Commit failed on dhcp37-121.lab.eng.blr.redhat.com. Please check log file for details. glusterd.log ---------- [2018-04-22 19:01:38.994769] I [MSGID: 106474] [glusterd-ganesha.c:433:check_host_list] 0-management: ganesha host found Hostname is dhcp46-116.lab.eng.blr.redhat.com [2018-04-22 19:03:34.675747] E [MSGID: 106153] [glusterd-syncop.c:113:gd_collate_errors] 0-glusterd: Commit failed on dhcp37-121.lab.eng.blr.redhat.com. Please check log file for details. -------- Version-Release number of selected component (if applicable): # rpm -qa | grep ganesha nfs-ganesha-gluster-2.5.5-5.el7rhgs.x86_64 glusterfs-ganesha-3.12.2-8.el7rhgs.x86_64 nfs-ganesha-debuginfo-2.5.5-4.el7rhgs.x86_64 nfs-ganesha-2.5.5-5.el7rhgs.x86_64 How reproducible: 2/2 Steps to Reproduce: 1.Create 4 node ganesha cluster out of 5 node gluster cluster Actual results: "gluster nfs-ganesha" command errors out.But if check on backend,pcs status output shows cluster is up and running on 4 nodes Expected results: "gluster nfs-ganesha" should not error out Additional info: # pcs status Cluster name: ganesha-ha-360 Stack: corosync Current DC: dhcp47-193.lab.eng.blr.redhat.com (version 1.1.18-11.el7-2b07d5c5a9) - partition with quorum Last updated: Mon Apr 23 00:05:48 2018 Last change: Sun Apr 22 23:53:58 2018 by root via cibadmin on dhcp47-193.lab.eng.blr.redhat.com 4 nodes configured 24 resources configured Online: [ dhcp46-116.lab.eng.blr.redhat.com dhcp46-184.lab.eng.blr.redhat.com dhcp47-193.lab.eng.blr.redhat.com dhcp47-2.lab.eng.blr.redhat.com ] Full list of resources: Clone Set: nfs_setup-clone [nfs_setup] Started: [ dhcp46-116.lab.eng.blr.redhat.com dhcp46-184.lab.eng.blr.redhat.com dhcp47-193.lab.eng.blr.redhat.com dhcp47-2.lab.eng.blr.redhat.com ] Clone Set: nfs-mon-clone [nfs-mon] Started: [ dhcp46-116.lab.eng.blr.redhat.com dhcp46-184.lab.eng.blr.redhat.com dhcp47-193.lab.eng.blr.redhat.com dhcp47-2.lab.eng.blr.redhat.com ] Clone Set: nfs-grace-clone [nfs-grace] Started: [ dhcp46-116.lab.eng.blr.redhat.com dhcp46-184.lab.eng.blr.redhat.com dhcp47-193.lab.eng.blr.redhat.com dhcp47-2.lab.eng.blr.redhat.com ] Resource Group: dhcp47-193.lab.eng.blr.redhat.com-group dhcp47-193.lab.eng.blr.redhat.com-nfs_block (ocf::heartbeat:portblock): Started dhcp47-193.lab.eng.blr.redhat.com dhcp47-193.lab.eng.blr.redhat.com-cluster_ip-1 (ocf::heartbeat:IPaddr): Started dhcp47-193.lab.eng.blr.redhat.com dhcp47-193.lab.eng.blr.redhat.com-nfs_unblock (ocf::heartbeat:portblock): Started dhcp47-193.lab.eng.blr.redhat.com Resource Group: dhcp46-116.lab.eng.blr.redhat.com-group dhcp46-116.lab.eng.blr.redhat.com-nfs_block (ocf::heartbeat:portblock): Started dhcp46-116.lab.eng.blr.redhat.com dhcp46-116.lab.eng.blr.redhat.com-cluster_ip-1 (ocf::heartbeat:IPaddr): Started dhcp46-116.lab.eng.blr.redhat.com dhcp46-116.lab.eng.blr.redhat.com-nfs_unblock (ocf::heartbeat:portblock): Started dhcp46-116.lab.eng.blr.redhat.com Resource Group: dhcp46-184.lab.eng.blr.redhat.com-group dhcp46-184.lab.eng.blr.redhat.com-nfs_block (ocf::heartbeat:portblock): Started dhcp46-184.lab.eng.blr.redhat.com dhcp46-184.lab.eng.blr.redhat.com-cluster_ip-1 (ocf::heartbeat:IPaddr): Started dhcp46-184.lab.eng.blr.redhat.com dhcp46-184.lab.eng.blr.redhat.com-nfs_unblock (ocf::heartbeat:portblock): Started dhcp46-184.lab.eng.blr.redhat.com Resource Group: dhcp47-2.lab.eng.blr.redhat.com-group dhcp47-2.lab.eng.blr.redhat.com-nfs_block (ocf::heartbeat:portblock): Started dhcp47-2.lab.eng.blr.redhat.com dhcp47-2.lab.eng.blr.redhat.com-cluster_ip-1 (ocf::heartbeat:IPaddr): Started dhcp47-2.lab.eng.blr.redhat.com dhcp47-2.lab.eng.blr.redhat.com-nfs_unblock (ocf::heartbeat:portblock): Started dhcp47-2.lab.eng.blr.redhat.com Daemon Status: corosync: active/disabled pacemaker: active/disabled pcsd: active/enabled ------------------------- Node on which ganesha was not set up--- [2018-04-22 19:03:33.723111] E [MSGID: 106470] [glusterd-ganesha.c:357:glusterd_op_set_ganesha] 0-management: Initial NFS-Ganesha set up failed [2018-04-22 19:03:33.723136] E [MSGID: 106123] [glusterd-op-sm.c:5959:glusterd_op_ac_commit_op] 0-management: Commit of operation 'Volume (null)' failed: -1 [2018-04-22 19:03:33.821219] E [MSGID: 106062] [glusterd-op-sm.c:4241:glusterd_op_ac_unlock] 0-management: Unable to acquire volname # cd /etc/ganesha/ [root@dhcp37-121 ganesha]# ll total 12 -rw-r--r--. 1 root root 1576 Apr 22 07:48 ganesha.conf -rw-r--r--. 1 root root 2345 Apr 19 10:32 ganesha.conf.example -rw-r--r--. 1 root root 776 Apr 4 05:32 ganesha-ha.conf.sample [root@dhcp37-121 ganesha]# cat /var/lib/glusterd/options global-option-version=90 cluster.enable-shared-storage=enable nfs-ganesha=disable # pcs status Error: cluster is not currently running on this node [root@dhcp37-121 ganesha]# gluster peer status Number of Peers: 4 Hostname: dhcp46-116.lab.eng.blr.redhat.com Uuid: 856cb4ed-c1d1-4da0-b753-b7adaaa46a4f State: Peer in Cluster (Connected) Hostname: dhcp47-193.lab.eng.blr.redhat.com Uuid: 97de04a8-5040-45e0-be11-0b891473d022 State: Peer in Cluster (Connected) Hostname: dhcp46-184.lab.eng.blr.redhat.com Uuid: f8c8ba49-c161-4502-a524-54e3b6f995e3 State: Peer in Cluster (Connected) Hostname: dhcp47-2.lab.eng.blr.redhat.com Uuid: afd776a4-3ad2-4aab-beec-f4697994be74 State: Peer in Cluster (Connected) --- Additional comment from Jiffin on 2018-04-24 00:22:04 EDT --- RCA : regression caused by https://review.gluster.org/16733. The pre_setup() function returns -1 if the node is not mentioned in the ganesha-ha.conf
REVIEW: https://review.gluster.org/19949 (glusterd/ganesha : Skip non-ganesha nodes properly for ganesha HA set up) posted (#1) for review on release-3.10 by jiffin tony Thottan
COMMIT: https://review.gluster.org/19949 committed in release-3.10 by "jiffin tony Thottan" <jthottan> with a commit message- glusterd/ganesha : Skip non-ganesha nodes properly for ganesha HA set up Change-Id: Iff7bc3ead43e97847219c5a5cc8b967bf0967903 BUG: 1573078 Signed-off-by: Jiffin Tony Thottan <jthottan>
This bug reported is against a version of Gluster that is no longer maintained (or has been EOL'd). See https://www.gluster.org/release-schedule/ for the versions currently maintained. As a result this bug is being closed. If the bug persists on a maintained version of gluster or against the mainline gluster repository, request that it be reopened and the Version field be marked appropriately.