Bug 1570541 - [Ganesha] Ganesha enable command errors out while setting up ganesha on 4 node out of 5 node gluster cluster
Summary: [Ganesha] Ganesha enable command errors out while setting up ganesha on 4 nod...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: nfs-ganesha
Version: rhgs-3.4
Hardware: Unspecified
OS: Unspecified
high
unspecified
Target Milestone: ---
: RHGS 3.4.0
Assignee: Jiffin
QA Contact: Manisha Saini
URL:
Whiteboard:
Depends On: 1573078
Blocks: 1503137
TreeView+ depends on / blocked
 
Reported: 2018-04-23 07:35 UTC by Manisha Saini
Modified: 2018-09-24 07:00 UTC (History)
9 users (show)

Fixed In Version: glusterfs-3.12.2-9
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1573078 (view as bug list)
Environment:
Last Closed: 2018-09-04 06:46:03 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2018:2607 None None None 2018-09-04 06:47:38 UTC

Description Manisha Saini 2018-04-23 07:35:08 UTC
Description of problem:

Ganesha enable command errors out while setting up ganesha on 4 node out of 5 node gluster cluster

# gluster nfs-ganesha enable
Enabling NFS-Ganesha requires Gluster-NFS to be disabled across the trusted pool. Do you still want to continue?
 (y/n) y
This will take a few minutes to complete. Please wait ..
nfs-ganesha: failed: Commit failed on dhcp37-121.lab.eng.blr.redhat.com. Please check log file for details.


glusterd.log
----------
[2018-04-22 19:01:38.994769] I [MSGID: 106474] [glusterd-ganesha.c:433:check_host_list] 0-management: ganesha host found Hostname is dhcp46-116.lab.eng.blr.redhat.com
[2018-04-22 19:03:34.675747] E [MSGID: 106153] [glusterd-syncop.c:113:gd_collate_errors] 0-glusterd: Commit failed on dhcp37-121.lab.eng.blr.redhat.com. Please check log file for details.
--------



Version-Release number of selected component (if applicable):

# rpm -qa | grep ganesha
nfs-ganesha-gluster-2.5.5-5.el7rhgs.x86_64
glusterfs-ganesha-3.12.2-8.el7rhgs.x86_64
nfs-ganesha-debuginfo-2.5.5-4.el7rhgs.x86_64
nfs-ganesha-2.5.5-5.el7rhgs.x86_64


How reproducible:
2/2


Steps to Reproduce:
1.Create 4 node ganesha cluster out of 5 node gluster cluster


Actual results:
"gluster nfs-ganesha" command errors out.But if check on backend,pcs status output shows cluster is up and running on 4 nodes




Expected results:
"gluster nfs-ganesha" should not error out

Additional info:


# pcs status
Cluster name: ganesha-ha-360
Stack: corosync
Current DC: dhcp47-193.lab.eng.blr.redhat.com (version 1.1.18-11.el7-2b07d5c5a9) - partition with quorum
Last updated: Mon Apr 23 00:05:48 2018
Last change: Sun Apr 22 23:53:58 2018 by root via cibadmin on dhcp47-193.lab.eng.blr.redhat.com

4 nodes configured
24 resources configured

Online: [ dhcp46-116.lab.eng.blr.redhat.com dhcp46-184.lab.eng.blr.redhat.com dhcp47-193.lab.eng.blr.redhat.com dhcp47-2.lab.eng.blr.redhat.com ]

Full list of resources:

 Clone Set: nfs_setup-clone [nfs_setup]
     Started: [ dhcp46-116.lab.eng.blr.redhat.com dhcp46-184.lab.eng.blr.redhat.com dhcp47-193.lab.eng.blr.redhat.com dhcp47-2.lab.eng.blr.redhat.com ]
 Clone Set: nfs-mon-clone [nfs-mon]
     Started: [ dhcp46-116.lab.eng.blr.redhat.com dhcp46-184.lab.eng.blr.redhat.com dhcp47-193.lab.eng.blr.redhat.com dhcp47-2.lab.eng.blr.redhat.com ]
 Clone Set: nfs-grace-clone [nfs-grace]
     Started: [ dhcp46-116.lab.eng.blr.redhat.com dhcp46-184.lab.eng.blr.redhat.com dhcp47-193.lab.eng.blr.redhat.com dhcp47-2.lab.eng.blr.redhat.com ]
 Resource Group: dhcp47-193.lab.eng.blr.redhat.com-group
     dhcp47-193.lab.eng.blr.redhat.com-nfs_block	(ocf::heartbeat:portblock):	Started dhcp47-193.lab.eng.blr.redhat.com
     dhcp47-193.lab.eng.blr.redhat.com-cluster_ip-1	(ocf::heartbeat:IPaddr):	Started dhcp47-193.lab.eng.blr.redhat.com
     dhcp47-193.lab.eng.blr.redhat.com-nfs_unblock	(ocf::heartbeat:portblock):	Started dhcp47-193.lab.eng.blr.redhat.com
 Resource Group: dhcp46-116.lab.eng.blr.redhat.com-group
     dhcp46-116.lab.eng.blr.redhat.com-nfs_block	(ocf::heartbeat:portblock):	Started dhcp46-116.lab.eng.blr.redhat.com
     dhcp46-116.lab.eng.blr.redhat.com-cluster_ip-1	(ocf::heartbeat:IPaddr):	Started dhcp46-116.lab.eng.blr.redhat.com
     dhcp46-116.lab.eng.blr.redhat.com-nfs_unblock	(ocf::heartbeat:portblock):	Started dhcp46-116.lab.eng.blr.redhat.com
 Resource Group: dhcp46-184.lab.eng.blr.redhat.com-group
     dhcp46-184.lab.eng.blr.redhat.com-nfs_block	(ocf::heartbeat:portblock):	Started dhcp46-184.lab.eng.blr.redhat.com
     dhcp46-184.lab.eng.blr.redhat.com-cluster_ip-1	(ocf::heartbeat:IPaddr):	Started dhcp46-184.lab.eng.blr.redhat.com
     dhcp46-184.lab.eng.blr.redhat.com-nfs_unblock	(ocf::heartbeat:portblock):	Started dhcp46-184.lab.eng.blr.redhat.com
 Resource Group: dhcp47-2.lab.eng.blr.redhat.com-group
     dhcp47-2.lab.eng.blr.redhat.com-nfs_block	(ocf::heartbeat:portblock):	Started dhcp47-2.lab.eng.blr.redhat.com
     dhcp47-2.lab.eng.blr.redhat.com-cluster_ip-1	(ocf::heartbeat:IPaddr):	Started dhcp47-2.lab.eng.blr.redhat.com
     dhcp47-2.lab.eng.blr.redhat.com-nfs_unblock	(ocf::heartbeat:portblock):	Started dhcp47-2.lab.eng.blr.redhat.com

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled



-------------------------

Node on which ganesha was not set up---



[2018-04-22 19:03:33.723111] E [MSGID: 106470] [glusterd-ganesha.c:357:glusterd_op_set_ganesha] 0-management: Initial NFS-Ganesha set up failed
[2018-04-22 19:03:33.723136] E [MSGID: 106123] [glusterd-op-sm.c:5959:glusterd_op_ac_commit_op] 0-management: Commit of operation 'Volume (null)' failed: -1
[2018-04-22 19:03:33.821219] E [MSGID: 106062] [glusterd-op-sm.c:4241:glusterd_op_ac_unlock] 0-management: Unable to acquire volname



# cd /etc/ganesha/
[root@dhcp37-121 ganesha]# ll
total 12
-rw-r--r--. 1 root root 1576 Apr 22 07:48 ganesha.conf
-rw-r--r--. 1 root root 2345 Apr 19 10:32 ganesha.conf.example
-rw-r--r--. 1 root root  776 Apr  4 05:32 ganesha-ha.conf.sample

[root@dhcp37-121 ganesha]# cat /var/lib/glusterd/options
global-option-version=90
cluster.enable-shared-storage=enable
nfs-ganesha=disable


# pcs status
Error: cluster is not currently running on this node

[root@dhcp37-121 ganesha]# gluster peer status
Number of Peers: 4

Hostname: dhcp46-116.lab.eng.blr.redhat.com
Uuid: 856cb4ed-c1d1-4da0-b753-b7adaaa46a4f
State: Peer in Cluster (Connected)

Hostname: dhcp47-193.lab.eng.blr.redhat.com
Uuid: 97de04a8-5040-45e0-be11-0b891473d022
State: Peer in Cluster (Connected)

Hostname: dhcp46-184.lab.eng.blr.redhat.com
Uuid: f8c8ba49-c161-4502-a524-54e3b6f995e3
State: Peer in Cluster (Connected)

Hostname: dhcp47-2.lab.eng.blr.redhat.com
Uuid: afd776a4-3ad2-4aab-beec-f4697994be74
State: Peer in Cluster (Connected)

Comment 11 errata-xmlrpc 2018-09-04 06:46:03 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2607

Comment 12 Manisha Saini 2018-09-24 07:00:28 UTC
Setting qe_test_coverage + with no BZ ID,since its been covered as part of every ganesha test case.


Note You need to log in before you can comment on or make changes to this bug.