Bug 1813166

Summary: [RHEL-8.1] NFS-Ganesha setup is failing via Gdeploy
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Manisha Saini <msaini>
Component: gdeployAssignee: Prajith <pkesavap>
Status: CLOSED ERRATA QA Contact: Manisha Saini <msaini>
Severity: urgent Docs Contact:
Priority: urgent    
Version: rhgs-3.5CC: godas, pkesavap, pprakash, puebele, rhs-bugs, storage-qa-internal
Target Milestone: ---Keywords: TestBlocker, ZStream
Target Release: RHGS 3.5.z Batch Update 2   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: block
Fixed In Version: gdeploy-3.0.0-4.el8rhgs Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-06-16 05:56:04 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1813908, 1821907    
Bug Blocks:    

Description Manisha Saini 2020-03-13 06:26:51 UTC
Description of problem:
=======================

Gdeploy fails while creating ganesha cluster


Version-Release number of selected component (if applicable):
=====================================# rpm -qa | grep gdeploy

gdeploy-3.0.0-3.el8rhgs.noarch

# rpm -qa | grep ansible
ansible-2.9.6-1.el8ae.noarch


How reproducible:
=================
1/1

Steps to Reproduce:
==================
1.Create config file for creating ganesha cluster via gdeploy

----
 cat ganesha.conf 
[hosts]
dhcp35-45.lab.eng.blr.redhat.com
dhcp35-146.lab.eng.blr.redhat.com
dhcp35-168.lab.eng.blr.redhat.com
dhcp35-144.lab.eng.blr.redhat.com

[firewalld]
action=add
ports=111/tcp,2049/tcp,54321/tcp,5900/tcp,5900-6923/tcp,5666/tcp,16514/tcp,662/tcp,662/udp
services=glusterfs,nlm,nfs,rpc-bind,high-availability,mountd,rquota

[nfs-ganesha]
action=create-cluster
ha-name=ganesha-ha-360
cluster-nodes=dhcp35-45.lab.eng.blr.redhat.com,dhcp35-146.lab.eng.blr.redhat.com,dhcp35-168.lab.eng.blr.redhat.com,dhcp35-144.lab.eng.blr.redhat.com
vip=10.70.35.21,10.70.35.22,10.70.35.23,10.70.35.24
#volname=ganesha
#ignore_ganesha_errors=no

----
2.Run config file 

Actual results:
==============

Gdeploy error out


---------
# gdeploy  -c ganesha.conf 

PLAY [gluster_servers] ***************************************************************************************************************************************************************

TASK [Start firewalld if not already started] ****************************************************************************************************************************************
ok: [dhcp35-45.lab.eng.blr.redhat.com]
ok: [dhcp35-146.lab.eng.blr.redhat.com]
ok: [dhcp35-144.lab.eng.blr.redhat.com]
ok: [dhcp35-168.lab.eng.blr.redhat.com]

TASK [Open/Close firewalld ports] ****************************************************************************************************************************************************
ok: [dhcp35-146.lab.eng.blr.redhat.com] => (item=111/tcp)
ok: [dhcp35-144.lab.eng.blr.redhat.com] => (item=111/tcp)
ok: [dhcp35-45.lab.eng.blr.redhat.com] => (item=111/tcp)
ok: [dhcp35-168.lab.eng.blr.redhat.com] => (item=111/tcp)
ok: [dhcp35-144.lab.eng.blr.redhat.com] => (item=2049/tcp)
ok: [dhcp35-45.lab.eng.blr.redhat.com] => (item=2049/tcp)
ok: [dhcp35-146.lab.eng.blr.redhat.com] => (item=2049/tcp)
ok: [dhcp35-168.lab.eng.blr.redhat.com] => (item=2049/tcp)
ok: [dhcp35-144.lab.eng.blr.redhat.com] => (item=54321/tcp)
ok: [dhcp35-45.lab.eng.blr.redhat.com] => (item=54321/tcp)
ok: [dhcp35-168.lab.eng.blr.redhat.com] => (item=54321/tcp)
ok: [dhcp35-146.lab.eng.blr.redhat.com] => (item=54321/tcp)
ok: [dhcp35-144.lab.eng.blr.redhat.com] => (item=5900/tcp)
ok: [dhcp35-45.lab.eng.blr.redhat.com] => (item=5900/tcp)
ok: [dhcp35-146.lab.eng.blr.redhat.com] => (item=5900/tcp)
ok: [dhcp35-168.lab.eng.blr.redhat.com] => (item=5900/tcp)
ok: [dhcp35-144.lab.eng.blr.redhat.com] => (item=5900-6923/tcp)
ok: [dhcp35-45.lab.eng.blr.redhat.com] => (item=5900-6923/tcp)
ok: [dhcp35-146.lab.eng.blr.redhat.com] => (item=5900-6923/tcp)
ok: [dhcp35-168.lab.eng.blr.redhat.com] => (item=5900-6923/tcp)
ok: [dhcp35-144.lab.eng.blr.redhat.com] => (item=5666/tcp)
ok: [dhcp35-45.lab.eng.blr.redhat.com] => (item=5666/tcp)
ok: [dhcp35-146.lab.eng.blr.redhat.com] => (item=5666/tcp)
ok: [dhcp35-168.lab.eng.blr.redhat.com] => (item=5666/tcp)
ok: [dhcp35-144.lab.eng.blr.redhat.com] => (item=16514/tcp)
ok: [dhcp35-45.lab.eng.blr.redhat.com] => (item=16514/tcp)
ok: [dhcp35-146.lab.eng.blr.redhat.com] => (item=16514/tcp)
ok: [dhcp35-168.lab.eng.blr.redhat.com] => (item=16514/tcp)
ok: [dhcp35-144.lab.eng.blr.redhat.com] => (item=662/tcp)
ok: [dhcp35-146.lab.eng.blr.redhat.com] => (item=662/tcp)
ok: [dhcp35-45.lab.eng.blr.redhat.com] => (item=662/tcp)
ok: [dhcp35-168.lab.eng.blr.redhat.com] => (item=662/tcp)
ok: [dhcp35-144.lab.eng.blr.redhat.com] => (item=662/udp)
ok: [dhcp35-146.lab.eng.blr.redhat.com] => (item=662/udp)
ok: [dhcp35-45.lab.eng.blr.redhat.com] => (item=662/udp)
ok: [dhcp35-168.lab.eng.blr.redhat.com] => (item=662/udp)

TASK [Reloads the firewall] **********************************************************************************************************************************************************
changed: [dhcp35-45.lab.eng.blr.redhat.com]
changed: [dhcp35-144.lab.eng.blr.redhat.com]
changed: [dhcp35-146.lab.eng.blr.redhat.com]
changed: [dhcp35-168.lab.eng.blr.redhat.com]

PLAY RECAP ***************************************************************************************************************************************************************************
dhcp35-144.lab.eng.blr.redhat.com : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
dhcp35-146.lab.eng.blr.redhat.com : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
dhcp35-168.lab.eng.blr.redhat.com : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
dhcp35-45.lab.eng.blr.redhat.com : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   


PLAY [gluster_servers] ***************************************************************************************************************************************************************

TASK [Start firewalld if not already started] ****************************************************************************************************************************************
ok: [dhcp35-146.lab.eng.blr.redhat.com]
ok: [dhcp35-45.lab.eng.blr.redhat.com]
ok: [dhcp35-144.lab.eng.blr.redhat.com]
ok: [dhcp35-168.lab.eng.blr.redhat.com]

TASK [Add/Delete services to firewalld rules] ****************************************************************************************************************************************
ok: [dhcp35-45.lab.eng.blr.redhat.com] => (item=glusterfs)
ok: [dhcp35-146.lab.eng.blr.redhat.com] => (item=glusterfs)
ok: [dhcp35-144.lab.eng.blr.redhat.com] => (item=glusterfs)
ok: [dhcp35-168.lab.eng.blr.redhat.com] => (item=glusterfs)
ok: [dhcp35-146.lab.eng.blr.redhat.com] => (item=nlm)
ok: [dhcp35-45.lab.eng.blr.redhat.com] => (item=nlm)
ok: [dhcp35-144.lab.eng.blr.redhat.com] => (item=nlm)
ok: [dhcp35-168.lab.eng.blr.redhat.com] => (item=nlm)
ok: [dhcp35-146.lab.eng.blr.redhat.com] => (item=nfs)
ok: [dhcp35-45.lab.eng.blr.redhat.com] => (item=nfs)
ok: [dhcp35-144.lab.eng.blr.redhat.com] => (item=nfs)
ok: [dhcp35-168.lab.eng.blr.redhat.com] => (item=nfs)
ok: [dhcp35-146.lab.eng.blr.redhat.com] => (item=rpc-bind)
ok: [dhcp35-45.lab.eng.blr.redhat.com] => (item=rpc-bind)
ok: [dhcp35-144.lab.eng.blr.redhat.com] => (item=rpc-bind)
ok: [dhcp35-168.lab.eng.blr.redhat.com] => (item=rpc-bind)
ok: [dhcp35-146.lab.eng.blr.redhat.com] => (item=high-availability)
ok: [dhcp35-45.lab.eng.blr.redhat.com] => (item=high-availability)
ok: [dhcp35-144.lab.eng.blr.redhat.com] => (item=high-availability)
ok: [dhcp35-168.lab.eng.blr.redhat.com] => (item=high-availability)
ok: [dhcp35-146.lab.eng.blr.redhat.com] => (item=mountd)
ok: [dhcp35-45.lab.eng.blr.redhat.com] => (item=mountd)
ok: [dhcp35-144.lab.eng.blr.redhat.com] => (item=mountd)
ok: [dhcp35-168.lab.eng.blr.redhat.com] => (item=mountd)
ok: [dhcp35-146.lab.eng.blr.redhat.com] => (item=rquota)
ok: [dhcp35-45.lab.eng.blr.redhat.com] => (item=rquota)
ok: [dhcp35-144.lab.eng.blr.redhat.com] => (item=rquota)
ok: [dhcp35-168.lab.eng.blr.redhat.com] => (item=rquota)

TASK [Reloads the firewall] **********************************************************************************************************************************************************
changed: [dhcp35-146.lab.eng.blr.redhat.com]
changed: [dhcp35-45.lab.eng.blr.redhat.com]
changed: [dhcp35-144.lab.eng.blr.redhat.com]
changed: [dhcp35-168.lab.eng.blr.redhat.com]

PLAY RECAP ***************************************************************************************************************************************************************************
dhcp35-144.lab.eng.blr.redhat.com : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
dhcp35-146.lab.eng.blr.redhat.com : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
dhcp35-168.lab.eng.blr.redhat.com : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
dhcp35-45.lab.eng.blr.redhat.com : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

Traceback (most recent call last):
  File "/usr/bin/gdeploy", line 211, in <module>
    main(sys.argv[1:])
  File "/usr/bin/gdeploy", line 200, in main
    call_features()
  File "/usr/lib/python3.6/site-packages/gdeploylib/call_features.py", line 39, in call_features
    list(map(get_feature_dir, Global.sections))
  File "/usr/lib/python3.6/site-packages/gdeploylib/call_features.py", line 95, in get_feature_dir
    section_dict, yml = feature_call(section_dict)
  File "/usr/lib/python3.6/site-packages/gdeployfeatures/nfs_ganesha/nfs_ganesha.py", line 32, in nfs_ganesha_create_cluster
    if section_dict.has_key('volname'):
AttributeError: 'dict' object has no attribute 'has_key'
------------


Expected results:
================
Gdeploy should not fail to create ganesha cluster


Additional info:

Comment 5 Prajith 2020-03-20 11:52:08 UTC
The packages can be downloaded from the below links:-

Task Info: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=27378429
Build Info: https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=1141714

Regards,
Prajith

Comment 7 Manisha Saini 2020-04-02 21:53:20 UTC
Verification of this BZ can be done only once the fix for BZ - https://bugzilla.redhat.com/show_bug.cgi?id=1813908 is available.

Comment 8 Manisha Saini 2020-04-07 19:43:04 UTC
Verification of this BZ is blocked until the fix for BZ 1821907 is available

Comment 9 Manisha Saini 2020-04-20 19:59:43 UTC
Verified this BZ with

# rpm -qa | grep gdeploy
gdeploy-3.0.0-6.el8rhgs.noarch


# rpm -qa | grep ganesha
nfs-ganesha-gluster-2.7.3-13.el8rhgs.x86_64
nfs-ganesha-debuginfo-2.7.3-13.el8rhgs.x86_64
nfs-ganesha-2.7.3-13.el8rhgs.x86_64
nfs-ganesha-selinux-2.7.3-13.el8rhgs.noarch
nfs-ganesha-debugsource-2.7.3-13.el8rhgs.x86_64
glusterfs-ganesha-6.0-32.el8rhgs.x86_64
nfs-ganesha-gluster-debuginfo-2.7.3-13.el8rhgs.x86_64


Ganesha setup is successful via gdeploy. Moving this BZ to verified state

wrt to incorrect (FAILOVER) status reflected at the end of gdeploy deployment BZ is raised - https://bugzilla.redhat.com/show_bug.cgi?id=1823706

Comment 11 errata-xmlrpc 2020-06-16 05:56:04 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2020:2577