Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
This project is now read‑only. Starting Monday, February 2, please use Jira Cloud for all bug tracking management.

Bug 1618678

Summary: Installation fails with error: 'dict object' has no attribute 'rgw_hostname'
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: shilpa <smanjara>
Component: Ceph-AnsibleAssignee: Sébastien Han <shan>
Status: CLOSED ERRATA QA Contact: shilpa <smanjara>
Severity: urgent Docs Contact: Aron Gunn <agunn>
Priority: high    
Version: 3.1CC: agunn, aschoen, ceph-eng-bugs, fhubik, gael_rehault, gfidente, gmeno, hgurav, hnallurv, jbrier, johfulto, kdreyer, nthomas, sankarshan, shan, tserlin, vashastr
Target Milestone: rcKeywords: Automation, AutomationBlocker, Regression
Target Release: 3.1   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: RHEL: ceph-ansible-3.1.0-0.1.rc21.el7cp Ubuntu: ceph-ansible_3.1.0~rc21-2redhat1 Doc Type: Bug Fix
Doc Text:
.Ceph installation no longer fails when trying to deploy the Object Gateway When deploying the Ceph Object Gateway using Ansible, the `rgw_hostname` variable was not being set on the Object Gateway node, but was incorrectly set on the Ceph Monitor node. In this release, the `rgw_hostname` variable is set properly and applied to the Ceph Object Gateway node.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2018-09-26 18:23:45 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1578730, 1584264    
Attachments:
Description Flags
/var/lib/mistral/xyz/ansible.log
none
ansible.log
none
inventory.yml
none
/var/lib/mistral/config-download-latest/ansible.log
none
/var/lib/mistral/config-download-latest/ceph-ansible/inventory.yml none

Description shilpa 2018-08-17 09:57:47 UTC 
Description of problem:
Playbook fails at the following stage:

TASK [ceph-config : generate ceph configuration file

Could have been introduced as a result of: 

https://github.com/ceph/ceph-ansible/commit/97cf08e89729db35264d0fb2c55ac01941761b9d

Version-Release number of selected component (if applicable):

3.1.0-0.1.rc18.el7cp 

How reproducible:
Always


Actual results:

INFO:teuthology.orchestra.run.clara012.stdout:TASK [ceph-config : generate ceph configuration file: c1.conf] *****************
INFO:teuthology.orchestra.run.clara012.stdout:task path: /home/ubuntu/ceph-ansible/roles/ceph-config/tasks/main.yml:12
INFO:teuthology.orchestra.run.clara012.stdout:Friday 17 August 2018  06:36:51 +0000 (0:00:00.351)       0:05:00.432 *********

INFO:teuthology.orchestra.run.clara012.stdout:fatal: [clara012.ceph.redhat.com]: FAILED! => {}
 MSG:
'dict object' has no attribute 'rgw_hostname'


INFO:teuthology.orchestra.run.clara012.stdout:PLAY RECAP *********************************************************************
INFO:teuthology.orchestra.run.clara012.stdout:clara012.ceph.redhat.com   : ok=44   changed=12   unreachable=0    failed=1
 INFO:teuthology.orchestra.run.clara012.stdout:pluto004.ceph.redhat.com   : ok=1    changed=0    unreachable=0    failed=0


Expected results:
The same cluster configuration works with rc17 version. 

Additional info:

Config parameters:

    ceph_ansible:
      rhbuild: '3.1'
      vars:
        ceph_conf_overrides:
          global:
            mon_max_pg_per_osd: 1024
            osd default pool size: 2
            osd pool default pg num: 64
            osd pool default pgp num: 64
        ceph_origin: distro
        ceph_repository: rhcs
        ceph_stable: true
        ceph_stable_release: luminous
        ceph_stable_rh_storage: true
        ceph_test: true
        journal_size: 1024
        osd_auto_discovery: true
        osd_scenario: collocated
Comment 4 Vasishta 2018-08-17 14:01:05 UTC 
RGW installation is blocked in scenario where ansible_hostname != ansible_fqdn in task "ceph-defaults : get current cluster status (if already running)" with message - 

"msg": "The conditional check 'ceph_release_num[ceph_release] >= ceph_release_num.luminous' failed. The error was: error while evaluating conditional (ceph_release_num[ceph_release] >= ceph_release_num.luminous): 'dict object' has no attribute u'dummy'\n\nThe error appears to have been in '/usr/share/ceph-ansible/roles/ceph-defaults/tasks/facts.yml': line 219, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n- block:\n    - name: get current cluster status (if already running)\n      ^ here\n"
}


Regards,
Vasishta Shastry
QE, Ceph
Comment 5 Sébastien Han 2018-08-20 22:08:39 UTC 
In https://github.com/ceph/ceph-ansible/releases/tag/v3.1.0rc19
Comment 6 Filip Hubík 2018-08-21 15:02:41 UTC 
I can not confirm this issue being fixed as part of OSP14 deployment, where we have ceph-ansible-3.1.0.0-0.rc19.1.el7.noarch available on UC.

I am attaching whole ansible.log from mistral, where it is apparent that last failure can be related.
Comment 7 Filip Hubík 2018-08-21 15:03:32 UTC 
Created attachment 1477591 [details]
/var/lib/mistral/xyz/ansible.log
Comment 8 Ken Dreyer (Red Hat) 2018-08-21 18:53:21 UTC 
(For clarity, Filip did some early testing before ceph-ansible-3.1.0rc19 is available in Brew.)

Since that build does not fix this for Filip, we'll need more investigation here.
Comment 9 Sébastien Han 2018-08-22 17:59:46 UTC 
In https://github.com/ceph/ceph-ansible/releases/tag/v3.1.0rc21
Comment 14 Giulio Fidente 2018-09-03 16:44:30 UTC 
*** Bug 1619736 has been marked as a duplicate of this bug. ***
Comment 17 Sébastien Han 2018-09-05 09:00:28 UTC 
Is this better now?
Comment 20 Giulio Fidente 2018-09-14 17:54:25 UTC 
it looks like we're still seeing the issue with 3.1.3 build; attaching ansible.log and inventory.yaml
Comment 21 Giulio Fidente 2018-09-14 17:55:15 UTC 
Created attachment 1483383 [details]
ansible.log
Comment 22 Giulio Fidente 2018-09-14 17:56:24 UTC 
Created attachment 1483384 [details]
inventory.yml
Comment 23 Filip Hubík 2018-09-17 09:26:44 UTC 
I can confirm this issue not being fixed in ceph-ansible-3.1.0-0.1.rc21.el7cp as part of OpenStack director 14 deployment.

openstack-mistral-executor contains this package, but ceph-ansible fails as part of mistral post-deployment on this error:

$ tail -n 300 /var/lib/mistral/config-download-latest/ansible.log
...
"Conditional result was False\"}\n\nTASK [ceph-config : ensure /etc/ceph exists] ***********************************\ntask path: /usr/share/ceph-ansible/roles/ceph-config/tasks/main.yml:76\nFriday 14 September 2018  10:43:50 -0400 (0:00:00.050)       0:01:04.055 ****** \nchanged: [controller-2] => {\"changed\": true, \"gid\": 167, \"group\": \"167\", \"mode\": \"0755\", \"owner\": \"167\", \"path\": \"/etc/ceph\", \"secontext\": \"unconfined_u:object_r:etc_t:s0\", \"size\": 6, \"state\": \"directory\", \"uid\": 167}\n\nTASK [ceph-config : generate ceph.conf configuration file] *********************\ntask path: /usr/share/ceph-ansible/roles/ceph-config/tasks/main.yml:84\nFriday 14 September 2018  10:43:51 -0400 (0:00:00.457)       0:01:04.513 ****** \nfatal: [controller-2]: FAILED! => {\"msg\": \"'dict object' has no attribute 'rgw_hostname'\"}\n\nPLAY RECAP *********************************************************************\nceph-0                     : ok=2    changed=0    unreachable=0    failed=0   \nceph-1                     : ok=2    changed=0    unreachable=0    failed=0   \nceph-2                     : ok=2    changed=0    unreachable=0    failed=0   \ncompute-0                  : ok=2    changed=0    unreachable=0    failed=0   \ncontroller-0               : ok=2    changed=0    unreachable=0    failed=0   \ncontroller-1               : ok=2    changed=0    unreachable=0    failed=0   \ncontroller-2               : ok=43   changed=4    unreachable=0    failed=1

This is HA deployment (3 controllers) with 3 ceph nodes, ceph rgw, lowmem and mds features enabled.

Attaching both /var/lib/mistral/config-download-latest/ansible.log and /var/lib/mistral/config-download-latest/ceph-ansible/inventory.yml.
Comment 24 Filip Hubík 2018-09-17 09:28:01 UTC 
Created attachment 1483944 [details]
/var/lib/mistral/config-download-latest/ansible.log
Comment 25 Filip Hubík 2018-09-17 09:28:36 UTC 
Created attachment 1483945 [details]
/var/lib/mistral/config-download-latest/ceph-ansible/inventory.yml
Comment 26 Filip Hubík 2018-09-17 09:31:24 UTC 
Also, if our automation/CI is not wrong, this is still not fixed in newest ceph-ansible-3.1.3-1.el7cp.noarch.rpm build.
Comment 27 Giulio Fidente 2018-09-17 10:14:47 UTC 
*** Bug 1622505 has been marked as a duplicate of this bug. ***
Comment 32 errata-xmlrpc 2018-09-26 18:23:45 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:2819