Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 1613255 - [UPGRADES][13]Failed to access hostvars - "|failed expects a dictionary"
[UPGRADES][13]Failed to access hostvars - "|failed expects a dictionary"
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-common (Show other bugs)
13.0 (Queens)
All All
urgent Severity urgent
: z2
: 13.0 (Queens)
Assigned To: Cédric Jeanneret
Gurenko Alex
: Triaged, ZStream
Depends On: 1613161
Blocks:
  Show dependency treegraph
 
Reported: 2018-08-07 06:13 EDT by Yurii Prokulevych
Modified: 2018-08-29 12:40 EDT (History)
9 users (show)

See Also:
Fixed In Version: openstack-tripleo-common-8.6.3-8.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2018-08-29 12:39:25 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
cjeanner: needinfo-


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
OpenStack gerrit 583285 None stable/queens: MERGED tripleo-common: Use hostnames in inventory (If834e5e60f79a7418cbc1441d2aaf85507ac7bd2) 2018-08-07 11:14 EDT
OpenStack gerrit 583286 None stable/queens: MERGED tripleo-common: Fix dynamic inventory (I035bb878508ceab06279cbaf07969aafeb66b9e5) 2018-08-07 11:14 EDT
OpenStack gerrit 583287 None stable/queens: MERGED tripleo-common: Include 'tripleo_role_name' in the inventory (I57c4eac87e2f96dfe5490b111cd2508505715d56) 2018-08-07 11:13 EDT
OpenStack gerrit 583949 None stable/queens: MERGED tripleo-common: Remove role_data from inventory (I06c9b3256567cc57d599bd4d9af617c718d2314f) 2018-08-07 11:14 EDT
Red Hat Product Errata RHBA-2018:2574 None None None 2018-08-29 12:40 EDT

  None (edit)
Description Yurii Prokulevych 2018-08-07 06:13:02 EDT
Description of problem:
-----------------------
While testing patch for rhbz1611016 upgrade failed:
...
 u'TASK [tripleo-ssh-known-hosts : Add hosts key in /etc/ssh/ssh_known_hosts for live/cold-migration] ***',
 u'Monday 06 August 2018  10:05:31 -0400 (0:00:00.521)       0:00:10.992 ********* ',
 u'fatal: [192.168.24.21]: FAILED! => {"failed": true, "msg": "|failed expects a dictionary"}',
 u'fatal: [192.168.24.24]: FAILED! => {"failed": true, "msg": "|failed expects a dictionary"}',
 u'fatal: [192.168.24.11]: FAILED! => {"failed": true, "msg": "|failed expects a dictionary"}',
 u'',
 u'NO MORE HOSTS LEFT *************************************************************',
 u'',
 u'PLAY RECAP *********************************************************************',
 u'192.168.24.11              : ok=4    changed=2    unreachable=0    failed=1   ',
 u'192.168.24.21              : ok=4    changed=2    unreachable=0    failed=1   ',
 u'192.168.24.24              : ok=4    changed=2    unreachable=0    failed=1   ',

The problem seems to be with hostvar[item]['ansible_hostname'] statement.




Version-Release number of selected component (if applicable):
-------------------------------------------------------------
ceph-ansible-3.1.0-0.1.rc10.el7cp.noarch
ansible-pacemaker-1.0.4-0.20180220234310.0e4d7c0.el7ost.noarch
ansible-2.4.6.0-1.el7ae.noarch
ansible-tripleo-ipsec-8.1.1-0.20180308133440.8f5369a.el7ost.noarch
ansible-role-redhat-subscription-1.0.1-1.el7ost.noarch

openstack-tripleo-heat-templates-8.0.4-4.el7ost.noarch


Steps to Reproduce:
-------------------
1. Upgrade UC to RHOS-13
2. Apply patch https://review.openstack.org/#/c/584994/
3. Prepare oc images/repos
4. Setup heat-outputs 
5. Start upgrade with Controller role, e.g.:
    openstack overcloud upgrade run \
        --stack overcloud \
        --roles ControllerOpenstack --playbook all 

Actual results:
---------------
Upgrade failed


Expected results:
-----------------
Upgrade succeeded

Additional info:
----------------
Try to check hostvars statement with inventory generated by tripleo-ansible-inventory. Example of playbook:
---
- name: check hostvars
  hosts: all
  become: true
  gather_facts: yes
  tasks:
    - name: print hostvar
      debug:
        msg: "{{ hostvars[item]['ansible_hostname'] + ' ssh-rsa ' + hostvars[item]['ansible_ssh_host_key_rsa_public'] }}"
      with_items: "{{ groups['all']|intersect(play_hosts) }}"
Comment 2 Jose Luis Franco 2018-08-07 06:32:59 EDT
We are missing several patches related to openstack-tripleo-common/inventory.py in the downstream branch rhos-13.0-patches which are present in tripleo_common stable/queens branch upstream. To be more precise, all of these:

[Include 'tripleo_role_name' in the inventory] https://github.com/openstack/tripleo-common/commit/f3970c2db0bb56b668da1ab82b557d52c458a28e#diff-354b2f588533c21d913db5dd03f9bade

[Fix dynamic inventory] https://github.com/openstack/tripleo-common/commit/18324fb412267bfb113731d92c4bd92f8ed40ea4#diff-354b2f588533c21d913db5dd03f9bade

[Use hostnames in inventory] https://github.com/openstack/tripleo-common/commit/7901fb43cb4db2d493349e649caf970ab72127bb#diff-354b2f588533c21d913db5dd03f9bade

[Remove role_data from inventory ] https://github.com/openstack/tripleo-common/commit/1e4f49f5563f153c9ef0bdd1a38734395084b0dc#diff-354b2f588533c21d913db5dd03f9bade

When generating the tripleo-ansible-inventory.yml using the current inventory.py ( https://raw.githubusercontent.com/openstack/tripleo-common/stable/queens/tripleo_common/inventory.py ) file from stable/queens, the errors dissapear and we can safely access hostvars variables in the playbooks. So, we would need to cherry-pick all those patches downstream, into rhos-13.0-patches, in order to solve this bug.
Comment 8 Joanne O'Flynn 2018-08-15 03:39:34 EDT
This bug is marked for inclusion in the errata but does not currently contain draft documentation text. To ensure the timely release of this advisory please provide draft documentation text for this bug as soon as possible.

If you do not think this bug requires errata documentation, set the requires_doc_text flag to "-".


To add draft documentation text:

* Select the documentation type from the "Doc Type" drop down field.

* A template will be provided in the "Doc Text" field based on the "Doc Type" value selected. Enter draft text in the "Doc Text" field.
Comment 11 errata-xmlrpc 2018-08-29 12:39:25 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:2574

Note You need to log in before you can comment on or make changes to this bug.