Description of problem: Unable to upgrade nodes to latest 3.6 version. The execution of the script yields the following error: The conditional check 'hostvars[item].openshift.common.hostname in nodes_to_upgrade.results.results[0]['items'] | map(attribute='metadata.name') | list' failed. The error was: error while evaluating conditional (hostvars[item].openshift.common.hostname in nodes_to_upgrade.results.results[0]['items'] | map(attribute='metadata.name') | list): 'dict object' has no attribute 'openshift' The error appears to have been in '/usr/share/ansible/openshift-ansible/playbooks/common/openshift-cluster/upgrades/initialize_nodes_to_upgrade.yml': line 25, column 7, but may be elsewhere in the file depending on the exact syntax problem. The offending line appears to be: # using their openshift.common.hostname fact. - name: Map labelled nodes to inventory hosts ^ here Version-Release number of the following components: $ rpm -aq | grep openshift-ansible openshift-ansible-roles-3.6.173.0.21-2.git.0.44a4038.el7.noarch openshift-ansible-callback-plugins-3.6.173.0.21-2.git.0.44a4038.el7.noarch openshift-ansible-lookup-plugins-3.6.173.0.21-2.git.0.44a4038.el7.noarch openshift-ansible-3.6.173.0.21-2.git.0.44a4038.el7.noarch openshift-ansible-filter-plugins-3.6.173.0.21-2.git.0.44a4038.el7.noarch openshift-ansible-docs-3.6.173.0.21-2.git.0.44a4038.el7.noarch openshift-ansible-playbooks-3.6.173.0.21-2.git.0.44a4038.el7.noarch How reproducible: Always on customer's environment Steps to Reproduce: 1. ansible-playbook -i /etc/ansible/hosts /usr/share/ansible/openshift-ansible/playbooks/byo/openshift-cluster/upgrades/v3_6/upgrade_control_plane.yml 2. ansible-playbook -i /etc/ansible/hosts /usr/share/ansible/openshift-ansible/playbooks/byo/openshift-cluster/upgrades/v3_6/upgrade_nodes.yml -e openshift_upgrade_nodes_label="purpose=infra" Actual results: TASK [Map labelled nodes to inventory hosts] ********************************************************************************************************************************************************************************************* skipping: [master1-prod.example.com] => (item=master1-prod.example.com) skipping: [master1-prod.example.com] => (item=master2-prod.example.com) skipping: [master1-prod.example.com] => (item=master3-prod.example.com) ok: [master1-prod.example.com] => (item=inode1-prod.example.com) ok: [master1-prod.example.com] => (item=inode2-prod.example.com) ok: [master1-prod.example.com] => (item=inode3-prod.example.com) skipping: [master1-prod.example.com] => (item=node1-prod.example.com) skipping: [master1-prod.example.com] => (item=node2-prod.example.com) fatal: [master1-prod.example.com]: FAILED! => { "failed": true } MSG: The conditional check 'hostvars[item].openshift.common.hostname in nodes_to_upgrade.results.results[0]['items'] | map(attribute='metadata.name') | list' failed. The error was: error while evaluating conditional (hostvars[item].openshift.common.hostname in nodes_to_upgrade.results.results[0]['items'] | map(attribute='metadata.name') | list): 'dict object' has no attribute 'openshift' The error appears to have been in '/usr/share/ansible/openshift-ansible/playbooks/common/openshift-cluster/upgrades/initialize_nodes_to_upgrade.yml': line 25, column 7, but may be elsewhere in the file depending on the exact syntax problem. The offending line appears to be: # using their openshift.common.hostname fact. - name: Map labelled nodes to inventory hosts ^ here Expected results: Target nodes to be successfully upgraded. Additional info: Please attach logs from ansible-playbook with the -vvv flag
The workaround for this is to not use openshift_upgrade_nodes_label and instead comment out all nodes except those you wish to upgrade. This of course is not ideal but it should get you around this problem in the short term while we address the bug.
@Sdodson, in the execution I see the node matching works but then it fails the retrieval of the values: 'hostvars[item].openshift.common.hostname in nodes_to_upgrade.results.results[0]['items'] | map(attribute='metadata.name') | list' Can be that only hostvars[item] is needed?
Host node3-prod was unreachable during fact gathering and therefore failed. Later when the task was run and stepped through each host to map hostnames, the failed host didn't have any facts defined and therefore caused that task to fail. Investigating a fix.
Proposed: https://github.com/openshift/openshift-ansible/pull/5728
Merged: https://github.com/openshift/openshift-ansible/pull/5728
@Russell, Thx. It can be re-produced with your smart way. Version: openshift-ansible-3.6.173.0.21-2.git.0.44a4038.el7.noarch 1. Install ocp3.5. 2. Upgrade masters. 3. Edit inventory host file to add a bogus node which is not labeled with targeted label. 4. Upgrade nodes with label "region=primary" Then hit the issue. MSG: The conditional check 'hostvars[item].openshift.common.hostname in nodes_to_upgrade.results.results[0]['items'] | map(attribute='metadata.name') | list' failed. The error was: error while evaluating conditional (hostvars[item].openshift.common.hostname in nodes_to_upgrade.results.results[0]['items'] | map(attribute='metadata.name') | list): 'dict object' has no attribute 'openshift' The error appears to have been in '/usr/share/ansible/openshift-ansible/playbooks/common/openshift-cluster/upgrades/initialize_nodes_to_upgrade.yml': line 25, column 7, but may be elsewhere in the file depending on the exact syntax problem. The offending line appears to be: # using their openshift.common.hostname fact. - name: Map labelled nodes to inventory hosts ^ here
Still no available build for verification in advisory 30978, change status to MODIFIED.
$ git tag --contains 3aeb222b5079f7789a8ba4fcb71345f77194cc58 openshift-ansible-3.6.173.0.50-1 openshift-ansible-3.6.173.0.51-1 openshift-ansible-3.6.173.0.52-1 openshift-ansible-3.6.173.0.53-1 openshift-ansible-3.6.173.0.54-1 openshift-ansible-3.6.173.0.55-1 openshift-ansible-3.6.173.0.56-1 openshift-ansible-3.6.173.0.57-1 openshift-ansible-3.6.173.0.58-1 openshift-ansible-3.6.173.0.59-1 openshift-ansible-3.6.173.0.60-1 Build available: OpenShift Version: v3.6.173.0.59-1
Version: openshift-ansible-3.6.173.0.59-1.git.0.0e31372.el7.noarch Steps: 1. Install ocp3.5. 2. Upgrade masters. 3. Edit inventory host file to add a bogus node which is not labeled with targeted label. 4. Upgrade nodes with label "region=primary" Upgrade succeed except the unreachable node.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:3255