Bug 1499254
| Summary: | [3.6] Unable to update nodes to 3.6 - 'dict object' has no attribute 'openshift' | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Ruben Romero Montes <rromerom> |
| Component: | Cluster Version Operator | Assignee: | Russell Teague <rteague> |
| Status: | CLOSED ERRATA | QA Contact: | liujia <jiajliu> |
| Severity: | urgent | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 3.6.0 | CC: | aos-bugs, bmcelvee, jokerman, mmccomas, rteague |
| Target Milestone: | --- | ||
| Target Release: | 3.6.z | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: |
Previously, a host was unreachable during fact gathering and failed. Later when the task ran and stepped through each host to map hostnames, the failed host did not have any facts defined causing that task to fail. With this bug fix, hosts are reachable and the task succeeds.
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2017-11-21 05:41:13 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
The workaround for this is to not use openshift_upgrade_nodes_label and instead comment out all nodes except those you wish to upgrade. This of course is not ideal but it should get you around this problem in the short term while we address the bug. @Sdodson, in the execution I see the node matching works but then it fails the retrieval of the values: 'hostvars[item].openshift.common.hostname in nodes_to_upgrade.results.results[0]['items'] | map(attribute='metadata.name') | list' Can be that only hostvars[item] is needed? Host node3-prod was unreachable during fact gathering and therefore failed. Later when the task was run and stepped through each host to map hostnames, the failed host didn't have any facts defined and therefore caused that task to fail. Investigating a fix. @Russell,
Thx. It can be re-produced with your smart way.
Version:
openshift-ansible-3.6.173.0.21-2.git.0.44a4038.el7.noarch
1. Install ocp3.5.
2. Upgrade masters.
3. Edit inventory host file to add a bogus node which is not labeled with targeted label.
4. Upgrade nodes with label "region=primary"
Then hit the issue.
MSG:
The conditional check 'hostvars[item].openshift.common.hostname in nodes_to_upgrade.results.results[0]['items'] | map(attribute='metadata.name') | list' failed. The error was: error while evaluating conditional (hostvars[item].openshift.common.hostname in nodes_to_upgrade.results.results[0]['items'] | map(attribute='metadata.name') | list): 'dict object' has no attribute 'openshift'
The error appears to have been in '/usr/share/ansible/openshift-ansible/playbooks/common/openshift-cluster/upgrades/initialize_nodes_to_upgrade.yml': line 25, column 7, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
# using their openshift.common.hostname fact.
- name: Map labelled nodes to inventory hosts
^ here
Still no available build for verification in advisory 30978, change status to MODIFIED. $ git tag --contains 3aeb222b5079f7789a8ba4fcb71345f77194cc58 openshift-ansible-3.6.173.0.50-1 openshift-ansible-3.6.173.0.51-1 openshift-ansible-3.6.173.0.52-1 openshift-ansible-3.6.173.0.53-1 openshift-ansible-3.6.173.0.54-1 openshift-ansible-3.6.173.0.55-1 openshift-ansible-3.6.173.0.56-1 openshift-ansible-3.6.173.0.57-1 openshift-ansible-3.6.173.0.58-1 openshift-ansible-3.6.173.0.59-1 openshift-ansible-3.6.173.0.60-1 Build available: OpenShift Version: v3.6.173.0.59-1 Version: openshift-ansible-3.6.173.0.59-1.git.0.0e31372.el7.noarch Steps: 1. Install ocp3.5. 2. Upgrade masters. 3. Edit inventory host file to add a bogus node which is not labeled with targeted label. 4. Upgrade nodes with label "region=primary" Upgrade succeed except the unreachable node. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:3255 |
Description of problem: Unable to upgrade nodes to latest 3.6 version. The execution of the script yields the following error: The conditional check 'hostvars[item].openshift.common.hostname in nodes_to_upgrade.results.results[0]['items'] | map(attribute='metadata.name') | list' failed. The error was: error while evaluating conditional (hostvars[item].openshift.common.hostname in nodes_to_upgrade.results.results[0]['items'] | map(attribute='metadata.name') | list): 'dict object' has no attribute 'openshift' The error appears to have been in '/usr/share/ansible/openshift-ansible/playbooks/common/openshift-cluster/upgrades/initialize_nodes_to_upgrade.yml': line 25, column 7, but may be elsewhere in the file depending on the exact syntax problem. The offending line appears to be: # using their openshift.common.hostname fact. - name: Map labelled nodes to inventory hosts ^ here Version-Release number of the following components: $ rpm -aq | grep openshift-ansible openshift-ansible-roles-3.6.173.0.21-2.git.0.44a4038.el7.noarch openshift-ansible-callback-plugins-3.6.173.0.21-2.git.0.44a4038.el7.noarch openshift-ansible-lookup-plugins-3.6.173.0.21-2.git.0.44a4038.el7.noarch openshift-ansible-3.6.173.0.21-2.git.0.44a4038.el7.noarch openshift-ansible-filter-plugins-3.6.173.0.21-2.git.0.44a4038.el7.noarch openshift-ansible-docs-3.6.173.0.21-2.git.0.44a4038.el7.noarch openshift-ansible-playbooks-3.6.173.0.21-2.git.0.44a4038.el7.noarch How reproducible: Always on customer's environment Steps to Reproduce: 1. ansible-playbook -i /etc/ansible/hosts /usr/share/ansible/openshift-ansible/playbooks/byo/openshift-cluster/upgrades/v3_6/upgrade_control_plane.yml 2. ansible-playbook -i /etc/ansible/hosts /usr/share/ansible/openshift-ansible/playbooks/byo/openshift-cluster/upgrades/v3_6/upgrade_nodes.yml -e openshift_upgrade_nodes_label="purpose=infra" Actual results: TASK [Map labelled nodes to inventory hosts] ********************************************************************************************************************************************************************************************* skipping: [master1-prod.example.com] => (item=master1-prod.example.com) skipping: [master1-prod.example.com] => (item=master2-prod.example.com) skipping: [master1-prod.example.com] => (item=master3-prod.example.com) ok: [master1-prod.example.com] => (item=inode1-prod.example.com) ok: [master1-prod.example.com] => (item=inode2-prod.example.com) ok: [master1-prod.example.com] => (item=inode3-prod.example.com) skipping: [master1-prod.example.com] => (item=node1-prod.example.com) skipping: [master1-prod.example.com] => (item=node2-prod.example.com) fatal: [master1-prod.example.com]: FAILED! => { "failed": true } MSG: The conditional check 'hostvars[item].openshift.common.hostname in nodes_to_upgrade.results.results[0]['items'] | map(attribute='metadata.name') | list' failed. The error was: error while evaluating conditional (hostvars[item].openshift.common.hostname in nodes_to_upgrade.results.results[0]['items'] | map(attribute='metadata.name') | list): 'dict object' has no attribute 'openshift' The error appears to have been in '/usr/share/ansible/openshift-ansible/playbooks/common/openshift-cluster/upgrades/initialize_nodes_to_upgrade.yml': line 25, column 7, but may be elsewhere in the file depending on the exact syntax problem. The offending line appears to be: # using their openshift.common.hostname fact. - name: Map labelled nodes to inventory hosts ^ here Expected results: Target nodes to be successfully upgraded. Additional info: Please attach logs from ansible-playbook with the -vvv flag