Bug 1586008
| Summary: | "Wait for all control plane pods to become ready" task failed when etcd is not co-located with master | ||||||
|---|---|---|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Johnny Liu <jialiu> | ||||
| Component: | Installer | Assignee: | Scott Dodson <sdodson> | ||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Johnny Liu <jialiu> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | high | ||||||
| Version: | 3.10.0 | CC: | aos-bugs, jokerman, mmccomas | ||||
| Target Milestone: | --- | Keywords: | Regression | ||||
| Target Release: | 3.10.0 | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | No Doc Update | |||||
| Doc Text: |
undefined
|
Story Points: | --- | ||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2018-06-18 17:03:40 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
Need to filter for etcd hosts that are also masters. Correction, need to stop using omit. Verified this bug with openshift-ansible-3.10.0-0.63.0.git.0.961c60d.el7.noarch, and PASS.
TASK [openshift_control_plane : Wait for all control plane pods to become ready] ***
Wednesday 06 June 2018 22:11:11 -0400 (0:00:00.061) 0:26:28.607 ********
skipping: [ec2-52-90-178-192.compute-1.amazonaws.com] => (item=) => {"changed": false, "item": "", "skip_reason": "Conditional result was False", "skipped": true}
skipping: [ec2-75-101-190-167.compute-1.amazonaws.com] => (item=) => {"changed": false, "item": "", "skip_reason": "Conditional result was False", "skipped": true}
skipping: [ec2-54-242-184-100.compute-1.amazonaws.com] => (item=) => {"changed": false, "item": "", "skip_reason": "Conditional result was False", "skipped": true}
<--snip-->
only check api and controller pod, etcd pod is skipped now.
|
Created attachment 1447786 [details] installation log with inventory file embedded Description of problem: In 93a2fcd9 commit, we have such code is merged: - name: Wait for all control plane pods to become ready oc_obj: state: list kind: pod name: "master-{{ item }}-{{ openshift.node.nodename | lower }}" namespace: kube-system <--snip--> retries: 60 delay: 5 with_items: - "{{ 'etcd' if inventory_hostname in groups['oo_etcd_to_config'] else omit }}" - api - controllers when etcd is not co-located with master, the "omit" would return some invalid value, which lead to the installation failure. Version-Release number of the following components: openshift-ansible-3.10.0-0.58.0.git.0.d8f6377.el7.noarch How reproducible: Always Steps to Reproduce: 1. define etcd group and master group, they are not co-located together. 2. trigger installation 3. Actual results: TASK [openshift_control_plane : Wait for all control plane pods to become ready] *** <--snip--> FAILED - RETRYING: Wait for all control plane pods to become ready (1 retries left). failed: [ec2-54-146-104-127.compute-1.amazonaws.com] (item=__omit_place_holder__ee64c3e15fab51456d4a1fd4ea054f1dc27b73d6) => {"attempts": 60, "changed": false, "failed": true, "item": "__omit_place_holder__ee64c3e15fab51456d4a1fd4ea054f1dc27b73d6", "results": {"cmd": "/usr/local/bin/oc get pod master-__omit_place_holder__ee64c3e15fab51456d4a1fd4ea054f1dc27b73d6-ip-172-18-6-122.ec2.internal -o json -n kube-system", "results": [{}], "returncode": 0, "stderr": "Error from server (NotFound): pods \"master-__omit_place_holder__ee64c3e15fab51456d4a1fd4ea054f1dc27b73d6-ip-172-18-6-122.ec2.internal\" not found\n", "stdout": ""}, "state": "list"} <--snip--> Expected results: installation should be passed. Additional info: Please attach logs from ansible-playbook with the -vvv flag