Bug 1549454
| Summary: | Etcd scale-up failed when running as system container on RHEL | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Gaoyun Pei <gpei> |
| Component: | Installer | Assignee: | Vadim Rutkovsky <vrutkovs> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Gaoyun Pei <gpei> |
| Severity: | low | Docs Contact: | |
| Priority: | low | ||
| Version: | 3.9.0 | CC: | aos-bugs, jokerman, mmccomas, smunilla, vrutkovs |
| Target Milestone: | --- | ||
| Target Release: | 3.9.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | openshift-ansible-3.9.2-1.git.0.1a855b3.el7.noarch | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2018-06-18 18:14:23 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
etcd playbooks use `etcdctl` when the installation is not atomic, I'll prepare a fix Fix is available in openshift-ansible-3.9.2-1.git.0.1a855b3.el7 Verify this bug with openshift-ansible-3.9.2-1.git.0.1a855b3.el7.noarch.rpm
Verify cluster step passed during the scale-up.
TASK [Verify cluster is stable] ************************************************
changed: [qe-gpei-auto-yunr-rhel-1.0305-t81.qe.rhcloud.com] => {"attempts": 1, "changed": true, "cmd": ["runc", "exec", "etcd", "etcdctl", "--cert-file", "/etc/etcd/peer.crt", "--key-file", "/etc/etcd/peer.key", "--ca-file", "/etc/etcd/ca.crt", "-C", "https://qe-gpei-auto-atfs-master-etcd-1:2379", "cluster-health"], "delta": "0:00:00.088975", "end": "2018-03-05 10:24:28.002649", "failed": false, "rc": 0, "start": "2018-03-05 10:24:27.913674", "stderr": "", "stderr_lines": [], "stdout": "member 3b2cb16416a55304 is healthy: got healthy result from https://10.240.0.215:2379\nmember eff658dbd0dd1181 is healthy: got healthy result from https://10.240.0.213:2379\ncluster is healthy", "stdout_lines": ["member 3b2cb16416a55304 is healthy: got healthy result from https://10.240.0.215:2379", "member eff658dbd0dd1181 is healthy: got healthy result from https://10.240.0.213:2379", "cluster is healthy"]}
|
Description of problem: Etcd scale-up playbook failed when trying to check cluster healthy on the first new etcd host: TASK [Verify cluster is stable] ************************************************ FAILED - RETRYING: Verify cluster is stable (3 retries left). FAILED - RETRYING: Verify cluster is stable (2 retries left). FAILED - RETRYING: Verify cluster is stable (1 retries left). fatal: [qe-gpei-auto-atqd-rhel-1.0227-r5a.qe.rhcloud.com]: FAILED! => {"attempts": 3, "changed": false, "cmd": "/usr/bin/etcdctl --cert-file /etc/etcd/peer.crt --key-file /etc/etcd/peer.key --ca-file /etc/etcd/ca.crt -C https://qe-gpei-auto-anwg-master-etcd-1:2379 cluster-health", "msg": "[Errno 2] No such file or directory", "rc": 2} No etcd package installed on etcd hosts, so no etcdctl command available. Version-Release number of the following components: openshift-ansible-3.9.0-0.53.0.git.0.f8f01ef.el7.noarch.rpm ansible-2.4.3.0-1.el7ae.noarch How reproducible: Always Steps to Reproduce: 1.Add two new etcd hosts under [new_etcd] group, run etcd scale-up playbook ansible-playbook playbooks/openshift-etcd/scaleup.yml -v Actual results: Expected results: Additional info: The full output will be attached.