Bug 1549454

Summary: Etcd scale-up failed when running as system container on RHEL
Product: OpenShift Container Platform Reporter: Gaoyun Pei <gpei>
Component: InstallerAssignee: Vadim Rutkovsky <vrutkovs>
Status: CLOSED CURRENTRELEASE QA Contact: Gaoyun Pei <gpei>
Severity: low Docs Contact:
Priority: low    
Version: 3.9.0CC: aos-bugs, jokerman, mmccomas, smunilla, vrutkovs
Target Milestone: ---   
Target Release: 3.9.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openshift-ansible-3.9.2-1.git.0.1a855b3.el7.noarch Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-06-18 18:14:23 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Gaoyun Pei 2018-02-27 06:57:06 UTC
Description of problem:
Etcd scale-up playbook failed when trying to check cluster healthy on the first new etcd host:

TASK [Verify cluster is stable] ************************************************
FAILED - RETRYING: Verify cluster is stable (3 retries left).
FAILED - RETRYING: Verify cluster is stable (2 retries left).
FAILED - RETRYING: Verify cluster is stable (1 retries left).
fatal: [qe-gpei-auto-atqd-rhel-1.0227-r5a.qe.rhcloud.com]: FAILED! => {"attempts": 3, "changed": false, "cmd": "/usr/bin/etcdctl --cert-file /etc/etcd/peer.crt --key-file /etc/etcd/peer.key --ca-file /etc/etcd/ca.crt -C https://qe-gpei-auto-anwg-master-etcd-1:2379 cluster-health", "msg": "[Errno 2] No such file or directory", "rc": 2}

No etcd package installed on etcd hosts, so no etcdctl command available.


Version-Release number of the following components:
openshift-ansible-3.9.0-0.53.0.git.0.f8f01ef.el7.noarch.rpm
ansible-2.4.3.0-1.el7ae.noarch

How reproducible:
Always

Steps to Reproduce:
1.Add two new etcd hosts under [new_etcd] group, run etcd scale-up playbook
ansible-playbook playbooks/openshift-etcd/scaleup.yml -v


Actual results:

Expected results:

Additional info:
The full output will be attached.

Comment 2 Vadim Rutkovsky 2018-02-28 12:08:19 UTC
etcd playbooks use `etcdctl` when the installation is not atomic, I'll prepare a fix

Comment 3 Vadim Rutkovsky 2018-03-01 12:32:17 UTC
Created https://github.com/openshift/openshift-ansible/pull/7340

Comment 4 Vadim Rutkovsky 2018-03-05 11:03:17 UTC
Fix is available in openshift-ansible-3.9.2-1.git.0.1a855b3.el7

Comment 5 Gaoyun Pei 2018-03-06 03:11:56 UTC
Verify this bug with openshift-ansible-3.9.2-1.git.0.1a855b3.el7.noarch.rpm

Verify cluster step passed during the scale-up. 

TASK [Verify cluster is stable] ************************************************
changed: [qe-gpei-auto-yunr-rhel-1.0305-t81.qe.rhcloud.com] => {"attempts": 1, "changed": true, "cmd": ["runc", "exec", "etcd", "etcdctl", "--cert-file", "/etc/etcd/peer.crt", "--key-file", "/etc/etcd/peer.key", "--ca-file", "/etc/etcd/ca.crt", "-C", "https://qe-gpei-auto-atfs-master-etcd-1:2379", "cluster-health"], "delta": "0:00:00.088975", "end": "2018-03-05 10:24:28.002649", "failed": false, "rc": 0, "start": "2018-03-05 10:24:27.913674", "stderr": "", "stderr_lines": [], "stdout": "member 3b2cb16416a55304 is healthy: got healthy result from https://10.240.0.215:2379\nmember eff658dbd0dd1181 is healthy: got healthy result from https://10.240.0.213:2379\ncluster is healthy", "stdout_lines": ["member 3b2cb16416a55304 is healthy: got healthy result from https://10.240.0.215:2379", "member eff658dbd0dd1181 is healthy: got healthy result from https://10.240.0.213:2379", "cluster is healthy"]}