Bug 1628201

Summary: OCP 3.11: etcd scaleup on CRI-O HA cluster fails with dict object has no attribute etcd_ip error
Product: OpenShift Container Platform Reporter: Russell Teague <rteague>
Component: InstallerAssignee: Russell Teague <rteague>
Status: CLOSED ERRATA QA Contact: ge liu <geliu>
Severity: high Docs Contact:
Priority: medium    
Version: 3.11.0CC: aos-bugs, gpei, jialiu, jkaur, jokerman, mifiedle, mmccomas, sdodson, vlaad, vrutkovs, wabouham, wmeng
Target Milestone: ---Keywords: NeedsTestCase
Target Release: 3.11.z   
Hardware: x86_64   
OS: Linux   
Whiteboard: aos-scalability-310
Fixed In Version: Doc Type: Bug Fix
Doc Text:
During etcd scaleup, facts about the etcd cluster are required in order to add new hosts. The necessary tasks have been added to ensure those facts are set before configuring new hosts and therefore allow the scaleup to complete as expected.
Story Points: ---
Clone Of: 1578482 Environment:
Last Closed: 2019-01-10 09:03:58 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Bug Depends On: 1578482    
Bug Blocks:    

Comment 1 Russell Teague 2018-09-12 12:56:09 UTC
release-3.11: https://github.com/openshift/openshift-ansible/pull/10021

Comment 2 Russell Teague 2018-09-13 11:59:51 UTC
openshift-ansible-3.11.2-1

Comment 3 Gaoyun Pei 2018-09-14 09:39:43 UTC
Test with openshift-ansible-3.11.5-1.git.0.5a01a3c.el7_5.noarch and ansible-2.6.4-1.el7ae.noarch, etcd scale-up playbook failed as below:

TASK [Verify cluster is stable] ************************************************
fatal: [ec2-54-208-238-208.compute-1.amazonaws.com]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'etcd_ca_host' is undefined\n\nThe error appears to have been in '/home/slave4/workspace/Run-Ansible-Playbooks/private-openshift-ansible/playbooks/openshift-etcd/private/scaleup.yml': line 48, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n  # etcd_hostname fact is set in add_new_member.yml called above.\n  - name: Verify cluster is stable\n    ^ here\n"}

Comment 5 Russell Teague 2018-09-20 12:13:38 UTC
Proposed: https://github.com/openshift/openshift-ansible/pull/10149 (master)

Comment 7 Scott Dodson 2018-10-04 14:01:51 UTC
https://github.com/openshift/openshift-ansible/pull/10258 release-3.11 backport

Comment 8 Vadim Rutkovsky 2018-10-15 09:21:15 UTC
Fix is available in openshift-ansible-3.11.23-1

Comment 9 ge liu 2018-10-24 09:04:42 UTC
Verified with openshift-ansible-3.11.28-1,

1). Setup ha env on aws(3 masters+1 etcd), then run scaleup playbook to setup other 2 etcd, and it works well

2). Setup env(1 master+1 etcd), prepare 2 vms, then scalup install etcd on those 2 vms, and it works well

Comment 11 errata-xmlrpc 2019-01-10 09:03:58 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0024