Bug 1564098

Summary: etcd3 migrate playbook fails with single master topology due to handler error
Product: OpenShift Container Platform Reporter: Kenjiro Nakayama <knakayam>
Component: InstallerAssignee: Vadim Rutkovsky <vrutkovs>
Status: CLOSED ERRATA QA Contact: Gaoyun Pei <gpei>
Severity: high Docs Contact:
Priority: unspecified    
Version: 3.7.0CC: aos-bugs, jokerman, mmccomas, wmeng, xtian
Target Milestone: ---   
Target Release: 3.7.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-06-27 07:59:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Kenjiro Nakayama 2018-04-05 11:14:18 UTC 
Description of problem:
- When upgrade single master to 3.7,
- It is a same error with bz#1523814, but task is different (this ticket is talking about handler.)

Version-Release number of the following components:
  openshift-ansible-callback-plugins-3.7.23-1.git.0.bc406aa.el7.noarch
  ansible-2.3.1.0-1.el7.noarch
  openshift-ansible-filter-plugins-3.7.23-1.git.0.bc406aa.el7.noarch
  openshift-ansible-roles-3.7.23-1.git.0.bc406aa.el7.noarch
  openshift-ansible-lookup-plugins-3.7.23-1.git.0.bc406aa.el7.noarch
  openshift-ansible-3.7.23-1.git.0.bc406aa.el7.noarch
  openshift-ansible-docs-3.7.23-1.git.0.bc406aa.el7.noarch
  openshift-ansible-playbooks-3.7.23-1.git.0.bc406aa.el7.noarch

Steps to Reproduce:
  1. upgrade openshift-ansible* packages to 3.7
  2. externized embedded etcd per [1]
  3. migrate etcd data v2 to v3

* Steps are described in - https://docs.openshift.com/container-platform/3.7/upgrading/automated_upgrades.html

Actual results:

(upload full logs as private attachment.)

  TASK [openshift_master : yedit] *******************************************************************************************************************************************************************************
  changed: [xxx.example.com]

  RUNNING HANDLER [openshift_master : restart master api] *******************************************************************************************************************************************************
  fatal: [xxx.example.com]: FAILED! => {"changed": false, "failed": true, "msg": "Could not find the requested service atomic-openshift-master-api: host"}

  RUNNING HANDLER [openshift_master : restart master controllers] ***********************************************************************************************************************************************
          to retry, use: --limit @/usr/share/ansible/openshift-ansible/playbooks/byo/openshift-etcd/migrate.retry

  PLAY RECAP ****************************************************************************************************************************************************************************************************
  localhost                  : ok=14   changed=0    unreachable=0    failed=0
  xxx.example.com : ok=94   changed=19   unreachable=0    failed=1
  ... snip ...

Expected results:
- No error
Comment 4 Kenjiro Nakayama 2018-04-05 11:24:10 UTC 
I think that bz#1523814 did not catch this issue, because openshift-ansible-3.7.18-1.git.0.a01e769.el7.noarch and ansible-2.4 were used for the test.

`static: true` was introduced by https://github.com/openshift/openshift-ansible/pull/6613 after a01e769.
Comment 7 Vadim Rutkovsky 2018-05-07 09:28:13 UTC 
Fix is available in openshift-ansible-3.7.46-1
Comment 8 Gaoyun Pei 2018-05-09 08:21:28 UTC 
Verify this bug with openshift-ansible-3.7.46-1.git.0.37f607e.el7.noarch and ansible-2.4.4.0-1.el7ae.noarch.


Setup an ocp-3.6 cluster using etcd v2 data. Migrate etcd v2 to v3 before running upgrade to 3.7. The migration and upgrade both passed.
Comment 10 errata-xmlrpc 2018-06-27 07:59:12 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:2009