Bug 1565201

Summary: 3.10.0-0.16.0: master services fail to start - can't load openshift-ansible generated master-config.yaml
Product: OpenShift Container Platform Reporter: Mike Fiedler <mifiedle>
Component: InstallerAssignee: Vadim Rutkovsky <vrutkovs>
Status: CLOSED ERRATA QA Contact: Mike Fiedler <mifiedle>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 3.10.0CC: aos-bugs, jokerman, mmccomas, vrutkovs, wabouham, wmeng, wsun
Target Milestone: ---   
Target Release: 3.10.0   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-07-30 19:12:20 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Redacted inventory
none
master-config.yaml
none
ansible log none

Description Mike Fiedler 2018-04-09 15:21:12 UTC 
Description of problem:   On 3.10.0-0.16.0 the master-api pod crashes immediately with this as the only line in the log.

F0409 15:04:49.386515       1 start_api.go:68] could not load config file "/etc/origin/master/master-config.yaml" due to an error: error reading config: [pos 3159]: only encoded map or array can be decoded into a slice (6)

Starting this with installation, please transfer if it is a master problem.


Version-Release number of the following components:
root@ip-172-31-14-26: ~ # rpm -q openshift-ansible                    
openshift-ansible-3.9.14-1.git.0.ca2cfc3.el7.noarch                   
root@ip-172-31-14-26: ~ # rpm -q ansible                              
ansible-2.4.3.0-1.el7ae.noarch     
root@ip-172-31-14-26: ~ # ansible --version                           
ansible 2.4.3.0                    
  config file = /etc/ansible/ansible.cfg                              
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']                                 
  ansible python module location = /usr/lib/python2.7/site-packages/ansible                                                                  
  executable location = /usr/bin/ansible                              
  python version = 2.7.5 (default, Feb 20 2018, 09:19:12) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] 

How reproducible:  Always, for this inventory

Steps to Reproduce:
1. Install v3.10.0-0.16.0 against an inventory equivalent to the one attached.


Actual results:
Please include the entire output from the last TASK line through the end of output if an error is generated

Expected results:

Additional info:
Please attach logs from ansible-playbook with the -vvv flag
Comment 1 Mike Fiedler 2018-04-09 15:21:42 UTC 
Created attachment 1419367 [details]
Redacted inventory
Comment 2 Mike Fiedler 2018-04-09 15:22:16 UTC 
Created attachment 1419368 [details]
master-config.yaml
Comment 3 Mike Fiedler 2018-04-09 15:22:43 UTC 
Created attachment 1419369 [details]
ansible log
Comment 4 Mike Fiedler 2018-04-09 15:37:19 UTC 
master controllers fails too:

F0409 15:35:48.383845       1 start_controllers.go:67] could not load config file "/etc/origin/master/master-config.yaml" due to an error: error reading config: [pos 3159]: only encoded map or array can be decoded into a slice (6)
Comment 5 Mike Fiedler 2018-04-10 15:30:00 UTC 
The offending entry is "disable-attach-detach-reconcile-sync: 'true'"  in the section below.   Trying to work around it with explicit config.

kubernetesMasterConfig:
  apiServerArguments:
    cloud-config:
    - /etc/origin/cloudprovider/aws.conf
    cloud-provider:
    - aws
    storage-backend:
    - etcd3
    storage-media-type:
    - application/vnd.kubernetes.protobuf
  controllerArguments:
    cloud-config:
    - /etc/origin/cloudprovider/aws.conf
    cloud-provider:
    - aws
    cluster-signing-cert-file:
    - /etc/origin/master/ca.crt
    cluster-signing-key-file:
    - /etc/origin/master/ca.key
    disable-attach-detach-reconcile-sync: 'true'
Comment 6 Mike Fiedler 2018-04-10 15:35:20 UTC 
Correct syntax is 

disable-attach-detach-reconcile-sync: 
- 'true'
Comment 7 Mike Fiedler 2018-04-10 18:47:45 UTC 
Fixed by https://github.com/openshift/openshift-ansible/pull/7854

The branch in that PR can be used for AWS installs.
Comment 8 Vadim Rutkovsky 2018-04-13 08:24:59 UTC 
Fix is available in openshift-ansible-3.10.0-0.21.0
Comment 9 Mike Fiedler 2018-04-19 11:58:23 UTC 
Verified on 3.10.0-0.22.0
Comment 11 errata-xmlrpc 2018-07-30 19:12:20 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:1816