Bug 1565201 - 3.10.0-0.16.0: master services fail to start - can't load openshift-ansible generated master-config.yaml
Summary: 3.10.0-0.16.0: master services fail to start - can't load openshift-ansible g...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 3.10.0
Hardware: x86_64
OS: Linux
unspecified
urgent
Target Milestone: ---
: 3.10.0
Assignee: Vadim Rutkovsky
QA Contact: Mike Fiedler
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-04-09 15:21 UTC by Mike Fiedler
Modified: 2018-07-30 19:12 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-07-30 19:12:20 UTC
Target Upstream Version:


Attachments (Terms of Use)
Redacted inventory (3.54 KB, text/plain)
2018-04-09 15:21 UTC, Mike Fiedler
no flags Details
master-config.yaml (6.01 KB, text/plain)
2018-04-09 15:22 UTC, Mike Fiedler
no flags Details
ansible log (780.42 KB, text/plain)
2018-04-09 15:22 UTC, Mike Fiedler
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:1816 None None None 2018-07-30 19:12:47 UTC

Description Mike Fiedler 2018-04-09 15:21:12 UTC
Description of problem:   On 3.10.0-0.16.0 the master-api pod crashes immediately with this as the only line in the log.

F0409 15:04:49.386515       1 start_api.go:68] could not load config file "/etc/origin/master/master-config.yaml" due to an error: error reading config: [pos 3159]: only encoded map or array can be decoded into a slice (6)

Starting this with installation, please transfer if it is a master problem.


Version-Release number of the following components:
root@ip-172-31-14-26: ~ # rpm -q openshift-ansible                    
openshift-ansible-3.9.14-1.git.0.ca2cfc3.el7.noarch                   
root@ip-172-31-14-26: ~ # rpm -q ansible                              
ansible-2.4.3.0-1.el7ae.noarch     
root@ip-172-31-14-26: ~ # ansible --version                           
ansible 2.4.3.0                    
  config file = /etc/ansible/ansible.cfg                              
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']                                 
  ansible python module location = /usr/lib/python2.7/site-packages/ansible                                                                  
  executable location = /usr/bin/ansible                              
  python version = 2.7.5 (default, Feb 20 2018, 09:19:12) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] 

How reproducible:  Always, for this inventory

Steps to Reproduce:
1. Install v3.10.0-0.16.0 against an inventory equivalent to the one attached.


Actual results:
Please include the entire output from the last TASK line through the end of output if an error is generated

Expected results:

Additional info:
Please attach logs from ansible-playbook with the -vvv flag

Comment 1 Mike Fiedler 2018-04-09 15:21:42 UTC
Created attachment 1419367 [details]
Redacted inventory

Comment 2 Mike Fiedler 2018-04-09 15:22:16 UTC
Created attachment 1419368 [details]
master-config.yaml

Comment 3 Mike Fiedler 2018-04-09 15:22:43 UTC
Created attachment 1419369 [details]
ansible log

Comment 4 Mike Fiedler 2018-04-09 15:37:19 UTC
master controllers fails too:

F0409 15:35:48.383845       1 start_controllers.go:67] could not load config file "/etc/origin/master/master-config.yaml" due to an error: error reading config: [pos 3159]: only encoded map or array can be decoded into a slice (6)

Comment 5 Mike Fiedler 2018-04-10 15:30:00 UTC
The offending entry is "disable-attach-detach-reconcile-sync: 'true'"  in the section below.   Trying to work around it with explicit config.

kubernetesMasterConfig:
  apiServerArguments:
    cloud-config:
    - /etc/origin/cloudprovider/aws.conf
    cloud-provider:
    - aws
    storage-backend:
    - etcd3
    storage-media-type:
    - application/vnd.kubernetes.protobuf
  controllerArguments:
    cloud-config:
    - /etc/origin/cloudprovider/aws.conf
    cloud-provider:
    - aws
    cluster-signing-cert-file:
    - /etc/origin/master/ca.crt
    cluster-signing-key-file:
    - /etc/origin/master/ca.key
    disable-attach-detach-reconcile-sync: 'true'

Comment 6 Mike Fiedler 2018-04-10 15:35:20 UTC
Correct syntax is 

disable-attach-detach-reconcile-sync: 
- 'true'

Comment 7 Mike Fiedler 2018-04-10 18:47:45 UTC
Fixed by https://github.com/openshift/openshift-ansible/pull/7854

The branch in that PR can be used for AWS installs.

Comment 8 Vadim Rutkovsky 2018-04-13 08:24:59 UTC
Fix is available in openshift-ansible-3.10.0-0.21.0

Comment 9 Mike Fiedler 2018-04-19 11:58:23 UTC
Verified on 3.10.0-0.22.0

Comment 11 errata-xmlrpc 2018-07-30 19:12:20 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:1816


Note You need to log in before you can comment on or make changes to this bug.