Bug 1492786 - Installer fails at Create OpenShift router step
Summary: Installer fails at Create OpenShift router step
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 3.6.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 3.7.0
Assignee: Andrew Butcher
QA Contact: Johnny Liu
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-09-18 15:43 UTC by Johan Swensson
Modified: 2017-11-28 22:11 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-11-28 22:11:25 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
ansible.log (903.77 KB, application/x-gzip)
2017-09-18 15:43 UTC, Johan Swensson
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2017:3188 0 normal SHIPPED_LIVE Moderate: Red Hat OpenShift Container Platform 3.7 security, bug, and enhancement update 2017-11-29 02:34:54 UTC

Description Johan Swensson 2017-09-18 15:43:48 UTC
Created attachment 1327518 [details]
ansible.log

Description of problem:
Installation of OCP fails when giving a wildcard certificate to the installer.


Version-Release number of the following components:

openshift-ansible-3.6.173.0.21-2.git.0.44a4038.el7.noarch
ansible-2.3.2.0-2.el7.noarch

How reproducible:
Every time


Steps to Reproduce:
1. Deploy OCP with openshift_hosted_router_certificate={"certfile": "/root/wildcard.apps.lab.kupo.se.crt", "keyfile": "/root/wildcard.apps.lab.kupo.se.key", 'cafile': '/root/ca-chain.pem' }

2
Actual results:

TASK [openshift_hosted : Create OpenShift router] ************************************************************************************************************
Monday 18 September 2017  09:28:38 +0000 (0:00:01.519)       1:07:19.473 ****** 
failed: [master1.lab.kupo.se] (item={u'name': u'router', u'certificate': {'keyfile': u'/root/wildcard.apps.lab.kupo.se.key', 'certfile': u'/root/wildcard.apps.lab.kupo.se.crt', 'cafile': u'/root/ca-chain.pem'}, u'replicas': u'3', u'serviceaccount': u'router', u'namespace': u'default', u'stats_port': 1936, u'edits': [{u'action': u'put', u'value': 1, u'key': u'spec.strategy.rollingParams.intervalSeconds'}, {u'action': u'put', u'value': 1, u'key': u'spec.strategy.rollingParams.updatePeriodSeconds'}, {u'action': u'put', u'value': 21600, u'key': u'spec.strategy.activeDeadlineSeconds'}], u'images': u'registry.lab.kupo.se:5000/openshift3/ose-${component}:${version}', u'selector': u'infra=true', u'ports': [u'80:80', u'443:443']}) => {
    "failed": true, 
    "item": {
        "certificate": {
            "cafile": "/root/ca-chain.pem", 
            "certfile": "/root/wildcard.apps.lab.kupo.se.crt", 
            "keyfile": "/root/wildcard.apps.lab.kupo.se.key"
        }, 
        "edits": [
            {
                "action": "put", 
                "key": "spec.strategy.rollingParams.intervalSeconds", 
                "value": 1
            }, 
            {
                "action": "put", 
                "key": "spec.strategy.rollingParams.updatePeriodSeconds", 
                "value": 1
            }, 
            {
                "action": "put", 
                "key": "spec.strategy.activeDeadlineSeconds", 
                "value": 21600
            }
        ], 
        "images": "registry.lab.kupo.se:5000/openshift3/ose-${component}:${version}", 
        "name": "router", 
        "namespace": "default", 
        "ports": [
            "80:80", 
            "443:443"
        ], 
        "replicas": "3", 
        "selector": "infra=true", 
        "serviceaccount": "router", 
        "stats_port": 1936
    }, 
    "module_stderr": "Shared connection to master1.lab.kupo.se closed.\r\n", 
    "module_stdout": "Traceback (most recent call last):\r\n  File \"/tmp/ansible_FoO2lC/ansible_module_oc_adm_router.py\", line 3185, in <module>\r\n    main()\r\n  File \"/tmp/ansible_FoO2lC/ansible_module_oc_adm_router.py\", line 3176, in main\r\n    results = Router.run_ansible(module.params, module.check_mode)\r\n  File \"/tmp/ansible_FoO2lC/ansible_module_oc_adm_router.py\", line 3093, in run_ansible\r\n    api_rval = ocrouter.create()\r\n  File \"/tmp/ansible_FoO2lC/ansible_module_oc_adm_router.py\", line 2895, in create\r\n    self.needs_update()\r\n  File \"/tmp/ansible_FoO2lC/ansible_module_oc_adm_router.py\", line 2940, in needs_update\r\n    not Utils.check_def_equal(self.prepared_router['ServiceAccount']['obj'].yaml_dict,\r\n  File \"/tmp/ansible_FoO2lC/ansible_module_oc_adm_router.py\", line 2665, in prepared_router\r\n    results = self._prepare_router()\r\n  File \"/tmp/ansible_FoO2lC/ansible_module_oc_adm_router.py\", line 2825, in _prepare_router\r\n    rfd.write(open(self.config.config_options['cert_file']['value']).read())\r\nIOError: [Errno 2] No such file or directory: '/etc/origin/master/wildcard.apps.lab.kupo.se.crt'\r\n", 
    "rc": 0
}

MSG:

MODULE FAILURE


Expected results:
The installer should copy the file to the needed location.

Additional info:
The same inventory file worked with v3.6.173.0.5

Comment 1 Kenny Woodson 2017-09-18 19:44:53 UTC
Johan,

Thanks for filing your issue.  I have a few questions so that we can get on the same page.

1. Do you have any further logs from the installation regarding the openshift_hosted role and the router.yml installation?

The previous step is what copies the certificates from the controller host onto the server that is receiving openshift.  These are the lines I'm referring to:
https://github.com/openshift/openshift-ansible/blob/master/roles/openshift_hosted/tasks/router/router.yml#L21-L28

Could you find the logs for this step?  Was this step successful?

2.  The error from the router explains that it was looking for the certificate but it was not copied to the destination upon where the oc_adm_router module was expecting it to be (/etc/origin/master/wildcard.apps.lab.kupo.se.crt)  Could you verify the contents of /etc/origin/master/ to see if the certificates are where they were copied from step #1?  This will help us know if they copy step was successful.  

3. Are the following inventory variables set?
openshift_hosted_router_create_certificate
openshift_hosted_router_certificate


Thanks.

Comment 2 Johan Swensson 2017-09-18 20:17:41 UTC
(In reply to Kenny Woodson from comment #1)
> Johan,
> 
> Thanks for filing your issue.  I have a few questions so that we can get on
> the same page.
> 
> 1. Do you have any further logs from the installation regarding the
> openshift_hosted role and the router.yml installation?
> 
> The previous step is what copies the certificates from the controller host
> onto the server that is receiving openshift.  These are the lines I'm
> referring to:
> https://github.com/openshift/openshift-ansible/blob/master/roles/
> openshift_hosted/tasks/router/router.yml#L21-L28
> 
> Could you find the logs for this step?  Was this step successful?
> 

Is this the one you're looking for? Seems like it skips the step for some reason.


2017-09-18 15:32:14,215 p=6001 u=root |  TASK [openshift_hosted : Get the certificate contents for router] ********************************************************************************************
2017-09-18 15:32:14,215 p=6001 u=root |  task path: /usr/share/ansible/openshift-ansible/roles/openshift_hosted/tasks/router/router.yml:42
2017-09-18 15:32:14,216 p=6001 u=root |  Monday 18 September 2017  15:32:14 +0000 (0:00:00.069)       0:53:16.629 ****** 
2017-09-18 15:32:14,276 p=6001 u=root |  skipping: [master1.lab.kupo.se] => (item=/root/wildcard.apps.lab.kupo.se.key)  => {
    "changed": false, 
    "item": "/root/wildcard.apps.lab.kupo.se.key", 
    "skip_reason": "Conditional result was False", 
    "skipped": true
}
2017-09-18 15:32:14,294 p=6001 u=root |  skipping: [master1.lab.kupo.se] => (item=/root/ca-chain.pem)  => {
    "changed": false, 
    "item": "/root/ca-chain.pem", 
    "skip_reason": "Conditional result was False", 
    "skipped": true
}
2017-09-18 15:32:14,295 p=6001 u=root |  skipping: [master1.lab.kupo.se] => (item=/root/wildcard.apps.lab.kupo.se.crt)  => {
    "changed": false, 
    "item": "/root/wildcard.apps.lab.kupo.se.crt", 
    "skip_reason": "Conditional result was False", 
    "skipped": true
}


> 2.  The error from the router explains that it was looking for the
> certificate but it was not copied to the destination upon where the
> oc_adm_router module was expecting it to be
> (/etc/origin/master/wildcard.apps.lab.kupo.se.crt)  Could you verify the
> contents of /etc/origin/master/ to see if the certificates are where they
> were copied from step #1?  This will help us know if they copy step was
> successful.  
> 

They are not there, as they were skipped in the previous step.


> 3. Are the following inventory variables set?
> openshift_hosted_router_create_certificate
> openshift_hosted_router_certificate
> 

openshift_hosted_router_certificate is set, openshift_hosted_router_create_certificate is not set.

Comment 3 Andrew Butcher 2017-09-19 18:09:26 UTC
Proposed fix: https://github.com/openshift/openshift-ansible/pull/5465

Comment 5 Johnny Liu 2017-10-13 08:14:16 UTC
Reproduce this bug with a rpm install using openshift-ansible-3.7.0-0.125.0.git.0.91043b6.el7.noarch, verified this bug with openshift-ansible-3.7.0-0.143.3.git.0.6c65767.el7.noarch, and PASS.

Comment 9 errata-xmlrpc 2017-11-28 22:11:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:3188


Note You need to log in before you can comment on or make changes to this bug.