Red Hat Bugzilla – Bug 1466727
broker resource fail to be created due to catalog api server is connecting a wrong etcd server url
Last modified: 2017-08-16 15:51 EDT
Description of problem: see the following for details. Version-Release number of selected component (if applicable): openshift-ansible-3.6.126.4-1.git.0.d25d828.el7 How reproducible: Always Steps to Reproduce: 1. prepare inventory host file with a external etcd server for cluster install 2. enable catalog service deployment with the following options in inventory host file: openshift_hosted_etcd_storage_kind=nfs openshift_hosted_etcd_storage_nfs_options="*(rw,root_squash,sync,no_wdelay)" openshift_hosted_etcd_storage_nfs_directory=/exports openshift_hosted_etcd_storage_volume_name=etcd openshift_hosted_etcd_storage_access_modes=["ReadWriteOnce"] openshift_hosted_etcd_storage_volume_size=10G openshift_hosted_etcd_storage_labels={'storage': 'etcd'} openshift_enable_service_catalog=true openshift_service_catalog_image_prefix=docker.io/openshift/origin- openshift_service_catalog_image_version=latest ansible_service_broker_image_prefix=ansibleplaybookbundle/ ansible_service_broker_image_tag=latest ansible_service_broker_etcd_image_prefix=quay.io/coreos/ ansible_service_broker_etcd_image_tag=latest 3. trigger installation 4. Actual results: Job failed at the following task: TASK [ansible_service_broker : Create the Broker resource in the catalog] ****** Friday 30 June 2017 07:49:22 +0000 (0:00:01.958) 1:04:18.052 *********** fatal: [openshift-155.lab.sjc.redhat.com]: FAILED! => { "changed": false, "failed": true } MSG: {u'returncode': 1, u'cmd': u'/usr/bin/oc create -f /tmp/brokerout-xPp27e -n default', u'results': {}, u'stderr': u'Error from server (Timeout): error when creating "/tmp/brokerout-xPp27e": Timeout: request did not complete within allowed duration\n', u'stdout': u''} After the above failure, log into cluster for a little debug. 1. "oc get broker" will hang there. 2. api server and controller-manager pod are running. # oc get po -n kube-service-catalog NAME READY STATUS RESTARTS AGE apiserver-9wwpf 1/1 Running 0 46m controller-manager-01s5g 1/1 Running 0 46m 3. checking api sever pod's etcd-server url # oc get po apiserver-9wwpf -o yaml <--snip--> - --etcd-servers - https://192.168.2.59:2379 <--snip--> It is connect master itself eth0 IP, but not external etcd IP. After correct it, oc get broker works. Expected results: installation is passed. Additional info:
Verified this bug with openshift-ansible-3.6.132-1.git.0.0d0f54a.el7.noarch, and PASS. For single external etcd: [root@openshift-126 ~]# oc edit ds apiserver -n kube-service-catalog <--snip--> containers: - args: - --storage-type - etcd - --secure-port - "6443" - --etcd-servers - https://openshift-140.lab.sjc.redhat.com:2379 - --etcd-cafile - /etc/origin/master/master.etcd-ca.crt - --etcd-certfile - /etc/origin/master/master.etcd-client.crt - --etcd-keyfile - /etc/origin/master/master.etcd-client.key <--snip--> [root@openshift-126 ~]# cat /etc/origin/master/master-config.yaml <--snip--> etcdClientInfo: ca: master.etcd-ca.crt certFile: master.etcd-client.crt keyFile: master.etcd-client.key urls: - https://openshift-140.lab.sjc.redhat.com:2379 <--snip-->
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:1716