Bug 1466727 - broker resource fail to be created due to catalog api server is connecting a wrong etcd server url
Summary: broker resource fail to be created due to catalog api server is connecting a ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 3.6.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: ---
Assignee: ewolinet
QA Contact: Johnny Liu
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-06-30 10:38 UTC by Johnny Liu
Modified: 2017-08-16 19:51 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
undefined
Clone Of:
Environment:
Last Closed: 2017-08-10 05:29:50 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2017:1716 0 normal SHIPPED_LIVE Red Hat OpenShift Container Platform 3.6 RPM Release Advisory 2017-08-10 09:02:50 UTC

Description Johnny Liu 2017-06-30 10:38:28 UTC
Description of problem:
see the following for details.

Version-Release number of selected component (if applicable):
openshift-ansible-3.6.126.4-1.git.0.d25d828.el7

How reproducible:
Always

Steps to Reproduce:
1. prepare inventory host file with a external etcd server for cluster install
2. enable catalog service deployment with the following options in inventory host file:
openshift_hosted_etcd_storage_kind=nfs
openshift_hosted_etcd_storage_nfs_options="*(rw,root_squash,sync,no_wdelay)"
openshift_hosted_etcd_storage_nfs_directory=/exports
openshift_hosted_etcd_storage_volume_name=etcd
openshift_hosted_etcd_storage_access_modes=["ReadWriteOnce"]
openshift_hosted_etcd_storage_volume_size=10G
openshift_hosted_etcd_storage_labels={'storage': 'etcd'}

openshift_enable_service_catalog=true
openshift_service_catalog_image_prefix=docker.io/openshift/origin-
openshift_service_catalog_image_version=latest
ansible_service_broker_image_prefix=ansibleplaybookbundle/
ansible_service_broker_image_tag=latest
ansible_service_broker_etcd_image_prefix=quay.io/coreos/
ansible_service_broker_etcd_image_tag=latest

3. trigger installation
4.

Actual results:
Job failed at the following task:
TASK [ansible_service_broker : Create the Broker resource in the catalog] ******
Friday 30 June 2017  07:49:22 +0000 (0:00:01.958)       1:04:18.052 *********** 

fatal: [openshift-155.lab.sjc.redhat.com]: FAILED! => {
    "changed": false, 
    "failed": true
}

MSG:

{u'returncode': 1, u'cmd': u'/usr/bin/oc create -f /tmp/brokerout-xPp27e -n default', u'results': {}, u'stderr': u'Error from server (Timeout): error when creating "/tmp/brokerout-xPp27e": Timeout: request did not complete within allowed duration\n', u'stdout': u''}


After the above failure, log into cluster for a little debug.
1. "oc get broker" will hang there.
2. api server and controller-manager pod are running.
# oc get po -n kube-service-catalog
NAME                       READY     STATUS    RESTARTS   AGE
apiserver-9wwpf            1/1       Running   0          46m
controller-manager-01s5g   1/1       Running   0          46m
3. checking api sever pod's etcd-server url
# oc get po apiserver-9wwpf -o yaml
<--snip-->
    - --etcd-servers
    - https://192.168.2.59:2379   
<--snip-->
It is connect master itself eth0 IP, but not external etcd IP.
After correct it, oc get broker works.


Expected results:
installation is passed.

Additional info:

Comment 4 Johnny Liu 2017-07-04 09:54:45 UTC
Verified this bug with openshift-ansible-3.6.132-1.git.0.0d0f54a.el7.noarch, and PASS.

For single external etcd:
[root@openshift-126 ~]# oc edit ds apiserver -n kube-service-catalog
<--snip-->
      containers:
      - args:
        - --storage-type
        - etcd
        - --secure-port
        - "6443"
        - --etcd-servers
        - https://openshift-140.lab.sjc.redhat.com:2379
        - --etcd-cafile
        - /etc/origin/master/master.etcd-ca.crt
        - --etcd-certfile
        - /etc/origin/master/master.etcd-client.crt
        - --etcd-keyfile
        - /etc/origin/master/master.etcd-client.key
<--snip-->


[root@openshift-126 ~]# cat /etc/origin/master/master-config.yaml
<--snip-->
etcdClientInfo:
  ca: master.etcd-ca.crt
  certFile: master.etcd-client.crt
  keyFile: master.etcd-client.key
  urls:
  - https://openshift-140.lab.sjc.redhat.com:2379
<--snip-->

Comment 6 errata-xmlrpc 2017-08-10 05:29:50 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1716


Note You need to log in before you can comment on or make changes to this bug.