Bug 1578300

Summary: Installation of service catalog fails on TASK [ansible_service_broker : Create the Broker resource in the catalog]
Product: OpenShift Container Platform Reporter: Rutvik <rkshirsa>
Component: Service CatalogAssignee: Jay Boyd <jaboyd>
Status: CLOSED NOTABUG QA Contact: Zhang Cheng <chezhang>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 3.7.1CC: emahoney, jaboyd, pmorie, rkshirsa, stwalter
Target Milestone: ---Keywords: Reopened
Target Release: 3.10.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-12-19 18:55:07 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Rutvik 2018-05-15 08:41:46 UTC
Description of problem:

1. Cluster upgraded from 3.6 to 3.7 successfully.

2. While running service_catalog.yml, its failing all the time on the same task


TASK [ansible_service_broker : Create the Broker resource in the catalog]

"msg": {
        "cmd": "/usr/bin/oc create -f /tmp/brokerout-** -n default", 
        "results": {}, 
        "returncode": 1, 
        "stderr": "Error from server (Forbidden): error when creating \"/tmp/brokerout-**\": clusterservicebrokers.servicecatalog.k8s.io \"ansible-service-broker\" is forbidden: not yet ready to handle request\n", 
        "stdout": ""
    }

3. POD's are up and running in both openshift-ansible-service-catalog & kube-service-catalog namespaces. But openshift-template-service-broker namespace is not present yet due to the incomplete installation of service_catalog.

4. There was a similar Bugzilla[1] which has been closed with an errata.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1523625

Comment 4 Paul Morie 2018-05-18 17:30:53 UTC
Would you please provide:

- The logs from the apiserver pod in kube-service-catalog ns
- The output of `oc get apiservices servicecatalog.k8s.io/v1beta1 -o yaml`

Thanks!

Comment 6 Steven Walter 2018-05-21 14:50:46 UTC
I have requested the yaml from the customer and provided the logs to the bz

Comment 7 Jay Boyd 2018-05-21 15:54:50 UTC
Still looking for output from `oc get apiservices servicecatalog.k8s.io/v1beta1 -o yaml`.  Also, I see both the Catalog API Server and Catalog Controller Manager pods are in a crash loop backoff.  Please also provide output of `oc describe` on both pods in the kube-service-catalog namespace.  Thanks.

Comment 8 Steven Walter 2018-05-23 14:01:29 UTC
$ oc get apiservices/v1beta1.servicecatalog.k8s.io -o yaml         
apiVersion: apiregistration.k8s.io/v1beta1
kind: APIService
metadata:
  creationTimestamp: 2018-05-15T12:42:48Z
  name: v1beta1.servicecatalog.k8s.io
  resourceVersion: "108270012"
  selfLink: /apis/apiregistration.k8s.io/v1beta1/apiservices/v1beta1.servicecatalog.k8s.io
  uid: 790165d3-583d-11e8-93f5-005056b7703f
spec:
  caBundle: .....=
  group: servicecatalog.k8s.io
  groupPriorityMinimum: 20
  service:
    name: apiserver
    namespace: kube-service-catalog
  version: v1beta1
  versionPriority: 10
status:
  conditions:
  - lastTransitionTime: 2018-05-22T13:04:16Z
    message: all checks passed
    reason: Passed
    status: "True"
    type: Available


Customer is attempting a redeploy of the service catalog. If this fails we will provide latest requested info.

Comment 10 Jay Boyd 2018-06-05 15:02:40 UTC
closing, we can't reproduce, looks to be environmental

Comment 17 Jay Boyd 2018-12-19 18:55:07 UTC
I'm going to close out this BZ again per comment #16.  Please open a new issue if you continue to have problems.

Comment 18 Red Hat Bugzilla 2023-09-15 00:08:09 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 500 days