Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1614711

Summary: automation-broker-apb should create servicebroker not clusterservicebroker
Product: OpenShift Container Platform Reporter: Zihan Tang <zitang>
Component: Service BrokerAssignee: David Zager <dzager>
Status: CLOSED ERRATA QA Contact: Zihan Tang <zitang>
Severity: high Docs Contact:
Priority: high    
Version: 3.11.0CC: aos-bugs, chezhang, dzager, jiazha, zitang
Target Milestone: ---   
Target Release: 3.11.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-10-11 07:24:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Zihan Tang 2018-08-10 09:28:42 UTC
Description of problem:
When using 'automation-broker-apb' to install NS-scoped broker, it didn't create servicebroker, but create clusterservicebroker ansible-service-broker

Version-Release number of selected component (if applicable):
service-catalog: v3.11.0-0.13.0;Upstream:v0.1.2
automation-broker-apb: docker.io/.../latest (image is not ready on downstream, but it blocks all ns-broker features test of ASB and SC, so using upstream to test.)

How reproducible:
always

Steps to Reproduce:
1. uninstall ansible-service-broker by openshift-ansible, and make sure asb resource is deleted.
# oc get clusterservicebroker
NAME                      CREATED AT
template-service-broker   2018-08-10T03:20:12Z
2. provision ns-broker by automation-broker-apb
# oc create -f install.yaml 
namespace/automation-broker-apb created
serviceaccount/automation-broker-apb created
clusterrolebinding.rbac.authorization.k8s.io/automation-broker-apb created
pod/automation-broker-apb created

# cat install.yaml 
---
apiVersion: v1
kind: Namespace
metadata:
  name: automation-broker-apb

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: automation-broker-apb
  namespace: automation-broker-apb

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: automation-broker-apb
roleRef:
  name: cluster-admin
  kind: ClusterRole
  apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
  name: automation-broker-apb
  namespace: automation-broker-apb

---
apiVersion: v1
kind: Pod
metadata:
  name: automation-broker-apb
  namespace: automation-broker-apb
spec:
  serviceAccount: automation-broker-apb
  containers:
    - name: apb
      image: docker.io/automationbroker/automation-broker-apb:latest
      args: [ "provision", "--extra-vars", '{ broker_kind": "ServiceBroker", "broker_namespace": "test-ns-broker", "create_broker_namespace": "true" }' ]
      imagePullPolicy: IfNotPresent
  restartPolicy: Never

3. check servicebroker resource

Actual results:
servicebroker pod are ready, 

# oc get pod -n automation-broker-apb
NAME                    READY     STATUS      RESTARTS   AGE
automation-broker-apb   0/1       Completed   0          9m

# oc get pod -n test-ns-broker
NAME                        READY     STATUS    RESTARTS   AGE
automation-broker-1-hg62m   1/1       Running   0          9m

but servicebroker and serviceclass/plan not created.
[root@qe-zitang-810-master-etcd-nfs-1 ~]# oc get servicebroker -n test-ns-broker
No resources found.
[root@qe-zitang-810-master-etcd-nfs-1 ~]# oc get serviceclass -n test-ns-broker
No resources found.

instead, it create clusterservicebroker and clusterserviceclass

# oc get clusterservicebroker ansible-service-broker -o yaml
apiVersion: servicecatalog.k8s.io/v1beta1
kind: ClusterServiceBroker
metadata:
  creationTimestamp: 2018-08-10T09:13:40Z
  finalizers:
  - kubernetes-incubator/service-catalog
  generation: 1
  name: ansible-service-broker
  resourceVersion: "49596"
  selfLink: /apis/servicecatalog.k8s.io/v1beta1/clusterservicebrokers/ansible-service-broker
  uid: abe9efd2-9c7d-11e8-8cc4-0a580a80008d
spec:
  authInfo:
    bearer:
      secretRef:
        name: automation-broker-client-token-67krt
        namespace: test-ns-broker
  caBundle: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM2akNDQWRLZ0F3SUJBZ0lCQVRBTkJna3Foa2lHOXcwQkFRc0ZBREFtTVNRd0lnWURWUVFEREJ0dmNHVnUKYzJocFpuUXRjMmxuYm1WeVFERTFNek00TnpBMk5qRXdIaGNOTVRnd09ERXdNRE14TVRBeFdoY05Nak13T0RBNQpNRE14TVRBeVdqQW1NU1F3SWdZRFZRUUREQnR2Y0dWdWMyaHBablF0YzJsbmJtVnlRREUxTXpNNE56QTJOakV3CmdnRWlNQTBHQ1NxR1NJYjNEUUVCQVFVQUE0SUJEd0F3Z2dFS0FvSUJBUUNLeTl1Qk4rZ0hVYlZzUFd5YytpM0QKS1VIeEdaL1EweWNxNWtlRFdjcG5XbXF3dGVNWndkMVg5aFZoRm1YTXcyblgwaG5IbG1YNmcySC84a0ZKeWxFTwp0QW5Yd2wvaXA1QTVuanJJZDJ4YTl3SGw1ZitPais3TzA3M0l0VndIUnJwVGpZL2p1QnYxM2UwTUxDRlZxeWdYCkp0UTZna0tLTzIzUDJqcEFnR2MvSUplMmVQWkY4VUFCWkRvWWRhdm1tR1c5MmxFRkhmUUt2UHRwWlRBQUVyd2gKc1RhcDVERlRPclVuRHZVT09GVk9jL0ZVS2tiNldWTmNFLzVtUHN2N3VGMjZDaGpaN0p6M1RaSCthMDdQaTlxMwo1TXVnUC9kZk1wS1RjVm9EblFNeWNyenc2cW1KMm5rQVFDaXlaOWVZdlc5K2N2ZXNqZEE1bEdTc3krakRnK0NICkFnTUJBQUdqSXpBaE1BNEdBMVVkRHdFQi93UUVBd0lDcERBUEJnTlZIUk1CQWY4RUJUQURBUUgvTUEwR0NTcUcKU0liM0RRRUJDd1VBQTRJQkFRQndHRVBJQU9iN29YdzlIMXp4VTdrbyt0NFlCOFNRalZpQ1d3Q3JudkQ2RGc3Ugp6NGh6bFlIRXgvUEx4SVNsQlgweE1xYnJnY1o3S0QvMzhrVitGSGxnK2FvbUZwYXNvbnhXL3VZS3pESGJmWTA0CnRHN2tteTZTOUx1OWFoR2NJaHNSTXVpSEo1WXJGejBwQWQyNTRyQ1hQQW5QM2VPeEVwL1krZ0duNEdndHFQckIKdEptZlVFdUFDMTBJaGlQZlBvQ29FRFRtREtDbG1pM3ZVQ21POWZxcGl4TGdCR2JnbVpUemNRaWsyVTdmZEFLeAp4Q0M4WmxpUnpyd0hDNEZaS0VJbjI1eFdlWlpVY0Q0NVltT2dCNzVrY3VueXpXOGxUWmVUVEIwSk5XQUgxeHh1ClIvMzBXVGpNTStsSjNnOEd6M2orRkRyU3NES1M3bms4L2NsSDdJTmgKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQoKLS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURDakNDQWZLZ0F3SUJBZ0lCQVRBTkJna3Foa2lHOXcwQkFRc0ZBREEyTVRRd01nWURWUVFEREN0dmNHVnUKYzJocFpuUXRjMlZ5ZG1salpTMXpaWEoyYVc1bkxYTnBaMjVsY2tBeE5UTXpPRGN3TmpZeU1CNFhEVEU0TURneApNREF6TVRFd01Wb1hEVEl6TURnd09UQXpNVEV3TWxvd05qRTBNRElHQTFVRUF3d3JiM0JsYm5Ob2FXWjBMWE5sCmNuWnBZMlV0YzJWeWRtbHVaeTF6YVdkdVpYSkFNVFV6TXpnM01EWTJNakNDQVNJd0RRWUpLb1pJaHZjTkFRRUIKQlFBRGdnRVBBRENDQVFvQ2dnRUJBTlJvbzFoU0tRaEZKQkpmL25wUCtCUlcyd2NPSHRhUWlWU0QxQXE0QzVhMAoyNnpUM3d4UE5zUnRvUEZlNzBDM3JlTWR1Rm9kYzZub0IyVFJSSzZsMzdpbExiZXc5K05qdGI3R2VBelVlMHN1ClB4VExqNFYxc2YzTzUyam5qMExGUWxxUHBkMFBhRFR6SVRieXZuRlYrSGF5M2daeUtDZ0pMWi9EajU4Visvc1AKWXFibVdMbS9aaEo2R3RZR3hBMzI2VTQ1QklVWXRhM1NPOHlmMTVPMVNBWFNHZXBsZEdxSjNLdFNEa1dONzAwTwp6aGNXdVNacmZxVTJ2by9oNFRaaUdlM3JPYmxBcTV2VlRNdG5sNVdFVXIrWlB3ZklvNzR3c0JJTzVRYmR1Ly80CjNDQ0xlZzZrZThia3hUWVo0YkxkUU0vaWZXZHJHYkh3cTNUQVMwOXNyMzhDQXdFQUFhTWpNQ0V3RGdZRFZSMFAKQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCL3dRRk1BTUJBZjh3RFFZSktvWklodmNOQVFFTEJRQURnZ0VCQUVyVgpFTWxsU0dOR0VxTTdIZThmcTVmQUpOZHJPU1JMRjB5SXFMMkE3ZFJ4dEZDclFaK3J2T2FuQlBZanRQcGxmWHFLCjE4b0k3RnpUNHZFS3cxQ1JNQjZNZHRDOTJhNkF2YWdTN1lScE9iK2h6UXdmdzlYeTZYRkxUSTEwVTRKcFpTcVMKVmFsTlhURUtXTXM4OC9jYzh0QXlBYXZRSnB0M3IwZmNKZi9PLzVxWlF5QnNlaFRLNDJOYlNDVFlsTUUzYnV4eQp1cHpRNERlV2FJbTJuUDdpQU1CbTIrWWQ3U2Z0YkRzeGRSMWpFYmlPa0FtQmhDaEFQQ1E2SVh5Y1Y1eVA0MytJCnd3dTZSNlM5UUtIWVlRVFFPajgvQzZZd2xUZklpZGVBWnZJNndaZUpUUzhnZ0FLb1c0NnBBNWZ1OFZoa2tRT3YKTDRVRHljYVJnK1AzQlhkWnJGdz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
  relistBehavior: Duration
  relistRequests: 0
  url: https://automation-broker.test-ns-broker.svc:1338/automation-broker/
status:
  conditions:
  - lastTransitionTime: 2018-08-10T09:13:43Z
    message: 'Error fetching catalog.Error getting broker catalog: Status: 404; ErrorMessage:
      <nil>; Description: <nil>; ResponseError: <nil>'
    reason: ErrorFetchingCatalog
    status: "False"
    type: Ready
  operationStartTime: 2018-08-10T09:13:46Z
  reconciledGeneration: 0

# oc get clusterserviceclass 
NAME                                   CREATED AT
a22f10e1-9c4b-11e8-bc1b-fa163eebd417   2018-08-10T03:20:24Z
a2306eff-9c4b-11e8-bc1b-fa163eebd417   2018-08-10T03:20:31Z
a232ced0-9c4b-11e8-bc1b-fa163eebd417   2018-08-10T03:20:41Z
.....

Expected results:
1. create servicebroker and serviceclass/plan in 'test-ns-broker'
2. should not create any clusterservicebroker related resource 

Additional info:
apb pod log
# oc logs -f automation-broker-apb -n automation-broker-apb
 [WARNING]: provided hosts list is empty, only localhost is available. Note
that the implicit localhost does not match 'all'

PLAY [automation-broker-apb provision] *****************************************

TASK [automation-broker-apb : Set facts] ***************************************
ok: [localhost]

TASK [automation-broker-apb : Debug important facts] ***************************
ok: [localhost] => {
    "msg": [
        "Cluster: openshift", 
        "broker_auto_escalate False", 
        "broker_local_openshift_enabled True"
    ]
}

TASK [automation-broker-apb : Set broker namespace state=present] **************
changed: [localhost]

TASK [automation-broker-apb : Verify preconditions] ****************************
ok: [localhost] => {
    "changed": false, 
    "msg": "All assertions passed"
}

TASK [automation-broker-apb : include_tasks] ***********************************
included: /opt/ansible/roles/automation-broker-apb/tasks/dao_crd.yaml for localhost

TASK [automation-broker-apb : Set broker clusterresourcedefinitions state=present] ***
ok: [localhost] => (item=bundle.crd.yaml)
ok: [localhost] => (item=bundlebindings.crd.yaml)
ok: [localhost] => (item=bundleinstances.crd.yaml)

TASK [automation-broker-apb : include_tasks] ***********************************
skipping: [localhost]

TASK [automation-broker-apb : Set broker objects state=present] ****************
changed: [localhost] => (item={u'name': u'broker.service.yaml'})
changed: [localhost] => (item={u'apply': True, u'name': u'broker.route.yaml'})
changed: [localhost] => (item={u'name': u'broker.serviceaccount.yaml'})
ok: [localhost] => (item={u'name': u'broker.clusterrolebinding.yaml'})
changed: [localhost] => (item={u'name': u'broker.configmap.yaml'})
ok: [localhost] => (item={u'name': u'broker-auth.clusterrole.yaml'})
ok: [localhost] => (item={u'name': u'broker-auth.clusterrolebinding.yaml'})
changed: [localhost] => (item={u'name': u'broker-client.serviceaccount.yaml'})
changed: [localhost] => (item={u'name': u'broker-client.secret.yaml'})
ok: [localhost] => (item={u'name': u'broker-client.clusterrolebinding.yaml'})
ok: [localhost] => (item={u'name': u'broker-access.clusterrole.yaml'})
skipping: [localhost] => (item={u'apply': False, u'name': u'broker-auth.secret.yaml'}) 
changed: [localhost] => (item={u'name': u'broker.deployment.yaml'})
changed: [localhost] => (item={u'name': u'broker.servicecatalog.yaml'})

TASK [automation-broker-apb : Wait for clusterservicebroker to become ready] ***
skipping: [localhost]

PLAY RECAP *********************************************************************
localhost                  : ok=7    changed=2    unreachable=0    failed=0

Comment 1 David Zager 2018-08-10 20:35:11 UTC
The docker.io/automationbroker/automation-broker-apb:latest did not support being namespaced until now, it has been updated.

Keep in mind, that the previous version of automation-broker-apb:latest was maintaining compatibility with the old python apb tool. That is no longer the case. So the default name of the broker will be 'automation-broker' and that will be used for the service catalog object also.

The downstream container project for the automation-broker-apb is now available. I intend to create a build of that early next week. Feel free to move this back to ASSIGNED if you would like to wait for the downstream image to be available.

Comment 2 Zihan Tang 2018-08-13 09:48:55 UTC
I tried to pre-test with the latest image in dockerhub,
it still created a clusterservicebroker type broker: automation-broker

[root@qe-zitang-r2-1-master-etcd-1 ~]# oc get clusterservicebroker
NAME                     CREATED AT
ansible-service-broker   2018-08-13T06:10:41Z
automation-broker        2018-08-13T08:49:27Z
[root@qe-zitang-r2-1-master-etcd-1 ~]# oc get servicebroker -n test-ns-broker
No resources found.

I'll wait for the image ready in downstream.

Comment 3 David Zager 2018-08-13 13:12:07 UTC
Looking back at the original comment, I notice that you have `broker_kind":` instead of `"broker_kind"`. That would cause the problem you are experiencing.

See the corrected Pod definition below:


---
apiVersion: v1
kind: Pod
metadata:
  name: automation-broker-apb
  namespace: automation-broker-apb
spec:
  serviceAccount: automation-broker-apb
  containers:
    - name: apb
      image: docker.io/automationbroker/automation-broker-apb:latest
      args: [ "provision", "--extra-vars", '{ "broker_kind": "ServiceBroker", "broker_namespace": "test-ns-broker", "create_broker_namespace": "true" }' ]
      imagePullPolicy: IfNotPresent
  restartPolicy: Never

Comment 4 Zihan Tang 2018-08-14 07:55:40 UTC
Thanks for your revision, this pod definition can create service-broker.
I'll use downstream image to double check.

Comment 5 David Zager 2018-08-15 19:57:57 UTC
Moving this to POST, all we need is the correct branch for automation-broker-apb (http://pkgs.devel.redhat.com/cgit/apbs/automation-broker-apb/log/?h=rhaos-3.11-rhel-7) **and** the container image to be built.

Comment 6 Zihan Tang 2018-08-28 08:07:17 UTC
downstream image is ready, change it to ON_QA

Comment 7 Zihan Tang 2018-08-28 08:08:05 UTC
Verified. 
it could create service-broker successfully.
version: automation-broker-apb: v3.11.0

Comment 9 errata-xmlrpc 2018-10-11 07:24:38 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:2652