Bug 1751650 - [ASB] automationbroker can not be installed by openshift-ansible-service-broker-operator in OCP4.2
Summary: [ASB] automationbroker can not be installed by openshift-ansible-service-brok...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Service Broker
Version: 4.2.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.2.0
Assignee: Fabian von Feilitzsch
QA Contact: Zhang Cheng
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-09-12 10:28 UTC by Cuiping HUO
Modified: 2019-10-16 06:41 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-10-16 06:41:11 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:2922 0 None None None 2019-10-16 06:41:19 UTC

Description Cuiping HUO 2019-09-12 10:28:29 UTC
Description of problem:
automationbroker can not be installed by openshift-ansible-service-broker-operator, error "Unable to create broker-admin clusterrolebinding"


Version-Release number of selected component (if applicable):
4.2.0-0.nightly-2019-09-12-034447
openshiftansibleservicebroker.4.2.0-201909081401

How reproducible:
Always

Steps to Reproduce:
1.install OpenShift Ansible Service Broker Operator from web console
2.install Automation Broker from web console

Actual results:
automationservicebroker can not be installed with error "Unable to create broker-admin clusterrolebinding"


Expected results:
automationservicebroker should be installed by operator.

Additional info:
$ oc get automationbroker ansible-service-broker -n openshift-ansible-service-broker -o yaml
apiVersion: osb.openshift.io/v1
kind: AutomationBroker
metadata:
  creationTimestamp: "2019-09-12T10:16:35Z"
  finalizers:
  - finalizer.osb.openshift.io
  generation: 1
  name: ansible-service-broker
  namespace: openshift-ansible-service-broker
  resourceVersion: "33602"
  selfLink: /apis/osb.openshift.io/v1/namespaces/openshift-ansible-service-broker/automationbrokers/ansible-service-broker
  uid: 666bfa1a-d546-11e9-a8a1-0a20c4448ad6
spec:
  createBrokerNamespace: "false"
  registries:
  - auth_name: asb-registry-auth
    auth_type: secret
    name: rhcc
    type: rhcc
    url: https://registry.redhat.io
    white_list:
    - .*-apb$
  waitForBroker: "false"
status:
  conditions:
  - lastTransitionTime: "2019-09-12T10:16:35Z"
    message: Running reconciliation
    reason: Running
    status: "False"
    type: Running
  - ansibleResult:
      changed: 0
      completion: 2019-09-12T10:16:41.366162
      failures: 1
      ok: 6
      skipped: 1
    lastTransitionTime: "2019-09-12T10:16:41Z"
    message: Unable to create broker-admin clusterrolebinding
    reason: Failed
    status: "True"
    type: Failure

$ oc get secret -n openshift-ansible-service-broker
NAME                                                        TYPE                                  DATA   AGE
asb-registry-auth                                           Opaque                                2      10m

$ oc logs -f openshift-ansible-service-broker-operator-86c5cc67c8-cwz86 -n openshift-ansible-service-broker
{"level":"info","ts":1568283299.7973933,"logger":"cmd","msg":"Go Version: go1.12.8"}
{"level":"info","ts":1568283299.7974343,"logger":"cmd","msg":"Go OS/Arch: linux/amd64"}
{"level":"info","ts":1568283299.7974432,"logger":"cmd","msg":"Version of operator-sdk: v0.10.0+git"}
{"level":"info","ts":1568283299.7974691,"logger":"cmd","msg":"Watching namespace.","Namespace":"openshift-ansible-service-broker"}
{"level":"info","ts":1568283299.9379592,"logger":"ansible-controller","msg":"Watching resource","Options.Group":"osb.openshift.io","Options.Version":"v1","Options.Kind":"AutomationBroker"}
{"level":"info","ts":1568283299.9382029,"logger":"kubebuilder.controller","msg":"Starting EventSource","controller":"automationbroker-controller","source":"kind source: osb.openshift.io/v1, Kind=AutomationBroker"}
{"level":"info","ts":1568283299.9383585,"logger":"leader","msg":"Trying to become the leader."}
{"level":"info","ts":1568283300.0855708,"logger":"leader","msg":"No pre-existing lock was found."}
{"level":"info","ts":1568283300.0921848,"logger":"leader","msg":"Became the leader."}
{"level":"info","ts":1568283300.2341294,"logger":"metrics","msg":"Metrics Service object created","Service.Name":"openshift-ansible-service-broker-operator-metrics","Service.Namespace":"openshift-ansible-service-broker"}
{"level":"info","ts":1568283300.2354884,"logger":"proxy","msg":"Starting to serve","Address":"127.0.0.1:8888"}
{"level":"info","ts":1568283300.3364913,"logger":"kubebuilder.controller","msg":"Starting Controller","controller":"automationbroker-controller"}
{"level":"info","ts":1568283300.4367683,"logger":"kubebuilder.controller","msg":"Starting workers","controller":"automationbroker-controller","worker count":1}
{"level":"info","ts":1568283398.6729143,"logger":"logging_event_handler","msg":"[playbook task]","name":"ansible-service-broker","namespace":"openshift-ansible-service-broker","gvk":"osb.openshift.io/v1, Kind=AutomationBroker","event_type":"playbook_on_task_start","job":"8674665223082153551","EventData.Name":"ansible-service-broker : Environment Validation"}
{"level":"info","ts":1568283398.744746,"logger":"logging_event_handler","msg":"[playbook task]","name":"ansible-service-broker","namespace":"openshift-ansible-service-broker","gvk":"osb.openshift.io/v1, Kind=AutomationBroker","event_type":"playbook_on_task_start","job":"8674665223082153551","EventData.Name":"ansible-service-broker : Verify service catalog is installed"}
{"level":"info","ts":1568283398.8067412,"logger":"logging_event_handler","msg":"[playbook task]","name":"ansible-service-broker","namespace":"openshift-ansible-service-broker","gvk":"osb.openshift.io/v1, Kind=AutomationBroker","event_type":"playbook_on_task_start","job":"8674665223082153551","EventData.Name":"ansible-service-broker : Set broker admin cluster rolebinding state=present"}
{"level":"info","ts":1568283400.3761544,"logger":"proxy","msg":"Injecting owner reference"}
{"level":"error","ts":1568283401.2736084,"logger":"logging_event_handler","msg":"","name":"ansible-service-broker","namespace":"openshift-ansible-service-broker","gvk":"osb.openshift.io/v1, Kind=AutomationBroker","event_type":"runner_on_failed","job":"8674665223082153551","EventData.Task":"Set broker admin cluster rolebinding state=present","EventData.TaskArgs":"","EventData.FailedTaskPath":"/opt/ansible/roles/ansible-service-broker/tasks/main.yml:40","error":"[playbook task failed]","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\tsrc/github.com/operator-framework/operator-sdk/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.com/operator-framework/operator-sdk/pkg/ansible/events.loggingEventHandler.Handle\n\tsrc/github.com/operator-framework/operator-sdk/pkg/ansible/events/log_events.go:84"}
{"level":"info","ts":1568283401.283837,"logger":"logging_event_handler","msg":"[playbook task]","name":"ansible-service-broker","namespace":"openshift-ansible-service-broker","gvk":"osb.openshift.io/v1, Kind=AutomationBroker","event_type":"playbook_on_task_start","job":"8674665223082153551","EventData.Name":"ansible-service-broker : fail"}
{"level":"info","ts":1568283401.3324745,"logger":"logging_event_handler","msg":"[playbook task]","name":"ansible-service-broker","namespace":"openshift-ansible-service-broker","gvk":"osb.openshift.io/v1, Kind=AutomationBroker","event_type":"playbook_on_task_start","job":"8674665223082153551","EventData.Name":"ansible-service-broker : fail"}
{"level":"error","ts":1568283401.3649004,"logger":"logging_event_handler","msg":"","name":"ansible-service-broker","namespace":"openshift-ansible-service-broker","gvk":"osb.openshift.io/v1, Kind=AutomationBroker","event_type":"runner_on_failed","job":"8674665223082153551","EventData.Task":"fail","EventData.TaskArgs":"","EventData.FailedTaskPath":"/opt/ansible/roles/ansible-service-broker/tasks/main.yml:51","error":"[playbook task failed]","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\tsrc/github.com/operator-framework/operator-sdk/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.com/operator-framework/operator-sdk/pkg/ansible/events.loggingEventHandler.Handle\n\tsrc/github.com/operator-framework/operator-sdk/pkg/ansible/events/log_events.go:84"}
{"level":"error","ts":1568283401.5534203,"logger":"runner","msg":"\u001b[0;34mansible-playbook 2.8.4\u001b[0m\r\n\u001b[0;34m  config file = /etc/ansible/ansible.cfg\u001b[0m\r\n\u001b[0;34m  configured module search path = [u'/usr/share/ansible/openshift']\u001b[0m\r\n\u001b[0;34m  ansible python module location = /usr/lib/python2.7/site-packages/ansible\u001b[0m\r\n\u001b[0;34m  executable location = /usr/bin/ansible-playbook\u001b[0m\r\n\u001b[0;34m  python version = 2.7.5 (default, Jun 11 2019, 14:33:56) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]\u001b[0m\r\n\u001b[0;34mUsing /etc/ansible/ansible.cfg as config file\u001b[0m\r\n\u001b[0;34mstatically imported: /opt/ansible/roles/ansible-service-broker/tasks/validate_present.yml\u001b[0m\r\n\u001b[0;34mstatically imported: /opt/ansible/roles/ansible-service-broker/tasks/tls_k8s.yml\u001b[0m\r\n\r\nPLAYBOOK: playbook.yaml ********************************************************\n\u001b[0;34m1 plays in /opt/ansible/playbook.yaml\u001b[0m\n\r\nPLAY [localhost] ***************************************************************\n\u001b[0;34mMETA: ran handlers\u001b[0m\n\u001b[1;30mtask path: /opt/ansible/roles/ansible-service-broker/tasks/main.yml:4\u001b[0m\r\n\r\nTASK [ansible-service-broker : Get cluster api_groups] *************************\r\n\u001b[1;30mtask path: /opt/ansible/roles/ansible-service-broker/tasks/main.yml:4\u001b[0m\n\u001b[0;32mok: [localhost] => {\"ansible_facts\": {\"api_groups\": [\"apiregistration.k8s.io\", \"extensions\", \"apps\", \"events.k8s.io\", \"authentication.k8s.io\", \"authorization.k8s.io\", \"autoscaling\", \"batch\", \"certificates.k8s.io\", \"networking.k8s.io\", \"policy\", \"rbac.authorization.k8s.io\", \"storage.k8s.io\", \"admissionregistration.k8s.io\", \"apiextensions.k8s.io\", \"scheduling.k8s.io\", \"coordination.k8s.io\", \"node.k8s.io\", \"apps.openshift.io\", \"authorization.openshift.io\", \"build.openshift.io\", \"image.openshift.io\", \"oauth.openshift.io\", \"project.openshift.io\", \"quota.openshift.io\", \"route.openshift.io\", \"security.openshift.io\", \"template.openshift.io\", \"user.openshift.io\", \"servicecatalog.k8s.io\", \"packages.operators.coreos.com\", \"config.openshift.io\", \"operator.openshift.io\", \"autoscaling.openshift.io\", \"cloudcredential.openshift.io\", \"console.openshift.io\", \"imageregistry.operator.openshift.io\", \"ingress.operator.openshift.io\", \"k8s.cni.cncf.io\", \"machineconfiguration.openshift.io\", \"monitoring.coreos.com\", \"network.openshift.io\", \"operators.coreos.com\", \"osb.openshift.io\", \"samples.operator.openshift.io\", \"tuned.openshift.io\", \"automationbroker.io\", \"healthchecking.openshift.io\", \"metal3.io\", \"machine.openshift.io\", \"metrics.k8s.io\"]}, \"changed\": false}\u001b[0m\n\r\nTASK [ansible-service-broker : Set reconciled_generation and generation facts] ***\r\n\u001b[1;30mtask path: /opt/ansible/roles/ansible-service-broker/tasks/main.yml:13\u001b[0m\n\u001b[0;32mok: [localhost] => {\"ansible_facts\": {\"generation\": \"1\", \"reconciled_generation\": \"\"}, \"changed\": false}\u001b[0m\n\r\nTASK [ansible-service-broker : Set pending_config_changes fact] ****************\r\n\u001b[1;30mtask path: /opt/ansible/roles/ansible-service-broker/tasks/main.yml:32\u001b[0m\n\u001b[0;32mok: [localhost] => {\"ansible_facts\": {\"pending_config_changes\": true}, \"changed\": false}\u001b[0m\n\r\nTASK [ansible-service-broker : Environment Validation] *************************\r\n\u001b[1;30mtask path: /opt/ansible/roles/ansible-service-broker/tasks/validate_present.yml:4\u001b[0m\n\u001b[0;32mok: [localhost] => {\u001b[0m\r\n\u001b[0;32m    \"changed\": false, \u001b[0m\r\n\u001b[0;32m    \"msg\": 
...
u001b[1;30mtask path: /opt/ansible/roles/ansible-service-broker/tasks/main.yml:47\u001b[0m\n\u001b[0;36mskipping: [localhost] => {\"changed\": false, \"skip_reason\": \"Conditional result was False\"}\u001b[0m\n\r\nTASK [ansible-service-broker : fail] *******************************************\r\n\u001b[1;30mtask path: /opt/ansible/roles/ansible-service-broker/tasks/main.yml:51\u001b[0m\n\u001b[0;31mfatal: [localhost]: FAILED! => {\"changed\": false, \"msg\": \"Unable to create broker-admin clusterrolebinding\"}\u001b[0m\n\r\nPLAY RECAP *********************************************************************\r\n\u001b[0;31mlocalhost\u001b[0m                  : \u001b[0;32mok=6   \u001b[0m changed=0    unreachable=0    \u001b[0;31mfailed=1   \u001b[0m \u001b[0;36mskipped=1   \u001b[0m rescued=0    \u001b[1;35mignored=1   \u001b[0m\r\n\n","job":"8674665223082153551","name":"ansible-service-broker","namespace":"openshift-ansible-service-broker","error":"exit status 2","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\tsrc/github.com/operator-framework/operator-sdk/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.com/operator-framework/operator-sdk/pkg/ansible/runner.(*runner).Run.func1\n\tsrc/github.com/operator-framework/operator-sdk/pkg/ansible/runner/runner.go:190"}

Comment 3 Fabian von Feilitzsch 2019-09-12 16:19:14 UTC
Confirmed with the provided cluster, it looks like the manual step to create a ClusterRoleBinding for the ansible-service-broker-operator serviceaccount was skipped, so this failure is expected. 

The process for creating the ClusterRoleBinding is detailed is step 2 of this document:
https://docs.openshift.com/container-platform/4.1/applications/service_brokers/installing-ansible-service-broker.html#sb-install-asb-operator_sb-installing-asb

(the doc is for 4.1 but the process has not changed for 4.2)

Comment 6 errata-xmlrpc 2019-10-16 06:41:11 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2922


Note You need to log in before you can comment on or make changes to this bug.