Description of problem: automationbroker can not be installed by openshift-ansible-service-broker-operator, error "Unable to create broker-admin clusterrolebinding" Version-Release number of selected component (if applicable): 4.2.0-0.nightly-2019-09-12-034447 openshiftansibleservicebroker.4.2.0-201909081401 How reproducible: Always Steps to Reproduce: 1.install OpenShift Ansible Service Broker Operator from web console 2.install Automation Broker from web console Actual results: automationservicebroker can not be installed with error "Unable to create broker-admin clusterrolebinding" Expected results: automationservicebroker should be installed by operator. Additional info: $ oc get automationbroker ansible-service-broker -n openshift-ansible-service-broker -o yaml apiVersion: osb.openshift.io/v1 kind: AutomationBroker metadata: creationTimestamp: "2019-09-12T10:16:35Z" finalizers: - finalizer.osb.openshift.io generation: 1 name: ansible-service-broker namespace: openshift-ansible-service-broker resourceVersion: "33602" selfLink: /apis/osb.openshift.io/v1/namespaces/openshift-ansible-service-broker/automationbrokers/ansible-service-broker uid: 666bfa1a-d546-11e9-a8a1-0a20c4448ad6 spec: createBrokerNamespace: "false" registries: - auth_name: asb-registry-auth auth_type: secret name: rhcc type: rhcc url: https://registry.redhat.io white_list: - .*-apb$ waitForBroker: "false" status: conditions: - lastTransitionTime: "2019-09-12T10:16:35Z" message: Running reconciliation reason: Running status: "False" type: Running - ansibleResult: changed: 0 completion: 2019-09-12T10:16:41.366162 failures: 1 ok: 6 skipped: 1 lastTransitionTime: "2019-09-12T10:16:41Z" message: Unable to create broker-admin clusterrolebinding reason: Failed status: "True" type: Failure $ oc get secret -n openshift-ansible-service-broker NAME TYPE DATA AGE asb-registry-auth Opaque 2 10m $ oc logs -f openshift-ansible-service-broker-operator-86c5cc67c8-cwz86 -n openshift-ansible-service-broker {"level":"info","ts":1568283299.7973933,"logger":"cmd","msg":"Go Version: go1.12.8"} {"level":"info","ts":1568283299.7974343,"logger":"cmd","msg":"Go OS/Arch: linux/amd64"} {"level":"info","ts":1568283299.7974432,"logger":"cmd","msg":"Version of operator-sdk: v0.10.0+git"} {"level":"info","ts":1568283299.7974691,"logger":"cmd","msg":"Watching namespace.","Namespace":"openshift-ansible-service-broker"} {"level":"info","ts":1568283299.9379592,"logger":"ansible-controller","msg":"Watching resource","Options.Group":"osb.openshift.io","Options.Version":"v1","Options.Kind":"AutomationBroker"} {"level":"info","ts":1568283299.9382029,"logger":"kubebuilder.controller","msg":"Starting EventSource","controller":"automationbroker-controller","source":"kind source: osb.openshift.io/v1, Kind=AutomationBroker"} {"level":"info","ts":1568283299.9383585,"logger":"leader","msg":"Trying to become the leader."} {"level":"info","ts":1568283300.0855708,"logger":"leader","msg":"No pre-existing lock was found."} {"level":"info","ts":1568283300.0921848,"logger":"leader","msg":"Became the leader."} {"level":"info","ts":1568283300.2341294,"logger":"metrics","msg":"Metrics Service object created","Service.Name":"openshift-ansible-service-broker-operator-metrics","Service.Namespace":"openshift-ansible-service-broker"} {"level":"info","ts":1568283300.2354884,"logger":"proxy","msg":"Starting to serve","Address":"127.0.0.1:8888"} {"level":"info","ts":1568283300.3364913,"logger":"kubebuilder.controller","msg":"Starting Controller","controller":"automationbroker-controller"} {"level":"info","ts":1568283300.4367683,"logger":"kubebuilder.controller","msg":"Starting workers","controller":"automationbroker-controller","worker count":1} {"level":"info","ts":1568283398.6729143,"logger":"logging_event_handler","msg":"[playbook task]","name":"ansible-service-broker","namespace":"openshift-ansible-service-broker","gvk":"osb.openshift.io/v1, Kind=AutomationBroker","event_type":"playbook_on_task_start","job":"8674665223082153551","EventData.Name":"ansible-service-broker : Environment Validation"} {"level":"info","ts":1568283398.744746,"logger":"logging_event_handler","msg":"[playbook task]","name":"ansible-service-broker","namespace":"openshift-ansible-service-broker","gvk":"osb.openshift.io/v1, Kind=AutomationBroker","event_type":"playbook_on_task_start","job":"8674665223082153551","EventData.Name":"ansible-service-broker : Verify service catalog is installed"} {"level":"info","ts":1568283398.8067412,"logger":"logging_event_handler","msg":"[playbook task]","name":"ansible-service-broker","namespace":"openshift-ansible-service-broker","gvk":"osb.openshift.io/v1, Kind=AutomationBroker","event_type":"playbook_on_task_start","job":"8674665223082153551","EventData.Name":"ansible-service-broker : Set broker admin cluster rolebinding state=present"} {"level":"info","ts":1568283400.3761544,"logger":"proxy","msg":"Injecting owner reference"} {"level":"error","ts":1568283401.2736084,"logger":"logging_event_handler","msg":"","name":"ansible-service-broker","namespace":"openshift-ansible-service-broker","gvk":"osb.openshift.io/v1, Kind=AutomationBroker","event_type":"runner_on_failed","job":"8674665223082153551","EventData.Task":"Set broker admin cluster rolebinding state=present","EventData.TaskArgs":"","EventData.FailedTaskPath":"/opt/ansible/roles/ansible-service-broker/tasks/main.yml:40","error":"[playbook task failed]","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\tsrc/github.com/operator-framework/operator-sdk/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.com/operator-framework/operator-sdk/pkg/ansible/events.loggingEventHandler.Handle\n\tsrc/github.com/operator-framework/operator-sdk/pkg/ansible/events/log_events.go:84"} {"level":"info","ts":1568283401.283837,"logger":"logging_event_handler","msg":"[playbook task]","name":"ansible-service-broker","namespace":"openshift-ansible-service-broker","gvk":"osb.openshift.io/v1, Kind=AutomationBroker","event_type":"playbook_on_task_start","job":"8674665223082153551","EventData.Name":"ansible-service-broker : fail"} {"level":"info","ts":1568283401.3324745,"logger":"logging_event_handler","msg":"[playbook task]","name":"ansible-service-broker","namespace":"openshift-ansible-service-broker","gvk":"osb.openshift.io/v1, Kind=AutomationBroker","event_type":"playbook_on_task_start","job":"8674665223082153551","EventData.Name":"ansible-service-broker : fail"} {"level":"error","ts":1568283401.3649004,"logger":"logging_event_handler","msg":"","name":"ansible-service-broker","namespace":"openshift-ansible-service-broker","gvk":"osb.openshift.io/v1, Kind=AutomationBroker","event_type":"runner_on_failed","job":"8674665223082153551","EventData.Task":"fail","EventData.TaskArgs":"","EventData.FailedTaskPath":"/opt/ansible/roles/ansible-service-broker/tasks/main.yml:51","error":"[playbook task failed]","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\tsrc/github.com/operator-framework/operator-sdk/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.com/operator-framework/operator-sdk/pkg/ansible/events.loggingEventHandler.Handle\n\tsrc/github.com/operator-framework/operator-sdk/pkg/ansible/events/log_events.go:84"} {"level":"error","ts":1568283401.5534203,"logger":"runner","msg":"\u001b[0;34mansible-playbook 2.8.4\u001b[0m\r\n\u001b[0;34m config file = /etc/ansible/ansible.cfg\u001b[0m\r\n\u001b[0;34m configured module search path = [u'/usr/share/ansible/openshift']\u001b[0m\r\n\u001b[0;34m ansible python module location = /usr/lib/python2.7/site-packages/ansible\u001b[0m\r\n\u001b[0;34m executable location = /usr/bin/ansible-playbook\u001b[0m\r\n\u001b[0;34m python version = 2.7.5 (default, Jun 11 2019, 14:33:56) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]\u001b[0m\r\n\u001b[0;34mUsing /etc/ansible/ansible.cfg as config file\u001b[0m\r\n\u001b[0;34mstatically imported: /opt/ansible/roles/ansible-service-broker/tasks/validate_present.yml\u001b[0m\r\n\u001b[0;34mstatically imported: /opt/ansible/roles/ansible-service-broker/tasks/tls_k8s.yml\u001b[0m\r\n\r\nPLAYBOOK: playbook.yaml ********************************************************\n\u001b[0;34m1 plays in /opt/ansible/playbook.yaml\u001b[0m\n\r\nPLAY [localhost] ***************************************************************\n\u001b[0;34mMETA: ran handlers\u001b[0m\n\u001b[1;30mtask path: /opt/ansible/roles/ansible-service-broker/tasks/main.yml:4\u001b[0m\r\n\r\nTASK [ansible-service-broker : Get cluster api_groups] *************************\r\n\u001b[1;30mtask path: /opt/ansible/roles/ansible-service-broker/tasks/main.yml:4\u001b[0m\n\u001b[0;32mok: [localhost] => {\"ansible_facts\": {\"api_groups\": [\"apiregistration.k8s.io\", \"extensions\", \"apps\", \"events.k8s.io\", \"authentication.k8s.io\", \"authorization.k8s.io\", \"autoscaling\", \"batch\", \"certificates.k8s.io\", \"networking.k8s.io\", \"policy\", \"rbac.authorization.k8s.io\", \"storage.k8s.io\", \"admissionregistration.k8s.io\", \"apiextensions.k8s.io\", \"scheduling.k8s.io\", \"coordination.k8s.io\", \"node.k8s.io\", \"apps.openshift.io\", \"authorization.openshift.io\", \"build.openshift.io\", \"image.openshift.io\", \"oauth.openshift.io\", \"project.openshift.io\", \"quota.openshift.io\", \"route.openshift.io\", \"security.openshift.io\", \"template.openshift.io\", \"user.openshift.io\", \"servicecatalog.k8s.io\", \"packages.operators.coreos.com\", \"config.openshift.io\", \"operator.openshift.io\", \"autoscaling.openshift.io\", \"cloudcredential.openshift.io\", \"console.openshift.io\", \"imageregistry.operator.openshift.io\", \"ingress.operator.openshift.io\", \"k8s.cni.cncf.io\", \"machineconfiguration.openshift.io\", \"monitoring.coreos.com\", \"network.openshift.io\", \"operators.coreos.com\", \"osb.openshift.io\", \"samples.operator.openshift.io\", \"tuned.openshift.io\", \"automationbroker.io\", \"healthchecking.openshift.io\", \"metal3.io\", \"machine.openshift.io\", \"metrics.k8s.io\"]}, \"changed\": false}\u001b[0m\n\r\nTASK [ansible-service-broker : Set reconciled_generation and generation facts] ***\r\n\u001b[1;30mtask path: /opt/ansible/roles/ansible-service-broker/tasks/main.yml:13\u001b[0m\n\u001b[0;32mok: [localhost] => {\"ansible_facts\": {\"generation\": \"1\", \"reconciled_generation\": \"\"}, \"changed\": false}\u001b[0m\n\r\nTASK [ansible-service-broker : Set pending_config_changes fact] ****************\r\n\u001b[1;30mtask path: /opt/ansible/roles/ansible-service-broker/tasks/main.yml:32\u001b[0m\n\u001b[0;32mok: [localhost] => {\"ansible_facts\": {\"pending_config_changes\": true}, \"changed\": false}\u001b[0m\n\r\nTASK [ansible-service-broker : Environment Validation] *************************\r\n\u001b[1;30mtask path: /opt/ansible/roles/ansible-service-broker/tasks/validate_present.yml:4\u001b[0m\n\u001b[0;32mok: [localhost] => {\u001b[0m\r\n\u001b[0;32m \"changed\": false, \u001b[0m\r\n\u001b[0;32m \"msg\": ... u001b[1;30mtask path: /opt/ansible/roles/ansible-service-broker/tasks/main.yml:47\u001b[0m\n\u001b[0;36mskipping: [localhost] => {\"changed\": false, \"skip_reason\": \"Conditional result was False\"}\u001b[0m\n\r\nTASK [ansible-service-broker : fail] *******************************************\r\n\u001b[1;30mtask path: /opt/ansible/roles/ansible-service-broker/tasks/main.yml:51\u001b[0m\n\u001b[0;31mfatal: [localhost]: FAILED! => {\"changed\": false, \"msg\": \"Unable to create broker-admin clusterrolebinding\"}\u001b[0m\n\r\nPLAY RECAP *********************************************************************\r\n\u001b[0;31mlocalhost\u001b[0m : \u001b[0;32mok=6 \u001b[0m changed=0 unreachable=0 \u001b[0;31mfailed=1 \u001b[0m \u001b[0;36mskipped=1 \u001b[0m rescued=0 \u001b[1;35mignored=1 \u001b[0m\r\n\n","job":"8674665223082153551","name":"ansible-service-broker","namespace":"openshift-ansible-service-broker","error":"exit status 2","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\tsrc/github.com/operator-framework/operator-sdk/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.com/operator-framework/operator-sdk/pkg/ansible/runner.(*runner).Run.func1\n\tsrc/github.com/operator-framework/operator-sdk/pkg/ansible/runner/runner.go:190"}
Confirmed with the provided cluster, it looks like the manual step to create a ClusterRoleBinding for the ansible-service-broker-operator serviceaccount was skipped, so this failure is expected. The process for creating the ClusterRoleBinding is detailed is step 2 of this document: https://docs.openshift.com/container-platform/4.1/applications/service_brokers/installing-ansible-service-broker.html#sb-install-asb-operator_sb-installing-asb (the doc is for 4.1 but the process has not changed for 4.2)
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:2922