Bug 1470624
Summary: | Can't delete broker resource after setup env by openshift-ansible | ||||||||
---|---|---|---|---|---|---|---|---|---|
Product: | OpenShift Container Platform | Reporter: | DeShuai Ma <dma> | ||||||
Component: | Installer | Assignee: | ewolinet | ||||||
Status: | CLOSED ERRATA | QA Contact: | DeShuai Ma <dma> | ||||||
Severity: | high | Docs Contact: | |||||||
Priority: | high | ||||||||
Version: | 3.6.0 | CC: | aos-bugs, decarr, dma, ewolinet, jokerman, jpeeler, mmccomas, pmorie | ||||||
Target Milestone: | --- | ||||||||
Target Release: | --- | ||||||||
Hardware: | Unspecified | ||||||||
OS: | Unspecified | ||||||||
Whiteboard: | |||||||||
Fixed In Version: | Doc Type: | If docs needed, set a value | |||||||
Doc Text: | Story Points: | --- | |||||||
Clone Of: | Environment: | ||||||||
Last Closed: | 2017-08-10 05:31:01 UTC | Type: | Bug | ||||||
Regression: | --- | Mount Type: | --- | ||||||
Documentation: | --- | CRM: | |||||||
Verified Versions: | Category: | --- | |||||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
Cloudforms Team: | --- | Target Upstream Version: | |||||||
Embargoed: | |||||||||
Attachments: |
|
Description
DeShuai Ma
2017-07-13 10:28:34 UTC
The logs didn't seem to make it. Created attachment 1298508 [details]
apiserver.log
Created attachment 1298510 [details]
controller-manager.log
Try to reproduce and debug. I find some logs: W0717 02:45:41.134178 1 controller_broker.go:276] Error deleting ServiceClass "mediawiki-apb" (Broker "ansible-service-broker"): User "system:serviceaccount:kube-service-catalog:service-catalog-controller" cannot delete serviceclasses.servicecatalog.k8s.io at the cluster scope I0717 02:45:41.134216 1 controller_broker.go:391] Found status change for Broker "ansible-service-broker" condition "Ready": "False" -> "Unknown"; setting lastTransitionTime to 2017-07-17 02:45:41.134207935 +0000 UTC I0717 02:45:41.134245 1 controller_broker.go:403] Updating ready condition for Broker ansible-service-broker to Unknown E0717 02:45:41.151083 1 controller_broker.go:406] Error updating ready condition for Broker ansible-service-broker: Operation cannot be fulfilled on brokers "ansible-service-broker": the object has been modified; please apply your changes to the latest version and try again I0717 02:45:41.151153 1 controller.go:195] Error syncing Broker ansible-service-broker: User "system:serviceaccount:kube-service-catalog:service-catalog-controller" cannot delete serviceclasses.servicecatalog.k8s.io at the cluster scope I0717 02:45:41.151380 1 event.go:217] Event(v1.ObjectReference{Kind:"Broker", Namespace:"", Name:"ansible-service-broker", UID:"8306dbc0-6a98-11e7-84c7-0a580a810003", APIVersion:"servicecatalog.k8s.io", ResourceVersion:"5038", FieldPath:""}): type: 'Warning' reason: 'ErrorDeletingServiceClass' Error deleting service class. Error deleting ServiceClass "mediawiki-apb" (Broker "ansible-service-broker"): User "system:serviceaccount:kube-service-catalog:service-catalog-controller" cannot delete serviceclasses.servicecatalog.k8s.io at the cluster scope I0717 02:45:41.162144 1 controller_broker.go:161] Processing Broker ansible-service-broker I0717 02:45:41.162174 1 controller_broker.go:182] Creating client for Broker ansible-service-broker, URL: http://asb.openshift-ansible-service-broker.svc:1338 I0717 02:45:41.162190 1 controller_broker.go:249] Finalizing Broker ansible-service-broker Version to reproduce: openshift-ansible-3.6.151-1.git.0.a82f0c2.el7.noarch.rpm openshift v3.6.151 kubernetes v1.6.1+5115d708d7 etcd 3.2.1 As I install the env by openshift-ansible. move to installer. This is target release 3.7 as catalog is tech preview in 3.6, and therefore, we will not block a release for any bugs associated. Jeff - can you look at the logs further to isolate the actual problem? My understanding is that if the controller is not available it's expected behavior for deletions to not be processed. But I will confirm later with additional investigation. My describe the issue clear again: After enable serivce-catalog by openshift-ansible. Then delete broker I met this error. Why I need delete broker? Because there are two bug about auto update serviceclass: https://bugzilla.redhat.com/show_bug.cgi?id=1468173 https://bugzilla.redhat.com/show_bug.cgi?id=1469448 I must delete broker then recreate. Otherwise serviceclass is empty. User can't see any apb service in console. From the controller-manage log: I0719 10:05:00.959306 1 controller.go:200] Dropping Broker "ansible-service-broker" out of the queue: User "system:serviceaccount:kube-service-catalog:service-catalog-controller" cannot delete serviceclasses.servicecatalog.k8s.io at the cluster scope So I think we have set wrong permission for the sa in installer, So I think we need fix for the installer in ocp36. aggregate that? If I'm wrong, please correct me, thanks. After grant permission (actually we don't need so large permission) I can delete broker successfully [root@host-8-175-72 ~]# oadm policy add-cluster-role-to-user cluster-admin system:serviceaccount:kube-service-catalog:service-catalog-controller cluster role "cluster-admin" added: "system:serviceaccount:kube-service-catalog:service-catalog-controller" [root@host-8-175-72 ~]# oc get broker NAME KIND ansible-service-broker Broker.v1alpha1.servicecatalog.k8s.io [root@host-8-175-72 ~]# oc delete broker ansible-service-broker broker "ansible-service-broker" deleted [root@host-8-175-72 ~]# oc get broker No resources found. Seems like a problem with the rbac setup created by the installer - reassigning to eric. I'd like to make sure we can get all missing permissions at once... should the service-catalog-controller be able to perform any other operations (update/patch/list/watch) other than create/delete on serviceclasses? It needs all of that actually. This link will be helpful: https://github.com/kubernetes-incubator/service-catalog/blob/8ec08745bf8c690c1b14a3da9a9e385d44739fb4/charts/catalog/templates/rbac.yaml#L90 Verify on openshift-ansible-3.6.162-1.git.0.50e29bd.el7.noarch.rpm. Now can delete broker successfully after install. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:1716 |