Bug 1476173
| Summary: | Delete project can't delete the instance/bindings and other user can get it if have same name project | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | DeShuai Ma <dma> |
| Component: | Service Broker | Assignee: | cchase |
| Status: | CLOSED ERRATA | QA Contact: | Jian Zhang <jiazha> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 3.6.0 | CC: | aos-bugs, cchase, chezhang, decarr, dma, jaboyd, jiazha, jmatthew, jokerman, mclerix, mmccomas, pmorie, shurley, xtian |
| Target Milestone: | --- | ||
| Target Release: | 3.7.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: |
Cause: During namespace deletion, Ansible Service Broker (ASB) is attempting to execute deprovision playbook actions using a namespace in a "terminating" state.
Consequence: The APB actions are rejected because the namespace is terminating. As a result deprovision fails, and both the APB deprovision sandbox and target namespace are not deleted.
Fix: Instead of executing APB actions on namespace deletion, clean up all records of the services to be deprovisioned, return success, and allow kubernetes to delete the resources normally.
Result: During namespace deletion:
- APB sandbox is not created
- APB deprovision actions are not executed
- ASB persisted data is cleaned up
- Target namespace is properly deleted by kubernetes
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2017-11-28 22:06:30 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
DeShuai Ma
2017-07-28 08:11:07 UTC
The env is installed by openshift-ansible Paul - please triage further, possibly want to doc this as known issue for 3.6, definitely must fix for 3.7. Hi guys, I experience the same issue. Is there currently a workaround to remove remaining bindings and instances ? Using oc delete does not work at all: [root@ocp-master01 archi]# oc get bindings.servicecatalog.k8s.io NAME KIND mongodb-ephemeral-4dzlt-d5nwm Binding.v1alpha1.servicecatalog.k8s.io [root@ocp-master01 archi]# oc delete bindings.servicecatalog.k8s.io mongodb-ephemeral-4dzlt-d5nwm binding "mongodb-ephemeral-4dzlt-d5nwm" deleted [root@ocp-master01 archi]# oc get bindings.servicecatalog.k8s.io NAME KIND mongodb-ephemeral-4dzlt-d5nwm Binding.v1alpha1.servicecatalog.k8s.io Thank you in advance Maxime- When you delete these resources, they have to be finalized before they are fully deleted. This requires communication with the service broker that offers the service, so you will rarely see these resources be fully deleted immediately. You can check the status of the ServiceBinding to get information about what's happening to the object. This seems to work properly with current latest build, but I will retest after https://github.com/openshift/origin/pull/16908 is merged. Several notes: if there is an error with the binding, the error condition may block the deletion of the instance and binding. I've got at least 2 example error conditions: 1) create an instance that references a serviceclass from TSB such as mysql-persistent (update service-catalog/go/src/github.com/kubernetes-incubator/service-catalog/contrib/examples/walkthrough/ups-instance.yaml and then oc create -f ups-instance.yaml). Then create a binding for this instance. If you look at the controller logs, you will see errors around the serviceclass, and the instance and binding are in a error condition, ready=false. 2) I installed Ansible Service Broker and then created a hastebin application. Once this was deployed I created a binding for it. In the controller logs, there are errors/warnings like this: type: 'Warning' reason: 'ErrorNonbindableServiceClass' References a non-bindable ClusterServiceClass (K8S: "ab24ffd54da0aefdea5277e0edce8425" ExternalName: "dh-hastebin-apb") and Plan ("default") combination in both cases, because the binding is in an error state, you can't delete it and it blocks cleaning up the associated instance. I'll create an upstream service catalog bug for this and come back with a link. upstream issue for items in comment 5 https://github.com/kubernetes-incubator/service-catalog/issues/1423 This all looks to work properly. If using an Ansible installed OpenShift, please ensure it has https://github.com/openshift/openshift-ansible/pull/5746 which was merged today. I'm not sure I understand this comment:
> I found there will always leave a new namespace after delete a target namespace.
Can you clarify what you meant here?
The pod in question appears to be a pod that the ansible broker left running. The ansible broker runs pods in transient namespaces and these are unrelated to the namespace that the service catalog resources (ServiceInstances, ServiceBindings) were created in.
Yeah...but the transient namespaces exist all the way, as above shows, the transient namespace "a5fc66c0-e4da-4b9b-a23c-ff7609068e7e" has kept alive 17 hours, is it normally? [root@preserve-jiazha-1024master-etcd-1 ~]# oc get ns | grep a5fc66c0-e4da-4b9b-a23c-ff7609068e7e a5fc66c0-e4da-4b9b-a23c-ff7609068e7e Active 17h In my opinion, the transient namespace should be deleted immediately after deleting the namespace that user created, right? Shawn, Please see if you can reproduce and gain insight into what might be wrong. 1. user deletes the project/namespace. 2. svc catalog sends us a deprovision 3. we attempt to create a svc-acct that has access to deleted namesapce 4. we create the Transient Namespace and run the pod, which can not access the Target Namespace because it does not exist 5. Pod fails and we keep the transient namespace around due to configuration. We can on deprovision request check if the target namespace is gone and then say the deprovision was successful. 1. user deletes the project/namespace. 2. svc catalog sends us a deprovision 3. we attempt to create a svc-acct that has access to deleted namesapce 4. we create the Transient Namespace and run the pod, which can not access the Target Namespace because it does not exist 5. Pod fails and we keep the transient namespace around due to configuration. We can on deprovision request check if the target namespace is gone and then say the deprovision was successful. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2017:3188 |