Bug 1623918

Summary:	Failed to unbind after deleting templateinstance with servicebinding existing
Product:	OpenShift Container Platform	Reporter:	Vladislav Walek <vwalek>
Component:	Service Catalog	Assignee:	Jay Boyd <jaboyd>
Status:	CLOSED NOTABUG	QA Contact:	Jian Zhang <jiazha>
Severity:	urgent	Docs Contact:
Priority:	urgent
Version:	3.7.0	CC:	aos-bugs, chezhang, hchatter, jaboyd, jiazha, jkim, jmatthew, openshift-bugs-escalate, zitang
Target Milestone:	---
Target Release:	4.1.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2018-10-25 13:11:21 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Vladislav Walek 2018-08-30 12:50:34 UTC

Description of problem:

reporting the same issue as in https://bugzilla.redhat.com/show_bug.cgi?id=1540819
customer is still facing the issue with 3.7.54

Version-Release number of selected component (if applicable):
OCP 3.7.54

How reproducible:
- I created a persistent mysql db from gui-template without having a pv available
- The project got stuck while it was requesting a pv.
- oc delete project undeletable-test

- gave this result:
- oc get projects|grep undelete

undeletable-test                    undeletable-test                   Terminating

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
I recreated the same project again and deleted it.... it's in "Terminating" state.
15 minutes: still Terminating.

also still terminating after 1 hour

Comment 9 jkim 2018-10-24 21:41:58 UTC

I was able to reproduce this in v3.7.69 

The provision does not fail, but it does not successfully complete and gets stuck in provision since there are no available PV’s.  When this happens, the only way to remove the project/namespace is to remove the finalizer on the ServiceInstance.  The finalizer prevents the ServiceInstance from being deleted so that the user has the chance to investigate the failure, and to clean up any remote resources that may still exist.  This is the expected behavior and I recommend that this be marked as NOTABUG.

Please see Jay's comment with more background and workaround: https://bugzilla.redhat.com/show_bug.cgi?id=1626351#c2

Comment 10 Jay Boyd 2018-10-25 13:11:21 UTC

The Broker development team and Service Catalog team have both reviewed & discussed these details and are in agreement that this scenario is functioning as designed.  

In cases like this where the Broker is unable to guarantee all allocated resources are freed, we don't want to potentially orphan resources that may continue to incur a charge to the end user.  There is no safe automated way to handle these cases (again in terms of ensuring backend resources are not abandoned without appropriate cleanup), users must review the details and if they still desire to remove the OpenShift resources, they must remove the finalizer.

We will be developing additional tooling to make the forced removal of "stuck" or "wedged" Instances or Bindings easier (https://github.com/kubernetes-incubator/service-catalog/issues/2268).