Bug 1745567

Summary:	Project stuck in Terminating
Product:	OpenShift Container Platform	Reporter:	Michal Fojtik <mfojtik>
Component:	kube-controller-manager	Assignee:	Maciej Szulik <maszulik>
Status:	CLOSED WORKSFORME	QA Contact:	zhou ying <yinzhou>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	4.2.0	CC:	aos-bugs, eparis, jnaess, jokerman, maszulik, mfojtik, xxia, yinzhou
Target Milestone:	---
Target Release:	4.2.0
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:	1743273	Environment:
Last Closed:	2019-09-04 13:28:59 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1743273

Description Michal Fojtik 2019-08-26 11:24:05 UTC

+++ This bug was initially created as a clone of Bug #1743273 +++

Description of problem: Namespace locked in "Terminating" state.


Version-Release number of selected component (if applicable): 4.1.6


How reproducible: unknown


Steps to Reproduce:
1. oc delete project codeready
2. oc get projects

Actual results:
Error from server (Conflict): Operation cannot be fulfilled on namespaces "codeready": The system is ensuring all content is removed from this namespace.  Upon completion, this namespace will automatically be purged by the system.

Expected results:
Project deleted.

Additional info:
Looking online, there are some issues reported on github regarding this.
https://github.com/kubernetes/kubernetes/issues/60807

It seems to be an issue with "finalizers" in the spec of the project.
Deleting the line "- kubernetes" via oc edit project does not work. However, doing the following does fix it: https://success.docker.com/article/kubernetes-namespace-stuck-in-terminating

Raised to get awareness of it in OpenShift 4.1.z

--- Additional comment from Michal Fojtik on 2019-08-26 09:55:35 UTC ---

Can you follow https://docs.google.com/document/d/13YIo4z8r2U5h03cwJVADALHN_dYT0Odp7rgvnUo3TdE/edit?usp=sharing this document, especially the "For Garbage Collector or not able to delete API resources" ?

Comment 1 Michal Fojtik 2019-08-26 11:25:08 UTC

Can QA verify if this bug is reproducible in 4.2?

Comment 3 zhou ying 2019-08-27 08:00:19 UTC

Confirmed with payload : 4.2.0-0.nightly-2019-08-25-233755, when delete project with finalizers, as normal user could not see the project , but can't recreate the same name project:
[root@dhcp-140-138 yamlfile]# oc whoami 
testuser-45
[root@dhcp-140-138 yamlfile]# oc get project
No resources found.
[root@dhcp-140-138 yamlfile]# oc new-project fftest
Error from server (AlreadyExists): project.project.openshift.io "fftest" already exists


As cluster-admin we still could see the project with status : Terminating.
[root@192 yamlfile]# oc whoami
system:admin
[root@192 yamlfile]# oc get project fftest
NAME     DISPLAY NAME   STATUS
fftest                  Terminating
[root@192 yamlfile]# oc get project fftest -o yaml
apiVersion: project.openshift.io/v1
kind: Project
metadata:
  annotations:
    openshift.io/description: ""
    openshift.io/display-name: ""
    openshift.io/requester: testuser-45
    openshift.io/sa.scc.mcs: s0:c28,c22
    openshift.io/sa.scc.supplemental-groups: 1000800000/10000
    openshift.io/sa.scc.uid-range: 1000800000/10000
  creationTimestamp: "2019-08-27T06:42:07Z"
  deletionTimestamp: "2019-08-27T07:54:42Z"
  name: fftest
  resourceVersion: "479543"
  selfLink: /apis/project.openshift.io/v1/projects/fftest
  uid: c9fbf43f-c895-11e9-baeb-02a5d4eb8c88
spec:
  finalizers:
  - kubernetes
status:
  phase: Terminating

Comment 4 zhou ying 2019-08-27 09:10:13 UTC

Ref https://bugzilla.redhat.com/show_bug.cgi?id=1745567#c3, Could verify this issue ?

Comment 5 Xingxing Xia 2019-09-04 13:28:59 UTC

Cannot reproduce in 4.2.0-0.nightly-2019-09-03-102130 env:
Let the project creator delete:
oc delete project xxia1-proj

Let cluster-admin watch:
oc get project -w
xxia1-proj                                                                     Terminating

After seconds, the project is gone:
oc get project -w # no xxia1-proj shown

Comment 6 Red Hat Bugzilla 2023-09-14 05:42:17 UTC

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days