Bug 1780809

Summary: [MSTR-485] co/kube-apiserver stuck in Progressing and Degraded after deleting openshift-kube-apiserver
Product: OpenShift Container Platform Reporter: Luis Sanchez <sanchezl>
Component: kube-apiserverAssignee: Luis Sanchez <sanchezl>
Status: CLOSED ERRATA QA Contact: Ke Wang <kewang>
Severity: medium Docs Contact:
Priority: medium    
Version: 4.4CC: aos-bugs, mfojtik, sttts, xxia
Target Milestone: ---   
Target Release: 4.4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1775580 Environment:
Last Closed: 2020-05-04 11:19:04 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 1775580    

Comment 2 Xingxing Xia 2019-12-30 10:48:01 UTC
Ke Wang, pls refer to the verification steps of the same bug with other version and verify in 4.4 env, thanks

Comment 3 Ke Wang 2020-01-06 09:02:24 UTC
Verified with the following ENV:
$ oc version
Client Version: v4.4.0
Server Version: 4.4.0-0.nightly-2020-01-05-221122
Kubernetes Version: v1.17.0

- Steps,
$ oc edit secret encryption-config encryption-config-{7..8} -n openshift-kube-apiserver
secret/encryption-config edited
secret/encryption-config-7 edited
secret/encryption-config-8 edited

$ oc delete project openshift-kube-apiserver
project.project.openshift.io "openshift-kube-apiserver" deleted

$ oc get ns openshift-kube-apiserver -w
NAME                       STATUS        AGE
openshift-kube-apiserver   Terminating   132m

To find the reason why openshift-kube-apiserver is in terminating
$ oc get ns openshift-kube-apiserver -o yaml
...
- lastTransitionTime: "2020-01-06T07:45:14Z"
    message: 'Some content in the namespace has finalizers remaining: encryption.apiserver.operator.openshift.io/deletion-protection
      in 1 resource instances'
    reason: SomeFinalizersRemain
    status: "True"
...

Find still one secret existing, deleted it.
$ oc get secret -n openshift-kube-apiserver
NAME                  TYPE     DATA   AGE
encryption-config-9   Opaque   1      59m

$ oc edit secret -n openshift-kube-apiserver

Check the openshift-kube-apiserver status again, it's activated
$ oc get ns openshift-kube-apiserver
NAME                       STATUS   AGE
openshift-kube-apiserver   Active   58s

After several minutes, check pod, ClusterOperator and node status,

 oc get po -A | grep -vE "(Running|Completed)"; oc get co; oc get no

$ oc get co --no-headers | grep -v samples| grep -v "True.*False.*False" # none

Comment 5 errata-xmlrpc 2020-05-04 11:19:04 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0581