Ke Wang, pls refer to the verification steps of the same bug with other version and verify in 4.4 env, thanks

Verified with the following ENV:
$ oc version
Client Version: v4.4.0
Server Version: 4.4.0-0.nightly-2020-01-05-221122
Kubernetes Version: v1.17.0

- Steps,
$ oc edit secret encryption-config encryption-config-{7..8} -n openshift-kube-apiserver
secret/encryption-config edited
secret/encryption-config-7 edited
secret/encryption-config-8 edited

$ oc delete project openshift-kube-apiserver
project.project.openshift.io "openshift-kube-apiserver" deleted

$ oc get ns openshift-kube-apiserver -w
NAME                       STATUS        AGE
openshift-kube-apiserver   Terminating   132m

To find the reason why openshift-kube-apiserver is in terminating
$ oc get ns openshift-kube-apiserver -o yaml
...
- lastTransitionTime: "2020-01-06T07:45:14Z"
    message: 'Some content in the namespace has finalizers remaining: encryption.apiserver.operator.openshift.io/deletion-protection
      in 1 resource instances'
    reason: SomeFinalizersRemain
    status: "True"
...

Find still one secret existing, deleted it.
$ oc get secret -n openshift-kube-apiserver
NAME                  TYPE     DATA   AGE
encryption-config-9   Opaque   1      59m

$ oc edit secret -n openshift-kube-apiserver

Check the openshift-kube-apiserver status again, it's activated
$ oc get ns openshift-kube-apiserver
NAME                       STATUS   AGE
openshift-kube-apiserver   Active   58s

After several minutes, check pod, ClusterOperator and node status,

 oc get po -A | grep -vE "(Running|Completed)"; oc get co; oc get no

$ oc get co --no-headers | grep -v samples| grep -v "True.*False.*False" # none

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0581