*** Bug 1793209 has been marked as a duplicate of this bug. ***
Hi Xingxing Xia, Can you share some details on how you are going to verify this bug? Thanks!
Hi xxia, We found an issue with CVO - it uses a protobuf client for `SecurityContextConstraints`. So we had to fix this issue in order for us to move the default SCC to cvo manifests in 'cluster-kube-apiserver-operator'. The CVO fix has the following BZ and PR https://bugzilla.redhat.com/show_bug.cgi?id=1793209 https://github.com/openshift/cluster-version-operator/pull/308 And the actual move of the default SCC is here - Thi BZ - https://bugzilla.redhat.com/show_bug.cgi?id=1794309 https://github.com/openshift/cluster-kube-apiserver-operator/pull/725 We are also back porting this fix to 4.3, for this we have the following PRs cherry-picked for 4.3. We cloned the BZs above as well. https://github.com/openshift/cluster-kube-apiserver-operator/pull/728 https://github.com/openshift/cluster-version-operator/pull/310 Once you validate that default SCC is being created by cvo when it deploys cluster-kube-apiserver-operator we want to disable default SCC bootstrap in openshift-apiserver. I have a PR open for that but it's in hold right now, waiting for qe to approve this this BZ first. https://github.com/openshift/openshift-apiserver/pull/67 Now, in terms of testing this BZ, in addition to doing scc related regression tests, I would recommend that we need to validate that the default SCC are being created by CVO not openshift-apiserver. - If you check the api audit log the create issued by openshift-apiserver for the default SCC shuold have a response of `AlreadyExists` - the api audit log will not have any create issued by cvo. For that you might need to check the api audit log of the bootstrap kube-apiserver
Thanks Abu Kashem for the detailed info. scc related regression tests not see issue in latest 4.4.0-0.nightly-2020-02-10-013941 daily test schedule. Checked audit, found oas scc creations with AlreadyExists which to be solved by https://github.com/openshift/openshift-apiserver/pull/67 : [root@xxiarebuild-zcpst-control-plane-0 log]# grep -nrE "securitycontext.*create" kube-apiserver/ kube-apiserver/audit.log:47696:{"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"1ec0ee24-a657-4236-bebe-958348ad85f1","stage":"ResponseComplete","requestURI":"/apis/security.openshift.io/v1/securitycontextconstraints","verb":"create","user":{"username":"system:serviceaccount:openshift-apiserver:openshift-apiserver-sa","uid":"a0bc3021-8481-4913-9712-a24a125c0d5c","groups":["system:serviceaccounts","system:serviceaccounts:openshift-apiserver","system:authenticated"]},"sourceIPs":["10.0.97.182"],"userAgent":"openshift-apiserver/v0.0.0 (linux/amd64) kubernetes/$Format","objectRef":{"resource":"securitycontextconstraints","name":"privileged","apiGroup":"security.openshift.io","apiVersion":"v1"},"responseStatus":{"metadata":{},"status":"Failure","reason":"AlreadyExists","code":409},"requestReceivedTimestamp":"2020-02-11T12:51:06.435715Z","stageTimestamp":"2020-02-11T12:51:06.459941Z","annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC: allowed by ClusterRoleBinding \"system:openshift:openshift-apiserver\" of ClusterRole \"cluster-admin\" to ServiceAccount \"openshift-apiserver-sa/openshift-apiserver\""}} kube-apiserver/audit.log:47698:{"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"c39356c7-266e-4c60-ac25-3304de314cc4","stage":"ResponseComplete","requestURI":"/apis/security.openshift.io/v1/securitycontextconstraints","verb":"create","user":{"username":"system:serviceaccount:openshift-apiserver:openshift-apiserver-sa","uid":"a0bc3021-8481-4913-9712-a24a125c0d5c","groups":["system:serviceaccounts","system:serviceaccounts:openshift-apiserver","system:authenticated"]},"sourceIPs":["10.0.97.182"],"userAgent":"openshift-apiserver/v0.0.0 (linux/amd64) kubernetes/$Format","objectRef":{"resource":"securitycontextconstraints","name":"nonroot","apiGroup":"security.openshift.io","apiVersion":"v1"},"responseStatus":{"metadata":{},"status":"Failure","reason":"AlreadyExists","code":409},"req ...snipped...
(In reply to Abu Kashem from comment #6) > ... validate that default SCC is being created by cvo when it deploys cluster-kube-apiserver-operator > ... you might need to check the api audit log of the bootstrap kube-apiserver Abu, On bootstrapper, I checked /var/log/bootstrap-control-plane/kube-apiserver.log also checked journalctl -b -f -u bootkube.service, not sure the right details. Could you give more details about where to check "the api audit log of the bootstrap kube-apiserver", how to check "SCC is being created by cvo when it deploys cluster-kube-apiserver-operator"? Thanks
"BTW, also found recreations: "username":"system:serviceaccount:openshift-apiserver:openshift-apiserver-sa", "userAgent":"openshift-apiserver, "409", I might file another bug later too:" Hi xxia, this is known, I mentioned this is in my comment https://bugzilla.redhat.com/show_bug.cgi?id=1794309#c6. "Once you validate that default SCC is being created by cvo when it deploys cluster-kube-apiserver-operator we want to disable default SCC bootstrap in openshift-apiserver. I have a PR open for that but it's in hold right now, waiting for qe to approve this this BZ first. https://github.com/openshift/openshift-apiserver/pull/67"
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:0581