Bug 1794309 - Move default SCC creation from openshift-apiserver to kube-apiserver operator
Summary: Move default SCC creation from openshift-apiserver to kube-apiserver operator
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: kube-apiserver
Version: 4.4
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: 4.4.0
Assignee: Stefan Schimanski
QA Contact: Xingxing Xia
URL:
Whiteboard:
Depends On: 1793209 1794454
Blocks: 1794468 1808602
TreeView+ depends on / blocked
 
Reported: 2020-01-23 09:17 UTC by Michal Fojtik
Modified: 2020-05-04 11:27 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1793209
: 1794468 (view as bug list)
Environment:
Last Closed: 2020-05-04 11:26:35 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Github openshift cluster-kube-apiserver-operator pull 725 None closed Bug 1794309: Copy default SCC into release image 2020-05-19 13:38:33 UTC
Red Hat Product Errata RHBA-2020:0581 None None None 2020-05-04 11:27:08 UTC

Comment 2 Xingxing Xia 2020-01-23 10:07:14 UTC
*** Bug 1793209 has been marked as a duplicate of this bug. ***

Comment 4 Abu Kashem 2020-01-27 14:46:54 UTC
Hi Xingxing Xia,
Can you share some details on how you are going to verify this bug?

Thanks!

Comment 6 Abu Kashem 2020-02-10 18:09:02 UTC
Hi xxia@redhat.com,

We found an issue with CVO - it uses a protobuf client for `SecurityContextConstraints`. So we had to fix this issue in order for us to move the default SCC to cvo manifests in 'cluster-kube-apiserver-operator'. 

The CVO fix has the following BZ and PR
https://bugzilla.redhat.com/show_bug.cgi?id=1793209
https://github.com/openshift/cluster-version-operator/pull/308

And the actual move of the default SCC is here -
Thi BZ - https://bugzilla.redhat.com/show_bug.cgi?id=1794309
https://github.com/openshift/cluster-kube-apiserver-operator/pull/725


We are also back porting this fix to 4.3, for this we have the following PRs cherry-picked for 4.3. We cloned the BZs above as well. 
https://github.com/openshift/cluster-kube-apiserver-operator/pull/728
https://github.com/openshift/cluster-version-operator/pull/310

Once you validate that default SCC is being created by cvo when it deploys cluster-kube-apiserver-operator we want to disable default SCC bootstrap in openshift-apiserver. I have a PR open for that but it's in hold right now, waiting for qe to approve this this BZ first.
https://github.com/openshift/openshift-apiserver/pull/67

Now, in terms of testing this BZ, in addition to doing scc related regression tests, I would recommend that we need to validate that the default SCC are being created by CVO not openshift-apiserver.
- If you check the api audit log the create issued by openshift-apiserver for the default SCC shuold have a response of `AlreadyExists`
- the api audit log will not have any create issued by cvo. For that you might need to check the api audit log of the bootstrap kube-apiserver

Comment 7 Xingxing Xia 2020-02-11 13:20:11 UTC
Thanks Abu Kashem for the detailed info.
scc related regression tests not see issue in latest 4.4.0-0.nightly-2020-02-10-013941 daily test schedule.
Checked audit, found oas scc creations with AlreadyExists which to be solved by https://github.com/openshift/openshift-apiserver/pull/67 :
[root@xxiarebuild-zcpst-control-plane-0 log]# grep -nrE "securitycontext.*create" kube-apiserver/
kube-apiserver/audit.log:47696:{"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"1ec0ee24-a657-4236-bebe-958348ad85f1","stage":"ResponseComplete","requestURI":"/apis/security.openshift.io/v1/securitycontextconstraints","verb":"create","user":{"username":"system:serviceaccount:openshift-apiserver:openshift-apiserver-sa","uid":"a0bc3021-8481-4913-9712-a24a125c0d5c","groups":["system:serviceaccounts","system:serviceaccounts:openshift-apiserver","system:authenticated"]},"sourceIPs":["10.0.97.182"],"userAgent":"openshift-apiserver/v0.0.0 (linux/amd64) kubernetes/$Format","objectRef":{"resource":"securitycontextconstraints","name":"privileged","apiGroup":"security.openshift.io","apiVersion":"v1"},"responseStatus":{"metadata":{},"status":"Failure","reason":"AlreadyExists","code":409},"requestReceivedTimestamp":"2020-02-11T12:51:06.435715Z","stageTimestamp":"2020-02-11T12:51:06.459941Z","annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC: allowed by ClusterRoleBinding \"system:openshift:openshift-apiserver\" of ClusterRole \"cluster-admin\" to ServiceAccount \"openshift-apiserver-sa/openshift-apiserver\""}}
kube-apiserver/audit.log:47698:{"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"c39356c7-266e-4c60-ac25-3304de314cc4","stage":"ResponseComplete","requestURI":"/apis/security.openshift.io/v1/securitycontextconstraints","verb":"create","user":{"username":"system:serviceaccount:openshift-apiserver:openshift-apiserver-sa","uid":"a0bc3021-8481-4913-9712-a24a125c0d5c","groups":["system:serviceaccounts","system:serviceaccounts:openshift-apiserver","system:authenticated"]},"sourceIPs":["10.0.97.182"],"userAgent":"openshift-apiserver/v0.0.0 (linux/amd64) kubernetes/$Format","objectRef":{"resource":"securitycontextconstraints","name":"nonroot","apiGroup":"security.openshift.io","apiVersion":"v1"},"responseStatus":{"metadata":{},"status":"Failure","reason":"AlreadyExists","code":409},"req
...snipped...

Comment 8 Xingxing Xia 2020-02-21 13:31:13 UTC
(In reply to Abu Kashem from comment #6)
> ... validate that default SCC is being created by cvo when it deploys cluster-kube-apiserver-operator
> ... you might need to check the api audit log of the bootstrap kube-apiserver
Abu, On bootstrapper, I checked /var/log/bootstrap-control-plane/kube-apiserver.log also checked journalctl -b -f -u bootkube.service, not sure the right details.
Could you give more details about where to check "the api audit log of the bootstrap kube-apiserver", how to check "SCC is being created by cvo when it deploys cluster-kube-apiserver-operator"? Thanks

Comment 11 Abu Kashem 2020-02-24 13:40:48 UTC
"BTW, also found recreations: "username":"system:serviceaccount:openshift-apiserver:openshift-apiserver-sa", "userAgent":"openshift-apiserver, "409", I might file another bug later too:"

Hi xxia@redhat.com,
this is known, I mentioned this is in my comment https://bugzilla.redhat.com/show_bug.cgi?id=1794309#c6. 

"Once you validate that default SCC is being created by cvo when it deploys cluster-kube-apiserver-operator we want to disable default SCC bootstrap in openshift-apiserver. I have a PR open for that but it's in hold right now, waiting for qe to approve this this BZ first.
https://github.com/openshift/openshift-apiserver/pull/67"

Comment 13 errata-xmlrpc 2020-05-04 11:26:35 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0581


Note You need to log in before you can comment on or make changes to this bug.