Description of problem: openshift-apiserver and Openshift-oauth-apiserver pods fails to initialize when a custom-SCC is created which disallows running as root(Whatever the priority may be). Usually openshift-apiserver and Openshift-oauth-apiserver pods uses node-exporter/Privileged SCC which has a priority of `null`. openshift-apiserver and Openshift-oauth-apiserver pods requires running as root but doesn't explicitly have it in their security context. This cause it to mistakenly apply the SCC which doesn't allow running as root. This usually fails the OCP upgrade Version-Release number of selected component (if applicable): 4.7 How reproducible: Consistently Reproducable Steps to Reproduce: 1. Verify the pods status and SCC applied ``` #oauth-apiserver pod status oc get pods -n openshift-oauth-apiserver NAME READY STATUS RESTARTS AGE apiserver-6568f5576b-9xjs4 1/1 Running 0 35m apiserver-6568f5576b-bz8z5 1/1 Running 0 35m apiserver-6568f5576b-wkhgs 1/1 Running 0 35m #applied SCC for the above pods oc get pod -o 'custom-columns=NAMESPACE:metadata.namespace,NAME:metadata.name,APPLIED SCC:metadata.annotations.openshift\.io/scc' -n openshift-oauth-apiserver NAMESPACE NAME APPLIED SCC openshift-oauth-apiserver apiserver-6568f5576b-9xjs4 node-exporter openshift-oauth-apiserver apiserver-6568f5576b-bz8z5 node-exporter openshift-oauth-apiserver apiserver-6568f5576b-wkhgs node-exporter ``` 2. Create a custom SCC as mentioned below ``` allowHostDirVolumePlugin: true allowHostIPC: false allowHostNetwork: true allowHostPID: true allowHostPorts: true allowPrivilegeEscalation: true allowPrivilegedContainer: true allowedCapabilities: null apiVersion: security.openshift.io/v1 defaultAddCapabilities: null fsGroup: type: RunAsAny groups: [] kind: SecurityContextConstraints metadata: name: custom-scc priority: 0 readOnlyRootFilesystem: false requiredDropCapabilities: null runAsUser: type: MustRunAsNonRoot seLinuxContext: type: RunAsAny supplementalGroups: type: RunAsAny users: [] volumes: - '*' ``` 3. Delete any of the pods in the namespace `openshift-apiserver` and `openshift-ouath-apiserver` 4. New pod fails to initialize. Check for the applied SCC ``` NAME READY STATUS RESTARTS AGE apiserver-6568f5576b-bz8z5 1/1 Running 0 45m apiserver-6568f5576b-w2lb9 0/1 Init:CreateContainerConfigError 0 4m8s apiserver-6568f5576b-wkhgs 1/1 Running 0 45m NAMESPACE NAME APPLIED SCC openshift-oauth-apiserver apiserver-6568f5576b-bz8z5 node-exporter openshift-oauth-apiserver apiserver-6568f5576b-w2lb9 custom-scc openshift-oauth-apiserver apiserver-6568f5576b-wkhgs node-exporter ``` Actual results: New pods doesn't come up. Expected results: apiserver and Oauth-apiserver pods should always apply the SCC that allows RunAsAny Additional info: List SCC ``` oc get scc NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP PRIORITY READONLYROOTFS VOLUMES anyuid false <no value> MustRunAs RunAsAny RunAsAny RunAsAny 10 false ["configMap","downwardAPI","emptyDir","persistentVolumeClaim","projected","secret"] custom-scc true <no value> RunAsAny MustRunAsNonRoot RunAsAny RunAsAny 0 false ["*"] hostaccess false <no value> MustRunAs MustRunAsRange MustRunAs RunAsAny <no value> false ["configMap","downwardAPI","emptyDir","hostPath","persistentVolumeClaim","projected","secret"] hostmount-anyuid false <no value> MustRunAs RunAsAny RunAsAny RunAsAny <no value> false ["configMap","downwardAPI","emptyDir","hostPath","nfs","persistentVolumeClaim","projected","secret"] hostnetwork false <no value> MustRunAs MustRunAsRange MustRunAs MustRunAs <no value> false ["configMap","downwardAPI","emptyDir","persistentVolumeClaim","projected","secret"] machine-api-termination-handler false <no value> MustRunAs RunAsAny MustRunAs MustRunAs <no value> false ["downwardAPI","hostPath"] node-exporter true <no value> RunAsAny RunAsAny RunAsAny RunAsAny <no value> false ["*"] nonroot false <no value> MustRunAs MustRunAsNonRoot RunAsAny RunAsAny <no value> false ["configMap","downwardAPI","emptyDir","persistentVolumeClaim","projected","secret"] privileged true ["*"] RunAsAny RunAsAny RunAsAny RunAsAny <no value> false ["*"] restricted false <no value> MustRunAs MustRunAsRange MustRunAs RunAsAny <no value> false ["configMap","downwardAPI","emptyDir","persistentVolumeClaim","projected","secret"] ```
This will be applicable for all the Openshift system pods which uses a serviceaccount that has a binding to clusterrole `cluster-admin`
@
@sur, Did you get a chance to look at this ?
@Sergiusz Urbaniak, Did you get a chance to look at this ?
sprint review: due to other higher priority work this wasn't being worked on yet.
Iām adding UpcomingSprint, because I was occupied by fixing bugs with higher priority/severity, developing new features with higher priority, or developing new features to improve stability at a macro level. I will revisit this bug next sprint.
oauth-apiserver has been fixed as part of https://bugzilla.redhat.com/show_bug.cgi?id=1973005. I am leaving this open specifically for openshift-apiserver.
*** This bug has been marked as a duplicate of bug 1960680 ***