Occasionally, some volume mount are not created with the correct security context. We have started two pods (of the same kind) in two different namespaces. Both pods ended up on the same worker. However, one of the pods did not start because one of the volume mounts (/data) had an incorrect security context:
*** Pod with an issue ***
$ oc project keep-vcu-operations-09
Now using project "keep-vcu-operations-09" on server "https://api.ocp065.exilis.npee.seki.gic.ericsson.se:6443".
[11:37]$ oc rsh -c init eric-data-distributed-coordinator-ed-0
sh-4.4$ ls -lZ /data
ls: cannot open directory '/data': Permission denied
*** After setting selinux mode to permissive on the worker ****
sh-4.4$ ls -lZd /data
drwxrwsrwx. 3 root 10000 system_u:object_r:unlabeled_t:s0 4096 Oct 25 03:58 /data
*** Working pod ***
[11:38]$ oc project keep-vcu-operations-10
Now using project "keep-vcu-operations-10" on server "https://api.ocp065.exilis.npee.seki.gic.ericsson.se:6443".
[11:38]$ oc rsh -c dced eric-data-distributed-coordinator-ed-0
sh-4.4$ ls -lZd /data
drwxrwsrwx. 6 root 10000 system_u:object_r:container_file_t:s0:c22,c28 4096 Oct 25 03:59 /data
I've attached the pod descriptions for both the working and the non working pod.
When we deleted the non working pod, it moved to another worker, and it started without any issues.
I've attached the sosreport for the worker and a must-gather for the cluster.
Big files here so all attachments are in the related case - 03065228
Working pod - https://attachments.access.redhat.com/hydra/rest/cases/03065228/attachments/c54651e1-1ddd-4ed0-83ef-314d532f50f3?usePresignedUrl=true
Not working pod - https://attachments.access.redhat.com/hydra/rest/cases/03065228/attachments/26472f5d-ed57-4b03-908b-dd107dbd338f?usePresignedUrl=true
Must-gather - https://attachments.access.redhat.com/hydra/rest/cases/03065228/attachments/88865b0d-89f9-49e5-b6c0-dd144f1a7044?usePresignedUrl=true
sosreport - https://attachments.access.redhat.com/hydra/rest/cases/03065228/attachments/6602c32b-e367-43f6-bdd0-aea9a0a0d98a?usePresignedUrl=true
Experimental PR: https://github.com/kubernetes/kubernetes/pull/105934
This needs some discussion upstream.
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.