Description of problem: Workaround in CRI-O in BZ 1987112 is not working. In crio code, file server/label_linux.go, line #15 [1], it's being compared mount point context and container context. Container context is given by Openshift, and MCS in Openshift are ordered High first. But in Linux (RHEL, CoreOS), MCS are ordered Low first. You can check in the basic example: --- oc rsh -n openshift-ingress router-default-65fdbf88-ff5cx ls -lZ | grep system | head -1 | awk '{print $5}' system_u:object_r:container_file_t:s0:c19,c24 $ oc get project openshift-ingress -o yaml | grep mcs openshift.io/sa.scc.mcs: s0:c24,c19 --- So, in line 15 of server/label_linux.go [1]: --- if err == nil && currentLabel == secLabel { --- It's comparing "system_u:object_r:container_file_t:s0:c19,c24" with "system_u:object_r:container_file_t:s0:c24,c19" which is different as MCS are ordered in a different way. But both contexts are the same. It's needed to implement a smarter way to compare SELinux contexts. Some of the functions in selinux_linux.go of containers/common repository [2] could be used to do this check. [1] https://github.com/haircommander/cri-o/blob/main/server/label_linux.go#L15 [2] https://github.com/containers/common/blob/main/vendor/github.com/opencontainers/selinux/go-selinux/selinux_linux.go Version-Release number of selected component (if applicable): 1.21.5-2 (Openshift 4.8.34) However, this is happening in Openshift 4.10 and 4.9. How reproducible: Follow steps in https://access.redhat.com/solutions/6221251 -> Skip SELinux Relabeling if already done with an annotation. Steps to Reproduce: 1. Follow steps in https://access.redhat.com/solutions/6221251 -> Skip SELinux Relabeling if already done with an annotation 2. Check SELinux is relabeled in the PV although mount point is already using the proper context 3. Check context in PV files and in Openshift which is the same, although the order of the MCS is different Actual results: Files are relabeled. Expected results: Files should not be relabeled. Additional info: This issue affects the PR done in 4.10 and backported to 4.9, 4.8 and 4.7: BZ 2029511 - Documentation update for Volumes with high file counts delay pod creation due to recursive SELinux file context relabeling BZ 2013773 - [4.9] Volumes with high file counts delay pod creation due to recursive SELinux file context relabeling BZ 2014174 - [4.8] Volumes with high file counts delay pod creation due to recursive SELinux file context relabeling BZ 2014695 - [4.7] Volumes with high file counts delay pod creation due to recursive SELinux file context relabeling
% oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.11.0-0.nightly-2022-06-15-222801 True False 5h1m Cluster version is 4.11.0-0.nightly-2022-06-15-222801 % oc get nodes NAME STATUS ROLES AGE VERSION sunil411-rdflg-master-0 Ready master 5h28m v1.24.0+cb71478 sunil411-rdflg-master-1 Ready master 5h28m v1.24.0+cb71478 sunil411-rdflg-master-2 Ready master 5h28m v1.24.0+cb71478 sunil411-rdflg-worker-southcentralus1-bqhh7 Ready worker 5h14m v1.24.0+cb71478 sunil411-rdflg-worker-southcentralus2-9b2jx Ready worker 5h14m v1.24.0+cb71478 sunil411-rdflg-worker-southcentralus3-2m94k Ready worker 5h14m v1.24.0+cb71478 % cat mcs.yaml apiVersion: machineconfiguration.openshift.io/v1 kind: MachineConfig metadata: labels: machineconfiguration.openshift.io/role: worker name: 99-worker-selinux-configuration spec: config: ignition: version: 3.2.0 storage: files: - contents: source: data:text/plain;charset=utf-8;base64,W2NyaW8ucnVudGltZS5ydW50aW1lcy5zZWxpbnV4XQpydW50aW1lX3BhdGggPSAiL3Vzci9iaW4vcnVuYyIKcnVudGltZV9yb290ID0gIi9ydW4vcnVuYyIKcnVudGltZV90eXBlID0gIm9jaSIKYWxsb3dlZF9hbm5vdGF0aW9ucyA9IFsiaW8ua3ViZXJuZXRlcy5jcmktby5UcnlTa2lwVm9sdW1lU0VMaW51eExhYmVsIl0K mode: 0640 overwrite: true path: /etc/crio/crio.conf.d/01-selinux.conf osImageURL: "" % oc create -f mcs.yaml machineconfig.machineconfiguration.openshift.io/99-worker-selinux-configuration created % oc get mc NAME GENERATEDBYCONTROLLER IGNITIONVERSION AGE 00-master 0bba861a74209e78177612789983183cdc77f1a5 3.2.0 5h27m 00-worker 0bba861a74209e78177612789983183cdc77f1a5 3.2.0 5h27m 01-master-container-runtime 0bba861a74209e78177612789983183cdc77f1a5 3.2.0 5h27m 01-master-kubelet 0bba861a74209e78177612789983183cdc77f1a5 3.2.0 5h27m 01-worker-container-runtime 0bba861a74209e78177612789983183cdc77f1a5 3.2.0 5h27m 01-worker-kubelet 0bba861a74209e78177612789983183cdc77f1a5 3.2.0 5h27m 99-master-generated-registries 0bba861a74209e78177612789983183cdc77f1a5 3.2.0 5h27m 99-master-ssh 3.2.0 5h35m 99-worker-generated-registries 0bba861a74209e78177612789983183cdc77f1a5 3.2.0 5h27m 99-worker-selinux-configuration 3.2.0 5s 99-worker-ssh 3.2.0 5h35m rendered-master-e270f1e75d98e3d54e36bc4a1a90d1de 0bba861a74209e78177612789983183cdc77f1a5 3.2.0 5h27m rendered-master-fbb32bd39ee43aa353d032d8b127460e 0bba861a74209e78177612789983183cdc77f1a5 3.2.0 4h55m rendered-worker-2cb22cf060d96238afa5d28f651e2e65 0bba861a74209e78177612789983183cdc77f1a5 3.2.0 5h27m rendered-worker-daa750a3ca06141347ad932a711e1ea3 0bba861a74209e78177612789983183cdc77f1a5 3.2.0 4h55m % oc debug node/sunil411-rdflg-worker-southcentralus1-bqhh7 ... Starting pod/sunil411-rdflg-worker-southcentralus1-bqhh7-debug ... sh-4.4# cat /etc/crio/crio.conf.d/01-selinux.conf [crio.runtime.runtimes.selinux] runtime_path = "/usr/bin/runc" runtime_root = "/run/runc" runtime_type = "oci" allowed_annotations = ["io.kubernetes.cri-o.TrySkipVolumeSELinuxLabel"] % cat selinuxrc.yaml apiVersion: node.k8s.io/v1 kind: RuntimeClass metadata: name: selinux handler: selinux % oc create -f selinuxrc.yaml runtimeclass.node.k8s.io/selinux created % oc create -f pv.yaml persistentvolume/pv0001 created % oc create -f pvc.yaml persistentvolumeclaim/myclaim created % oc create -f deploy.yaml deployment.apps/pv-deploy created % oc get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES pv-deploy-76d8bff7f6-qz24f 1/1 Running 0 32s 10.129.2.20 sunil411-rdflg-worker-southcentralus1-bqhh7 <none> <none> sh-4.4# runc list | grep -i a0cd566a48dd7 a0cd566a48dd722b949d76a6133f5bf79462fcb8d8c8fb65b7a7f1085b5ea4fb 15856 running /run/containers/storage/overlay-containers/a0cd566a48dd722b949d76a6133f5bf79462fcb8d8c8fb65b7a7f1085b5ea4fb/userdata 2022-06-16T11:12:26.710600896Z root sh-4.4# chcon -t unlabeled_t /run/containers/storage/overlay-containers/a0cd566a48dd722b949d76a6133f5bf79462fcb8d8c8fb65b7a7f1085b5ea4fb/userdata/config.json sh-4.4# ls -lZ /run/containers/storage/overlay-containers/a0cd566a48dd722b949d76a6133f5bf79462fcb8d8c8fb65b7a7f1085b5ea4fb/userdata/config.json -rw-r--r--. 1 root root system_u:object_r:unlabeled_t:s0 9840 Jun 16 11:12 /run/containers/storage/overlay-containers/a0cd566a48dd722b949d76a6133f5bf79462fcb8d8c8fb65b7a7f1085b5ea4fb/userdata/config.json
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: OpenShift Container Platform 4.11.0 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:5069