Bug 2071033

Summary:	conditionally relabel volumes given annotation not working - SELinux context match is wrong
Product:	OpenShift Container Platform	Reporter:	Alberto Gonzalez de Dios <algonzal>
Component:	Node	Assignee:	Peter Hunt <pehunt>
Node sub component:	CRI-O	QA Contact:	Sunil Choudhary <schoudha>
Status:	CLOSED ERRATA	Docs Contact:
Severity:	high
Priority:	medium	CC:	dmunneor, pehunt, ramon.gordillo
Version:	4.8
Target Milestone:	---
Target Release:	4.11.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:
Clones:	2079454 (view as bug list)		Environment:
Last Closed:	2022-08-10 11:03:06 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	2079454, 2079461, 2086098

Description Alberto Gonzalez de Dios 2022-04-01 16:12:29 UTC

Description of problem:

Workaround in CRI-O in BZ 1987112 is not working. In crio code, file server/label_linux.go, line #15 [1], it's being compared mount point context and container context. Container context is given by Openshift, and MCS in Openshift are ordered High first. But in Linux (RHEL, CoreOS), MCS are ordered Low first.

You can check in the basic example:

---
oc rsh -n openshift-ingress router-default-65fdbf88-ff5cx ls -lZ | grep system | head -1 | awk '{print $5}'
system_u:object_r:container_file_t:s0:c19,c24

$ oc get project openshift-ingress -o yaml | grep mcs
openshift.io/sa.scc.mcs: s0:c24,c19
---

So, in line 15 of server/label_linux.go [1]:

---
if err == nil && currentLabel == secLabel {
---

It's comparing "system_u:object_r:container_file_t:s0:c19,c24" with "system_u:object_r:container_file_t:s0:c24,c19" which is different as MCS are ordered in a different way. But both contexts are the same.

It's needed to implement a smarter way to compare SELinux contexts. Some of the functions in selinux_linux.go of containers/common repository [2] could be used to do this check.

[1] https://github.com/haircommander/cri-o/blob/main/server/label_linux.go#L15

[2] https://github.com/containers/common/blob/main/vendor/github.com/opencontainers/selinux/go-selinux/selinux_linux.go

Version-Release number of selected component (if applicable):
1.21.5-2 (Openshift 4.8.34)
However, this is happening in Openshift 4.10 and 4.9.

How reproducible:
Follow steps in https://access.redhat.com/solutions/6221251 -> Skip SELinux Relabeling if already done with an annotation.

Steps to Reproduce:
1. Follow steps in https://access.redhat.com/solutions/6221251 -> Skip SELinux Relabeling if already done with an annotation
2. Check SELinux is relabeled in the PV although mount point is already using the proper context
3. Check context in PV files and in Openshift which is the same, although the order of the MCS is different

Actual results:
Files are relabeled.

Expected results:
Files should not be relabeled.

Additional info:
This issue affects the PR done in 4.10 and backported to 4.9, 4.8 and 4.7:

BZ 2029511 - Documentation update for Volumes with high file counts delay pod creation due to recursive SELinux file context relabeling

BZ 2013773 - [4.9] Volumes with high file counts delay pod creation due to recursive SELinux file context relabeling

BZ 2014174 - [4.8] Volumes with high file counts delay pod creation due to recursive SELinux file context relabeling

BZ 2014695 - [4.7] Volumes with high file counts delay pod creation due to recursive SELinux file context relabeling

Comment 15 Sunil Choudhary 2022-06-16 11:16:32 UTC

 % oc get clusterversion
NAME      VERSION                              AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.11.0-0.nightly-2022-06-15-222801   True        False         5h1m    Cluster version is 4.11.0-0.nightly-2022-06-15-222801

% oc get nodes 
NAME                                          STATUS   ROLES    AGE     VERSION
sunil411-rdflg-master-0                       Ready    master   5h28m   v1.24.0+cb71478
sunil411-rdflg-master-1                       Ready    master   5h28m   v1.24.0+cb71478
sunil411-rdflg-master-2                       Ready    master   5h28m   v1.24.0+cb71478
sunil411-rdflg-worker-southcentralus1-bqhh7   Ready    worker   5h14m   v1.24.0+cb71478
sunil411-rdflg-worker-southcentralus2-9b2jx   Ready    worker   5h14m   v1.24.0+cb71478
sunil411-rdflg-worker-southcentralus3-2m94k   Ready    worker   5h14m   v1.24.0+cb71478

% cat mcs.yaml 
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
  labels:
    machineconfiguration.openshift.io/role: worker
  name: 99-worker-selinux-configuration
spec:
  config:
    ignition:
      version: 3.2.0
    storage:
      files:
      - contents:
          source: data:text/plain;charset=utf-8;base64,W2NyaW8ucnVudGltZS5ydW50aW1lcy5zZWxpbnV4XQpydW50aW1lX3BhdGggPSAiL3Vzci9iaW4vcnVuYyIKcnVudGltZV9yb290ID0gIi9ydW4vcnVuYyIKcnVudGltZV90eXBlID0gIm9jaSIKYWxsb3dlZF9hbm5vdGF0aW9ucyA9IFsiaW8ua3ViZXJuZXRlcy5jcmktby5UcnlTa2lwVm9sdW1lU0VMaW51eExhYmVsIl0K
        mode: 0640
        overwrite: true
        path: /etc/crio/crio.conf.d/01-selinux.conf
  osImageURL: ""

% oc create -f mcs.yaml 
machineconfig.machineconfiguration.openshift.io/99-worker-selinux-configuration created

 % oc get mc
NAME                                               GENERATEDBYCONTROLLER                      IGNITIONVERSION   AGE
00-master                                          0bba861a74209e78177612789983183cdc77f1a5   3.2.0             5h27m
00-worker                                          0bba861a74209e78177612789983183cdc77f1a5   3.2.0             5h27m
01-master-container-runtime                        0bba861a74209e78177612789983183cdc77f1a5   3.2.0             5h27m
01-master-kubelet                                  0bba861a74209e78177612789983183cdc77f1a5   3.2.0             5h27m
01-worker-container-runtime                        0bba861a74209e78177612789983183cdc77f1a5   3.2.0             5h27m
01-worker-kubelet                                  0bba861a74209e78177612789983183cdc77f1a5   3.2.0             5h27m
99-master-generated-registries                     0bba861a74209e78177612789983183cdc77f1a5   3.2.0             5h27m
99-master-ssh                                                                                 3.2.0             5h35m
99-worker-generated-registries                     0bba861a74209e78177612789983183cdc77f1a5   3.2.0             5h27m
99-worker-selinux-configuration                                                               3.2.0             5s
99-worker-ssh                                                                                 3.2.0             5h35m
rendered-master-e270f1e75d98e3d54e36bc4a1a90d1de   0bba861a74209e78177612789983183cdc77f1a5   3.2.0             5h27m
rendered-master-fbb32bd39ee43aa353d032d8b127460e   0bba861a74209e78177612789983183cdc77f1a5   3.2.0             4h55m
rendered-worker-2cb22cf060d96238afa5d28f651e2e65   0bba861a74209e78177612789983183cdc77f1a5   3.2.0             5h27m
rendered-worker-daa750a3ca06141347ad932a711e1ea3   0bba861a74209e78177612789983183cdc77f1a5   3.2.0             4h55m


% oc debug node/sunil411-rdflg-worker-southcentralus1-bqhh7
...
Starting pod/sunil411-rdflg-worker-southcentralus1-bqhh7-debug ...

sh-4.4# cat /etc/crio/crio.conf.d/01-selinux.conf 
[crio.runtime.runtimes.selinux]
runtime_path = "/usr/bin/runc"
runtime_root = "/run/runc"
runtime_type = "oci"
allowed_annotations = ["io.kubernetes.cri-o.TrySkipVolumeSELinuxLabel"]


% cat selinuxrc.yaml 
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: selinux
handler: selinux

% oc create -f selinuxrc.yaml 
runtimeclass.node.k8s.io/selinux created

% oc create -f pv.yaml 
persistentvolume/pv0001 created

% oc create -f pvc.yaml
persistentvolumeclaim/myclaim created

% oc create -f deploy.yaml 
deployment.apps/pv-deploy created

% oc get pods -o wide
NAME                         READY   STATUS    RESTARTS   AGE   IP            NODE                                          NOMINATED NODE   READINESS GATES
pv-deploy-76d8bff7f6-qz24f   1/1     Running   0          32s   10.129.2.20   sunil411-rdflg-worker-southcentralus1-bqhh7   <none>           <none>


sh-4.4# runc list | grep -i a0cd566a48dd7
a0cd566a48dd722b949d76a6133f5bf79462fcb8d8c8fb65b7a7f1085b5ea4fb   15856       running     /run/containers/storage/overlay-containers/a0cd566a48dd722b949d76a6133f5bf79462fcb8d8c8fb65b7a7f1085b5ea4fb/userdata   2022-06-16T11:12:26.710600896Z   root

sh-4.4# chcon -t unlabeled_t   /run/containers/storage/overlay-containers/a0cd566a48dd722b949d76a6133f5bf79462fcb8d8c8fb65b7a7f1085b5ea4fb/userdata/config.json

sh-4.4# ls -lZ  /run/containers/storage/overlay-containers/a0cd566a48dd722b949d76a6133f5bf79462fcb8d8c8fb65b7a7f1085b5ea4fb/userdata/config.json
-rw-r--r--. 1 root root system_u:object_r:unlabeled_t:s0 9840 Jun 16 11:12 /run/containers/storage/overlay-containers/a0cd566a48dd722b949d76a6133f5bf79462fcb8d8c8fb65b7a7f1085b5ea4fb/userdata/config.json

Comment 16 errata-xmlrpc 2022-08-10 11:03:06 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Container Platform 4.11.0 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:5069