Bug 1957840

Note that crio requires the same configuration but gets it using another script that preforms`sed --in-place` on the crio.conf

```
sh-4.4# audit2allow  -a


#============= init_t ==============
allow init_t kubernetes_file_t:file read;
```

@dwalsh Do you think this should be allowed by default?

When installing with openshift-installer (kubelet can read the file) this is the selinux context:
ls -lahZ /etc/kubernetes/kubelet-pause-image-override
-rw-r--r--. 1 root root system_u:object_r:container_file_t:s0 146 May  6 17:51 /etc/kubernetes/kubelet-pause-image-override

When installing with assisted-installer:
[root@openshift-master-1 core]# ls -lahZ /etc/kubernetes/kubelet-pause-image-override
-rw-r--r--. 1 root root system_u:object_r:kubernetes_file_t:s0 147 May  6 13:38 /etc/kubernetes/kubelet-pause-image-override 

This is with the same OCP release image (quay.io/openshift-release-dev/ocp-release:4.8.0-fc.0-x86_64)

I think this should be allowed by default in policy.

But there's an easy workaround: just store the environment override outside of `/etc/kubernetes`, say in `/etc/kubelet.env` or something that will be `etc_t`.  Since kubelet itself isn't reading this file, it doesn't need to be in `/etc/kubernetes`.

Started sno installation with assisted-service.
changed Env params in service to:
EnvironmentFile=/etc/kubernetes/kubelet-env
EnvironmentFile=/etc/kubernetes/kubelet-pause-image-override

reloaded daemon and restarted kubelet. Kubelet started as expected and i don't see any errors.

Verified context of env file :
-rw-r--r--. 1 root root system_u:object_r:container_file_t:s0 146 May  9 14:34 /etc/kubernetes/kubelet-pause-image-override

Seems to be related to this: https://bugzilla.redhat.com/show_bug.cgi?id=1904693

In my testing on the bootstrap node I made env file /etc/kubernetes/kubelet-pause-image-override mandatory in kubelet service by removing '-'

EnvironmentFile=/etc/kubernetes/kubelet-pause-image-override

If you try to restart the kubelet service after that, it fails because of the reported issue. 

The original SELinux label on the /etc/kubernetes/kubelet-pause-image-override on the bootstrap node was, 

-rw-r--r--. 1 root root system_u:object_r:kubernetes_file_t:s0 147 May 10 08:12 /etc/kubernetes/kubelet-pause-image-override

But when I changed the labels to system_u:object_r:container_file_t:s0 the kubelet service restart successfully. 



I am not sure why the installer is setting those labels incorrectly. Moving this back to RHCOS, should this be sent to the installer team?

As a data point I see the same with rhcos-48.84.202104271417-0-qemu.x86_64.qcow2.gz

[core@localhost ~]$ ls -lahZ /etc/kubernetes/kubelet-pause-image-override
-rw-r--r--. 1 root root system_u:object_r:kubernetes_file_t:s0 146 May 10 10:15 /etc/kubernetes/kubelet-pause-image-override

Doing a restorecon switches back to container_file_t, so it may be a label issue in the image?

[core@localhost ~]$ sudo restorecon -R /etc
[core@localhost ~]$ ls -lahZ /etc/kubernetes/kubelet-pause-image-override
-rw-r--r--. 1 root root system_u:object_r:container_file_t:s0 146 May 10 10:15 /etc/kubernetes/kubelet-pause-image-override

I did not see this with rhcos-48.83.202103221318-0-qemu.x86_64.qcow2.gz

Another note, the file is created by this installer script https://github.com/openshift/installer/blob/master/data/data/bootstrap/files/usr/local/bin/kubelet-pause-image.sh.template#L13

However I couldn't reproduce the mis-labled file by re-running that script, so I'm unclear why the file initially has the kubernetes_file_t label.

Ok, testing with the rhcos-4.8.0-fc.3-x86_64-live.x86_64 ISO, it happens in the same way, but with a 4.7 does not happen. this is the policy oer the /etc/kubernetes folder:

[root@openshift-master-1 core]# ls -alhZ /etc | grep kubernetes
drwxr-xr-x.  8 root        root        system_u:object_r:kubernetes_file_t:s0          191 May 10 10:36 kubernetes


So that explains that the placed files over/etc/kubernetes has this context.

Summary:

- Running the OCP Assistend Installer on an RHCOS node based on RHEL 8.3 creates a file `/etc/kubernetes/kubelet-pause-image-override` labeled `system_u:object_r:container_file_t:s0`
- Running the OCP Assistend Installer on an RHCOS node based on RHEL 8.4 creates the same file with the label `system_u:object_r:kubernetes_file_t:s0`
- When the `/etc/kubernetes/kubelet-pause-image-override` file is labeled `system_u:object_r:kubernetes_file_t:s0`, the `kubelet` cannot access the file with an SELinux denial:

type=AVC msg=audit(1620311565.546:147): avc:  denied  { read } for  pid=1 comm="systemd" name="kubelet-pause-image-override" dev="loop0" ino=34417753 scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:kubernetes_file_t:s0 tclass=file permissive=0

- `audit2allow` provides the following rule:


```
sh-4.4# audit2allow  -a


#============= init_t ==============
allow init_t kubernetes_file_t:file read;
```

- We think this transition should be allowed, so we are re-routing the BZ to the SELinux team for additional triage.

Please provide a list of policy modules which are active on your machine:

# semodule -lfull

Please tell us more about the kubernetes_file_t type:

# seinfo -tkubernetes_file_t -x


SELinux policy packages shipped in RHEL-8.x do not define kubernetes_file_t type, even if the container-selinux package is installed:

# seinfo -tkubernetes_file_t -x

Types: 0
# semodule -lfull | grep container
200 container         pp         
#

Created attachment 1781757 [details]
semodule -l full

```
sh-4.4# seinfo -tkubernetes_file_t -x

Types: 1
   type kubernetes_file_t, file_type, non_auth_file_type, non_security_file_type;
```

See the attachment with the semodules, too.

kubernetes_file_t is a part of container-selinux and I think this problem needs to be addressed there:

# rpm -q container-selinux
container-selinux-2.158.0-1.module+el8.4.0+10607+f4da7515.noarch

# seinfo -tkubernetes_file_t -x

Types: 1
   type kubernetes_file_t, file_type, non_auth_file_type, non_security_file_type;

https://github.com/containers/container-selinux/releases/tag/v2.162.0

Now we need to get this packaged up for RHEL.

This should be in the current version of container-selinux.

Fix for this was released via RHSA-2021:2371: https://errata.devel.redhat.com/advisory/76279/builds

Will be in v2.165.1. (I had a typo in the previous version).

This should show up in RHEL 8.5

I would be fine with backporting the entire container-selinux to RHEL8.4

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: container-tools:rhel8 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:4154

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 500 days