Description of problem ---------------------- The software TPM emulator binary ("/usr/bin/swtpm") running inside a container currently does not have write/append permissions. As a result, the 'swtpm' binary fails to start, and that in turn causes a virtual machine configured with vTPM failing to boot. That's the SELinux denial in question (see further below for fuller denial): $> ausearch -m avc -m user_avc -m selinux_err -i -c swtpm [...] type=AVC msg=audit(06/02/2022 13:42:20.778:43441) : avc: denied { append } for pid=339697 comm=swtpm name=instance-000000b0-swtpm.log dev="overlay" ino=40378064 scontext=system_u:system_r:svirt_t:s0:c146,c635 tcontext=system_u:object_r:container_ro_file_t:s0 tclass=file permissive=0 In short, the log file directory under /var/log/swtpm should have the label "container_file_t" and _not_ "container_ro_file_t". SELinux labels -------------- Looking at the SELinx labels for 'swtpm' binary, we see that it has the label "container_ro_file_t" (correctly so): [heat-admin@compute-1 ~]$ sudo podman exec -it nova_virtqemud /bin/bash [root@compute-1 /]# ls -lZ /usr/bin/swtpm -rwxr-xr-x. 1 root root system_u:object_r:container_ro_file_t:s0 42312 Nov 12 2021 /usr/bin/swtpm And the log file (and the directory) has "container_ro_file_t", i.e. read-only (notice the "_ro_" in the SELinux label): [root@compute-1 /]# ls -lZ /var/log/swtpm/libvirt/qemu/instance-000000b0-swtpm.log -rw-r--r--. 1 tss tss system_u:object_r:container_ro_file_t:s0 3385 Jun 2 13:42 /var/log/swtpm/libvirt/qemu/instance-000000b0-swtpm.log For comparison, if you see the QEMU binary, it also has "container_ro_file_t" (correctly so): [root@compute-1 /]# ls -lZ /usr/libexec/qemu-kvm -rwxr-xr-x. 1 root root system_u:object_r:container_ro_file_t:s0 18574328 Mar 22 13:34 /usr/libexec/qemu-kvm However, the log files it handles have the label "container_file_t" (notice, there's no"_ro_" here, i.e. they _are_ writeable): [root@compute-1 /]# ls -lZ /var/log/libvirt/qemu/instance-000000b0.log -rw-------. 1 root root unconfined_u:object_r:container_file_t:s0 0 Jun 3 00:00 /var/log/libvirt/qemu/instance-000000b0.log Environment ----------- This is an OpenStack setup with RHEL-9 host. And the virtual machines will be launched inside a container. Version ------- - container-selinux-2.179.1-1.el9_0.noarch - swtpm-0.7.0-1.20211109gitb79fd91.el9.x86_64 [inside the container] - qemu-kvm-6.2.0-11.el9_0.2.x86_64 [inside the container] How reproducible: Consistently. Steps to Reproduce ------------------ This bug was found in a QE environment (thanks: James Parker) of Red Hat OpenStack on RHEL9. It's a fairly involved setup, and can be re-run if really required. However, the information in this bug should be sufficient. As the solution here seems to be for '/usr/bin/swtpm' to be able to write to log files under /var/log/swtpm/ directory. Actual results -------------- A QEMU instance (i.e. a virtual machine) running inside a container fails to start as 'swtpm' binary couldn't be started: ----------------------------------------------------------------------- $> cat instance-000000b0.log.1 2022-06-02 13:42:20.765+0000: Starting external device: TPM Emulator /usr/bin/swtpm socket --daemon --ctrl type=unixio,path=/run/libvirt/qemu/swtpm/115-instance-000000b0-swtpm.sock,mode=0600 --tpmstate dir=/var/lib/libvirt/swtpm/febdee00-d829-4f92-8441-4abbf6768c77/tpm2,mode=0600 --log file=/var/log/swtpm/libvirt/qemu/instance-000000b0-swtpm.log -- terminate --tpm2 --pid file=/run/libvirt/qemu/swtpm/115-instance-000000b0-swtpm.pid --key pwdfd=31,mode=aes-256-cbc --migration-key pwdfd=33,mode=aes- 256-cbc 2022-06-02 13:42:20.780+0000: shutting down, reason=failed ----------------------------------------------------------------------- And that's the fuller SELinux denial for the 'swtpm' binary (I've manually line-wrapped the logs below): ----------------------------------------------------------------------- $> ausearch -m avc -m user_avc -m selinux_err -i -c swtpm ---- type=PROCTITLE msg=audit(06/02/2022 13:42:20.778:43441) : proctitle=/usr/bin/swtpm socket --daemon --ctrl type=unixio,path=/run/libvirt/qemu/swtpm/115-instance-000000b0-swtpm.sock,mode=0600 --tpms type=SYSCALL msg=audit(06/02/2022 13:42:20.778:43441) : arch=x86_64 syscall=openat success=no exit=EACCES(Permission denied) a0=AT_FDCWD a1=0x5598ba4fad50 a2=O_WRONLY|O_CREAT|O_APPEND|O_NOFOLLOW a3=0x180 items=0 ppid=339696 pid=339697 auid=heat-admin uid=tss gid=tss euid=tss suid=tss fsuid=tss egid=tss sgid=tss fsgid=tss tty=(none) ses=3 comm=swtpm exe=/usr/bin/swtpm subj=system_u:system_r:svirt_t:s0:c146,c635 key=(null) type=AVC msg=audit(06/02/2022 13:42:20.778:43441) : avc: denied { append } for pid=339697 comm=swtpm name=instance-000000b0-swtpm.log dev="overlay" ino=40378064 scontext=system_u:system_r:svirt_t:s0:c146,c635 tcontext=system_u:object_r:container_ro_file_t:s0 tclass=file permissive=0 ----------------------------------------------------------------------- Expected results ---------------- The 'swtpm' binary running in a container should be able to append to log files under /var/log/swtpm/libvirt/qemu/ directory.
Hm, afaik, containers will always expose container_ro_file_t inside the container itself. Here, it looks more like svirt_t isn't allowed to write on the container_ro_file_t type, which kind of makes sense. Since it's more than probably restricted to our (weird?) OSP usage, we may just add the required policy in openstack-selinux[1]. Cheers, C. [1] https://github.com/redhat-openstack/openstack-selinux/blob/master/os-virt.te
I don't think we should allow svirt_t to write on the container_ro_file_t. We need to follow the described above qemu binary vs logs example instead. For logs having container_ro_file_t is an obvious mistake
@dwalsh: Hi, Dan — any opinions here?
I agree with cjeanner that this is likely to be specific to the way OSP is deployed, and I also agree with bogdando that we don't want to allow writing to container_ro_file_t. Looking into THT, it looks like the container_file_t type for /var/log/libvirt/qemu/ is defined here: https://github.com/openstack/tripleo-heat-templates/blob/321e740/deployment/nova/nova-modular-libvirt-container-puppet.yaml#L940 Meanwhile, while I see a container_file_t type for /var/log/containers/libvirt/swtpm at https://github.com/openstack/tripleo-heat-templates/blob/321e740/deployment/nova/nova-modular-libvirt-container-puppet.yaml#L941, the logs seem to actually exist elsewhere based on the description for this bug, as it mentions /var/log/swtpm/libvirt/ instead. I'm not familiar enough with THT or swtpm to be sure, or confirm if this needs a new definition or if the existing log definition in THT is a typo, but my hunch is that this should be resolved in the tripleo-heat-templates component.
In tripleo we use /var/log/containers/<element> for the containerized element logging. We cannot change that as containerized logrotate also depends on that. Could you please clarify what should be changed in t-h-t, /var/log/containers/libvirt/swtpm path to /var/log/swtpm/libvirt/:/var/log/swtpm/libvirt on the host? Unfortunately we cannot do that
I'm wondering if adding something like "{ 'path': /var/log/swtpm/, 'setype': container_file_t, 'mode': '0750' }" after L941 would help, since that seems to be the path we want to change the type of? (Or with /var/log/swtpm/libvirt/qemu, maybe) From what I can tell, for the qemu logs we have this to mount the volume: https://github.com/openstack/tripleo-heat-templates/blob/321e740d/deployment/logging/files/nova-libvirt.yaml#L156 - /var/log/containers/libvirt:/var/log/libvirt:shared,z # +qemu subdir and this to set the type: https://github.com/openstack/tripleo-heat-templates/blob/321e740/deployment/nova/nova-modular-libvirt-container-puppet.yaml#L940 - { 'path': /var/log/libvirt/qemu, 'setype': container_file_t } (Note: Not "/var/log/containers") While the swtpm mount and logs look like this: https://github.com/openstack/tripleo-heat-templates/blob/321e740/deployment/nova/nova-modular-libvirt-container-puppet.yaml#L681 - /var/log/containers/libvirt/swtpm:/var/log/swtpm:z https://github.com/openstack/tripleo-heat-templates/blob/321e740/deployment/nova/nova-modular-libvirt-container-puppet.yaml#L941 - { 'path': /var/log/containers/libvirt/swtpm, 'setype': container_file_t, 'mode': '0750' } I'm seeing now that this says "puppet" in the filename so I'm not sure if that's the file actually used, but even if this is the wrong place, it seems to me like this is something that could be resolved at deployment time in OSP in the same way it's done for the other qemu logs, by setting the type explicitly in THT?
> I'm wondering if adding something like "{ 'path': /var/log/swtpm/, 'setype': container_file_t, 'mode': '0750' }" after L941 would help, since that seems to be the path we want to change the type of? (Or with /var/log/swtpm/libvirt/qemu, maybe) That would create a host path, which we cannot use as the container bind-mount (see the /var/log/containers/<element> logging architecture explained). So that would not fix the issue. > From what I can tell, for the qemu logs we have this to mount the volume: ? https://github.com/openstack/tripleo-heat-templates/blob/321e740d/deployment/logging/files/nova-libvirt.yaml#L156 > - /var/log/containers/libvirt:/var/log/libvirt:shared,z # +qemu subdir yes, this also covers /var/log/containers/libvirt/{qemu,swtpm} inside of container. > and this to set the type: > https://github.com/openstack/tripleo-heat-templates/blob/321e740/deployment/nova/nova-modular-libvirt-container-puppet.yaml#L940 > - { 'path': /var/log/libvirt/qemu, 'setype': container_file_t } > (Note: Not "/var/log/containers") That should be a leftover, we can remove this, it is no longer used. > While the swtpm mount and logs look like this: > https://github.com/openstack/tripleo-heat-templates/blob/321e740/deployment/nova/nova-modular-libvirt-container-puppet.yaml#L681 > - /var/log/containers/libvirt/swtpm:/var/log/swtpm:z > https://github.com/openstack/tripleo-heat-templates/blob/321e740/deployment/nova/nova-modular-libvirt-container-puppet.yaml#L941 > - { 'path': /var/log/containers/libvirt/swtpm, 'setype': container_file_t, 'mode': '0750' } Yes, it follows the same approach that other containerized services logging takes. We cannot change that. SELinux does not complain for those, we should do the same to swtpm logs in /var/log/containers/libvirt
This is svirt policy not container-selinux.
Hi berrange, can you please look at this bz?
(In reply to Nikola Knazekova from comment #10) > Hi berrange, > > can you please look at this bz? I'm not sure what you're asking me to say ? The problem looks pretty clearly understood in that the /var/log/swtpm location inside the container is labelled incorrectly with container_ro_file_t, rather than container_file_t OSP deployment tools needs to decide how to configure the container and any passthrough bind mounts, such that the /var/log/swtpm location is writable.
I think this bug should be re-assigned to some component under OSP, not selinux-policy, but I don't know what OSP tool is relevant. Kashyap can help .....
In which rhel version can we have that container_logwriter_t, so that we can use it in OSP as well?
Similar to: https://bugzilla.redhat.com/show_bug.cgi?id=2007314
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days