Hide Forgot
Description of problem: When checking audit.log on compute node, got several "denied" lines: 1. Setup with 2 overclouds, without additional ceph, 1 controller, 1 compute on each overcloud, rebooted compute node: [heat-admin@overcloud2-novacompute-0 ~]$ sudo cat /var/log/audit/audit.log | grep denied type=AVC msg=audit(1562468761.649:33825): avc: denied { dac_override } for pid=558318 comm="tmpwatch" capability=1 scontext=system_u:system_r:tmpreaper_t:s0-s0:c0.c1023 tcontext=system_u:system_r:tmpreaper_t:s0-s0:c0.c1023 tclass=capability permissive=0 type=AVC msg=audit(1562468761.649:33826): avc: denied { dac_override } for pid=558318 comm="tmpwatch" capability=1 scontext=system_u:system_r:tmpreaper_t:s0-s0:c0.c1023 tcontext=system_u:system_r:tmpreaper_t:s0-s0:c0.c1023 tclass=capability permissive=0 type=AVC msg=audit(1562468761.650:33827): avc: denied { dac_override } for pid=558318 comm="tmpwatch" capability=1 scontext=system_u:system_r:tmpreaper_t:s0-s0:c0.c1023 tcontext=system_u:system_r:tmpreaper_t:s0-s0:c0.c1023 tclass=capability permissive=0 type=AVC msg=audit(1562468761.672:33828): avc: denied { read } for pid=558347 comm="logrotate" name="qemu" dev="vda2" ino=1012139 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:container_file_t:s0 tclass=dir permissive=0 type=AVC msg=audit(1562468761.673:33829): avc: denied { read } for pid=558347 comm="logrotate" name="openvswitch" dev="vda2" ino=1012142 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:container_file_t:s0 tclass=dir permissive=0 type=AVC msg=audit(1562557081.933:45121): avc: denied { dac_override } for pid=758799 comm="tmpwatch" capability=1 scontext=system_u:system_r:tmpreaper_t:s0-s0:c0.c1023 tcontext=system_u:system_r:tmpreaper_t:s0-s0:c0.c1023 tclass=capability permissive=0 type=AVC msg=audit(1562557081.933:45122): avc: denied { dac_override } for pid=758799 comm="tmpwatch" capability=1 scontext=system_u:system_r:tmpreaper_t:s0-s0:c0.c1023 tcontext=system_u:system_r:tmpreaper_t:s0-s0:c0.c1023 tclass=capability permissive=0 type=AVC msg=audit(1562557081.934:45123): avc: denied { dac_override } for pid=758799 comm="tmpwatch" capability=1 scontext=system_u:system_r:tmpreaper_t:s0-s0:c0.c1023 tcontext=system_u:system_r:tmpreaper_t:s0-s0:c0.c1023 tclass=capability permissive=0 type=AVC msg=audit(1562557081.934:45124): avc: denied { dac_override } for pid=758799 comm="tmpwatch" capability=1 scontext=system_u:system_r:tmpreaper_t:s0-s0:c0.c1023 tcontext=system_u:system_r:tmpreaper_t:s0-s0:c0.c1023 tclass=capability permissive=0 type=AVC msg=audit(1562557081.948:45125): avc: denied { read } for pid=758807 comm="logrotate" name="qemu" dev="vda2" ino=1012139 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:container_file_t:s0 tclass=dir permissive=0 type=AVC msg=audit(1562557081.949:45126): avc: denied { read } for pid=758807 comm="logrotate" name="openvswitch" dev="vda2" ino=1012142 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:container_file_t:s0 tclass=dir permissive=0 type=AVC msg=audit(1562587479.922:49211): avc: denied { connectto } for pid=828775 comm="plymouth" path=002F6F72672F667265656465736B746F702F706C796D6F75746864 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:system_r:plymouthd_t:s0 tclass=unix_stream_socket permissive=0 type=AVC msg=audit(1562587479.939:49213): avc: denied { connectto } for pid=828808 comm="plymouth" path=002F6F72672F667265656465736B746F702F706C796D6F75746864 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:system_r:plymouthd_t:s0 tclass=unix_stream_socket permissive=0 type=AVC msg=audit(1562587793.241:215): avc: denied { read } for pid=8208 comm="systemd-user-ru" name="libpod" dev="tmpfs" ino=73106 scontext=system_u:system_r:init_t:s0 tcontext=unconfined_u:object_r:container_runtime_tmpfs_t:s0 tclass=dir permissive=0 type=AVC msg=audit(1562587793.241:216): avc: denied { read } for pid=8208 comm="systemd-user-ru" name="overlay-containers" dev="tmpfs" ino=73105 scontext=system_u:system_r:init_t:s0 tcontext=unconfined_u:object_r:container_runtime_tmpfs_t:s0 tclass=dir permissive=0 type=AVC msg=audit(1562587793.241:217): avc: denied { read } for pid=8208 comm="systemd-user-ru" name="overlay-layers" dev="tmpfs" ino=73104 scontext=system_u:system_r:init_t:s0 tcontext=unconfined_u:object_r:container_runtime_tmpfs_t:s0 tclass=dir permissive=0 2. Setup 3 controllers, 2 computes, 3 cephs, without any reboots: [heat-admin@compute-1 ~]$ sudo cat /var/log/audit/audit.log | grep denied type=AVC msg=audit(1562508743.188:39): avc: denied { execute_no_trans } for pid=1219 comm="modprobe" path="/usr/bin/bash" dev="vda2" ino=4215568 scontext=system_u:system_r:openvswitch_load_module_t:s0 tcontext=system_u:object_r:shell_exec_t:s0 tclass=file permissive=1 type=AVC msg=audit(1562508745.961:81): avc: denied { sendto } for pid=865 comm="chronyd" path="/run/chrony/chronyc.2707.sock" scontext=system_u:system_r:chronyd_t:s0 tcontext=system_u:system_r:cloud_init_t:s0 tclass=unix_dgram_socket permissive=1 type=AVC msg=audit(1562508756.013:83): avc: denied { sendto } for pid=865 comm="chronyd" path="/run/chrony/chronyc.2719.sock" scontext=system_u:system_r:chronyd_t:s0 tcontext=system_u:system_r:cloud_init_t:s0 tclass=unix_dgram_socket permissive=1 type=AVC msg=audit(1562513521.954:5767): avc: denied { read } for pid=54302 comm="logrotate" name="qemu" dev="vda2" ino=1012139 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:container_file_t:s0 tclass=dir permissive=0 type=AVC msg=audit(1562513521.955:5768): avc: denied { read } for pid=54302 comm="logrotate" name="openvswitch" dev="vda2" ino=1012142 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:container_file_t:s0 tclass=dir permissive=0 type=AVC msg=audit(1562556122.068:12339): avc: denied { read } for pid=178739 comm="logrotate" name="qemu" dev="vda2" ino=1012139 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:container_file_t:s0 tclass=dir permissive=0 type=AVC msg=audit(1562556122.069:12340): avc: denied { read } for pid=178739 comm="logrotate" name="openvswitch" dev="vda2" ino=1012142 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:container_file_t:s0 tclass=dir permissive=0 [heat-admin@ceph-0 ~]$ sudo cat /var/log/audit/audit.log | grep denied type=AVC msg=audit(1562508548.724:40): avc: denied { execute_no_trans } for pid=1212 comm="modprobe" path="/usr/bin/bash" dev="vda2" ino=4215568 scontext=system_u:system_r:openvswitch_load_module_t:s0 tcontext=system_u:object_r:shell_exec_t:s0 tclass=file permissive=1 type=AVC msg=audit(1562508551.990:82): avc: denied { sendto } for pid=877 comm="chronyd" path="/run/chrony/chronyc.2725.sock" scontext=system_u:system_r:chronyd_t:s0 tcontext=system_u:system_r:cloud_init_t:s0 tclass=unix_dgram_socket permissive=1 type=AVC msg=audit(1562508552.402:83): avc: denied { sendto } for pid=877 comm="chronyd" path="/run/chrony/chronyc.2763.sock" scontext=system_u:system_r:chronyd_t:s0 tcontext=system_u:system_r:cloud_init_t:s0 tclass=unix_dgram_socket permissive=1 Version-Release number of selected component (if applicable): tmpwatch-2.11-14.el8.x86_64 logrotate-3.14.0-3.el8.x86_64 plymouth-scripts-0.9.3-12.el8.x86_64 plymouth-0.9.3-12.el8.x86_64 plymouth-core-libs-0.9.3-12.el8.x86_64 chrony-3.3-3.el8.x86_64 RHOS_TRUNK-15.0-RHEL-8-20190701.n.0 How reproducible: 100% Steps to Reproduce: 1. Deploy 2. Enter compute or ceph node 3. sudo cat /var/log/audit/audit.log | grep denied Actual results: Get "denied" from various containers Expected results: No "denied" in log Additional info:
So, I think we can already push some of the AVC to other projects: type=AVC msg=audit(1562468761.649:33825): avc: denied { dac_override } for pid=558318 comm="tmpwatch" capability=1 scontext=system_u:system_r:tmpreaper_t:s0-s0:c0.c1023 tcontext=system_u:system_r:tmpreaper_t:s0-s0:c0.c1023 tclass=capability permissive=0 That's for tmpwatch - although the issue itself will be sorted out by a selinux ruleset type=AVC msg=audit(1562587479.922:49211): avc: denied { connectto } for pid=828775 comm="plymouth" path=002F6F72672F667265656465736B746F702F706C796D6F75746864 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:system_r:plymouthd_t:s0 tclass=unix_stream_socket permissive=0 That's for plymouth - although, here again, the issue itself will probably be sorted out by a selinux ruleset. On might ask "why plymouth on a server" though...... "Graphical Boot Animation and Logger" type=AVC msg=audit(1562587793.241:215): avc: denied { read } for pid=8208 comm="systemd-user-ru" name="libpod" dev="tmpfs" ino=73106 scontext=system_u:system_r:init_t:s0 tcontext=unconfined_u:object_r:container_runtime_tmpfs_t:s0 tclass=dir permissive=0 type=AVC msg=audit(1562587793.241:216): avc: denied { read } for pid=8208 comm="systemd-user-ru" name="overlay-containers" dev="tmpfs" ino=73105 scontext=system_u:system_r:init_t:s0 tcontext=unconfined_u:object_r:container_runtime_tmpfs_t:s0 tclass=dir permissive=0 type=AVC msg=audit(1562587793.241:217): avc: denied { read } for pid=8208 comm="systemd-user-ru" name="overlay-layers" dev="tmpfs" ino=73104 scontext=system_u:system_r:init_t:s0 tcontext=unconfined_u:object_r:container_runtime_tmpfs_t:s0 tclass=dir permissive=0 That's probably for podman/libpod/runc, and, if legit accesses, will lead to a new rulset in container-selinux ======= type=AVC msg=audit(1562508548.724:40): avc: denied { execute_no_trans } for pid=1212 comm="modprobe" path="/usr/bin/bash" dev="vda2" ino=4215568 scontext=system_u:system_r:openvswitch_load_module_t:s0 tcontext=system_u:object_r:shell_exec_t:s0 tclass=file permissive=1 There, no idea for now, need some investigations. type=AVC msg=audit(1562508551.990:82): avc: denied { sendto } for pid=877 comm="chronyd" path="/run/chrony/chronyc.2725.sock" scontext=system_u:system_r:chronyd_t:s0 tcontext=system_u:system_r:cloud_init_t:s0 tclass=unix_dgram_socket permissive=1 Probably due to some file created during bootstrap with cloudinit, needs some investigations Finally, the logrotate will need an addition to openstack-selinux, I'll take care of that one now. I'll push the SELinux issues related to podman on their github - care to push the others to the right components and link back the BZ here so that we can have a fine status? Do you have an accessible env so that I can test the generated selinux rules in order to ensure it covers all? Cheers, C.
Just pushed a change that will get rid of the logrotate AVCs. Some details: - it's the *host* logrotate, not the containerized one - apparently we have a bunch of logs written with the container_file_t context outside of /var/log/containers - using a simple SELinux boolean allows to correct that AVC, making the whole thing far, far easier to manage - imho we shouldn't push logrotate in a container, since it's installed by default on the nodes.... But that's another question/issue ;)
It looks like a couple of the AVCs moved to bug 1728196 may be relevant here after all: - The modprobe denial references openvswitch / openvswitch_load_module_t type=AVC msg=audit(1562508548.724:40): avc: denied { execute_no_trans } for pid=1212 comm="modprobe" path="/usr/bin/bash" dev="vda2" ino=4215568 scontext=system_u:system_r:openvswitch_load_module_t:s0 tcontext=system_u:object_r:shell_exec_t:s0 tclass=file permissive=1 - The tmpwatch denial references /var/log/containers (cf. https://bugzilla.redhat.com/show_bug.cgi?id=1728196#c3 ) type=PROCTITLE msg=audit(07/09/2019 05:07:01.625:5580) : proctitle=tmpwatch --nodirs -X /var/log/containers/*/*log -X /var/log/containers/*/*/*log -X /var/log/containers/*/*err 15 /var/log/contai type=SYSCALL msg=audit(07/09/2019 05:07:01.625:5580) : arch=x86_64 syscall=unlink success=no exit=EACCES(Permission denied) a0=0x55cd23fc7a6b a1=0x55cd238b0044 a2=0x55cd23fc6930 a3=0x0 items=0 ppid=118753 pid=118755 auid=unset uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=(none) ses=unset comm=tmpwatch exe=/usr/sbin/tmpwatch subj=system_u:system_r:tmpreaper_t:s0-s0:c0.c1023 key=(null) type=AVC msg=audit(07/09/2019 05:07:01.625:5580) : avc: denied { dac_override } for pid=118755 comm=tmpwatch capability=dac_override scontext=system_u:system_r:tmpreaper_t:s0-s0:c0.c1023 tcontext=system_u:system_r:tmpreaper_t:s0-s0:c0.c1023 tclass=capability permissive=0
With latest compose it looks like we still have a denial for openvswitch: [root@overcloud-novacompute-0 heat-admin]# grep denied /var/log/audit/audit.log type=AVC msg=audit(1567480861.728:31696): avc: denied { write } for pid=683284 comm="logrotate" name="openvswitch" dev="sda2" ino=2881762 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:container_file_t:s0 tclass=dir permissive=0 type=AVC msg=audit(1567568341.529:44396): avc: denied { write } for pid=989380 comm="logrotate" name="openvswitch" dev="sda2" ino=2881762 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:container_file_t:s0 tclass=dir permissive=0 [root@overcloud-novacompute-0 heat-admin]# ls -l /var/log/openvswitch/ total 28 -rw-r-----. 1 openvswitch hugetlbfs 2490 Aug 31 23:31 ovsdb-server.log -rw-r-----. 1 openvswitch hugetlbfs 16947 Sep 4 02:39 ovs-vswitchd.log -rw-r--r--. 1 root root 90 Aug 31 22:14 readme.txt
So https://github.com/redhat-openstack/openstack-selinux/pull/42 should sort this issue.
Adding build info that should fix the logrotate issues. If there are other types of denials causing problems, please file separate bugs for them. Thank you.
[root@overcloud-novacompute-0 heat-admin]# rpm -q openstack-selinux openstack-selinux-0.8.20-0.20190904140454.936ea4f.el8ost.noarch [root@overcloud-novacompute-0 heat-admin]# grep denied /var/log/audit/audit.log [root@overcloud-novacompute-0 heat-admin]#
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2019:2811