Description of problem: For OSP8 Ceph storage nodes where OSDs are running have SELinux in permissive mode. While all other OpenStack nodes (controllers and compute) are in SELinux enforcing mode. That includes HA Controllers that run Ceph MON, Ceph calamari client, and RGW. Version-Release number of selected component (if applicable): Ceph 1.3.2 & OSP8 How reproducible: Always Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info: For OSP7 with Ceph 1.3.1 OSD nodes had SELinux in enforcing mode but MON nodes where in permissive mode. We used Foreman for OSP7 deployment and not OSPd director.
Would love it to be fixed for OSP8 but will settle for OSP9. If we have a workaround we should be able to do it JS scripts if you want to delay incorporating it into overcloud base image till later version of OSP director.
Here the output of audit2why to show the AVCs that are seen on the OSD node: [heat-admin@iaas-cephstorage-1 ~]$ sudo audit2why -a type=AVC msg=audit(1466539054.021:23): avc: denied { write } for pid=1886 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file Was caused by: Missing type enforcement (TE) allow rule. You can use audit2allow to generate a loadable module to allow this access. type=AVC msg=audit(1466539054.187:25): avc: denied { write } for pid=1936 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file Was caused by: Missing type enforcement (TE) allow rule. You can use audit2allow to generate a loadable module to allow this access. type=AVC msg=audit(1466539054.306:26): avc: denied { write } for pid=1952 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file Was caused by: Missing type enforcement (TE) allow rule. You can use audit2allow to generate a loadable module to allow this access. type=AVC msg=audit(1466539054.448:27): avc: denied { write } for pid=1975 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file Was caused by: Missing type enforcement (TE) allow rule. You can use audit2allow to generate a loadable module to allow this access. type=AVC msg=audit(1466539054.590:29): avc: denied { write } for pid=2003 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file Was caused by: Missing type enforcement (TE) allow rule. You can use audit2allow to generate a loadable module to allow this access. type=AVC msg=audit(1466539054.656:30): avc: denied { write } for pid=2020 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file Was caused by: Missing type enforcement (TE) allow rule. You can use audit2allow to generate a loadable module to allow this access. type=AVC msg=audit(1466539054.796:32): avc: denied { write } for pid=2036 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file Was caused by: Missing type enforcement (TE) allow rule. You can use audit2allow to generate a loadable module to allow this access. type=AVC msg=audit(1466539054.912:33): avc: denied { write } for pid=2049 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file Was caused by: Missing type enforcement (TE) allow rule. You can use audit2allow to generate a loadable module to allow this access. type=AVC msg=audit(1466539055.049:34): avc: denied { write } for pid=2065 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file Was caused by: Missing type enforcement (TE) allow rule. You can use audit2allow to generate a loadable module to allow this access. type=AVC msg=audit(1466539055.182:35): avc: denied { write } for pid=2086 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file Was caused by: Missing type enforcement (TE) allow rule. You can use audit2allow to generate a loadable module to allow this access. type=AVC msg=audit(1466539055.259:36): avc: denied { write } for pid=2104 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file Was caused by: Missing type enforcement (TE) allow rule. You can use audit2allow to generate a loadable module to allow this access. type=AVC msg=audit(1466539055.310:37): avc: denied { write } for pid=2117 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file Was caused by: Missing type enforcement (TE) allow rule. You can use audit2allow to generate a loadable module to allow this access. type=AVC msg=audit(1466539055.427:38): avc: denied { write } for pid=2130 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file Was caused by: Missing type enforcement (TE) allow rule. You can use audit2allow to generate a loadable module to allow this access. type=AVC msg=audit(1466539055.543:39): avc: denied { write } for pid=2143 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file Was caused by: Missing type enforcement (TE) allow rule. You can use audit2allow to generate a loadable module to allow this access. type=AVC msg=audit(1466539055.614:40): avc: denied { write } for pid=2159 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file Was caused by: Missing type enforcement (TE) allow rule. You can use audit2allow to generate a loadable module to allow this access. type=AVC msg=audit(1466539055.674:41): avc: denied { write } for pid=2172 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file Was caused by: Missing type enforcement (TE) allow rule. You can use audit2allow to generate a loadable module to allow this access. type=AVC msg=audit(1466539055.739:42): avc: denied { write } for pid=2192 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file Was caused by: Missing type enforcement (TE) allow rule. You can use audit2allow to generate a loadable module to allow this access. type=AVC msg=audit(1466539055.809:43): avc: denied { write } for pid=2208 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file Was caused by: Missing type enforcement (TE) allow rule. You can use audit2allow to generate a loadable module to allow this access. type=AVC msg=audit(1466539055.939:44): avc: denied { write } for pid=2227 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file Was caused by: Missing type enforcement (TE) allow rule. You can use audit2allow to generate a loadable module to allow this access. type=AVC msg=audit(1466539055.991:45): avc: denied { write } for pid=2244 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file Was caused by: Missing type enforcement (TE) allow rule. You can use audit2allow to generate a loadable module to allow this access. type=AVC msg=audit(1466539056.049:47): avc: denied { write } for pid=2260 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file Was caused by: Missing type enforcement (TE) allow rule. You can use audit2allow to generate a loadable module to allow this access. type=AVC msg=audit(1466539056.107:48): avc: denied { write } for pid=2277 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file Was caused by: Missing type enforcement (TE) allow rule. You can use audit2allow to generate a loadable module to allow this access. type=AVC msg=audit(1466539056.190:50): avc: denied { write } for pid=2300 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file Was caused by: Missing type enforcement (TE) allow rule. You can use audit2allow to generate a loadable module to allow this access. type=AVC msg=audit(1466539056.246:51): avc: denied { write } for pid=2313 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file Was caused by: Missing type enforcement (TE) allow rule. You can use audit2allow to generate a loadable module to allow this access. type=AVC msg=audit(1466539063.454:101): avc: denied { read } for pid=5502 comm="sshd" name="sshd_config" dev="sda2" ino=16824239 scontext=system_u:system_r:sshd_t:s0-s0:c0.c1023 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file Was caused by: Missing type enforcement (TE) allow rule. You can use audit2allow to generate a loadable module to allow this access. [heat-admin@iaas-cephstorage-1 ~]$
The workaround for now is to include the following in one of your environment yaml: parameter_defaults: CephStorageExtraConfig: ceph_osd_selinux_permissive: false
Is this landing in OSP9 director? Had anybody validation that SElinux for OSDs for Ceph 1.3.2 and Ceph 2.0 do work?
*** This bug has been marked as a duplicate of bug 1349194 ***
As an alternative approach to the puppet-ceph patch (which might not be accepted upstream), given Swift is already tagging /srv/node as swift_data_t , it looks like we could tag /srv/data for Ceph in ceph-selinux
verified on puppet-ceph-2.2.0-1.el7ost.noarch
*** Bug 1367926 has been marked as a duplicate of this bug. ***
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHEA-2016-2948.html