Bug 1346401

Summary:	SELinux on Ceph OSD nodes should be in enforcing mode
Product:	Red Hat OpenStack	Reporter:	arkady kanevsky <arkady_kanevsky>
Component:	puppet-ceph	Assignee:	Giulio Fidente <gfidente>
Status:	CLOSED ERRATA	QA Contact:	Arik Chernetsky <achernet>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	8.0 (Liberty)	CC:	cdevine, christopher_dearborn, david_paterson, dmacpher, gfidente, jdurgin, jjoyce, joherr, johfulto, John_walsh, jschluet, kazen, kbader, kschinck, kurt_hey, lhh, mburns, morazi, nlevine, randy_perryman, rhallise, rsussman, scohen, slinaber, sreichar, srevivo, sumedh_sathaye, tvignaud, yrabl
Target Milestone:	rc	Keywords:	Reopened, Triaged
Target Release:	10.0 (Newton)
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:	puppet-ceph-2.1.0-0.20160926220714.c764ef8.el7ost	Doc Type:	Enhancement
Doc Text:	It is now possible to confine 'ceph-osd' instances with SELinux policies. In OSP10, new deployments have SELinux configured in 'enforcing' mode on the Ceph Storage nodes.	Story Points:	---
Clone Of:
Clones:	1349194 (view as bug list)		Environment:
Last Closed:	2016-12-14 15:39:31 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1305654

Description arkady kanevsky 2016-06-14 17:42:30 UTC

Description of problem:
For OSP8 Ceph storage nodes where OSDs are running have SELinux in permissive mode. While all other OpenStack nodes (controllers and compute) are in SELinux enforcing mode. That includes HA Controllers that run Ceph MON, Ceph calamari client, and RGW.

Version-Release number of selected component (if applicable):
Ceph 1.3.2 & OSP8

How reproducible:
Always

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
For OSP7 with Ceph 1.3.1 OSD nodes had SELinux in enforcing mode but MON nodes where in permissive mode. We used Foreman for OSP7 deployment and not OSPd director.

Comment 2 arkady kanevsky 2016-06-14 17:55:17 UTC

Would love it to be fixed for OSP8 but will settle for OSP9.
If we have a workaround we should be able to do it JS scripts if you want to delay incorporating it into overcloud base image till later version of OSP director.

Comment 4 Steve Reichard 2016-06-22 14:27:11 UTC

Here the output of audit2why to show the AVCs that are seen on the OSD node:

[heat-admin@iaas-cephstorage-1 ~]$ sudo audit2why -a
type=AVC msg=audit(1466539054.021:23): avc:  denied  { write } for  pid=1886 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file
	Was caused by:
		Missing type enforcement (TE) allow rule.

		You can use audit2allow to generate a loadable module to allow this access.

type=AVC msg=audit(1466539054.187:25): avc:  denied  { write } for  pid=1936 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file
	Was caused by:
		Missing type enforcement (TE) allow rule.

		You can use audit2allow to generate a loadable module to allow this access.

type=AVC msg=audit(1466539054.306:26): avc:  denied  { write } for  pid=1952 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file
	Was caused by:
		Missing type enforcement (TE) allow rule.

		You can use audit2allow to generate a loadable module to allow this access.

type=AVC msg=audit(1466539054.448:27): avc:  denied  { write } for  pid=1975 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file
	Was caused by:
		Missing type enforcement (TE) allow rule.

		You can use audit2allow to generate a loadable module to allow this access.

type=AVC msg=audit(1466539054.590:29): avc:  denied  { write } for  pid=2003 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file
	Was caused by:
		Missing type enforcement (TE) allow rule.

		You can use audit2allow to generate a loadable module to allow this access.

type=AVC msg=audit(1466539054.656:30): avc:  denied  { write } for  pid=2020 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file
	Was caused by:
		Missing type enforcement (TE) allow rule.

		You can use audit2allow to generate a loadable module to allow this access.

type=AVC msg=audit(1466539054.796:32): avc:  denied  { write } for  pid=2036 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file
	Was caused by:
		Missing type enforcement (TE) allow rule.

		You can use audit2allow to generate a loadable module to allow this access.

type=AVC msg=audit(1466539054.912:33): avc:  denied  { write } for  pid=2049 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file
	Was caused by:
		Missing type enforcement (TE) allow rule.

		You can use audit2allow to generate a loadable module to allow this access.

type=AVC msg=audit(1466539055.049:34): avc:  denied  { write } for  pid=2065 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file
	Was caused by:
		Missing type enforcement (TE) allow rule.

		You can use audit2allow to generate a loadable module to allow this access.

type=AVC msg=audit(1466539055.182:35): avc:  denied  { write } for  pid=2086 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file
	Was caused by:
		Missing type enforcement (TE) allow rule.

		You can use audit2allow to generate a loadable module to allow this access.

type=AVC msg=audit(1466539055.259:36): avc:  denied  { write } for  pid=2104 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file
	Was caused by:
		Missing type enforcement (TE) allow rule.

		You can use audit2allow to generate a loadable module to allow this access.

type=AVC msg=audit(1466539055.310:37): avc:  denied  { write } for  pid=2117 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file
	Was caused by:
		Missing type enforcement (TE) allow rule.

		You can use audit2allow to generate a loadable module to allow this access.

type=AVC msg=audit(1466539055.427:38): avc:  denied  { write } for  pid=2130 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file
	Was caused by:
		Missing type enforcement (TE) allow rule.

		You can use audit2allow to generate a loadable module to allow this access.

type=AVC msg=audit(1466539055.543:39): avc:  denied  { write } for  pid=2143 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file
	Was caused by:
		Missing type enforcement (TE) allow rule.

		You can use audit2allow to generate a loadable module to allow this access.

type=AVC msg=audit(1466539055.614:40): avc:  denied  { write } for  pid=2159 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file
	Was caused by:
		Missing type enforcement (TE) allow rule.

		You can use audit2allow to generate a loadable module to allow this access.

type=AVC msg=audit(1466539055.674:41): avc:  denied  { write } for  pid=2172 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file
	Was caused by:
		Missing type enforcement (TE) allow rule.

		You can use audit2allow to generate a loadable module to allow this access.

type=AVC msg=audit(1466539055.739:42): avc:  denied  { write } for  pid=2192 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file
	Was caused by:
		Missing type enforcement (TE) allow rule.

		You can use audit2allow to generate a loadable module to allow this access.

type=AVC msg=audit(1466539055.809:43): avc:  denied  { write } for  pid=2208 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file
	Was caused by:
		Missing type enforcement (TE) allow rule.

		You can use audit2allow to generate a loadable module to allow this access.

type=AVC msg=audit(1466539055.939:44): avc:  denied  { write } for  pid=2227 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file
	Was caused by:
		Missing type enforcement (TE) allow rule.

		You can use audit2allow to generate a loadable module to allow this access.

type=AVC msg=audit(1466539055.991:45): avc:  denied  { write } for  pid=2244 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file
	Was caused by:
		Missing type enforcement (TE) allow rule.

		You can use audit2allow to generate a loadable module to allow this access.

type=AVC msg=audit(1466539056.049:47): avc:  denied  { write } for  pid=2260 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file
	Was caused by:
		Missing type enforcement (TE) allow rule.

		You can use audit2allow to generate a loadable module to allow this access.

type=AVC msg=audit(1466539056.107:48): avc:  denied  { write } for  pid=2277 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file
	Was caused by:
		Missing type enforcement (TE) allow rule.

		You can use audit2allow to generate a loadable module to allow this access.

type=AVC msg=audit(1466539056.190:50): avc:  denied  { write } for  pid=2300 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file
	Was caused by:
		Missing type enforcement (TE) allow rule.

		You can use audit2allow to generate a loadable module to allow this access.

type=AVC msg=audit(1466539056.246:51): avc:  denied  { write } for  pid=2313 comm="restorecon" path="/var/lib/ceph/tmp/ceph-disk.activate.lock" dev="sda2" ino=19936944 scontext=system_u:system_r:setfiles_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lib_t:s0 tclass=file
	Was caused by:
		Missing type enforcement (TE) allow rule.

		You can use audit2allow to generate a loadable module to allow this access.

type=AVC msg=audit(1466539063.454:101): avc:  denied  { read } for  pid=5502 comm="sshd" name="sshd_config" dev="sda2" ino=16824239 scontext=system_u:system_r:sshd_t:s0-s0:c0.c1023 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file
	Was caused by:
		Missing type enforcement (TE) allow rule.

		You can use audit2allow to generate a loadable module to allow this access.

[heat-admin@iaas-cephstorage-1 ~]$

Comment 5 Mike Burns 2016-06-23 18:14:46 UTC

The workaround for now is to include the following in one of your environment yaml:

parameter_defaults:
    CephStorageExtraConfig:
      ceph_osd_selinux_permissive: false

Comment 6 arkady kanevsky 2016-07-11 19:43:35 UTC

Is this landing in OSP9 director?
Had anybody validation that SElinux for OSDs for Ceph 1.3.2 and Ceph 2.0 do work?

Comment 7 Sean Cohen 2016-07-13 19:18:22 UTC


*** This bug has been marked as a duplicate of bug 1349194 ***

Comment 9 Giulio Fidente 2016-08-11 15:14:47 UTC

As an alternative approach to the puppet-ceph patch (which might not be accepted upstream), given Swift is already tagging /srv/node as swift_data_t , it looks like we could tag /srv/data for Ceph in ceph-selinux

Comment 11 Yogev Rabl 2016-10-07 13:04:10 UTC

verified on puppet-ceph-2.2.0-1.el7ost.noarch

Comment 12 jomurphy 2016-10-13 20:19:25 UTC

*** Bug 1367926 has been marked as a duplicate of this bug. ***

Comment 15 errata-xmlrpc 2016-12-14 15:39:31 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-2948.html