Bug 1303799 - [RHCeph 1.3.2 / 0.94.5-6] /var/lib/ceph Child dir secontext is unlabeled_t
[RHCeph 1.3.2 / 0.94.5-6] /var/lib/ceph Child dir secontext is unlabeled_t
Status: CLOSED ERRATA
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: Distribution (Show other bugs)
1.3.2
Unspecified Unspecified
urgent Severity urgent
: rc
: 1.3.2
Assigned To: Boris Ranto
ceph-qe-bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-02-01 20:58 EST by Vasu Kulkarni
Modified: 2016-02-29 09:45 EST (History)
3 users (show)

See Also:
Fixed In Version: RHEL: ceph-0.94.5-7.el7cp Ubuntu: ceph_0.94.5-5redhat1trusty
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-02-29 09:45:42 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Vasu Kulkarni 2016-02-01 20:58:09 EST
Description of problem:

Child dir's secontext is system_u or unconfined_t for child dir's in /var/lib/ceph/


> [ubuntu@clara002 ~]$ ls -Z /var/lib/ceph/
> drwxr-xr-x. root root unconfined_u:object_r:ceph_var_lib_t:s0
> bootstrap-mds
> drwxr-xr-x. root root system_u:object_r:ceph_var_lib_t:s0 bootstrap-
> osd
> drwxr-xr-x. root root unconfined_u:object_r:ceph_var_lib_t:s0
> bootstrap-rgw
> drwxr-xr-x. root root system_u:object_r:ceph_var_lib_t:s0 mon
> drwxr-xr-x. root root system_u:object_r:ceph_var_lib_t:s0 osd
> drwxr-xr-x. root root system_u:object_r:ceph_var_lib_t:s0 tmp
>


So these are ok


> [ubuntu@clara002 ~]$ ls -Z /var/lib/ceph/osd/
> drwxr-xr-x. root root system_u:object_r:unlabeled_t:s0 ceph-0
> drwxr-xr-x. root root system_u:object_r:unlabeled_t:s0 ceph-1
>


but these are not ok


> [ubuntu@clara002 ~]$ ls -Z /var/lib/ceph/osd/ceph-0/
> -rw-r--r--. root root unconfined_u:object_r:unlabeled_t:s0
> activate.monmap
> -rw-r--r--. root root unconfined_u:object_r:unlabeled_t:s0 active
> -rw-r--r--. root root unconfined_u:object_r:unlabeled_t:s0 ceph_fsid
> drwxr-xr-x. root root unconfined_u:object_r:unlabeled_t:s0 current
> -rw-r--r--. root root unconfined_u:object_r:unlabeled_t:s0 fsid
> lrwxrwxrwx. root root unconfined_u:object_r:unlabeled_t:s0 journal ->
> /dev/disk/by-partuuid/57d1200b-b303-46f4-bad1-e90ab2e71845
> -rw-r--r--. root root unconfined_u:object_r:unlabeled_t:s0
> journal_uuid
> -rw-------. root root unconfined_u:object_r:unlabeled_t:s0 keyring
> -rw-r--r--. root root unconfined_u:object_r:unlabeled_t:s0 magic
> -rw-r--r--. root root unconfined_u:object_r:unlabeled_t:s0 ready
> -rw-r--r--. root root unconfined_u:object_r:unlabeled_t:s0
> store_version
> -rw-r--r--. root root unconfined_u:object_r:unlabeled_t:s0 superblock
> -rw-r--r--. root root unconfined_u:object_r:unlabeled_t:s0 sysvinit
> -rw-r--r--. root root unconfined_u:object_r:unlabeled_t:s0 whoami
>


and neither are these. We might need to back-port some more bits if you
hit this with the current builds and ceph-selinux installed.


Version-Release number of selected component (if applicable):
1.3.2

How reproducible:
1/1

Steps to Reproduce:
Install verify child dir's secontext
Comment 3 Boris Ranto 2016-02-02 11:23:08 EST
@Ken: Will you back-port the referenced patches or should I do that?
Comment 4 Vasu Kulkarni 2016-02-02 18:07:39 EST
Boris,

There are some denials which I am seeing during rbd test run in permissive mode, Some might be related to above issue, would you check if there might be addition fix required for this denials specifically the devfs and lttng 

http://fpaste.org/317728/54326145/raw/
Comment 5 Boris Ranto 2016-02-03 12:40:09 EST
(summing up the IRC discussion)

Most of the denials seem to be related to this bugzilla. However, the lttng denials do refer to another issue:

https://bugzilla.redhat.com/show_bug.cgi?id=1304455
Comment 6 Boris Ranto 2016-02-03 13:21:04 EST
dist-git commits related to build ceph-0.94.5-5.el7cp:
http://pkgs.devel.redhat.com/cgit/rpms/ceph/commit/?h=ceph-1.3-rhel-7&id=2d7376fcc541255dbd71f9c316f306fd39ae54e0
Comment 7 Vasu Kulkarni 2016-02-03 13:26:46 EST
Thanks will rerun with new build and update here.
Comment 8 Boris Ranto 2016-02-03 13:34:47 EST
@Vasu: You'll need to wait a while before it is built:

https://brewweb.devel.redhat.com/taskinfo?taskID=10437360

All but the lttng-related denials should disappear afterwards.
Comment 9 Boris Ranto 2016-02-03 14:07:40 EST
The packages were built successfully, moving to ON_QA.
Comment 10 Ken Dreyer (Red Hat) 2016-02-03 19:59:27 EST
Ubuntu not yet built -> MODIFIED
Comment 12 Vasu Kulkarni 2016-02-08 14:53:49 EST
Boris, this is from one of the test, other tests are still running, but would list those denials in this bz itself

ceph build: 0.94.5-8.el7cp 

SELinuxError: SELinux denials found on ubuntu@clara005.ceph.redhat.com: ['type=AVC msg=audit(1454957380.457:4785): avc:  denied  { create } for  pid=16083 comm="ceph-osd" name="ceph-osd.0.asok" scontext=system_u:system_r:ceph_t:s0 tcontext=system_u:object_r:var_run_t:s0 tclass=sock_file', 'type=AVC msg=audit(1454957463.387:4921): avc:  denied  { unlink } for  pid=15118 comm="ceph-mon" name="ceph-mon.clara005.asok" dev="tmpfs" ino=115072 scontext=system_u:system_r:ceph_t:s0 tcontext=system_u:object_r:var_run_t:s0 tclass=sock_file', 'type=AVC msg=audit(1454957456.663:4905): avc:  denied  { open } for  pid=18386 comm="radosgw" path="/var/log/radosgw/ceph-client.rgw.clara005.log" dev="sda1" ino=2230449 scontext=system_u:system_r:ceph_t:s0 tcontext=system_u:object_r:var_log_t:s0 tclass=file', 'type=AVC msg=audit(1454957463.389:4922): avc:  denied  { read } for  pid=15118 comm="ceph-mon" name="mon.clara005.pid" dev="tmpfs" ino=115069 scontext=system_u:system_r:ceph_t:s0 tcontext=system_u:object_r:var_run_t:s0 tclass=file', 'type=AVC msg=audit(1454957360.439:4729): avc:  denied  { create } for  pid=15118 comm="ceph-mon" name="ceph-mon.clara005.asok" scontext=system_u:system_r:ceph_t:s0 tcontext=system_u:object_r:var_run_t:s0 tclass=sock_file', 'type=AVC msg=audit(1454957402.659:4823): avc:  denied  { create } for  pid=16960 comm="ceph-osd" name="ceph-osd.1.asok" scontext=system_u:system_r:ceph_t:s0 tcontext=system_u:object_r:var_run_t:s0 tclass=sock_file', 'type=AVC msg=audit(1454957463.389:4923): avc:  denied  { unlink } for  pid=15118 comm="ceph-mon" name="mon.clara005.pid" dev="tmpfs" ino=115069 scontext=system_u:system_r:ceph_t:s0 tcontext=system_u:object_r:var_run_t:s0 tclass=file', 'type=AVC msg=audit(1454957424.619:4854): avc:  denied  { create } for  pid=17838 comm="ceph-osd" name="ceph-osd.2.asok" scontext=system_u:system_r:ceph_t:s0 tcontext=system_u:object_r:var_run_t:s0 tclass=sock_file', 'type=AVC msg=audit(1454957456.699:4908): avc:  denied  { create } for  pid=18392 comm="radosgw" name="ceph-client.rgw.clara005.asok" scontext=system_u:system_r:ceph_t:s0 tcontext=system_u:object_r:var_run_t:s0 tclass=sock_file']
Comment 13 Boris Ranto 2016-02-08 15:35:02 EST
That looks more like bz1223731. The log itself suggests that the issue referenced here was fixed (if that is all you have hit) and you are having trouble with socket files and log files which should probably be set up in ceph.conf to be located in /var/run/ceph and /var/log/ceph, respectively to get the proper context.
Comment 14 Vasu Kulkarni 2016-02-08 18:48:54 EST
Other than below which is already tracked in bz1223731, I dont see anything else.

Marking this as fixed.


SELinux denials found on ubuntu@clara007.ceph.redhat.com: ['type=AVC msg=audit(1454972591.340:3774): avc: denied { create } for pid=17608 comm="ceph-osd" name="ceph-osd.4.asok" scontext=system_u:system_r:ceph_t:s0 tcontext=system_u:object_r:var_run_t:s0 tclass=sock_file', 'type=AVC msg=audit(1454972985.663:3913): avc: denied { unlink } for pid=15126 comm="ceph-mon" name="mon.clara007.pid" dev="tmpfs" ino=72143 scontext=system_u:system_r:ceph_t:s0 tcontext=system_u:object_r:var_run_t:s0 tclass=file', 'type=AVC msg=audit(1454972985.663:3912): avc: denied { read } for pid=15126 comm="ceph-mon" name="mon.clara007.pid" dev="tmpfs" ino=72143 scontext=system_u:system_r:ceph_t:s0 tcontext=system_u:object_r:var_run_t:s0 tclass=file', 'type=AVC msg=audit(1454972985.659:3911): avc: denied { unlink } for pid=15126 comm="ceph-mon" name="ceph-mon.clara007.asok" dev="tmpfs" ino=72146 scontext=system_u:system_r:ceph_t:s0 tcontext=system_u:object_r:var_run_t:s0 tclass=sock_file', 'type=AVC msg=audit(1454972612.950:3796): avc: denied { create } for pid=18538 comm="ceph-osd" name="ceph-osd.5.asok" scontext=system_u:system_r:ceph_t:s0 tcontext=system_u:object_r:var_run_t:s0 tclass=sock_file', 'type=AVC msg=audit(1454972569.656:3752): avc: denied { create } for pid=16700 comm="ceph-osd" name="ceph-osd.3.asok" scontext=system_u:system_r:ceph_t:s0 tcontext=system_u:object_r:var_run_t:s0 tclass=sock_file', 'type=AVC msg=audit(1454972466.805:3637): avc: denied { create } for pid=15126 comm="ceph-mon" name="ceph-mon.clara007.asok" scontext=system_u:system_r:ceph_t:s0 tcontext=system_u:object_r:var_run_t:s0 tclass=sock_file']
Comment 15 Boris Ranto 2016-02-09 10:34:39 EST
Looking at this more closely, there are two more things that I can see that are going on in here:

1.) You do put mon.clara007.pid outside the /var/run/ceph directory (it looks like it is located directly in /var/run/ directory) which we should not do as we have designated /var/run/ceph/ directory for these files. If this is the default location, we should tune the policy to cover these files. We will need a format specification for that, though -- i.e. is it anything that matches {mon,osd,mds,radosgw}.*.pid or is there something else to it?

2.) You do store rgw logs in '/var/log/radosgw/ceph-client.rgw.clara005.log'. We should not do that as we have designated /var/log/ceph/ directory for ceph-related logs (we do not even package /var/log/radosgw directory). Again, let me know if this is a default (not configured) behaviour somewhere in the source code so that I could update the policy accordingly.
Comment 16 Vasu Kulkarni 2016-02-09 12:39:26 EST
Thanks Boris, raised bz1305978 and bz1305981 to fix the issue instead of working around in policy
Comment 17 Boris Ranto 2016-02-09 16:30:46 EST
@Vasu: Thanks, do you have ceph.conf lying around somewhere? Looking at it should help me with the debugging quite a lot. (feel free to attach it to any of the two new bzs)
Comment 19 errata-xmlrpc 2016-02-29 09:45:42 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:0313

Note You need to log in before you can comment on or make changes to this bug.