Bug 1775303 - selinux policy prevents host logrotate from rotating logs written by ceph containers
Summary: selinux policy prevents host logrotate from rotating logs written by ceph con...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat Storage
Component: Container
Version: 4.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: rc
: 4.*
Assignee: Dimitri Savineau
QA Contact: Vasishta
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-11-21 17:10 UTC by Sage Weil
Modified: 2023-09-15 00:19 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-11-25 14:18:32 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Ceph Project Bug Tracker 43229 0 None None None 2019-12-10 14:36:02 UTC
Red Hat Issue Tracker RHCEPH-3221 0 None None None 2022-02-21 18:05:29 UTC

Description Sage Weil 2019-11-21 17:10:37 UTC
Description of problem:

The new ceph-daemon tool allows Ceph daemons to (optionally) write to legacy-style logs in /var/log/ceph.  It does this by passing the host's /var/log/ceph/$fsid through to /var/log/ceph with the ":z" annotation/mode.

The logs, if enabled, are written by the daemon.  However, log rotation is configured by adding an /etc/logrotate.d/ceph-$fsid file on the host.  When log rotation runs, we get the following selinux denials:

type=AVC msg=audit(1574355721.524:278): avc:  denied  { read } for  pid=18185 comm="logrotate" name="e0f31562-0c7f-11ea-9a6a-00259009e0b6" dev="sda1" ino=396658 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:container_file_t:s0 tclass=dir permissive=1
type=AVC msg=audit(1574355721.524:279): avc:  denied  { getattr } for  pid=18185 comm="logrotate" path="/var/log/ceph/e0f31562-0c7f-11ea-9a6a-00259009e0b6/ceph.audit.log" dev="sda1" ino=396665 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:container_file_t:s0 tclass=file permissive=1

We could deploy a container to rotate logs, but this is awkward to deploy.  It would be preferable if the policy could be constructed to allow the host logrotate to access these log files.


Version-Release number of selected component (if applicable):

This is Ceph master (upcoming octopus release, eventually RHCS 5).


How reproducible:

Every time.


Steps to Reproduce:
1. curl --silent --remote-name --location https://github.com/ceph/ceph/raw/master/src/ceph-daemon/ceph-daemon
2. chmod +x ceph-daemon 
3. ./ceph-daemon bootstrap --mon-ip 172.21.6.136   # change the IP here
4. enter the shell container using the command output by the bootstrap step, which will look something like

 sudo ./ceph-daemon shell -c ceph.conf -k ceph.client.admin.keyring

5. ceph config set global log_to_file true
6. exit shell back to host
7. add a line to /etc/crontab to run /etc/cron.daily/logrotate as root in a minute or two from now
8. tail -f /var/log/audit/audit.log | grep avc

Actual results:

type=AVC msg=audit(1574355721.524:278): avc:  denied  { read } for  pid=18185 comm="logrotate" name="e0f31562-0c7f-11ea-9a6a-00259009e0b6" dev="sda1" ino=396658 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:container_file_t:s0 tclass=dir permissive=1
type=AVC msg=audit(1574355721.524:279): avc:  denied  { getattr } for  pid=18185 comm="logrotate" path="/var/log/ceph/e0f31562-0c7f-11ea-9a6a-00259009e0b6/ceph.audit.log" dev="sda1" ino=396665 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:container_file_t:s0 tclass=file permissive=1

and logs aren't rotated,

[root@mira057 ~]# ls -al /var/log/ceph/*
total 140
drwxrwx---. 2  167  167  4096 Nov 21 16:59 .
drwxr-x---. 3 root root  4096 Nov 21 16:57 ..
-rw-------. 1  167  167 21182 Nov 21 16:59 ceph.audit.log
-rw-------. 1  167  167 52703 Nov 21 17:08 ceph.log
-rw-r--r--. 1  167  167 36397 Nov 21 17:08 ceph-mgr.mira057.front.sepia.ceph.com.log
-rw-r--r--. 1  167  167 13314 Nov 21 17:07 ceph-mon.mira057.front.sepia.ceph.com.log


Expected results:

No denials.


Additional info:

Ceph selinux policy lives here:

https://github.com/ceph/ceph/tree/master/selinux

Comment 1 RHEL Program Management 2019-11-21 17:10:44 UTC
Please specify the severity of this bug. Severity is defined here:
https://bugzilla.redhat.com/page.cgi?id=fields.html#bug_severity.

Comment 2 Dimitri Savineau 2019-11-21 18:46:05 UTC
Trying to reproduce on my side but it seems to come from the logrotate script because the ceph group is hardcoded [1]

The ceph user/group aren't present on the host and I don't think you can use uid/gid with the logrotate su parameter.

# logrotate -f /etc/logrotate.d/ceph-bd95ccf4-0c89-11ea-ac5a-fa163e66a417 
error: /etc/logrotate.d/ceph-bd95ccf4-0c89-11ea-ac5a-fa163e66a417:12 unknown group 'ceph'
error: found error in /var/log/ceph/bd95ccf4-0c89-11ea-ac5a-fa163e66a417/*.log , skipping

Am I missing something ?

[1] https://github.com/ceph/ceph/blob/master/src/ceph-daemon/ceph-daemon#L666

Comment 3 Sage Weil 2019-11-22 20:13:38 UTC
I changed the 'su root ceph' line to 'su root root' and logrotate -f success without denials, but when run from cron, I still see

type=AVC msg=audit(1574453402.035:326): avc:  denied  { read } for  pid=28394 comm="logrotate" name="a274b0fe-0d63-11ea-a7c1-0cc47a8fd3d5" dev="sda1" ino=396619 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:container_file_t:s0 tclass=dir permissive=1
type=AVC msg=audit(1574453402.035:327): avc:  denied  { getattr } for  pid=28394 comm="logrotate" path="/var/log/ceph/a274b0fe-0d63-11ea-a7c1-0cc47a8fd3d5/ceph.audit.log" dev="sda1" ino=396647 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:container_file_t:s0 tclass=file permissive=1

Comment 6 Red Hat Bugzilla 2023-09-15 00:19:46 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 500 days


Note You need to log in before you can comment on or make changes to this bug.