Bug 1413775

Summary: rabbitmqctl rotate_logs issues with selinux in OSP 9 and 10
Product: Red Hat OpenStack Reporter: Andreas Karis <akaris>
Component: rabbitmq-serverAssignee: Peter Lemenkov <plemenko>
Status: CLOSED CURRENTRELEASE QA Contact: Udi Shkalim <ushkalim>
Severity: low Docs Contact:
Priority: low    
Version: 9.0 (Mitaka)CC: akaris, apevec, chjones, dabarzil, fdinitto, jeckersb, jjoyce, lhh, mflusche, michele, mwinkler, plemenko, sclewis, srevivo
Target Milestone: ---Keywords: Triaged, ZStream
Target Release: 9.0 (Mitaka)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-07-20 08:59:32 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1426600    

Description Andreas Karis 2017-01-16 23:34:38 UTC 
Description of problem:
rabbitmqctl rotate_logs issues with selinux in OSP 9 and 10

Logging issue seems to be caused by selinux. This cannot be reproduced manually simply by running 'rabbitmqctl rotate_logs'. Likely, selinux context is different just running as root.

From a lab system, this example is OSP 10, but I see the same issue on OSP 9 and rabbitmq-server-3.6.3-5.el7ost.noarch:
 
[root@undercloud10 rabbitmq]# grep logrotate /var/log/messages
Jan 16 03:17:18 undercloud10 logrotate: ALERT exited abnormally with [1]

[root@undercloud10 rabbitmq]# ls -l /var/log/rabbitmq/
total 44
-rw-r--r--. 1 rabbitmq rabbitmq     0 Jan 16 03:17 rabbit
-rw-r--r--. 1 rabbitmq rabbitmq  9916 Dec 19 10:49 rabbit.gz
-rw-r--r--. 1 rabbitmq rabbitmq  3113 Dec 20 16:30 rabbit.gz
-rw-r--r--. 1 rabbitmq rabbitmq  1995 Jan  4 13:22 rabbit.gz
-rw-r--r--. 1 rabbitmq rabbitmq  2428 Jan 10 10:58 rabbit.gz
-rw-r--r--. 1 rabbitmq rabbitmq 19325 Jan 16 13:32 rabbit
-rw-r--r--. 1 rabbitmq rabbitmq     0 Dec  7 15:18 rabbit

[root@undercloud10 rabbitmq]# grep rabbit /var/log/audit/audit.log 

type=USER_AUTH msg=audit(1484554626.328:1529): pid=29567 uid=0 auid=0 ses=163 subj=system_u:system_r:logrotate_t:s0-s0:c0.c1023 msg='op=PAM:authentication grantors=? acct="rabbitmq" exe="/usr/bin/su" hostname=? addr=? terminal=? res=failed'

[root@undercloud10 rabbitmq]# date -d @1484554626
Mon Jan 16 03:17:06 EST 2017
Comment 1 Andreas Karis 2017-01-16 23:48:15 UTC 
OSP 9:

    [root@undercloud9 ~]# grep logrotate /var/log/messages
    Jan 16 12:42:21 undercloud9 logrotate: ALERT exited abnormally with [1]
     
    [root@undercloud9 ~]# grep logrotate /var/log/audit/audit.log
    type=AVC msg=audit(1484588529.273:324): avc:  denied  { bind } for  pid=25741 comm="su" scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tclass=netlink_selinux_socket
    type=SYSCALL msg=audit(1484588529.273:324): arch=c000003e syscall=49 success=no exit=-13 a0=3 a1=7ffc72ec5610 a2=c a3=1 items=0 ppid=25727 pid=25741 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=5 comm="su" exe="/usr/bin/su" subj=system_u:system_r:logrotate_t:s0-s0:c0.c1023 key=(null)
    type=USER_AVC msg=audit(1484588529.274:325): pid=25741 uid=0 auid=0 ses=5 subj=system_u:system_r:logrotate_t:s0-s0:c0.c1023 msg='avc:  can't open netlink socket: 13 (Permission denied)  exe="/usr/bin/su" sauid=0 hostname=? addr=? terminal=?'
    type=ANOM_ABEND msg=audit(1484588529.274:326): auid=0 uid=0 gid=0 ses=5 subj=system_u:system_r:logrotate_t:s0-s0:c0.c1023 pid=25741 comm="su" reason="memory violation" sig=6
     
     
    [root@undercloud9 ~]# date -d @1484588529
    Mon Jan 16 12:42:09 EST 2017
Comment 4 Matt Flusche 2017-01-17 16:34:46 UTC 
Peter,

Here are the details with selinux in permissive mode; I changed logrotate to roll the rabbit log every hour so we don't have to wait.

[root@undercloud10 ~]# cat /var/log/audit/audit.log |audit2allow 

#============= logrotate_t ==============
allow logrotate_t epmd_port_t:tcp_socket name_connect;
allow logrotate_t rabbitmq_port_t:tcp_socket name_connect;
allow logrotate_t rabbitmq_var_lib_t:dir { read write };
allow logrotate_t rabbitmq_var_lib_t:file { getattr open read };
allow logrotate_t security_t:security compute_av;
allow logrotate_t self:passwd passwd;
allow logrotate_t self:process setrlimit;

[root@undercloud10 ~]# getenforce
Permissive
Comment 7 Peter Lemenkov 2017-05-26 16:16:03 UTC 
I've finally got a patch for SELinux against Fedora Rawhide. I'll provide a build as soon as it will be approved by upstream and merged. Hopefully soon enough.
Comment 10 Peter Lemenkov 2018-03-12 15:24:46 UTC 
It seems that it takes a lot of time to move these patches through the upstream selinux-policy repository, so I decided to switch to a more short path and push them into openstack-selinux repository. Expect a proper solution soon.
Comment 11 Peter Lemenkov 2018-04-12 06:15:06 UTC 
This issue is fixed in openstack-selinux-0.8.14-1.el7ost which is available for all RHOS-10, RHOS-11, RHOS-12, RHOS-13.