Bug 1261747

Summary: Upgrade of systemd leaves service enablement failing with selinux error
Product: Red Hat Enterprise Linux 7 Reporter: Ian Wienand <iwienand>
Component: selinux-policyAssignee: Miroslav Grepl <mgrepl>
Status: CLOSED WONTFIX QA Contact: Milos Malik <mmalik>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.0CC: jsynacek, lnykryn, lvrabec, mgrepl, mmalik, plautrba, pvrabec, ssekidde, systemd-maint-list
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-02-21 14:57:44 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
log of upgrade session, showing failure and selinux error none

Description Ian Wienand 2015-09-10 06:13:32 UTC
Starting out from a freshly booted base system of

===
[root@testing centos]# cat /etc/redhat-release 
CentOS Linux release 7.0.1406 (Core) 
===

(this is the "CentOS 7 x86_64 (2014-09-29) - Partner Image" provided by hp's cloud service) I run an install of systemd, which pulls down the following

===
Running transaction
  Updating   : systemd-libs-208-20.el7_1.5.x86_64 
  Updating   : systemd-208-20.el7_1.5.x86_64 
  Updating   : systemd-sysv-208-20.el7_1.5.x86_64 
  Updating   : libgudev1-208-20.el7_1.5.x86_64
===

This goes fine, but immediately after this, we can not enable services due to selinux failures; e.g.

===
[root@testing centos]# /bin/yum -d 0 -e 0 -y install iptables-services
[root@testing centos]# /bin/systemctl enable iptables
Failed to issue method call: Access denied
[root@testing centos]# journalctl -xn
Sep 10 05:32:31 testing systemd[1]: SELinux policy denies access.
===

In fact, it seems the rpm install also had a selinux failure with "systemctl preset" during install, then you can see the enable done afterwards:

===
[root@testing centos]# ausearch -m avc -m user_avc -m selinux_err
----
time->Thu Sep 10 05:32:31 2015
type=USER_AVC msg=audit(1441863151.298:84): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='avc:  denied  { enable } for auid=1000 uid=0 gid=0 cmdline="/usr/bin/systemctl preset iptables.service ip6tables.service" scontext=unconfined_u:system_r:rpm_script_t:s0-s0:c0.c1023 tcontext=system_u:system_r:init_t:s0 tclass=service  exe="/usr/lib/systemd/systemd" sauid=0 hostname=? addr=? terminal=?'
----
time->Thu Sep 10 05:32:36 2015
type=USER_AVC msg=audit(1441863156.979:85): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='avc:  denied  { enable } for auid=1000 uid=0 gid=0 cmdline="/bin/systemctl enable iptables" scontext=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 tcontext=system_u:system_r:init_t:s0 tclass=service  exe="/usr/lib/systemd/systemd" sauid=0 hostname=? addr=? terminal=?'
----
===

After this, a daemon-reload does get things going again

===
[root@testing centos]# systemctl daemon-reload
[root@testing centos]# /bin/systemctl enable iptables
ln -s '/usr/lib/systemd/system/iptables.service' '/etc/systemd/system/basic.target.wants/iptables.service'
===

So although that's an easy work-around, it's kind of annoying because this is all happening via puppet of course, so it means extra hacks to the scripts.

Comment 2 Ian Wienand 2015-09-10 06:15:27 UTC
Created attachment 1072025 [details]
log of upgrade session, showing failure and selinux error

Comment 3 Ian Wienand 2015-09-17 05:38:18 UTC
So maybe this is a selinux issue?

When this fails, I see the following when stracing systemd

If I do the following
1. install systemd
2. install iptables-services
3. $ systemctl enable iptables

I get the following strace out of systemd for the enable.

---
1     read(25, "systemctl\0enable\0iptables\0", 1024) = 26
1     read(25, "", 1024)                = 0
1     close(25)                         = 0
1     munmap(0x7faf54d3d000, 4096)      = 0
1     open("/proc/self/task/1/attr/current", O_RDONLY|O_CLOEXEC) = 25
1     read(25, "system_u:system_r:init_t:s0\0", 4095) = 28
1     close(25)                         = 0
1     socket(PF_LOCAL, SOCK_STREAM|SOCK_CLOEXEC, 0) = 25
1     connect(25, {sa_family=AF_LOCAL, sun_path="/var/run/setrans/.setrans-unix"}, 110) = -1 ENOENT (No such file or directory)
1     close(25)                         = 0
1     socket(PF_LOCAL, SOCK_STREAM|SOCK_CLOEXEC, 0) = 25
1     connect(25, {sa_family=AF_LOCAL, sun_path="/var/run/setrans/.setrans-unix"}, 110) = -1 ENOENT (No such file or directory)
1     close(25)                         = 0
1     poll([{fd=29, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
1     open("/sys/fs/selinux/access", O_RDWR) = 25
1     write(25, "unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 system_u:system_r:init_t:s0 82 40", 87) = 87
1     read(25, "f ffffffff 0 ffffffff 1 0", 4095) = 25
1     close(25)                         = 0
[continue to failure]
---

If, however, I do a "daemon-reload" of systemd *after* I install iptables-services, I get the following

---
1     read(25, "systemctl\0enable\0iptables\0", 1024) = 26
1     read(25, "", 1024)                = 0
1     close(25)                         = 0
1     munmap(0x7faf54d3d000, 4096)      = 0
1     open("/proc/self/task/1/attr/current", O_RDONLY|O_CLOEXEC) = 25
1     read(25, "system_u:system_r:init_t:s0\0", 4095) = 28
1     close(25)                         = 0
1     socket(PF_LOCAL, SOCK_STREAM|SOCK_CLOEXEC, 0) = 25
1     connect(25, {sa_family=AF_LOCAL, sun_path="/var/run/setrans/.setrans-unix"}, 110) = -1 ENOENT (No such file or directory)
1     close(25)                         = 0
1     socket(PF_LOCAL, SOCK_STREAM|SOCK_CLOEXEC, 0) = 25
1     connect(25, {sa_family=AF_LOCAL, sun_path="/var/run/setrans/.setrans-unix"}, 110) = -1 ENOENT (No such file or directory)
1     close(25)                         = 0
1     poll([{fd=29, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
1     open("/sys/fs/selinux/access", O_RDWR) = 25
1     write(25, "unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 system_u:object_r:iptables_unit_file_t:s0 82 40", 101) = 101
1     read(25, "ffffffff ffffffff 0 ffffffff 1 0", 4095) = 32
1     close(25)  
[continue to work]
---

Note, iptables-services has already done "systemctl preset iptables" as part of the RPM install scripts (you can see the avc denial there).  I don't know why a "daemon-reload" after the package is installed gets it working, but the "systemctl preset" doesn't.

I don't really have a good idea where it's getting these selinux permissions from, but that's what happens.

I also note that if I install selinux-policy first, then it seems to "just work"

Comment 4 Lukáš Nykrýn 2015-09-17 08:00:43 UTC
Can you try this with rhel-7.2 beta, I believe, that we have fixed that.

Comment 5 Ian Wienand 2015-09-17 21:15:11 UTC
(In reply to Lukáš Nykrýn from comment #4)
> Can you try this with rhel-7.2 beta, I believe, that we have fixed that.

Unfortunately, I'm mostly at the mercy of the cloud provider and the base image they provide, at this point

Can you give a pointer on what was fixed/how?  I'm fine with workarounds, but it's not clear to me what that is (is it systemd, selinux-policy, iptables, iptables-services, some combination?)

Comment 6 Ian Wienand 2015-09-20 23:53:29 UTC
What ended up working around this was installing selinux-policy first, then systemd and restarting [1]

Also, when my F22 system couldn't reboot, I noticed [2] which seems to be in a similar situation of policy updates conflicting.  Might be related

[1] https://review.openstack.org/#/c/224433/2
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1224211

Comment 7 Jan Synacek 2016-11-23 09:03:15 UTC
(In reply to Ian Wienand from comment #6)
> ...
> [2] https://bugzilla.redhat.com/show_bug.cgi?id=1224211

This bug had two sides - systemd and selinux-policy. The systemd part of the fix mentioned in that bug was backported when fixing https://bugzilla.redhat.com/show_bug.cgi?id=1230190 and is now present in RHEL-7.3.

I don't know if the selinux-policy part was fixed in RHEL-7.3. Changing component.

Comment 8 Miroslav Grepl 2019-02-21 14:57:27 UTC
This issue was not selected to be included in Red Hat Enterprise Linux 7.7 because it is seen either as low or moderate impact to a small number of use-cases. The next release will be in Maintenance Support 1 Phase, which means that qualified Critical and Important Security errata advisories (RHSAs) and Urgent Priority Bug Fix errata advisories (RHBAs) may be released as they become available. We will now close this issue, but if you believe that it qualifies for the Maintenance Support 1 Phase, please re-open; otherwise, we recommend moving the request to Red Hat Enterprise Linux 8 if applicable.