Bug 1640528

Summary: On RHEL 7.6 Undercloud installation fails on nova-api: sudo in nova-rootwrap blocked by SELinux
Product: Red Hat Enterprise Linux 7 Reporter: Pavel Sedlák <psedlak>
Component: selinux-policyAssignee: Lukas Vrabec <lvrabec>
Status: CLOSED ERRATA QA Contact: Milos Malik <mmalik>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 7.6CC: asoni, jmelvin, jpichon, jschluet, lhh, lvrabec, mmalik, pkomarov, plautrba, salmy, ssekidde, toneata, vmojzis, zcaplovi
Target Milestone: rcKeywords: Triaged, ZStream
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: selinux-policy-3.13.1-229.el7_6.6 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1638547
: 1645270 (view as bug list) Environment:
Last Closed: 2019-08-06 12:52:32 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1638547, 1638548, 1641671, 1641743, 1641746, 1645270, 1647587, 1651357, 1653106    

Description Pavel Sedlák 2018-10-18 09:44:57 UTC
+++ This bug was initially created as a clone of Bug #1638547 +++

Description of problem:

During installation of undercloud for OSP8 and/or OSP9 using:
> openstack undercloud install
it fails on nova-api
> Error: Could not start Service[nova-api]: Execution of '/bin/systemctl start openstack-nova-api' returned 1: Job for openstack-nova-api.service failed because a timeout was exceeded. See "systemctl status openstack-nova-api.service" and "journalctl -xe" for details.

in nova.log exception show failure of sudo nova-rootwrap:
> 2018-10-11 13:55:47.710 4825 DEBUG oslo_concurrency.processutils [-] u'sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-save -c' failed. Not Retrying. execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:375
> 2018-10-11 13:55:47.711 4825 DEBUG oslo_concurrency.lockutils [-] Lock "iptables" released by "nova.network.linux_net._apply" :: held 3.790s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:265
> 2018-10-11 13:55:47.713 4825 CRITICAL nova [-] ProcessExecutionError: Unexpected error while running command.
> Command: sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-save -c
> Exit code: 1
> Stdout: u''
> Stderr: u'sudo: PAM account management error: Authentication service cannot retrieve authentication info\n'
> 2018-10-11 13:55:47.713 4825 ERROR nova Traceback (most recent call last):
> 2018-10-11 13:55:47.713 4825 ERROR nova   File "/usr/bin/nova-api", line 10, in <module>
> 2018-10-11 13:55:47.713 4825 ERROR nova     sys.exit(main())
> ...
> 2018-10-11 13:55:47.713 4825 ERROR nova   File "/usr/lib/python2.7/site-packages/nova/utils.py", line 272, in execute
> 2018-10-11 13:55:47.713 4825 ERROR nova     return processutils.execute(*cmd, **kwargs)
> 2018-10-11 13:55:47.713 4825 ERROR nova   File "/usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py", line 342, in execute
> 2018-10-11 13:55:47.713 4825 ERROR nova     cmd=sanitized_cmd)
> 2018-10-11 13:55:47.713 4825 ERROR nova ProcessExecutionError: Unexpected error while running command.
> 2018-10-11 13:55:47.713 4825 ERROR nova Command: sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-save -c
> 2018-10-11 13:55:47.713 4825 ERROR nova Exit code: 1
> 2018-10-11 13:55:47.713 4825 ERROR nova Stdout: u''
> 2018-10-11 13:55:47.713 4825 ERROR nova Stderr: u'sudo: PAM account management error: Authentication service cannot retrieve authentication info\n'

in audit.log is visible about 65 entries like:
> type=AVC msg=audit(1539280257.488:1159): avc:  denied  { execute } for  pid=1782 comm="sudo" name="unix_chkpwd" dev="vda1" ino=4531529 scontext=system_u:system_r:nova_t:s0 tcontext=system_u:object_r:chkpwd_exec_t:s0 tclass=file permissive=0


Version-Release number of selected component (if applicable):
this happens on two osp versions, OSP8:
> openstack-selinux.noarch         0.8.14-15.el7ost       @rhelosp-8.0-puddle
> selinux-policy.noarch            3.13.1-229.el7         @rhelosp-rhel-7.6-server
and in case of OSP9:
> openstack-selinux.noarch             0.8.14-15.el7ost   @rhelosp-9.0-puddle     
> selinux-policy.noarch                3.13.1-229.el7     @rhelosp-rhel-7.6-server


How reproducible:
always

Steps to Reproduce:
1. on rhel-7.6 machine add RHOSP-8 repositories
2. install python-tripleoclient
3. openstack undercloud install

Actual results:
it fails, and in output there is error about systemctl start nova-api failed

Expected results:
undercloud installation succeeded without errors

Comment 1 Pavel Sedlák 2018-10-18 09:53:24 UTC
Also another nova_t related denials from run OpenStack packstack installer in permissive mode are:

> type=AVC msg=audit(...): avc:  denied  { connectto } for  pid=... comm="sudo" path="/run/dbus/system_bus_socket" scontext=system_u:system_r:nova_t:s0 tcontext=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 tclass=unix_stream_socket permissive=1
> type=AVC msg=audit(...): avc:  denied  { execute } for  pid=... comm="sudo" name="unix_chkpwd" dev="vda1" ino=... scontext=system_u:system_r:nova_t:s0 tcontext=system_u:object_r:chkpwd_exec_t:s0 tclass=file permissive=1
> type=AVC msg=audit(...): avc:  denied  { execute_no_trans } for  pid=... comm="sudo" path="/usr/sbin/unix_chkpwd" dev="vda1" ino=... scontext=system_u:system_r:nova_t:s0 tcontext=system_u:object_r:chkpwd_exec_t:s0 tclass=file permissive=1
> type=AVC msg=audit(...): avc:  denied  { getattr } for  pid=... comm="unix_chkpwd" path="/etc/shadow" dev="vda1" ino=... scontext=system_u:system_r:nova_t:s0 tcontext=system_u:object_r:shadow_t:s0 tclass=file permissive=1
> type=AVC msg=audit(...): avc:  denied  { open } for  pid=... comm="unix_chkpwd" path="/etc/shadow" dev="vda1" ino=... scontext=system_u:system_r:nova_t:s0 tcontext=system_u:object_r:shadow_t:s0 tclass=file permissive=1
> type=AVC msg=audit(...): avc:  denied  { read } for  pid=... comm="unix_chkpwd" name="shadow" dev="vda1" ino=... scontext=system_u:system_r:nova_t:s0 tcontext=system_u:object_r:shadow_t:s0 tclass=file permissive=1
> type=AVC msg=audit(...): avc:  denied  { read open } for  pid=... comm="sudo" path="/usr/sbin/unix_chkpwd" dev="vda1" ino=... scontext=system_u:system_r:nova_t:s0 tcontext=system_u:object_r:chkpwd_exec_t:s0 tclass=file permissive=1
> type=USER_AVC msg=audit(): pid=... uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc:  denied  { send_msg } for msgtype=method_call interface=org.freedesktop.DBus member=Hello dest=org.freedesktop.DBus spid=... scontext=system_u:system_r:nova_t:s0 tcontext=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 tclass=dbus  exe="/usr/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?'
> type=USER_AVC msg=audit(): pid=... uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc:  denied  { send_msg } for msgtype=method_call interface=org.freedesktop.login1.Manager member=CreateSession dest=org.freedesktop.login1 spid=... tpid=... scontext=system_u:system_r:nova_t:s0 tcontext=system_u:system_r:systemd_logind_t:s0 tclass=dbus  exe="/usr/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?'
> type=USER_AVC msg=audit(): pid=... uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc:  denied  { send_msg } for msgtype=method_return dest=... spid=... tpid=... scontext=system_u:system_r:systemd_logind_t:s0 tcontext=system_u:system_r:nova_t:s0 tclass=dbus  exe="/usr/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?'

Comment 4 Lukas Vrabec 2018-10-19 08:38:05 UTC
commit 9684362a3b17030829324948190061f48f2c2126 (HEAD -> rhel7.7-contrib, origin/rhel7.7-contrib)
Author: Lukas Vrabec <lvrabec>
Date:   Fri Oct 19 10:29:36 2018 +0200

    Allow nova_t domain to use pam
    Resolves: rhbz:#1640528

Comment 8 Ollie Walsh 2018-11-23 16:14:33 UTC
*** Bug 1652035 has been marked as a duplicate of this bug. ***

Comment 18 errata-xmlrpc 2019-08-06 12:52:32 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2127