Bug 1561711

Summary:	[OSP13] Got lots OVS daemon ERRs while starting a OVS-dpdk guest
Product:	Red Hat OpenStack	Reporter:	Lon Hohberger <lhh>
Component:	openstack-selinux	Assignee:	Lon Hohberger <lhh>
Status:	CLOSED ERRATA	QA Contact:	Maxim Babushkin <mbabushk>
Severity:	high	Docs Contact:
Priority:	urgent
Version:	13.0 (Queens)	CC:	aconole, atragler, berrange, ctrautma, fleitner, jherrman, jhsiao, jraju, jsuchane, juzhang, ktraynor, kzhang, maxime.coquelin, mbabushk, mgrepl, pezhang, rbalakri, rcain, skramaja, srevivo, tredaelli
Target Milestone:	rc	Keywords:	SELinux, Triaged
Target Release:	13.0 (Queens)
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:	openstack-selinux-0.8.14-12.el7ost	Doc Type:	Bug Fix
Doc Text:	Previously, the virtlogd service logged redundant AVC denial errors when a guest virtual machine was started. With this update, the virtlogd service no longer attempts to send shutdown inhibition calls to systemd, which prevents the described errors from occurring.	Story Points:	---
Clone Of:	1547250
Clones:	1561727 1561728 1561729 (view as bug list)		Environment:
Last Closed:	2018-06-27 13:49:05 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1561727, 1561728, 1561729

Description Lon Hohberger 2018-03-28 18:04:50 UTC

+++ This bug was initially created as a clone of Bug #1547250 +++

[snip]

Description of problem:

Got lots OVS daemon ERRs while starting a OVS-dpdk guest

[snip]

--- Additional comment from Jean-Tsung Hsiao on 2018-02-21 11:00:25 EST ---

Selinux could be the issue here.

On netqe19 when guest ran in CLIENT mode 2.9.0-1 fdP and qemu-kvm-rhev-2.10.0-20. If Selinux=Permissive, there was no such issue.

But, if Selinux=Enforcing, the issue happened --- lots of "truncted msg" ERRs seen in ovs-vswitchd.log.

See below for a USER_AVC.

[root@netqe19 ~]# tail -f /var/log/audit/audit.log | grep AVC
type=USER_AVC msg=audit(1519227919.365:2627): pid=1104 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc:  denied  { send_msg } for msgtype=method_call interface=org.freedesktop.login1.Manager member=Inhibit dest=org.freedesktop.login1 spid=2650 tpid=1095 scontext=system_u:system_r:virtlogd_t:s0-s0:c0.c1023 tcontext=system_u:system_r:systemd_logind_t:s0 tclass=dbus  exe="/usr/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?'


2018-02-21T15:54:30.709Z|1446065|dpdk|ERR|VHOST_CONFIG: truncted msg
2018-02-21T15:54:30.709Z|1446066|dpdk|ERR|VHOST_CONFIG: vhost read message failed
2018-02-21T15:54:30.709Z|1446067|dpdk|INFO|VHOST_CONFIG: new vhost user connection is 62
2018-02-21T15:54:30.709Z|1446068|dpdk|INFO|VHOST_CONFIG: new device, handle is 0
2018-02-21T15:54:30.709Z|1446069|dpdk|INFO|VHOST_CONFIG: read message VHOST_USER_GET_FEATURES
2018-02-21T15:54:30.709Z|1446070|dpdk|INFO|VHOST_CONFIG: read message VHOST_USER_GET_PROTOCOL_FEATURES
2018-02-21T15:54:30.709Z|1446071|dpdk|INFO|VHOST_CONFIG: read message VHOST_USER_SET_PROTOCOL_FEATURES
2018-02-21T15:54:30.709Z|1446072|dpdk|INFO|VHOST_CONFIG: read message VHOST_USER_GET_QUEUE_NUM
2018-02-21T15:54:30.709Z|1446073|dpdk|ERR|VHOST_CONFIG: truncted msg

[snip]

--- Additional comment from Daniel Berrange on 2018-03-06 11:17:02 EST ---

The virNetDaemon class that's used by virtlogd (and libvirtd) calls virNetDaemonCallInhibit() when it wants to prevent shutdown of the login session. This invokes the Inhibit message on logind over DBus, hence why this AVC is triggered. 

virtlogd inhibits shutdown whenever it has a log file for a running guest open, though. So the AVC being reported here is a gap in the policy.

That said, I think we could reasonably argue that virtlogd should not try to inhibit shutdown itself. libvirtd can already inhibit shutdown when QEMU is running, if required, so virtlogd is really not adding value in this respect.

So I'd suggest we can probably just remove the inhibit logic from src/logging/log_handler.c

[snip]


Goal is to simply work around this USER_AVC while this is fixed in a future RHEL7 update.

Comment 1 Lon Hohberger 2018-03-28 18:29:56 UTC

https://github.com/redhat-openstack/openstack-selinux/commit/bc744f2300da53e3f3b39b2b233a15a7e6197adf

Comment 6 Lon Hohberger 2018-05-23 13:45:46 UTC

/var/log/audit/audit.log.1:type=USER_AVC msg=audit(1527075220.353:14540): pid=581 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc:  denied  { send_msg } for msgtype=method_return dest=:1.866 spid=575 tpid=11664 scontext=system_u:system_r:systemd_logind_t:s0 tcontext=system_u:system_r:virtlogd_t:s0-s0:c0.c1023 tclass=dbus  exe="/usr/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?'

was turned up in CI, which is the opposite from the original AVC reported

Comment 10 Lon Hohberger 2018-05-29 11:24:16 UTC

0001-Allow-virtlogd-to-write-to-systemd_logind-FIFOs.patch also showed up, but may not be affecting this bug.

Comment 11 Lon Hohberger 2018-05-29 11:25:19 UTC

Bad paste:

type=AVC msg=audit(1527492439.572:13842): avc:  denied  { write } for  pid=10949 comm=\"virtlogd\" path=\"/run/systemd/inhibit/4.ref\" dev=\"tmpfs\" ino=251799 scontext=system_u:system_r:virtlogd_t:s0-s0:c0.c1023 tcontext=system_u:object_r:systemd_logind_inhibit_var_run_t:s0 tclass=fifo_file

showed up during CI runs

Comment 13 errata-xmlrpc 2018-06-27 13:49:05 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:2086