Bug 1945646

"audit2allow -a" mentioned that it needs "allow systemd_hostnamed_t initrc_t:dbus send_msg" in the policy.

Since RHCOS is just consuming the SELinux policy from RHEL, going to move this to RHEL/selinux-policy for additional triage

The above-mentioned SELinux denial contains the word "granted" which means that nothing was denied.

Normally, the tuned service runs under tuned_t context and SELinux policy contains these rules:

allow tuned_t security_t:security setsecparam;
auditallow tuned_t security_t:security setsecparam;

(In reply to Qian Cai from comment #1)
> "audit2allow -a" mentioned that it needs "allow systemd_hostnamed_t initrc_t:dbus send_msg" in the policy.

Also note this is a problem different to the one in the bz description. Can you find out which process is running in initrc_t context? Was it started properly as a service?

ps -eo pid,ppid,command,context | grep -e CONTEXT -e initrc_t

# ps -eo pid,ppid,command,context | grep -e CONTEXT -e initrc_t
    PID    PPID COMMAND                     CONTEXT
   1298       1 /bin/bash /usr/sbin/gcp-rou system_u:system_r:initrc_t:s0
 419607    1298 sleep 5                     system_u:system_r:initrc_t:s0
 419731  419375 grep -e CONTEXT -e initrc_t system_u:system_r:spc_t:s0

# cat /usr/sbin/gcp-routes.sh
#!/bin/bash

# Update iptables rules based on google cloud load balancer VIPS
#
# This is needed because the GCP L3 load balancer doesn't actually do DNAT; 
# the destination IP address is still the VIP. Normally, there is an agent that
# adds the vip to the local routing table, tricking the kernel in to thinking
# it's a local IP and allowing processes doing an accept(0.0.0.0) to receive
# the packets. Clever.
#
# We don't do that. Instead, we DNAT with conntrack. This is so we don't break
# existing connections when the vip is removed. This is useful for draining
# connections - take ourselves out of the vip, but service existing conns.
#
# Additionally, clients can write a file to /run/gcp-routes/$IP.down to force
# a VIP as down. This is useful for graceful shutdown / upgrade.
#
# ~cdc~
...

(In reply to Milos Malik from comment #3)
> The above-mentioned SELinux denial contains the word "granted" which means
> that nothing was denied.

Is there a way to silent those "granted" messages to not "pollute" the audit logs?

(In reply to Qian Cai from comment #7)
> (In reply to Milos Malik from comment #3)
> > The above-mentioned SELinux denial contains the word "granted" which means
> > that nothing was denied.
> 
> Is there a way to silent those "granted" messages to not "pollute" the audit
> logs?
The purpose of auditallow rules is to notify the administrator as this is of particular importance, I am not aware of a way of silencing the rules. There are only a few of such rules in the policy.

> # ps -eo pid,ppid,command,context | grep -e CONTEXT -e initrc_t
>    PID    PPID COMMAND                     CONTEXT
>   1298       1 /bin/bash /usr/sbin/gcp-rou system_u:system_r:initrc_t:s0
> 419607    1298 sleep 5                     system_u:system_r:initrc_t:s0
> 419731  419375 grep -e CONTEXT -e initrc_t system_u:system_r:spc_t:s0
How is the script started and which package it is a part of? If it needs some additional permissions like direct communication with hostnamed, it needs to be SELinux confined.

(In reply to Zdenek Pytela from comment #8)
> (In reply to Qian Cai from comment #7)
> > (In reply to Milos Malik from comment #3)
> > > The above-mentioned SELinux denial contains the word "granted" which means
> > > that nothing was denied.
> > 
> > Is there a way to silent those "granted" messages to not "pollute" the audit
> > logs?
> The purpose of auditallow rules is to notify the administrator as this is of
> particular importance, I am not aware of a way of silencing the rules. There
> are only a few of such rules in the policy.

Okay, so admin is now aware of this. What should admins do? If they reviewed those and acknowledged that those are the same things to do. Then, Isn't it annoying that they will see other over and over again which could render selinux less useful?

> 
> > # ps -eo pid,ppid,command,context | grep -e CONTEXT -e initrc_t
> >    PID    PPID COMMAND                     CONTEXT
> >   1298       1 /bin/bash /usr/sbin/gcp-rou system_u:system_r:initrc_t:s0
> > 419607    1298 sleep 5                     system_u:system_r:initrc_t:s0
> > 419731  419375 grep -e CONTEXT -e initrc_t system_u:system_r:spc_t:s0
> How is the script started and which package it is a part of? If it needs
> some additional permissions like direct communication with hostnamed, it
> needs to be SELinux confined.

I think that is something that need input from OCP/RHCOS side. Micah, is that something you can find out more about gcp-routes.sh for default OCP installation on GCP.

(In reply to Qian Cai from comment #9)

> I think that is something that need input from OCP/RHCOS side. Micah, is
> that something you can find out more about gcp-routes.sh for default OCP
> installation on GCP.

The `gcp-routes.sh` script is something that we include in RHCOS at build time; it's used to manage iptables rules related to the GCP load balancer VIPs.

It's started as a simple systemd service.  See the service definition and script at the links below:

https://github.com/openshift/os/blob/a06951cb0b9ec63d79087ca257a2135601e027dd/overlay.d/05rhcos/usr/lib/systemd/system/gcp-routes.service
https://github.com/openshift/os/blob/a06951cb0b9ec63d79087ca257a2135601e027dd/overlay.d/05rhcos/usr/sbin/gcp-routes.sh

> > The purpose of auditallow rules is to notify the administrator as this is of
> > particular importance, I am not aware of a way of silencing the rules. There
> > are only a few of such rules in the policy.
> 
> Okay, so admin is now aware of this. What should admins do? If they reviewed those and acknowledged that those are the same
> things to do. Then, Isn't it annoying that they will see other over and over again which could render selinux less useful?
These type of audit records is not frequent (there is total of 5 such rule) so it should not bother excessively. The sole purpose of an auditallow record is not to be overlooked.

> The `gcp-routes.sh` script is something that we include in RHCOS at build time; it's used to manage iptables rules related to the GCP load balancer VIPs.
> 
> It's started as a simple systemd service.  See the service definition and script at the links below:
> 
> https://github.com/openshift/os/blob/a06951cb0b9ec63d79087ca257a2135601e027dd/overlay.d/05rhcos/usr/lib/systemd/system/gcp-routes.service
> https://github.com/openshift/os/blob/a06951cb0b9ec63d79087ca257a2135601e027dd/overlay.d/05rhcos/usr/sbin/gcp-routes.sh
There is a substantial difference between
ExecStart=/bin/bash /usr/sbin/gcp-routes.sh start
and
ExecStart=/usr/sbin/gcp-routes.sh start
from selinux point of view: in the former case the service starts in initrc_t domain while in the latter one it is unconfined_service_t which is allowed most of possible permission. I suggest make the script executable and start directly, not preceded by bash. If still additional permissions were required, the script would need to be confined.

(In reply to Zdenek Pytela from comment #11)
> These type of audit records is not frequent (there is total of 5 such rule)
> so it should not bother excessively. The sole purpose of an auditallow
> record is not to be overlooked.

As mentioned, this looks like plague every single OCP installations. Since we noticed the issues has been there for a few OCP releases, I'll leave it to OCP support organizations to escalate further more once we hear more from customers. 

> 
> > The `gcp-routes.sh` script is something that we include in RHCOS at build time; it's used to manage iptables rules related to the GCP load balancer VIPs.
> > 
> > It's started as a simple systemd service.  See the service definition and script at the links below:
> > 
> > https://github.com/openshift/os/blob/a06951cb0b9ec63d79087ca257a2135601e027dd/overlay.d/05rhcos/usr/lib/systemd/system/gcp-routes.service
> > https://github.com/openshift/os/blob/a06951cb0b9ec63d79087ca257a2135601e027dd/overlay.d/05rhcos/usr/sbin/gcp-routes.sh
> There is a substantial difference between
> ExecStart=/bin/bash /usr/sbin/gcp-routes.sh start
> and
> ExecStart=/usr/sbin/gcp-routes.sh start
> from selinux point of view: in the former case the service starts in
> initrc_t domain while in the latter one it is unconfined_service_t which is
> allowed most of possible permission. I suggest make the script executable
> and start directly, not preceded by bash. If still additional permissions
> were required, the script would need to be confined.

Okay, so looks like we should flip the bug to OCP CoreOS again, so they could fix the script?

(In reply to Qian Cai from comment #12)

> Okay, so looks like we should flip the bug to OCP CoreOS again, so they
> could fix the script?

Yup, we'll take it.  I have a patch to the service file that I've tested on an RHCOS node in GCP:

```
$ systemctl cat gcp-routes
# /usr/lib/systemd/system/gcp-routes.service
[Unit]
Description=Update GCP routes for forwarded IPs.
ConditionKernelCommandLine=|ignition.platform.id=gce
ConditionKernelCommandLine=|ignition.platform.id=gcp
After=network.target

[Service]
Type=simple
ExecStart=/usr/sbin/gcp-routes.sh start
ExecStopPost=/usr/sbin/gcp-routes.sh cleanup
User=root
RestartSec=30
Restart=always


[Install]
WantedBy=multi-user.target

$ systemctl status gcp-routes
● gcp-routes.service - Update GCP routes for forwarded IPs.
   Loaded: loaded (/usr/lib/systemd/system/gcp-routes.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2021-04-16 18:23:17 UTC; 4min 7s ago
 Main PID: 1293 (gcp-routes.sh)
    Tasks: 2 (limit: 22948)
   Memory: 1.1M
   CGroup: /system.slice/gcp-routes.service
           ├─1293 /bin/bash /usr/sbin/gcp-routes.sh start
           └─1651 sleep 5

Apr 16 18:23:17 miabbott-rhcos-48.c.openshift-rhcos-devel.internal systemd[1]: Started Update GCP routes for forwarded IPs..
Apr 16 18:23:18 miabbott-rhcos-48.c.openshift-rhcos-devel.internal gcp-routes.sh[1293]: done applying vip rules
Apr 16 18:27:18 miabbott-rhcos-48.c.openshift-rhcos-devel.internal gcp-routes.sh[1293]: done applying vip rules

$ ps -eo pid,ppid,command,context | grep -e CONTEXT -e initrc_t
    PID    PPID COMMAND                     CONTEXT

$ ps -eo pid,ppid,command,context | grep -e CONTEXT -e unconfined_service_t
    PID    PPID COMMAND                     CONTEXT
   1293       1 /bin/bash /usr/sbin/gcp-rou system_u:system_r:unconfined_service_t:s0
   1657    1293 sleep 5                     system_u:system_r:unconfined_service_t:s0

$ sudo ausearch -m avc
<no matches>

```

Please also make sure that “audit2allow -a” does not report anything.

This landed in RHCOS as part of https://gitlab.cee.redhat.com/coreos/redhat-coreos/-/merge_requests/1255/

First included in RHCOS 48.84.202104221420-0

Should be part of the nightly payloads now

miabbott  There seems to be two issues discussed in this BZ--one described in the title and one described in the notes.  

The issue in the title ( systemd_hostnamed_t initrc_t:dbus send_msg) which is shown in `audit2allow -a` output is caused by gcp-routes running as initrc_t.  I am still seeing this in the `audit2allow` output in 4.8.0-0.nightly-2021-04-30-201824.  Looking at the context of gcp-routes, it is now running as `unconfined_service_t, but still showing up in `audit2allow -a` output.  Is there something I need to do to reset this message?  See output below.

The second issue, as described in the comments `ausearch -m avc`, is caused by the openshift-cluster-node-tuning-operator.  A separate issue needs to be filed for this.


$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.8.0-0.nightly-2021-04-30-201824   True        False         28m     Cluster version is 4.8.0-0.nightly-2021-04-30-201824
$ oc get nodes
NAME                                       STATUS   ROLES    AGE   VERSION
ci-ln-h710m42-f76d1-tqdqv-master-0         Ready    master   53m   v1.21.0-rc.0+aa1dc1f
ci-ln-h710m42-f76d1-tqdqv-master-1         Ready    master   53m   v1.21.0-rc.0+aa1dc1f
ci-ln-h710m42-f76d1-tqdqv-master-2         Ready    master   53m   v1.21.0-rc.0+aa1dc1f
ci-ln-h710m42-f76d1-tqdqv-worker-b-lsjmm   Ready    worker   45m   v1.21.0-rc.0+aa1dc1f
ci-ln-h710m42-f76d1-tqdqv-worker-c-v4jh6   Ready    worker   45m   v1.21.0-rc.0+aa1dc1f
ci-ln-h710m42-f76d1-tqdqv-worker-d-fbvtd   Ready    worker   45m   v1.21.0-rc.0+aa1dc1f
$ oc debug node/ci-ln-h710m42-f76d1-tqdqv-worker-b-lsjmm
Starting pod/ci-ln-h710m42-f76d1-tqdqv-worker-b-lsjmm-debug ...
To use host binaries, run `chroot /host`
If you don't see a command prompt, try pressing enter.
sh-4.2# chroot /host
sh-4.4# systemctl cat gcp-routes
# /usr/lib/systemd/system/gcp-routes.service
[Unit]
Description=Update GCP routes for forwarded IPs.
ConditionKernelCommandLine=|ignition.platform.id=gce
ConditionKernelCommandLine=|ignition.platform.id=gcp
After=network.target

[Service]
Type=simple
ExecStart=/usr/sbin/gcp-routes.sh start
ExecStopPost=/usr/sbin/gcp-routes.sh cleanup
User=root
RestartSec=30
Restart=always

[Install]
WantedBy=multi-user.target
sh-4.4# systemctl status gcp-routes
● gcp-routes.service - Update GCP routes for forwarded IPs.
   Loaded: loaded (/usr/lib/systemd/system/gcp-routes.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2021-05-03 16:43:42 UTC; 46min ago
 Main PID: 1242 (gcp-routes.sh)
    Tasks: 2 (limit: 95515)
   Memory: 1.1M
      CPU: 1.342s
   CGroup: /system.slice/gcp-routes.service
           ├─ 1242 /bin/bash /usr/sbin/gcp-routes.sh start
           └─82775 sleep 5

May 03 17:15:43 ci-ln-h710m42-f76d1-tqdqv-worker-b-lsjmm gcp-routes.sh[1242]: done applying vip rules
May 03 17:19:43 ci-ln-h710m42-f76d1-tqdqv-worker-b-lsjmm gcp-routes.sh[1242]: Processing route for NIC 0/42:01:0a:00:80:04 for 35.229.42.125
May 03 17:19:43 ci-ln-h710m42-f76d1-tqdqv-worker-b-lsjmm gcp-routes.sh[1242]: ensuring rule for 35.229.42.125
May 03 17:19:43 ci-ln-h710m42-f76d1-tqdqv-worker-b-lsjmm gcp-routes.sh[1242]: done applying vip rules
May 03 17:23:43 ci-ln-h710m42-f76d1-tqdqv-worker-b-lsjmm gcp-routes.sh[1242]: Processing route for NIC 0/42:01:0a:00:80:04 for 35.229.42.125
May 03 17:23:43 ci-ln-h710m42-f76d1-tqdqv-worker-b-lsjmm gcp-routes.sh[1242]: ensuring rule for 35.229.42.125
May 03 17:23:43 ci-ln-h710m42-f76d1-tqdqv-worker-b-lsjmm gcp-routes.sh[1242]: done applying vip rules
May 03 17:27:43 ci-ln-h710m42-f76d1-tqdqv-worker-b-lsjmm gcp-routes.sh[1242]: Processing route for NIC 0/42:01:0a:00:80:04 for 35.229.42.125
May 03 17:27:43 ci-ln-h710m42-f76d1-tqdqv-worker-b-lsjmm gcp-routes.sh[1242]: ensuring rule for 35.229.42.125
May 03 17:27:43 ci-ln-h710m42-f76d1-tqdqv-worker-b-lsjmm gcp-routes.sh[1242]: done applying vip rules

sh-4.4# ps -eo pid,ppid,command,context | grep -e CONTEXT -e initrc_t
    PID    PPID COMMAND                     CONTEXT
  84707   82256 grep -e CONTEXT -e initrc_t system_u:system_r:spc_t:s0
sh-4.4# ps -eo pid,ppid,command,context | grep -e CONTEXT -e unconfined_service_t
    PID    PPID COMMAND                     CONTEXT
   1242       1 /bin/bash /usr/sbin/gcp-rou system_u:system_r:unconfined_service_t:s0
   1472       1 kubelet --config=/etc/kuber system_u:system_r:unconfined_service_t:s0
  84890    1242 sleep 5                     system_u:system_r:unconfined_service_t:s0
  84892   82256 grep -e CONTEXT -e unconfin system_u:system_r:spc_t:s0

sh-4.4# sudo ausearch -m avc
----
time->Mon May  3 16:44:38 2021
type=PROCTITLE msg=audit(1620060278.578:181): proctitle=2F7573722F6C6962657865632F706C6174666F726D2D707974686F6E002D4573002F7573722F7362696E2F74756E6564002D2D6E6F2D64627573
type=SYSCALL msg=audit(1620060278.578:181): arch=c000003e syscall=1 success=yes exit=4 a0=7 a1=7fa854015b80 a2=4 a3=0 items=0 ppid=2566 pid=3416 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="tuned" exe="/usr/libexec/platform-python3.6" subj=system_u:system_r:spc_t:s0 key=(null)
type=AVC msg=audit(1620060278.578:181): avc:  granted  { setsecparam } for  pid=3416 comm="tuned" scontext=system_u:system_r:spc_t:s0 tcontext=system_u:object_r:security_t:s0 tclass=security
sh-4.4# 
sh-4.4# rpm-ostree status
State: idle
Deployments:
* pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:42d24e77a34e22a89dcd467e227bedab3dfb81b5bfec4ea9686f617680bcb9d9
              CustomOrigin: Managed by machine-config-operator
                   Version: 48.84.202104301525-0 (2021-04-30T15:29:10Z)

  ostree://328a44d7c259ca1e3ed31ae020f09d922f460be998657a92f684f6760443077b
                   Version: 48.83.202103221318-0 (2021-03-22T13:22:02Z)
sh-4.4# audit2allow -a


#============= systemd_hostnamed_t ==============
allow systemd_hostnamed_t initrc_t:dbus send_msg;

Since the fix attached to this BZ was for the gcp-routes.sh script, I'll retitle this BZ to scope the fix to just that script/problem.

I'll create a clone of this BZ for the original problem related to the output in `audit2allow`

I've opened another BZ for the original problem at BZ#1956836

Closing as verified based on https://bugzilla.redhat.com/show_bug.cgi?id=1945646#c19 @ 4.8.0-0.nightly-2021-04-30-201824

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:2438