Bug 1945644
| Summary: | systemd provider in /etc/nsswitch.conf introduces 25s lockups for NFS share mount | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 8 | Reporter: | Leonid Titov <ltitov> |
| Component: | selinux-policy | Assignee: | Zdenek Pytela <zpytela> |
| Status: | CLOSED DUPLICATE | QA Contact: | BaseOS QE Security Team <qe-baseos-security> |
| Severity: | high | Docs Contact: | |
| Priority: | medium | ||
| Version: | 8.2 | CC: | atikhono, bbreard, grajaiya, imcleod, jhrozek, jligon, lslebodn, lvrabec, miabbott, mmalik, mzidek, nstielau, pbrezina, plautrba, sbose, ssekidde, sssd-maint, tscherf |
| Target Milestone: | beta | Keywords: | Triaged |
| Target Release: | --- | Flags: | pm-rhel:
mirror+
|
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | No Doc Update | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-04-08 07:08:05 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Leonid Titov
2021-04-01 14:01:50 UTC
Looking at BZ#1694681, the symptoms look very similar. For confirmation, is are there any audit messages in /var/log/audit.log on the node? Or reported via `ausearch -m avc`? If there is a denial similar to what is reported in https://bugzilla.redhat.com/show_bug.cgi?id=1694681#c17, it seems like this would be the same problem. ``` type=USER_AVC msg=audit(1589400931.223:5344): pid=993 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc: denied { send_msg } for msgtype=error error_name=org.freedesktop.systemd1.NoSuchDynamicUser dest=:1.4536 spid=1 tpid=2770 scontext=system_u:system_r:init_t:s0 tcontext=system_u:system_r:kernel_t:s0 tclass=dbus permissive=0 exe="/usr/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?'UID="dbus" AUID="unset" SAUID="dbus" ``` That BZ appears to be fixed in RHEL 8.4; if we want it fixed for OCP 4.6, we would need to request a backport all the way to RHEL 8.2 EUS Hello, Unfortunately I don't see logged failure in audit logs: $grep dbusd_t var/log/audit/audit.log | wc -l 0 You can try to workaround the problem by dropping a custom `/etc/nsswitch.conf` via Machine config like so:
```
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
labels:
machineconfiguration.openshift.io/role: worker
name: 99-nsswitch-remove-systemd
spec:
config:
ignition:
version: 3.2.0
storage:
files:
- path: /etc/nsswitch.conf
contents:
source: data:text/plain;charset=utf-8;base64,IyBXcml0dGVuIHZpYSBNYWNoaW5lQ29uZmlnIDk5LW5zc3N3aXRjaC1yZW1vdmUtc3lzdGVtZApwYXNzd2Q6IHNzcyBmaWxlcyBhbHRmaWxlcwpncm91cDogc3NzIGZpbGVzIGFsdGZpbGVzIHN5c3RlbWQKbmV0Z3JvdXA6ICAgc3NzIGZpbGVzCmF1dG9tb3VudDogIHNzcyBmaWxlcwpzZXJ2aWNlczogICBzc3MgZmlsZXMKc2hhZG93OiAgICAgZmlsZXMgc3NzCmhvc3RzOiAgICAgIGZpbGVzIGRucyBteWhvc3RuYW1lCmFsaWFzZXM6ICAgIGZpbGVzCmV0aGVyczogICAgIGZpbGVzCmdzaGFkb3c6ICAgIGZpbGVzCm5ldHdvcmtzOiAgIGZpbGVzIGRucwpwcm90b2NvbHM6ICBmaWxlcwpwdWJsaWNrZXk6ICBmaWxlcwpycGM6ICAgICAgICBmaWxlcwoK
user:
name: root
mode: 0644
```
I took the default `/etc/nsswitch.conf` from a node on OCP 4.7, stripped out all the comments, and removed `systemd` from the `passwd:` line.
```
$ echo "IyBXcml0dGVuIHZpYSBNYWNoaW5lQ29uZmlnIDk5LW5zc3N3aXRjaC1yZW1vdmUtc3lzdGVtZApwYXNzd2Q6IHNzcyBmaWxlcyBhbHRmaWxlcwpncm91cDogc3NzIGZpbGVzIGFsdGZpbGVzIHN5c3RlbWQKbmV0Z3JvdXA6ICAgc3NzIGZpbGVzCmF1dG9tb3VudDogIHNzcyBmaWxlcwpzZXJ2aWNlczogICBzc3MgZmlsZXMKc2hhZG93OiAgICAgZmlsZXMgc3NzCmhvc3RzOiAgICAgIGZpbGVzIGRucyBteWhvc3RuYW1lCmFsaWFzZXM6ICAgIGZpbGVzCmV0aGVyczogICAgIGZpbGVzCmdzaGFkb3c6ICAgIGZpbGVzCm5ldHdvcmtzOiAgIGZpbGVzIGRucwpwcm90b2NvbHM6ICBmaWxlcwpwdWJsaWNrZXk6ICBmaWxlcwpycGM6ICAgICAgICBmaWxlcwoK" | base64 -d
# Written via MachineConfig 99-nssswitch-remove-systemd
passwd: sss files altfiles
group: sss files altfiles systemd
netgroup: sss files
automount: sss files
services: sss files
shadow: files sss
hosts: files dns myhostname
aliases: files
ethers: files
gshadow: files
networks: files dns
protocols: files
publickey: files
rpc: files
```
And I'm going to send this over to the RHEL team for further investigation. Please be prepared to provide additional information from affected nodes and to reproduce the problem.
Hi, can you check the output of the sesearch commands from https://bugzilla.redhat.com/show_bug.cgi?id=1694681#c24 and https://bugzilla.redhat.com/show_bug.cgi?id=1694681#c33 and change to component of this ticket to selinux-policy is the sesearch output indicates that it is the same issue? bye, Sumit @sbose here they are: sh-5.0# sesearch --dontaudit -s init_t -t kernel_t -c dbus -p send_msg dontaudit init_t kernel_t:dbus send_msg; sh-5.0# sesearch -A -s init_t -t kernel_t -c dbus -p send_msg sh-5.0# sesearch -A -t init_t -s kernel_t -c dbus -p send_msg allow dbusd_unconfined init_t:dbus send_msg; allow dbusd_unconfined nsswitch_domain:dbus send_msg; (In reply to Leonid Titov from comment #8) > @sbose here they are: > > sh-5.0# sesearch --dontaudit -s init_t -t kernel_t -c dbus -p send_msg > dontaudit init_t kernel_t:dbus send_msg; > > sh-5.0# sesearch -A -s init_t -t kernel_t -c dbus -p send_msg > sh-5.0# sesearch -A -t init_t -s kernel_t -c dbus -p send_msg > allow dbusd_unconfined init_t:dbus send_msg; > allow dbusd_unconfined nsswitch_domain:dbus send_msg; Hi, thanks, this really looks like https://bugzilla.redhat.com/show_bug.cgi?id=1694681, I'll change the component to selinux-policy. Maybe the selinux maintainers even know a workaround to modify the running policy to allow those operations. bye, Sumit I can confirm the symptoms look the same, hence I am about to close this bz a duplicate. If a fix for EUS is required, please follow the acceleration fixes workflow. To work around the problem, a local selinux policy module can be installed: # cat local_init_kernel_dbus.cil (allow init_t kernel_t (dbus (send_msg))) # semodule -i local_init_kernel_dbus.cil *** This bug has been marked as a duplicate of bug 1694681 *** |