Bug 2223568
| Summary: | Regression: denied { ipc_lock sqpoll } for comm="pmproxy" | ||
|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Martin Pitt <mpitt> |
| Component: | pcp | Assignee: | Nathan Scott <nathans> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
| Severity: | medium | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 39 | CC: | agerstmayr, dwalsh, jkurik, lvrabec, mgoodwin, mmalik, nathans, nknazeko, omosnacek, pkoncity, sfeifer, vmojzis, zpytela |
| Target Milestone: | --- | Keywords: | Reopened |
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Linux | ||
| URL: | https://cockpit-logs.us-east-1.linodeobjects.com/pull-5014-20230718-032327-b98bfddb-fedora-testing-cockpit-project-cockpit/log.html | ||
| Whiteboard: | CockpitTest | ||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2024-03-18 06:20:41 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Martin, Do you know what exactly is needed to trigger this issue? I'd like to see more debugging information. FWIW, I am not aware of any related change in policy and no problem was reported in our (simple) pcp tests. Just `systemctl start pmproxy`. That immediately triggers the AVCs here. No other pm* service is running (the package does not enable them by default). pcp-6.0.5-1.fc38.x86_64 selinux-policy-38.21-1.fc38.noarch kernel-core-6.3.12-200.fc38.x86_64 The same denials appear when the policy is downgraded to selinux-policy-38.20-1
pmproxy 24332 [000] 78695.474349: avc:selinux_audited: requested=0x4000 denied=0x4000 audited=0x40>
BFD: DWARF error: could not find variable specification at offset 0x825
ffffffff866e9715 avc_audit_post_callback+0x215 ([kernel.kallsyms])
ffffffff866e9715 avc_audit_post_callback+0x215 ([kernel.kallsyms])
ffffffff86712f5f common_lsm_audit+0x2af ([kernel.kallsyms])
ffffffff866ea9ae slow_avc_audit+0xce ([kernel.kallsyms])
ffffffff866efe4e cred_has_capability.isra.0+0x12e ([kernel.kallsyms])
ffffffff866e3f01 security_capable+0x41 ([kernel.kallsyms])
ffffffff8611e1bf capable+0x2f ([kernel.kallsyms])
ffffffff867ce71b io_uring_setup+0x4ab ([kernel.kallsyms])
ffffffff86f9a2dd do_syscall_64+0x5d ([kernel.kallsyms])
ffffffff870000ae entry_SYSCALL_64_after_hwframe+0x72 ([kernel.kallsyms])
10ab5d syscall+0x1d (/usr/lib64/libc.so.6)
26813 [unknown] (/usr/lib64/libuv.so.1.0.0)
26b49 [unknown] (/usr/lib64/libuv.so.1.0.0)
22498 uv_loop_init+0x248 (/usr/lib64/libuv.so.1.0.0)
140de uv_default_loop+0x2e (/usr/lib64/libuv.so.1.0.0)
f3a1 open_request_ports+0x211 (/usr/libexec/pcp/bin/pmproxy)
9918 main+0x438 (/usr/libexec/pcp/bin/pmproxy)
27b49 __libc_start_call_main+0x79 (/usr/lib64/libc.so.6)
27c0a __libc_start_main_alias_2+0x8a (inlined)
a024 _start+0x24 (/usr/libexec/pcp/bin/pmproxy)
pmproxy 24332 [000] 78695.474535: avc:selinux_audited: requested=0x2 denied=0x2 audited=0x2 result>
ffffffff866e9715 avc_audit_post_callback+0x215 ([kernel.kallsyms])
ffffffff866e9715 avc_audit_post_callback+0x215 ([kernel.kallsyms])
ffffffff86712f5f common_lsm_audit+0x2af ([kernel.kallsyms])
ffffffff866ea9ae slow_avc_audit+0xce ([kernel.kallsyms])
ffffffff866eb200 avc_has_perm+0xd0 ([kernel.kallsyms])
ffffffff866e8ba6 security_uring_sqpoll+0x26 ([kernel.kallsyms])
ffffffff867dbbe1 io_sq_offload_create+0x71 ([kernel.kallsyms])
ffffffff867ce7ff io_uring_setup+0x58f ([kernel.kallsyms])
ffffffff86f9a2dd do_syscall_64+0x5d ([kernel.kallsyms])
ffffffff870000ae entry_SYSCALL_64_after_hwframe+0x72 ([kernel.kallsyms])
10ab5d syscall+0x1d (/usr/lib64/libc.so.6)
26813 [unknown] (/usr/lib64/libuv.so.1.0.0)
26b49 [unknown] (/usr/lib64/libuv.so.1.0.0)
22498 uv_loop_init+0x248 (/usr/lib64/libuv.so.1.0.0)
140de uv_default_loop+0x2e (/usr/lib64/libuv.so.1.0.0)
f3a1 open_request_ports+0x211 (/usr/libexec/pcp/bin/pmproxy)
9918 main+0x438 (/usr/libexec/pcp/bin/pmproxy)
27b49 __libc_start_call_main+0x79 (/usr/lib64/libc.so.6)
27c0a __libc_start_main_alias_2+0x8a (inlined)
a024 _start+0x24 (/usr/libexec/pcp/bin/pmproxy)
Kernel perhaps? Looking further.
See the image build log (link in description). The totality of updated packages was this: fwupd (1.9.2-1.fc38 -> 1.9.3-2.fc38) fwupd-plugin-flashrom (1.9.2-1.fc38 -> 1.9.3-2.fc38) fwupd-plugin-modem-manager (1.9.2-1.fc38 -> 1.9.3-2.fc38) fwupd-plugin-uefi-capsule-data (1.9.2-1.fc38 -> 1.9.3-2.fc38) libuv (1:1.44.2-3.fc38 -> 1:1.46.0-1.fc38) llvm-libs (16.0.5-1.fc38 -> 16.0.6-1.fc38) podman (5:4.6.0~rc1-1.fc38 -> 5:4.6.0~rc2-1.fc38) podman-gvproxy (5:4.6.0~rc1-1.fc38 -> 5:4.6.0~rc2-1.fc38) python3-urllib3 (1.26.15-1.fc38 -> 1.26.16-1.fc38) python3-urllib3+socks (1.26.15-1.fc38 -> 1.26.16-1.fc38) selinux-policy (38.20-1.fc38 -> 38.21-1.fc38) selinux-policy-devel (38.20-1.fc38 -> 38.21-1.fc38) selinux-policy-targeted (38.20-1.fc38 -> 38.21-1.fc38) There is no kernel update. I didn't mention fwupd, podman, and urllib3 as they didn't seem plausible; but one never knows.. Oh, but your trace *does* go via libuv, so that may be it? libuv (1:1.44.2-3.fc38 -> 1:1.46.0-1.fc38) Yes, this is the culprit: https://github.com/libuv/libuv/commit/d2c31f429b87b476a7f1344d145dad4752a406d4 I guess we will need to start allowing `<domain> self:io_uring sqpoll;` to all services that use libuv... The ipc_lock denial is likely caused by bug 2193317. I've managed to find only one service (pmproxy) linked with libuv, but many can run just a command which is not SELinux confined which is the case of sssd: https://bugzilla.redhat.com/show_bug.cgi?id=2223441 pcp folks, please add the required permission to pmproxy, refer to https://github.com/fedora-selinux/selinux-policy/pull/1784 Thanks Martin and Zdenek, I've opened a PR to start fixing things: https://github.com/performancecopilot/pcp/pull/1776 Unfortunately the PCP rawhide CI has been impacted by the python upgrade for several days now, so we missed this. The upstream PR landed in July, and we haven't seen this issue any more in the last 24 days, so this seems fixed now. Argh, no, just spotted again in the wild on latest Fedora 39: https://cockpit-logs.us-east-1.linodeobjects.com/pull-5403-20231017-230011-01ce28fe-fedora-39-other-cockpit-project-cockpit/log.html#54 audit: type=1400 audit(1697584332.406:3819): avc: denied { sqpoll } for pid=13851 comm="pmproxy" scontext=system_u:system_r:pcp_pmproxy_t:s0 tcontext=system_u:system_r:pcp_pmproxy_t:s0 tclass=io_uring permissive=0 We haven't seen this any more since December, so another hopeful attempt at closing this. |
The latest update [1] of selinux-policy regressed pmproxy. We noticed that in our cockpit tests with our regular run against updates-testing [2]. Unfortunately [1] already got promoted to stable, so the damage is done now. pmproxy now triggers these violations: avc: denied { ipc_lock } for pid=79320 comm="pmproxy" capability=14 scontext=system_u:system_r:pcp_pmproxy_t:s0 tcontext=system_u:system_r:pcp_pmproxy_t:s0 tclass=capability permissive=0 avc: denied { sqpoll } for pid=79320 comm="pmproxy" scontext=system_u:system_r:pcp_pmproxy_t:s0 tcontext=system_u:system_r:pcp_pmproxy_t:s0 tclass=io_uring permissive=0 [1] https://bodhi.fedoraproject.org/updates/FEDORA-2023-2663818afd [2] https://github.com/cockpit-project/bots/pull/5014 Reproducible: Always The image build log [1] shows that the only relevant update here was selinux-policy. PCP did not change. selinux-policy (38.20-1.fc38 -> 38.21-1.fc38) [1] https://cockpit-logs.us-east-1.linodeobjects.com/image-refresh-logs/fedora-testing-20230717-225757.log