2280935 – SELinux is preventing /usr/lib/systemd/system-generators/nfs-server-generator from map_read, map_write access on the bpf /lib64/ld-linux-x86-64.so.2.

Bug 2280935 - SELinux is preventing /usr/lib/systemd/system-generators/nfs-server-generator from map_read, map_write access on the bpf /lib64/ld-linux-x86-64.so.2.

Summary: SELinux is preventing /usr/lib/systemd/system-generators/nfs-server-generator...

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Fedora
Classification:	Fedora
Component:	systemd
Sub Component:
Version:	rawhide
Hardware:	x86_64
OS:	Unspecified
Priority:	unspecified
Severity:	high
Target Milestone:	---
Assignee:	systemd-maint
QA Contact:	Fedora Extras Quality Assurance
Docs Contact:
URL:
Whiteboard:	abrt_hash:16c4fada81e20fa40061261e586...
Duplicates (5):	2280934 2280985 2281797 2282396 2283205 (view as bug list)
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2024-05-17 06:54 UTC by Mikhail
Modified:	2024-05-29 18:20 UTC (History)
CC List:	20 users (show)
Fixed In Version:	systemd-256~rc3-3.fc41
Clone Of:
Environment:
Last Closed:	2024-05-29 18:20:22 UTC
Type:	---
Embargoed:
Dependent Products:

Attachments	(Terms of Use)
File: description (3.15 KB, text/plain) 2024-05-17 06:55 UTC, Mikhail	no flags	Details
File: os_info (770 bytes, text/plain) 2024-05-17 06:55 UTC, Mikhail	no flags	Details
View All

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Bugzilla	2250930	0	unspecified	CLOSED	SELinux is preventing systemd-userdbd from map_read, map_write access on the bpf labeled init_t.	2024-05-29 17:23:39 UTC

Description Mikhail 2024-05-17 06:54:59 UTC

Description of problem:
 Dnf running trigger-install scriptlet
SELinux is preventing /usr/lib/systemd/system-generators/nfs-server-generator from map_read, map_write access on the bpf /lib64/ld-linux-x86-64.so.2.

*****  Plugin catchall (100. confidence) suggests   **************************

If you believe that nfs-server-generator should be allowed map_read map_write access on the ld-linux-x86-64.so.2 bpf by default.
Then you should report this as a bug.
You can generate a local policy module to allow this access.
Do
allow this access for now by executing:
# ausearch -c 'nfs-server-gene' --raw | audit2allow -M my-nfsservergene
# semodule -X 300 -i my-nfsservergene.pp

Additional Information:
Source Context                system_u:system_r:nfsd_t:s0
Target Context                system_u:system_r:init_t:s0
Target Objects                /lib64/ld-linux-x86-64.so.2 [ bpf ]
Source                        nfs-server-gene
Source Path                   /usr/lib/systemd/system-generators/nfs-server-
                              generator
Port                          <Unknown>
Host                          (removed)
Source RPM Packages           nfs-utils-2.6.4-0.rc6.fc41.x86_64
Target RPM Packages           glibc-2.39.9000-18.fc41.x86_64
SELinux Policy RPM            selinux-policy-targeted-40.18-2.fc41.noarch
Local Policy RPM              selinux-policy-targeted-40.18-2.fc41.noarch
Selinux Enabled               True
Policy Type                   targeted
Enforcing Mode                Permissive
Host Name                     (removed)
Platform                      Linux (removed) 6.10.0-
                              0.rc0.20240516git3c999d1ae3c7.5.fc41.x86_64+debug
                              #1 SMP PREEMPT_DYNAMIC Thu May 16 13:41:37 UTC
                              2024 x86_64
Alert Count                   5
First Seen                    2024-05-17 11:52:52 +05
Last Seen                     2024-05-17 11:53:04 +05
Local ID                      43998e33-54b6-4d12-aa9e-df051e00b813

Raw Audit Messages
type=AVC msg=audit(1715928784.806:1134): avc:  denied  { map_read map_write } for  pid=122445 comm="nfs-server-gene" scontext=system_u:system_r:nfsd_t:s0 tcontext=system_u:system_r:init_t:s0 tclass=bpf permissive=1


type=SYSCALL msg=audit(1715928784.806:1134): arch=x86_64 syscall=execve success=yes exit=0 a0=558ee76494c0 a1=7ffc793bedb0 a2=558ee7312080 a3=558ee75712b0 items=2 ppid=122442 pid=122445 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm=nfs-server-gene exe=/usr/lib/systemd/system-generators/nfs-server-generator subj=system_u:system_r:nfsd_t:s0 key=(null)

type=CWD msg=audit(1715928784.806:1134): cwd=/

type=PATH msg=audit(1715928784.806:1134): item=0 name=/usr/lib/systemd/system-generators/nfs-server-generator inode=35959 dev=00:28 mode=0100755 ouid=0 ogid=0 rdev=00:00 obj=system_u:object_r:nfsd_exec_t:s0 nametype=NORMAL cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0

type=PATH msg=audit(1715928784.806:1134): item=1 name=/lib64/ld-linux-x86-64.so.2 inode=37659 dev=00:28 mode=0100755 ouid=0 ogid=0 rdev=00:00 obj=system_u:object_r:ld_so_t:s0 nametype=NORMAL cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0

Hash: nfs-server-gene,nfsd_t,init_t,bpf,map_read,map_write

Version-Release number of selected component:
selinux-policy-targeted-40.18-2.fc41.noarch

Additional info:
reporter:       libreport-2.17.15
reason:         SELinux is preventing /usr/lib/systemd/system-generators/nfs-server-generator from map_read, map_write access on the bpf /lib64/ld-linux-x86-64.so.2.
package:        selinux-policy-targeted-40.18-2.fc41.noarch
component:      selinux-policy
hashmarkername: setroubleshoot
type:           libreport
kernel:         6.10.0-0.rc0.20240516git3c999d1ae3c7.5.fc41.x86_64+debug
comment:         Dnf running trigger-install scriptlet
component:      selinux-policy

Comment 1 Mikhail 2024-05-17 06:55:01 UTC

Created attachment 2033649 [details]
File: description

Comment 2 Mikhail 2024-05-17 06:55:04 UTC

Created attachment 2033650 [details]
File: os_info

Comment 3 Zdenek Pytela 2024-05-17 17:56:44 UTC

*** Bug 2280934 has been marked as a duplicate of this bug. ***

Comment 4 Zdenek Pytela 2024-05-17 17:56:59 UTC

*** Bug 2280985 has been marked as a duplicate of this bug. ***

Comment 5 Zdenek Pytela 2024-05-20 14:55:14 UTC

*** Bug 2281797 has been marked as a duplicate of this bug. ***

Comment 6 Zdenek Pytela 2024-05-22 13:59:18 UTC

*** Bug 2282396 has been marked as a duplicate of this bug. ***

Comment 7 Zdenek Pytela 2024-05-22 14:18:21 UTC

Juraj,

Every systemd system generator, including those not provided by the systemd package,
hits a similar denial since v255, but is seems to have started not a long time
ago. No systemd service raises such denials.

Some more facts and speculations:
- systemd v255 started to use the new systemd-executor binary as a transient
  intermediate executable to start system services which, among others, closes
  unnecessary open file descriptors
- generators are not started through executor
- systemd runs a lot of bpf code when it detects libbpf
- there *seem* to be internal systemd function which *may* try to close open
  bpf fds (but why is it then on exec and in the service domain then?)

Anyway this problem does not look like it was in selinux-policy, but rather between
systemd and kernel (e.g. why map_read/write?). Can you please take a look?

This is the perf trace:
nfs-server-gene     993 [000]   773.453572: avc:selinux_audited: requested=0x6 denied=0x6>
        ffffffff937678e6 avc_audit_post_callback+0x216 ([kernel.kallsyms])
        ffffffff937678e6 avc_audit_post_callback+0x216 ([kernel.kallsyms])
        ffffffff9379309b common_lsm_audit+0x2ab ([kernel.kallsyms])
        ffffffff93768db3 slow_avc_audit+0xb3 ([kernel.kallsyms])
        ffffffff9376966f avc_has_perm+0xbf ([kernel.kallsyms])
        ffffffff9376ae9c file_has_perm+0xac ([kernel.kallsyms])
        ffffffff9376f421 match_file+0x31 ([kernel.kallsyms])
        ffffffff934ebcc1 iterate_fd+0x61 ([kernel.kallsyms])
        ffffffff9376cf8b selinux_bprm_committing_creds+0xfb ([kernel.kallsyms])
        ffffffff93761a93 security_bprm_committing_creds+0x23 ([kernel.kallsyms])
        ffffffff934cc6c2 begin_new_exec+0x672 ([kernel.kallsyms])
        ffffffff93555a7c load_elf_binary+0x2fc ([kernel.kallsyms])
        ffffffff934ca231 bprm_execve+0x241 ([kernel.kallsyms])
        ffffffff934cbc3d do_execveat_common.isra.0+0x1bd ([kernel.kallsyms])
        ffffffff934ccb76 __x64_sys_execve+0x36 ([kernel.kallsyms])
        ffffffff9416e4b2 do_syscall_64+0x82 ([kernel.kallsyms])
        ffffffff9420012f entry_SYSCALL_64_after_hwframe+0x76 ([kernel.kallsyms])

https://github.com/systemd/systemd/releases/tag/v255-rc2
https://github.com/systemd/systemd/releases/tag/v256-rc2

Comment 8 Juraj Marcin 2024-05-27 12:37:58 UTC

(In reply to Zdenek Pytela from comment #7)
> Juraj,
> 
> Every systemd system generator, including those not provided by the systemd
> package,
> hits a similar denial since v255, but is seems to have started not a long
> time
> ago. No systemd service raises such denials.
> 
> Some more facts and speculations:
> - systemd v255 started to use the new systemd-executor binary as a transient
>   intermediate executable to start system services which, among others,
> closes
>   unnecessary open file descriptors
> - generators are not started through executor
> - systemd runs a lot of bpf code when it detects libbpf
> - there *seem* to be internal systemd function which *may* try to close open
>   bpf fds (but why is it then on exec and in the service domain then?)
> 
> Anyway this problem does not look like it was in selinux-policy, but rather
> between
> systemd and kernel (e.g. why map_read/write?). Can you please take a look?
> 
> This is the perf trace:
> nfs-server-gene     993 [000]   773.453572: avc:selinux_audited:
> requested=0x6 denied=0x6>
>         ffffffff937678e6 avc_audit_post_callback+0x216 ([kernel.kallsyms])
>         ffffffff937678e6 avc_audit_post_callback+0x216 ([kernel.kallsyms])
>         ffffffff9379309b common_lsm_audit+0x2ab ([kernel.kallsyms])
>         ffffffff93768db3 slow_avc_audit+0xb3 ([kernel.kallsyms])
>         ffffffff9376966f avc_has_perm+0xbf ([kernel.kallsyms])
>         ffffffff9376ae9c file_has_perm+0xac ([kernel.kallsyms])
>         ffffffff9376f421 match_file+0x31 ([kernel.kallsyms])
>         ffffffff934ebcc1 iterate_fd+0x61 ([kernel.kallsyms])
>         ffffffff9376cf8b selinux_bprm_committing_creds+0xfb
> ([kernel.kallsyms])
>         ffffffff93761a93 security_bprm_committing_creds+0x23
> ([kernel.kallsyms])
>         ffffffff934cc6c2 begin_new_exec+0x672 ([kernel.kallsyms])
>         ffffffff93555a7c load_elf_binary+0x2fc ([kernel.kallsyms])
>         ffffffff934ca231 bprm_execve+0x241 ([kernel.kallsyms])
>         ffffffff934cbc3d do_execveat_common.isra.0+0x1bd ([kernel.kallsyms])
>         ffffffff934ccb76 __x64_sys_execve+0x36 ([kernel.kallsyms])
>         ffffffff9416e4b2 do_syscall_64+0x82 ([kernel.kallsyms])
>         ffffffff9420012f entry_SYSCALL_64_after_hwframe+0x76
> ([kernel.kallsyms])
> 
> https://github.com/systemd/systemd/releases/tag/v255-rc2
> https://github.com/systemd/systemd/releases/tag/v256-rc2

I have managed to reproduce the issue with systemd 256~rc3-1. The problem is caused by the generator inhering a file descriptor to a BPF map. This looks like an error in systemd 256~rc3-1, as 254.5-2 and 255.6-1 correctly close this file descriptor (and all other BPF file descriptors) before executing the generator binary.

This file descriptor is obtained using bpf() call with the command BPF_MAP_CREATE. In systemd this is called only from src/core/bpf-firewall.c and these calls have not changed since v254 in upstream, so the issue is most likely in that internal systemd function that should close all BPF file descriptors.


File descriptors open in nfs-server-generator:

systemd 254.5-2 (Fedora 39):

COMMAND    PID USER   FD   TYPE DEVICE SIZE/OFF  NODE NAME
nfs-serve 1787 root    0u   CHR    1,3      0t0     4 /dev/null
nfs-serve 1787 root    1u   CHR    1,3      0t0     4 /dev/null
nfs-serve 1787 root    2u   CHR    1,3      0t0     4 /dev/null
nfs-serve 1787 root  255r   REG   0,32      156 41075 /usr/lib/systemd/system-generators/nfs-server-generator

systemd 255.6-1 (Fedora 40):

COMMAND    PID USER   FD   TYPE DEVICE SIZE/OFF  NODE NAME
nfs-serve 1743 root    0u   CHR    1,3      0t0     4 /dev/null
nfs-serve 1743 root    1u   CHR    1,3      0t0     4 /dev/null
nfs-serve 1743 root    2u   CHR    1,3      0t0     4 /dev/null
nfs-serve 1743 root  255r   REG   0,32      156 33528 /usr/lib/systemd/system-generators/nfs-server-generator

systemd 256~rc3-1 (Fedora Rawhide):

COMMAND    PID USER   FD      TYPE DEVICE SIZE/OFF  NODE NAME
nfs-serve 3786 root    0u      CHR    1,3      0t0     4 /dev/null
nfs-serve 3786 root    1u      CHR    1,3      0t0     4 /dev/null
nfs-serve 3786 root    2u      CHR    1,3      0t0     4 /dev/null
nfs-serve 3786 root    9u  a_inode   0,16        0  2083 bpf-map       ## <-- this should not be open
nfs-serve 3786 root  255r      REG   0,39      156 32470 /usr/lib/systemd/system-generators/nfs-server-generator

Comment 9 Zdenek Pytela 2024-05-27 12:54:14 UTC

Thank you, switching component.

Comment 10 Zdenek Pytela 2024-05-29 07:51:51 UTC

*** Bug 2283205 has been marked as a duplicate of this bug. ***

Comment 11 Zbigniew Jędrzejewski-Szmek 2024-05-29 09:49:29 UTC

I tried to reproduce this on my laptop (by having a binary that just sleeps inserted as a generator and calling systemctl daemon-reexec and checking what fds it has open). Unfortunately this isn't enough to reproduce. I'll need to try some more.

Comment 12 Zbigniew Jędrzejewski-Szmek 2024-05-29 12:59:51 UTC

Zdenek, thank you for the excellent debug work.

The bug is really in libbpf, see https://github.com/libbpf/libbpf/issues/812.
We have a work-around in systemd: https://github.com/systemd/systemd/pull/33072
and systemd-256~rc3-3.fc41 will have the patch.

To check the issue:
$ sudo grep -r '^flags:.02$' /proc/1/fdinfo/
/proc/1/fdinfo/9:flags:	02

Instead, 02000002 is expected (O_CLOEXEC set).

Comment 13 Fedora Update System 2024-05-29 15:24:52 UTC

FEDORA-2024-2d439ff763 (systemd-256~rc3-3.fc41) has been submitted as an update to Fedora 41.
https://bodhi.fedoraproject.org/updates/FEDORA-2024-2d439ff763

Comment 14 Zdenek Pytela 2024-05-29 17:23:40 UTC

Linking an older bz just in case.

Comment 15 Fedora Update System 2024-05-29 18:20:22 UTC

FEDORA-2024-2d439ff763 (systemd-256~rc3-3.fc41) has been pushed to the Fedora 41 stable repository.
If problem still persists, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.

dwalsh
fedoraproject
jmarcin
ksrot
lnykryn
lvrabec
mikhail.v.gavrilov
mmalik
msekleta
omosnacek
pkoncity
ryncsn
systemd-maint
szidek
thozza
vmojzis
yuwatana
zbyszek
zdohnal
zpytela