This is about the update in https://bodhi.fedoraproject.org/updates/FEDORA-2025-e7a319968a. Cockpit's tests (https://github.com/cockpit-project/cockpit-machines/issues/1983) and other users spotted regressions, so it was -1'ed. But as it is still in updates-testing, and has broken our nightly test runs ever since, I file a formal bug so that we can mark it as "known broken". When updating *only* selinux-policy 41.28-1.fc41 to 41.29-1.fc41 on Fedora 41, things work. However, when updating it together with another SELinux module, it fails. Two weeks ago this was "pcp-selinux" from https://bodhi.fedoraproject.org/updates/FEDORA-2025-f69b50954b . That went into -updates fast, but now it's freeipa-selinux from https://bodhi.fedoraproject.org/updates/FEDORA-2025-83633f8bbb. I.e.: - `dnf update --enablerepo=updates-testing selinux-policy` → passes - `dnf update --enablerepo=updates-testing selinux-policy freeipa-selinux` → fails - `dnf update --enablerepo=updates-testing selinux-policy`, run the test (passes), `dnf update --enablerepo=updates-testing freeipa-selinux` → passes (!) When it fails, we get this rejection: avc: denied { connectto } for pid=7024 comm="nbd-connect" path="/var/lib/libvirt/qemu/domain-1-subVmTestCreate8/nbdkit-libvirt-1-storage.socket" scontext=system_u:system_r:svirt_tcg_t:s0:c531,c721 tcontext=system_u:system_r:nbdkit_t:s0:c531,c721 tclass=unix_stream_socket permissive=0 (There are 9 other libvirt related ones, but they are all permissive=1, and supposedly WIP). `semodule -l` diff before/after updating just selinux-policy is empty. After updating both packages together it is "+extra_binsbin". After updating the packages separately, it is *still* empty. This all corroborates that there is something "magic" going on with updating two packages together. Also, the libvirt policy in general still works, as it's only one test that fails (I assume that's the only one touching nbd-connect). I can reproduce that at will in VMs. I'm also happy to add some developer's SSH key to them for investigation. I can't say whether this is a regression specific to 41.29-1.fc41 or whether that extra_binsbin stuff already happened before and it's just bad circumstances. Reproducible: Always
Thanks Martin for the investigation. However, whatever I do, I cannot find any issue. I tried your steps with dnf install https://kojipkgs.fedoraproject.org//packages/freeipa/4.12.2/6.fc41/noarch/freeipa-selinux-4.12.2-6.fc41.noarch.rpm and then guess what: ... Running transaction [ 1/10] Verify package files 100% | 100.0 B/s | 4.0 B | 00m00s [ 2/10] Prepare transaction 100% | 81.0 B/s | 8.0 B | 00m00s [ 3/10] Upgrading selinux-policy-0:41.29-1.fc41 100% | 4.7 KiB/s | 33.0 KiB | 00m07s [ 4/10] Upgrading selinux-policy-targeted-0:41. 100% | 33.6 MiB/s | 14.8 MiB | 00m00s [ 5/10] Upgrading freeipa-selinux-0:4.12.2-7.fc 100% | 1.3 KiB/s | 17.3 KiB | 00m13s [ 6/10] Upgrading selinux-policy-devel-0:41.29- 100% | 5.2 MiB/s | 23.0 MiB | 00m04s [ 7/10] Removing freeipa-selinux-0:4.12.2-6.fc4 100% | 153.0 B/s | 2.0 B | 00m00s [ 8/10] Removing selinux-policy-devel-0:41.28-1 100% | 54.6 KiB/s | 1.4 KiB | 00m00s [ 9/10] Removing selinux-policy-0:41.28-1.fc41. 100% | 923.0 B/s | 12.0 B | 00m00s [10/10] Removing selinux-policy-targeted-0:41.2 100% | 49.0 B/s | 1.7 KiB | 00m35s Complete! That is: no error during update, no avc denial, no related journal entry. It was a fresh update F41 1MT system. I guess the result may also depend on the installed packages?
Zdenek: Sorry if that was unclear -- the AVC denial doesn't happen during the dnf update. That just leaves the policy in a broken state where cockpit-machine's TestMachinesCreate.testCreateUrlSource fails with the given denial. Is there something I can run on the affected VM to pry out some more information? About that "extra_binsbin" policy in particular?
I am not sure if the example above is a good one since the permission is not allowed in the policy: f41# sesearch -A -s svirt_tcg_t -t nbdkit_t -c unix_stream_socket -p connectto f41# so it's hard to tell if this particular can be is a symptom of a systematic error. I believe extra_* policies are a red herring, but you can check e.g. bunzip2 < /var/lib/selinux/targeted/active/modules/400/extra_binsbin/cil bunzip2 < /var/lib/selinux/targeted/active/modules/400/extra_varrun/cil before and after. If there are some modules disabled, then semodule -lfull will (should?) list them. I usually save (again for before/after checks) semodule -lfull | grep -v ^100 To check if just "something" failed or is in an inconsistent state, semodule -B will rebuild the policy. Also note the policy content in rawhide is currently the same so should run into the same issues. Does it?
> since the permission is not allowed in the policy: Hmm, so did the new sepol version turn some permissive=1 checks into permissive=0? Without updates, there are a few permissive=1 nbd related violations: # journalctl -ocat -b | grep 'denied.*nbd' AVC avc: denied { add_name } for pid=873 comm="rpc-virtqemud" name="nbdkitcapabilities" scontext=system_u:system_r:virtqemud_t:s0 tcontext=system_u:object_r:var_t:s0 tclass=dir permissive=1 AVC avc: denied { create } for pid=873 comm="rpc-virtqemud" name="nbdkitcapabilities" scontext=system_u:system_r:virtqemud_t:s0 tcontext=system_u:object_r:var_t:s0 tclass=dir permissive=1 AVC avc: denied { write } for pid=873 comm="rpc-virtqemud" path="/var/cache/libvirt/qemu/nbdkitcapabilities/9c2babbed3958767ab90100f70c66b13a5ad348b268824fca02cf5e50f67eee3.xml" dev="vda4" ino=122375 scontext=system_u:system_r:virtqemud_t:s0 tcontext=system_u:object_r:var_t:s0 tclass=file permissive=1 AVC avc: denied { transition } for pid=4909 comm="rpc-virtqemud" path="/usr/sbin/nbdkit" dev="vda4" ino=45856 scontext=system_u:system_r:virtqemud_t:s0 tcontext=system_u:system_r:virtqemud_t:s0:c682,c988 tclass=process permissive=1 AVC avc: denied { entrypoint } for pid=4909 comm="rpc-virtqemud" path="/usr/sbin/nbdkit" dev="vda4" ino=45856 scontext=system_u:system_r:virtqemud_t:s0:c682,c988 tcontext=system_u:object_r:bin_t:s0 tclass=file permissive=1 AVC avc: denied { name_connect } for pid=4909 comm="nbdkit" dest=443 scontext=system_u:system_r:virtqemud_t:s0:c682,c988 tcontext=system_u:object_r:http_port_t:s0 tclass=tcp_socket permissive=1 AVC avc: denied { name_connect } for pid=4909 comm="nbdkit" dest=443 scontext=system_u:system_r:virtqemud_t:s0:c682,c988 tcontext=system_u:object_r:http_port_t:s0 tclass=tcp_socket permissive=1 but none of them exactly match the new "connectto" failure here. Starting from a clean F41 image without any updates-testing packages, i.e. selinux-policy-41.28-1.fc41.noarch and freeipa-selinux-4.12.2-6.fc41.noarch. I had to run "dnf install /usr/bin/sesearch" # bunzip2 < /var/lib/selinux/targeted/active/modules/400/extra_binsbin/cil; bunzip2 < /var/lib/selinux/targeted/active/modules/400/extra_varrun/cil (optional extra_binsbin_1 (filecon "/usr/bin/tlshd" file (system_u object_r ktlshd_exec_t ((s0) (s0)))) ) (optional extra_binsbin_2 (filecon "/usr/bin/pcm-sensor-server" file (system_u object_r pcmsensor_exec_t ((s0) (s0)))) ) (optional extra_var_run_1 (filecon "/run/ipa(/.*)?" any (system_u object_r ipa_var_run_t ((s0) (s0)))) ) (optional extra_var_run_2 (filecon "/run/pcp(/.*)?" any (system_u object_r pcp_var_run_t ((s0) (s0)))) ) (optional extra_var_run_3 (filecon "/run/pasta.pid" any (system_u object_r pasta_pid_t ((s0) (s0)))) ) # semodule -lfull | grep -v ^100 400 extra_binsbin cil 400 extra_varrun cil 200 cockpit pp 200 container pp 200 ipa pp 200 nbdkit pp 200 passt pp 200 pasta pp 200 pcp pp 200 swtpm pp 200 swtpm_libvirt pp 200 swtpm_svirt pp Also, as that is *very* relevant to this bug: # ls -lZ /usr/sbin/nbdkit -rwxr-xr-x. 1 root root system_u:object_r:bin_t:s0 166512 Sep 27 00:00 /usr/sbin/nbdkit Running the test passes, and `journalctl -b | grep 'avc.*denied.*permissive=0'` only shows an ancient and completely unrelated issue #2112782 (which we can safely ignore here). Now install the two updates together: dnf update --enablerepo=updates-testing selinux-policy freeipa-selinux (note that your command in #1 doesn't work -- as I wrote, installing the packages *separately* doesn't fail). This updates to selinux-policy-0:41.29-1.fc41.noarch and freeipa-selinux-0:4.12.2-6.fc41.noarch. Now running the c-machines test fails due to the already mentioned AVC avc: denied { connectto } for pid=12283 comm="nbd-connect" path="/var/lib/libvirt/qemu/domain-1-subVmTestCreate8/nbdkit-libvirt-1-storage.socket" scontext=system_u:system_r:svirt_tcg_t:s0:c58,c568 tcontext=system_u:system_r:nbdkit_t:s0:c58,c568 tclass=unix_stream_socket permissive=0 Now the two dynamic extra policies: extra_varrun is the same, but extra_binsbin is different! # bunzip2 < /var/lib/selinux/targeted/active/modules/400/extra_binsbin/cil; bunzip2 < /var/lib/selinux/targeted/active/modules/400/extra_varrun/cil (optional extra_binsbin_1 (filecon "/usr/bin/nbdkit" file (system_u object_r nbdkit_exec_t ((s0) (s0)))) ) (optional extra_var_run_1 (filecon "/run/ipa(/.*)?" any (system_u object_r ipa_var_run_t ((s0) (s0)))) ) (optional extra_var_run_2 (filecon "/run/pcp(/.*)?" any (system_u object_r pcp_var_run_t ((s0) (s0)))) ) (optional extra_var_run_3 (filecon "/run/pasta.pid" any (system_u object_r pasta_pid_t ((s0) (s0)))) ) That new filecon rule is too specific to be a coincidence. After the update: # ls -lZ /usr/sbin/nbdkit -rwxr-xr-x. 1 root root system_u:object_r:nbdkit_exec_t:s0 166512 Sep 27 00:00 /usr/sbin/nbdkit It was bin_t before. The "semodule -lfull | grep -v ^100" list is the same. After "semodule -B" there is no change in any of these commands, and the c-machines test still fails. "restorecon -v /usr/sbin/nbdkit" doesn't change permissions. After "chcon -t bin_t /usr/sbin/nbdkit" the test works again. So the problem is somehow that the dynamic extra_binsbin policy changes the file context of nbdkit. The change feels correct, but it puts nbdkit under a different policy which has to allow that connectto action. But I suppose the bigger mysteries are (1) why is the file context wrong on current F41?, and (2) why is this handled in such a brittle way with dynamic extra policies, which are so very sensitive to how you install package updates? (as group or individual) Thanks!
While it looked tangled at the beginning, now I think the explanation is quite easy. > (note that your command in #1 doesn't work -- as I wrote, installing the packages *separately* doesn't fail) This was needed so that freeipa-selinux can be updated in the next step. Anyway the real problem is different: nbdkit is now confined when started by virtqemud, but some additional permissions seem to be needed. We do not have test coverage for that, it was not reported by revdeps tests in github, even the attempt to run cockpit-machines test did not help to catch there is some issue. Can you try the copr build from here? https://github.com/fedora-selinux/selinux-policy/pull/2549 Checks -> rpm build rawhide with f42# ls -lZ /usr/sbin/nbdkit -rwxr-xr-x. 1 root root system_u:object_r:nbdkit_exec_t:s0 162536 Jan 16 19:00 /usr/sbin/nbdkit The extra_* modules were meant as a temporary workaround till all other packages are changed. Given the latest development in rawhide it will take time until things get right.
FEDORA-2025-62c612355c (selinux-policy-41.31-1.fc41) has been submitted as an update to Fedora 41. https://bodhi.fedoraproject.org/updates/FEDORA-2025-62c612355c
FEDORA-2025-62c612355c has been pushed to the Fedora 41 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2025-62c612355c` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2025-62c612355c See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2025-62c612355c (selinux-policy-41.31-1.fc41) has been pushed to the Fedora 41 stable repository. If problem still persists, please make note of it in this bug report.
Thanks! Last night's updates-testing run [1] had selinux-policy 41.31-1.fc41 and also updated freeipa-selinux. The previously failing testCreateUrlSource now passed. Thanks! [1] https://cockpit-logs.us-east-1.linodeobjects.com/pull-0-a3a536d5-20250203-021147-fedora-41-updates-testing/log.html