Attempt to upgrade to the latest util-linux upstream release v2.41 failed. For more details, see Adam's report at https://bodhi.fedoraproject.org/updates/FEDORA-2025-dd6b6950ff I could be wrong, as I have minimal experience with LiveCD, but the relevant change in the upgrade is new native support for EROFS images. The recent kernels can mount EROFS directly without loop devices, and this feature is enabled in Fedora kernels (>=6.12; CONFIG_EROFS_FS_BACKED_BY_FILE=y). This feature is now supported by mount/libmount too. The difference is how mount(8) reacts to # mount foo.img /mnt The old version creates a loop device and mounts it. The new mount(8) version directly mounts the image. Now, from the running system: https://adamwill.fedorapeople.org/05017930-Fedora-Workstation-Live-x86_64-FEDORA-2025-dd6b6950ff.iso You can see LiveOS/squashfs.img is directly mounted, but there is also /dev/loop0 associated with the image. This is probably an unwanted situation (and potentially risky). $ findmnt --submounts /run TARGET SOURCE FSTYPE OPTI /run tmpfs tmpfs rw,n ├─/run/initramfs/live /dev/sr0 iso9660 ro,r ├─/run/rootfsbase /run/initramfs/live/LiveOS/squashfs.img erofs ro,r ├─/run/credentials/systemd-journald.service tmpfs tmpfs ro,n ├─/run/credentials/systemd-resolved.service tmpfs tmpfs ro,n └─/run/user/1000 tmpfs tmpfs rw,n ├─/run/user/1000/doc portal fuse.portal rw,n └─/run/user/1000/gvfs gvfsd-fuse fuse.gvfsd-fuse rw,n $ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS loop0 7:0 0 2.1G 1 loop sr0 11:0 1 2.3G 0 rom /run/initramfs/live zram0 251:0 0 3.8G 0 disk [SWAP] vda 253:0 0 10G 0 disk $ losetup NAME SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE DIO LOG-SEC /dev/loop0 0 0 0 1 /LiveOS/squashfs.img 0 512 I'm not sure if this is entirely related to this problem, but LiveCD is almost useless due to SELinux May 22 09:42:37 localhost-live audit[1348]: AVC avc: denied { read } for pid=1348 comm="firewalld" path="/LiveOS/squashfs.img" dev="sr0" ino=4294 scontext=system_u:system_r:firewalld_t:s0 tcontext=system_u:object_r:iso9660_t:s0 tclass=file permissive=1 May 22 09:42:48 localhost-live audit[2484]: AVC avc: denied { read } for pid=2484 comm="sssd_kcm" path="/LiveOS/squashfs.img" dev="sr0" ino=4294 scontext=system_u:system_r:sssd_t:s0 tcontext=system_u:object_r:iso9660_t:s0 tclass=file permissive=1 May 22 09:42:49 localhost-live audit[2515]: AVC avc: denied { read } for pid=2515 comm="systemd-localed" path="/LiveOS/squashfs.img" dev="sr0" ino=4294 scontext=system_u:system_r:systemd_localed_t:s0 tcontext=system_u:object_r:iso9660_t:s0 tclass=file permissive=1 May 22 09:42:50 localhost-live audit[2562]: AVC avc: denied { read } for pid=2562 comm="pkla-check-auth" path="/LiveOS/squashfs.img" dev="sr0" ino=4294 scontext=system_u:system_r:policykit_auth_t:s0 tcontext=system_u:object_r:iso9660_t:s0 tclass=file permissive=1 May 22 09:42:50 localhost-live audit[2555]: AVC avc: denied { read } for pid=2555 comm="geoclue" path="/LiveOS/squashfs.img" dev="sr0" ino=4294 scontext=system_u:system_r:geoclue_t:s0 tcontext=system_u:object_r:iso9660_t:s0 tclass=file permissive=1 May 22 09:42:51 localhost-live audit[2555]: AVC avc: denied { read } for pid=2555 comm="geoclue" path="/LiveOS/squashfs.img" dev="sr0" ino=4294 scontext=system_u:system_r:geoclue_t:s0 tcontext=system_u:object_r:iso9660_t:s0 tclass=file permissive=1 May 22 09:42:54 localhost-live audit[2765]: AVC avc: denied { read } for pid=2765 comm="pkla-check-auth" path="/LiveOS/squashfs.img" dev="sr0" ino=4294 scontext=system_u:system_r:policykit_auth_t:s0 tcontext=system_u:object_r:iso9660_t:s0 tclass=file permissive=1 May 22 09:43:01 localhost-live audit[2607]: AVC avc: denied { read } for pid=2607 comm="wpa_supplicant" path="/LiveOS/squashfs.img" dev="sr0" ino=4294 scontext=system_u:system_r:NetworkManager_t:s0 tcontext=system_u:object_r:iso9660_t:s0 tclass=file permissive=1 May 22 09:43:18 localhost-live audit[2270]: AVC avc: denied { read } for pid=2270 comm="cupsd" path="/LiveOS/squashfs.img" dev="sr0" ino=4294 scontext=system_u:system_r:cupsd_t:s0-s0:c0.c1023 tcontext=system_u:object_r:iso9660_t:s0 tclass=file permissive=1 I'll report this also to the SELinux guys. Reproducible: Always
SElinux BZ: https://bugzilla.redhat.com/show_bug.cgi?id=2367958
LiveCD is pretty much never the right component for anything any more, we should get rid of it :P I *think* it's dracut that does this mounting.
This is dracut yes, unfortunately I am not a dracut maintainer in Fedora and have no power to do anything about dracut in Fedora due to its usage of a restrictive source-git implementation.
I am a dracut maintainer, but I'm however not entirely familiar with how erofs / squashfs / live works -- but when you get to initramfs, it [initramfs part] should be already mounted (or unpacked in case of CPIO); so dracut takes care only of mounting & switching to the rootfs -- for liveCD, it probably runs https://github.com/redhat-plumbers/dracut-fedora/blob/4684cfdb7a18ba359f44f217bb2211c691743919/modules.d/90dmsquash-live/module-setup.sh#L27 -- it's hard to say without a log with `rd.debug` on kernel cmdline what happened. Also noteworhy might be that selinux does not run in initramfs. Do I understand it correctly that pivot_root (or any alternative for livecd) succeeded, but the mount is somehow wrong? TBH, I have never debugged a LiveCD, but I can forward this upstream & ping someone from erofs to take a look. > This is dracut yes, unfortunately I am not a dracut maintainer in Fedora and have no power to do anything about dracut in Fedora due to its usage of a restrictive source-git implementation. To reiterate the situation with source-git: we're not restricting anything; really ... if you find creating a PR on github restrictive, feel free to use the dist-git in the same way as with any other package... everything gets synced both ways. The benefit of source-git (and therefore the preference) is just the CI we have there runs upstream test suite.
The default way that EROFS is mounted has changed such that it no longer requires a loop device. This breaks dracut, and down the road, this also breaks Anaconda's ability to install a live system, since we rely on the specific mount setup to sync over only the pristine OS, not the the live OS.
Note that you can still create a loop device associated with the erofs image by losetup and call `mount /dev/loop0 /mnt`. In this case, mount(8) will not do anything unusual and will not directly mount the image or create the loop device. This scenario is backwardly compatible. I can implement a workaround in libmount to disable this EROFS feature (or it can be disabled in the kernel), but I think it would be better to adapt to the change :-)
Also, note that the main issue is a read-deny from SELinux. On systems with SELinux disabled, it boots and works as expected. The issue (mess) with the loop device doesn't seem so critical from my point of view.
The important bit is that "/run/rootfsbase" exists for Anaconda and Calamares to be able to grab the pristine rootfs to install.
I don't see anything wrong with the boot.iso, the dracut mount code for erofs/squashfs live is doing a losetup first and then mounting the loop device. This is likely an issue with how kiwi is building the live or configuring it.
(In reply to Brian Lane from comment #9) > I don't see anything wrong with the boot.iso, the dracut mount code for > erofs/squashfs live is doing a losetup first and then mounting the loop > device. This is likely an issue with how kiwi is building the live or > configuring it. We do literally the exact same thing since we use dracut's dmsquash code and the same flags. Please actually *look* before insinuating that this is kiwi's fault.
Can we run the liveCD in Permissive mode in the meantime as a workaround?
Personally I'd much rather we come up with a proper fix for this.
Looks like it might be this? https://lore.kernel.org/all/20250519175640.2fcac001@leda.eworm.net/ This came up in the upstream dracut discussion: https://github.com/dracut-ng/dracut-ng/issues/1342
(In reply to Neal Gompa from comment #13) > Looks like it might be this? > https://lore.kernel.org/all/20250519175640.2fcac001@leda.eworm.net/ > > This came up in the upstream dracut discussion: > https://github.com/dracut-ng/dracut-ng/issues/1342 There is no problem creating /dev/loop0 (see my first comment, the device exists). The minor problem is that the loop is unnecessary (the kernel can mount the image directly without the loop device), and a fatal problem is that SELInux reports that /LiveOS/squashfs.img is directly read by some system tools. The file is marked as iso9660_t, which makes SELinux unhappy (see #2357958). May 22 09:43:18 localhost-live audit[2270]: AVC avc: denied { read } for pid=2270 comm="cupsd" path="/LiveOS/squashfs.img" dev="sr0" ino=4294 scontext=system_u:system_r:cupsd_t:s0-s0:c0.c1023 tcontext=system_u:object_r:iso9660_t:s0 tclass=file permissive=1 This is something I do not understand, why "cupsd" wants to read /LiveOS/squashfs.img? The file is marked as iso9660_t and located on device /dev/sr0. It seems like a mess.
Well it's not just cupsd, it's tons of things. What I find kinda interesting is that it even works enough for a desktop to start up. So it's not like *everything* looks to SELinux like it's trying to read the squashfs.img, but a *lot* of things do. I'm not quite sure what the criteria are. Is it 'everything using dbus'? 'everything using udev'?
Created attachment 2093109 [details] complete boot log with enabled rd.debug
I have attached the complete boot log with rd.debug enabled. Here are the relevant lines (grep -E '(losetup|mount -[a-z])' log): losetup -r /dev/loop0 /run/initramfs/live/LiveOS/squashfs.img mount -n -o ro /dev/loop0 /run/initramfs/squashfs mount -r /run/initramfs/live/LiveOS/squashfs.img /run/rootfsbase umount -l /run/initramfs/squashfs You can see that modules.d/70dmsquash-live/dmsquash-live-root.sh tries to mount the same image twice, first by losetup + mount; second as "mount squashfs.img". This worked as expected in the old version because libmount is smart and detects that the same backing file (squashfs.img) is already associated with any loop device and then it reuses the device rather than creating a new device. After updating to util-linux v2.41, it works too, but the second mount does not create (or reuse) a loop device, but mounts squashfs.img directly. I think the best approach would be to avoid the second mount altogether, independently of the util-linux version. It is more robust to use a bind mount there than to try to create a second instance of the same filesystem. Something like: diff --git a/modules.d/70dmsquash-live/dmsquash-live-root.sh b/modules.d/70dmsquash-live/dmsq index 2ea393e1..68c26a3e 100755 --- a/modules.d/70dmsquash-live/dmsquash-live-root.sh +++ b/modules.d/70dmsquash-live/dmsquash-live-root.sh @@ -422,7 +422,11 @@ fi if [ -n "$overlayfs" ]; then if [ -n "$FSIMG" ]; then mkdir -m 0755 -p /run/rootfsbase - mount -r "$FSIMG" /run/rootfsbase + if [ "$FSIMG" = "$SQUASHED" ]; then + mount --bind /run/initramfs/squashfs /run/rootfsbase + else + mount -r "$FSIMG" /run/rootfsbase + fi else ln -sf /run/initramfs/live /run/rootfsbase fi This change should be portable and allow avoiding direct erofs image mounts. It will also avoid dependence on libmount duplicate loopdev backing file detection. Maybe it will also help with SELinux, as the mount table will again contain a loop device. I have tried searching for the reason why SELinux complains, but I'm not sure. It seems that some services sandboxed by systemd are affected. Maybe systemd uses the mount table to gather devices necessary for the sandbox, and it's confused that the EROFS image is there as the mount source (device), but it's just my guess.
Reported to upstream: https://github.com/dracut-ng/dracut-ng/issues/1384