This can't possibly be failing as intended: (see next comment for more legible shell cut-n-paste output of df + dnf failure) Reproducible: Always Steps to Reproduce: 1. dnf upgrade Actual Results: dnf fails due to lack of disk space on /boot - even though it has 853M free and the previous kernel only uses 54M. Expected Results: dnf upgrade completes. I guess it's confusing / vs /boot disk requirements?
Once again, hopefully more readable: [root@vault ~]# df -hl; dnf upgrade Filesystem Size Used Avail Use% Mounted on /dev/mapper/vault-root 69G 18G 48G 27% / devtmpfs 4.0M 0 4.0M 0% /dev tmpfs 32G 0 32G 0% /dev/shm efivarfs 128K 38K 86K 31% /sys/firmware/efi/efivars tmpfs 13G 968K 13G 1% /run tmpfs 32G 0 32G 0% /tmp /dev/sda2 974M 54M 853M 6% /boot /dev/sda1 200M 18M 183M 9% /boot/efi tmpfs 6.3G 4.0K 6.3G 1% /run/user/1000 Last metadata expiration check: 0:58:35 ago on Mon 13 Nov 2023 06:28:44 PM PST. Dependencies resolved. ===================================================================================== Package Architecture Version Repository Size ===================================================================================== Installing: kernel-core x86_64 6.5.11-300.fc39 updates 16 M kernel-modules x86_64 6.5.11-300.fc39 updates 57 M kernel-modules-core x86_64 6.5.11-300.fc39 updates 31 M kernel-modules-extra x86_64 6.5.11-300.fc39 updates 2.6 M Transaction Summary ===================================================================================== Install 4 Packages Total size: 107 M Installed size: 154 M Is this ok [y/N]: y Downloading Packages: [SKIPPED] kernel-core-6.5.11-300.fc39.x86_64.rpm: Already downloaded [SKIPPED] kernel-modules-6.5.11-300.fc39.x86_64.rpm: Already downloaded [SKIPPED] kernel-modules-core-6.5.11-300.fc39.x86_64.rpm: Already downloaded [SKIPPED] kernel-modules-extra-6.5.11-300.fc39.x86_64.rpm: Already downloaded Running transaction check Transaction check succeeded. Running transaction test The downloaded packages were saved in cache until the next successful transaction. You can remove cached packages by executing 'dnf clean packages'. Error: Transaction test error: installing package kernel-core-6.5.11-300.fc39.x86_64 needs 45MB more space on the /boot filesystem Error Summary ------------- Disk Requirements: At least 45MB more space needed on the /boot filesystem.
Actually this appears to be the error message when /boot is R/O. And appears to be coming from rpm - so perhaps it should be filed there, as returning a better error message?
I'm not sure why /boot is defaulting to R/O mount on boot... some fsck issue? [root@vault boot]# egrep -v '^#|^ *$' /etc/fstab UUID=37bc6d91-9a66-494c-bfb9-bc0241c17f6f / ext4 defaults,x-systemd.device-timeout=0 1 1 UUID=aaa845d9-7648-43b8-80ba-1a1f8a936673 /boot ext4 defaults 1 2 UUID=A087-E89B /boot/efi vfat umask=0077,shortname=winnt 0 2 [root@vault boot]# dmesg | egrep mount [ 6.812189] EXT4-fs (sda2): mounted filesystem aaa845d9-7648-43b8-80ba-1a1f8a936673 ro with ordered data mode. Quota mode: none. [ 11.326157] EXT4-fs (sda2): unmounting filesystem aaa845d9-7648-43b8-80ba-1a1f8a936673. [ 11.572519] EXT4-fs (dm-1): mounted filesystem 37bc6d91-9a66-494c-bfb9-bc0241c17f6f ro with ordered data mode. Quota mode: none. [ 13.585283] systemd[1]: Set up automount proc-sys-fs-binfmt_misc.automount - Arbitrary Executable File Formats File System Automount Point. [ 13.796323] systemd[1]: Mounting dev-hugepages.mount - Huge Pages File System... [ 13.807112] systemd[1]: Mounting dev-mqueue.mount - POSIX Message Queue File System... [ 13.817850] systemd[1]: Mounting sys-kernel-debug.mount - Kernel Debug File System... [ 13.828486] systemd[1]: Mounting sys-kernel-tracing.mount - Kernel Trace File System... [ 13.837786] systemd[1]: tmp.mount: Directory /tmp to mount over is not empty, mounting anyway. [ 13.848767] systemd[1]: Mounting tmp.mount - Temporary Directory /tmp... [ 14.026295] systemd[1]: Starting systemd-remount-fs.service - Remount Root and Kernel File Systems... [ 14.045439] EXT4-fs (dm-1): re-mounted 37bc6d91-9a66-494c-bfb9-bc0241c17f6f r/w. Quota mode: none. [ 14.809637] EXT4-fs (sda2): mounted filesystem aaa845d9-7648-43b8-80ba-1a1f8a936673 ro with ordered data mode. Quota mode: none. [ 345.791543] EXT4-fs (sda2): re-mounted aaa845d9-7648-43b8-80ba-1a1f8a936673 r/w. Quota mode: none.
Note that that last log line (with the sda2 r/w mount) was done manually by me. Also I unmounted /boot/efi & /boot, fsck'ed them, remounted them (they came up r/w), rebooted - and /boot is again r/o... looking in journalctl I do see boot time attempts to fsck /dev/sda2 failing with 8 (operational error) due to it being in use. That appears to be the root cause, tweaking the final field in /etc/fstab from 2 to 0 makes it come up r/w. # egrep -v '^#|^ *$' /etc/fstab UUID=37bc6d91-9a66-494c-bfb9-bc0241c17f6f / ext4 defaults,x-systemd.device-timeout=0 1 1 UUID=aaa845d9-7648-43b8-80ba-1a1f8a936673 /boot ext4 defaults 1 0 <-- changed this final '1 2' to '1 0' UUID=A087-E89B /boot/efi vfat umask=0077,shortname=winnt 0 2
I believe the AI for rpm is to log a more useful error for a readonly filesystem (instead of complaining about lack of diskspace, it should have said /boot is read-only) There might also be something to better handle this in the boot/fsck scripts... but not sure exactly what... umount, fsck, mount again?
Yes, rpm lacks the means to specifically report read-only mount through the disk space checking machinery. The existing behavior was still considered an improvement over proceeding with the transaction anyway. This isn't Fedora specific so it's an upstream RFE matter.
Moved to upstream: https://github.com/rpm-software-management/rpm/issues/2782