The latest package updates on Fedora 40 regress kdump. Starting the service fails to rebuild the initrd, without any useful error message. This was spotted by Cockpit's tests in [1]. There are a lot of package updates (see end of [2]); specifically, kexec-tools was not updated, but there is a new kernel, new selinux (but still fails with `setenforce 0`), new systemd, etc. [1] https://github.com/cockpit-project/bots/pull/6101 [2] https://cockpit-logs.us-east-1.linodeobjects.com/image-refresh-fedora-40-ddc79c7b-20240318-223715/log.html Reproducible: Always Steps to Reproduce: 1. Boot a standard Fedora 40 cloud image 2. Enable kdump with: `kdumpctl reset-crashkernel; reboot` 3. After reboot, validate that /proc/cmdline has crashkernel 4. systemctl start kdump Actual Results: Job for kdump.service failed because the control process exited with error code. kdumpctl[1322]: kdump: Rebuilding /boot/initramfs-6.8.0-63.fc40.1.x86_64kdump.img kdumpctl[1322]: kdump: mkdumprd: failed to make kdump initrd kdumpctl[1322]: kdump: Starting kdump: [FAILED] systemd[1]: kdump.service: Main process exited, code=exited, status=1/FAILURE `sh -x kdumpctl start` does not give more useful information either unfortunately: + dinfo 'No kdump initial ramdisk found.' + set +x kdump: No kdump initial ramdisk found. kdump: Rebuilding /boot/initramfs-6.8.0-63.fc40.1.x86_64kdump.img kdump: mkdumprd: failed to make kdump initrd kdump: Starting kdump: [FAILED] + ret=1 Expected Results: kdump works
That seems to be because F40 cloud image uses btrfs subvol for /var, and kexec-tools doesn't handle it properly.
FYI, this problem is gone with latest rawhide. I'm not sure if it's fixed in latest dracut (dracut-101-1.fc41) or systemd (systemd-256~rc3-3.fc41).
I'm not sure if I was mistaken before. But the problem still exists in latest rawhide. I notice F38's / is also a subvol but F40&41's /var also becomes an independent subvolume. If I modify mkdumprd to call add_dracut_mount manually, kdump will work, ``` add_dracut_mount '/dev/disk/by-uuid/1344093c-5f70-444d-b568-a91242aed979 /sysroot btrfs rw,relatime,seclabel,compress=zstd:1,discard=async,space_cache=v2,subvolid=258,subvol=/var' ``` The / and /var subvolumes have the same UUID and they only differ in subvolid, ``` # mount /dev/vda4 on / type btrfs (rw,relatime,seclabel,compress=zstd:1,discard=async,space_cache=v2,subvolid=256,subvol=/root) /dev/vda4 on /var type btrfs (rw,relatime,seclabel,compress=zstd:1,discard=async,space_cache=v2,subvolid=258,subvol=/var) # lsblk -f NAME FSTYPE FSVER LABEL UUID FSAVAIL FSUSE% MOUNTPOINTS zram0 [SWAP] vda ├─vda1 ├─vda2 vfat FAT16 EFI 1B53-0818 83.7M 16% /boot/efi ├─vda3 ext4 1.0 BOOT c59ecb53-cd2d-4d6b-a5a1-c65e9dbcf660 791.4M 11% /boot └─vda4 btrfs fedora 1344093c-5f70-444d-b568-a91242aed979 2.7G 20% /var /home / ```
(In reply to Coiby from comment #3) Hi Coiby, this issue still exists, I think it is because we dropped useful extra information when we handled mount info in get_bind_mount_source()/get_mount_info(). We dropped the fsroot and only picked the first output of `findmnt`. I think bz2284097 is also caused by these functions.
For the record, this issue still exists in current Fedora 40. Example run: https://cockpit-logs.us-east-1.linodeobjects.com/pull-0-5bc61331-20240807-013032-fedora-40-updates-testing/log.html
Our automatic tracker in https://github.com/cockpit-project/bots/issues/6114 says that this was last observed on February 11. Was something changed/fixed since then or is that sheer luck? Thanks!
Oh, correction: We only ever observed this on Fedora 40, not 41/42/rawhide. And we stopped testing on Fedora 40 a few weeks ago.
This message is a reminder that Fedora Linux 40 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora Linux 40 on 2025-05-13. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a 'version' of '40'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, change the 'version' to a later Fedora Linux version. Note that the version field may be hidden. Click the "Show advanced fields" button if you do not see it. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora Linux 40 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora Linux, you are encouraged to change the 'version' to a later version prior to this bug being closed.
Fedora Linux 40 entered end-of-life (EOL) status on 2025-05-13. Fedora Linux 40 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora Linux please feel free to reopen this bug against that version. Note that the version field may be hidden. Click the "Show advanced fields" button if you do not see the version field. If you are unable to reopen this bug, please file a new report against an active release. Thank you for reporting this bug and we are sorry it could not be fixed.