Bug 2087133

Summary: kdump: kernel 5.17.8-300.fc36.x86_64 doesn't exist
Product: [Fedora] Fedora Reporter: Cristian Ciupitu <cristian.ciupitu>
Component: kexec-toolsAssignee: Lichen Liu <lichliu>
Status: NEW --- QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 39CC: acaringi, adscvr, airlied, alciregi, bhe, bskeggs, coxu, hdegoede, hpa, jarodwilson, jforbes, jglisse, jonathan, josef, kernel-maint, lgoncalv, lichliu, linville, masami256, mchehab, ptalbert, robatino, ruyang, ryncsn, steved
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Cristian Ciupitu 2022-05-17 12:10:10 UTC
1. Please describe the problem:
dnf upgrade
...
Transaction test succeeded.
Running transaction
  Preparing        :                                                   1/1
  Installing       : kernel-core-5.17.8-300.fc36.x86_64                1/13
  Running scriptlet: kernel-core-5.17.8-300.fc36.x86_64                1/13
  Installing       : kernel-modules-5.17.8-300.fc36.x86_64             2/13
  Running scriptlet: kernel-modules-5.17.8-300.fc36.x86_64             2/13
  Upgrading        : blivet-data-1:3.4.4-1.fc36.noarch                 3/13
  Upgrading        : python3-blivet-1:3.4.4-1.fc36.noarch              4/13
  Installing       : kernel-modules-extra-5.17.8-300.fc36.x86_64       5/13
  Running scriptlet: kernel-modules-extra-5.17.8-300.fc36.x86_64       5/13
  Upgrading        : perl-XML-LibXSLT-2.001.000-1.fc36.x86_64          6/13
  Upgrading        : librsvg2-2.54.3-1.fc36.x86_64                     7/13
  Running scriptlet: dbus-broker-31-1.fc36.x86_64                      8/13
  Upgrading        : dbus-broker-31-1.fc36.x86_64                      8/13
  Running scriptlet: dbus-broker-31-1.fc36.x86_64                      8/13
  Cleanup          : python3-blivet-1:3.4.3-2.fc36.noarch              9/13
  Cleanup          : blivet-data-1:3.4.3-2.fc36.noarch                10/13
  Cleanup          : perl-XML-LibXSLT-2.000.000-1.fc36.x86_64         11/13
  Cleanup          : librsvg2-2.54.2-1.fc36.x86_64                    12/13
  Running scriptlet: dbus-broker-29-5.fc36.x86_64                     13/13
  Cleanup          : dbus-broker-29-5.fc36.x86_64                     13/13
  Running scriptlet: dbus-broker-29-5.fc36.x86_64                     13/13
  Running scriptlet: kernel-core-5.17.8-300.fc36.x86_64               13/13
kdump: kernel 5.17.8-300.fc36.x86_64 doesn't exist

  Running scriptlet: kernel-modules-5.17.8-300.fc36.x86_64            13/13
  Running scriptlet: dbus-broker-29-5.fc36.x86_64                     13/13
  Verifying        : kernel-core-5.17.8-300.fc36.x86_64                1/13
  Verifying        : kernel-modules-5.17.8-300.fc36.x86_64             2/13
  Verifying        : kernel-modules-extra-5.17.8-300.fc36.x86_64       3/13
  Verifying        : blivet-data-1:3.4.4-1.fc36.noarch                 4/13
  Verifying        : blivet-data-1:3.4.3-2.fc36.noarch                 5/13
  Verifying        : dbus-broker-31-1.fc36.x86_64                      6/13
  Verifying        : dbus-broker-29-5.fc36.x86_64                      7/13
  Verifying        : librsvg2-2.54.3-1.fc36.x86_64                     8/13
  Verifying        : librsvg2-2.54.2-1.fc36.x86_64                     9/13
  Verifying        : perl-XML-LibXSLT-2.001.000-1.fc36.x86_64         10/13
  Verifying        : perl-XML-LibXSLT-2.000.000-1.fc36.x86_64         11/13
  Verifying        : python3-blivet-1:3.4.4-1.fc36.noarch             12/13
  Verifying        : python3-blivet-1:3.4.3-2.fc36.noarch             13/13

Upgraded:
  blivet-data-1:3.4.4-1.fc36.noarch
  dbus-broker-31-1.fc36.x86_64
  librsvg2-2.54.3-1.fc36.x86_64
  perl-XML-LibXSLT-2.001.000-1.fc36.x86_64
  python3-blivet-1:3.4.4-1.fc36.noarch
Installed:
  kernel-core-5.17.8-300.fc36.x86_64
  kernel-modules-5.17.8-300.fc36.x86_64
  kernel-modules-extra-5.17.8-300.fc36.x86_64



2. What is the Version-Release number of the kernel:
kernel-core-5.17.8-300.fc36.x86_64

3. Did it work previously in Fedora? If so, what kernel version did the issue
   *first* appear?  Old kernels are available for download at
   https://koji.fedoraproject.org/koji/packageinfo?packageID=8 :
NO


4. Can you reproduce this issue? If so, please provide the steps to reproduce
   the issue below:
I have no idea.

5. Does this problem occur with the latest Rawhide kernel? To install the
   Rawhide kernel, run ``sudo dnf install fedora-repos-rawhide`` followed by
   ``sudo dnf update --enablerepo=rawhide kernel``:
I have no idea.

6. Are you running any modules that not shipped with directly Fedora's kernel?:
No.

7. Please attach the kernel logs. You can get the complete kernel log
   for a boot with ``journalctl --no-hostname -k > dmesg.txt``. If the
   issue occurred on a previous boot, use the journalctl ``-b`` flag.


EXTRA:
I currently have 
 kernel-core-5.17.7-300.fc36.x86_64 kexec-tools-2.0.23-6.fc36.x86_64
 kexec-tools-2.0.23-6.fc36.x86_64

Comment 1 Justin M. Forbes 2022-05-17 18:01:40 UTC
Where is this coming from? There is nothing in %post that calls kdump at all on a typical system, so it is likely something in /etc/kernel/* but I do have kexec-tools installed and still do not see an entry.

Comment 2 Cristian Ciupitu 2022-05-17 23:00:52 UTC
I don't have the slightest idea where it's coming from.


# find /etc/kernel/
/etc/kernel/
/etc/kernel/postinst.d
/etc/kernel/install.d

Comment 3 Justin M. Forbes 2022-05-18 14:23:24 UTC
Interesting.  The kernel itself only calls kernel-install as the post. That goes through the /etc/kernel directories for plugins (dkms uses this), but I see no reference to kdump in kernel-install, nor do I see one in dracut. What is your kernel command line?

Comment 4 Andre Robatino 2022-05-18 19:31:43 UTC
Upon doing a Google search for "kdump: kernel 5.17.8-300.fc36.x86_64 doesn't exist" I got this page which includes a reference to "kdump: kernel 5.17.6-300.fc36.x86_64 doesn't exist".

https://forums.fedoraforum.org/showthread.php?328435-After-33-gt-35-36-upgrades-still-using-33-kernel&p=1859062

Comment 5 Andre Robatino 2022-05-18 19:56:23 UTC
Oh, and an adjacent reference to 5.17.5 as well, suggesting that wherever this is coming from is not version-specific.

Comment 6 Cristian Ciupitu 2022-05-18 20:22:28 UTC
$ cat /proc/cmdline 
initrd=\cde44f2e256c4d7f816b58632df8afa1\5.17.7-300.fc36.x86_64\initrd root=UUID=126898db-f61b-402b-a9ca-4442b79f41a4 ro rhgb quiet rd.lvm=0 rd.luks=0 rd.md=0 rd.dm=0 resume=PARTUUID=ef7284ab-13f9-4698-9555-33fab31dc6c6 intel_iommu=on video=VGA-1:d

Comment 7 Cristian Ciupitu 2022-05-18 21:09:55 UTC
In case these are relevant.

I'm not using GRUB (2), I'm using systemd.

# rpm -q -a | fgrep kernel | sort
abrt-addon-kerneloops-2.15.1-1.fc36.x86_64
kernel-core-5.17.7-300.fc36.x86_64
kernel-core-5.17.8-300.fc36.x86_64
kernel-headers-5.17.6-300.fc36.x86_64
kernel-modules-5.17.7-300.fc36.x86_64
kernel-modules-5.17.8-300.fc36.x86_64
kernel-modules-extra-5.17.7-300.fc36.x86_64
kernel-modules-extra-5.17.8-300.fc36.x86_64
kernel-srpm-macros-1.0-14.fc36.noarch
kernel-tools-5.17.6-300.fc36.x86_64
kernel-tools-libs-5.17.6-300.fc36.x86_64
libreport-plugin-kerneloops-2.17.1-1.fc36.x86_64
python3-ipykernel-6.6.1-2.fc36.noarch

`dnf check` does not complain.

# rpm -V -a | tee /tmp/rpm-V-a
.M.......    /boot/efi/EFI
.M.......  g /usr/share/icons/Humanity-Dark/icon-theme.cache
.M.......  g /usr/share/icons/Humanity/icon-theme.cache
S.5....T.  c /etc/security/access.conf
.M.......    /var/lib/AccountsService/icons
S.5....T.  c /var/lib/unbound/root.key
S.5....T.  c /etc/sudoers
.M.......  g /var/lib/colord/mapping.db
.M.......  g /var/lib/colord/storage.db
S.5....T.  c /etc/sysconfig/lm_sensors
.M.......  g /var/lib/plymouth/boot-duration
.M...UG..  g /run/lightdm
..5....T.  c /etc/yum.repos.d/google-chrome.repo
S.5....T.  c /root/.bash_profile
SM5....T.  c /root/.bashrc
.M.......  g /etc/udev/hwdb.bin
.M.......  g /var/lib/systemd/random-seed
.M.......  g /var/lib/selinux/targeted/active/modules/200/smartmon
S.5....T.  c /etc/dhcp/dhcpd.conf
.M.......  g /etc/containers/storage.conf
.......T.    /boot/efi/EFI/efifs/bfs.efi
.......T.    /boot/efi/EFI/efifs/btrfs.efi
.......T.    /boot/efi/EFI/efifs/cpio.efi
.......T.    /boot/efi/EFI/efifs/cpio_be.efi
.......T.    /boot/efi/EFI/efifs/ext2.efi
.......T.    /boot/efi/EFI/efifs/fat.efi
.......T.    /boot/efi/EFI/efifs/hfs.efi
.......T.    /boot/efi/EFI/efifs/iso9660.efi
.......T.    /boot/efi/EFI/efifs/minix.efi
.......T.    /boot/efi/EFI/efifs/minix2.efi
.......T.    /boot/efi/EFI/efifs/minix_be.efi
.......T.    /boot/efi/EFI/efifs/ntfs.efi
.......T.    /boot/efi/EFI/efifs/odc.efi
.......T.    /boot/efi/EFI/efifs/procfs.efi
.......T.    /boot/efi/EFI/efifs/squash4.efi
.......T.    /boot/efi/EFI/efifs/udf.efi
.......T.    /boot/efi/EFI/efifs/ufs1.efi
.....UG..  g /var/lib/nfs/statd/state
.M.......  g /run/libvirt/common
.M...UG..  g /run/libvirt/qemu/dbus
.....UG..  g /run/libvirt/qemu/slirp
S.5....T.  c /etc/dnf/dnf.conf
S.5....T.  c /etc/environment
S.5....T.  c /etc/samba/smb.conf
S.5....T.  c /etc/selinux/targeted/contexts/files/file_contexts.local

Comment 8 Andre Robatino 2022-05-19 16:15:09 UTC
I installed kernel-5.17.8-300.fc36.x86_64 in F36 with kexec-tools installed and did not see this message.

# find /etc/kernel/
/etc/kernel/
/etc/kernel/install.d
/etc/kernel/install.d/dkms
/etc/kernel/postinst.d
/etc/kernel/postinst.d/dkms
/etc/kernel/prerm.d
/etc/kernel/prerm.d/dkms

Comment 9 Andre Robatino 2022-05-19 20:41:47 UTC
(In reply to Cristian Ciupitu from comment #7)
> In case these are relevant.
> 
> I'm not using GRUB (2), I'm using systemd.
> 
> # rpm -q -a | fgrep kernel | sort
> abrt-addon-kerneloops-2.15.1-1.fc36.x86_64
> kernel-core-5.17.7-300.fc36.x86_64
> kernel-core-5.17.8-300.fc36.x86_64
> kernel-headers-5.17.6-300.fc36.x86_64
> kernel-modules-5.17.7-300.fc36.x86_64
> kernel-modules-5.17.8-300.fc36.x86_64
> kernel-modules-extra-5.17.7-300.fc36.x86_64
> kernel-modules-extra-5.17.8-300.fc36.x86_64
> kernel-srpm-macros-1.0-14.fc36.noarch
> kernel-tools-5.17.6-300.fc36.x86_64
> kernel-tools-libs-5.17.6-300.fc36.x86_64
> libreport-plugin-kerneloops-2.17.1-1.fc36.x86_64
> python3-ipykernel-6.6.1-2.fc36.noarch

Just noticed something odd, you don't appear to actually have kernel-5.17.8-300.fc36.x86_64 installed. On my machines, for each installed kernel version I have a (0-byte) package named kernel. And if I attempt to remove them with dnf, it's allowed (though I didn't actually do it), so not sure what their purpose is.

$ rpm -q kernel
kernel-5.17.6-300.fc36.x86_64
kernel-5.17.7-300.fc36.x86_64
kernel-5.17.8-300.fc36.x86_64

Comment 10 Cristian Ciupitu 2022-05-21 11:42:03 UTC
Yeah, you're right. I've removed the plain kernel package from my system long time ago after noticing that it's empty anyway.

Anyway for what it's worth I had the same problem with kernel-core-5.17.9-300.fc36.x86_64.

Also in the mean time I've upgraded more packages and I have also rebooted the system.
After booting into 5.17.9-300.fc36.x86_64 I rerun `dnf install kernel-core-5.17.8-300.fc36.x86_64` and there were no issues.
I run the command multiple times and still no issues, so I guess the issue is sort of solved (though something fishy is or was going on).

Comment 11 Andre Robatino 2022-05-21 17:56:14 UTC
I'm guessing the 0-byte package is there to make it easy for applications to check if a kernel is installed (jforbes, is that true?). In any case, regardless of where your kdump message is coming from, it's literally true, if it's referring to that specific package.

I tried removing the 0-byte kernel package for one of my kernels and then reinstalling the corresponding kernel-core and was not able to trigger that message, though.

Comment 12 Cristian Ciupitu 2022-05-26 19:20:06 UTC
It happened again today. I booted 5.17.9-300.fc36.x86_64 (according to `uname -r`) then I run

    # dnf upgrade

    Downloading Packages:
    Running transaction check
    Transaction check succeeded.
    Running transaction test
    Transaction test succeeded.
    Running transaction
      Preparing        :                                               1/1 
      Installing       : kernel-core-5.17.11-300.fc36.x86_64           1/3 
      Running scriptlet: kernel-core-5.17.11-300.fc36.x86_64           1/3 
      Installing       : kernel-modules-5.17.11-300.fc36.x86_64        2/3 
      Running scriptlet: kernel-modules-5.17.11-300.fc36.x86_64        2/3 
      Installing       : kernel-modules-extra-5.17.11-300.fc36.x86_64  3/3 
      Running scriptlet: kernel-modules-extra-5.17.11-300.fc36.x86_64  3/3 
      Running scriptlet: kernel-core-5.17.11-300.fc36.x86_64           3/3 
    kdump: kernel 5.17.11-300.fc36.x86_64 doesn't exist
    
      Running scriptlet: kernel-modules-5.17.11-300.fc36.x86_64        3/3 
      Running scriptlet: kernel-modules-extra-5.17.11-300.fc36.x86_64  3/3 
      Verifying        : kernel-core-5.17.11-300.fc36.x86_64           1/3 

If I uninstall kexec-tools, then uninstall the new kernel and reinstall the new kernel, there's no error message.

Comment 13 Cristian Ciupitu 2022-05-26 21:25:10 UTC
After some debugging I think there is a bug in the  _find_kernel_path_by_release function  from  /usr/bin/kdumpctl

	local _release="$1" _grubby_kernel_str _kernel_path
	_grubby_kernel_str=$(grubby --info ALL | grep "^kernel=.*$_release")
	_kernel_path=$(_filter_grubby_kernel_str "$_grubby_kernel_str")
	if [[ -z $_kernel_path ]]; then
		derror "kernel $_release doesn't exist"
		return 1
	fi
	echo -n "$_kernel_path"

I'm using systemd-boot instead of GRUB because it does enough for me. The kernel is installed on /boot/efi of course.

I run `grubby --info ALL` separately and it doesn't print anything.

This is what I have installed:

  grub2-common-2.06-40.fc36.noarch          
  grub2-efi-x64-modules-2.06-40.fc36.noarch
  grub2-tools-2.06-40.fc36.x86_64
  grub2-tools-efi-2.06-40.fc36.x86_64
  grub2-tools-extra-2.06-40.fc36.x86_64
  grub2-tools-minimal-2.06-40.fc36.x86_64
  grubby-8.40-57.fc36.x86_64
  systemd-250.6-1.fc36.x86_64
  systemd-bootchart-233-11.fc36.x86_64
  systemd-container-250.6-1.fc36.x86_64
  systemd-libs-250.6-1.fc36.x86_64
  systemd-networkd-250.6-1.fc36.x86_64
  systemd-oomd-defaults-250.6-1.fc36.noarch
  systemd-pam-250.6-1.fc36.x86_64
  systemd-resolved-250.6-1.fc36.x86_64
  systemd-rpm-macros-250.6-1.fc36.noarch
  systemd-udev-250.6-1.fc36.x86_64

Comment 14 Justin M. Forbes 2022-05-26 21:35:26 UTC
Either way, sounds very much like a kexec-tools issue, and not a kernel issue. Reassigning.

Comment 15 Cristian Ciupitu 2022-08-02 14:57:35 UTC
It's still happening with kexec-tools-2.0.24-4.fc36.x86_64

Comment 16 Cristian Ciupitu 2022-09-06 19:19:12 UTC
Not sure if it's related, but I'm also getting this error after getting rid of grubby because I don't need it (sd-boot FTW) and reinstalling kexec-tools-2.0.24-4.fc36.x86_64.rpm:
    
      Running scriptlet: kexec-tools-2.0.24-4.fc36.x86_64        5/5 
    /bin/kdumpctl: line 1393: grubby: command not found

Comment 17 Cristian Ciupitu 2022-10-13 20:07:42 UTC
I was talking about bug #2121912

Comment 18 Ben Cotton 2023-04-25 18:24:17 UTC
This message is a reminder that Fedora Linux 36 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 36 on 2023-05-16.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '36'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version. Note that the version field may be hidden.
Click the "Show advanced fields" button if you do not see it.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 36 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Comment 19 Fedora Release Engineering 2023-08-16 07:05:37 UTC
This bug appears to have been reported against 'rawhide' during the Fedora Linux 39 development cycle.
Changing version to 39.