Bug 1581478 - dnf system upgrade does not upgrade rescue kernel
Summary: dnf system upgrade does not upgrade rescue kernel
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: dracut
Version: 28
Hardware: x86_64
OS: Linux
unspecified
low
Target Milestone: ---
Assignee: dracut-maint-list
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-05-22 21:45 UTC by John Griffiths
Modified: 2018-08-03 07:41 UTC (History)
30 users (show)

Fixed In Version: dracut-048-14.git20180726.fc28
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-07-29 03:25:13 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description John Griffiths 2018-05-22 21:45:07 UTC
Description of problem:
dnf does not upgrade the rescue kernel to the current kernel version.

Upgraded from Fedora 26 to Fedora 28 and the rescue kernel is Fedora 26.

Version-Release number of selected component (if applicable):
dnf-2.7.5-12.fc28.noarch
dracut-047-8.git20180305.fc28.x86_64


How reproducible:
always

Steps to Reproduce:
1. dnf upgrade --refersh
2. dnf install dnf-plugin-system-upgrade (if not already intalled)
3. dnf system-upgrade download --refresh --releasever=28
4. dnf system-upgrade reboot

Actual results:
vmlinuz-0-rescue-51331025f1934688ac507601a35ee806 and initramfs-0-rescue-51331025f1934688ac507601a35ee806.img are still version 26

Expected results:
Should upgrade to the version being installed.

Additional info:
Keeping old rescue kernel may be reasonable behavior if there are still older version kernels present, but at the point there are not any older version kernels installed, then dracut should be called when the last older major version is removed to update the rescue kernel to the current version.

Comment 1 Daniel Mach 2018-05-28 11:30:10 UTC
the rescue kernel is created during %post or %posttran
we believe this has to be fixed in the kernel package rather than in the dnf plugin code

Comment 2 Laura Abbott 2018-05-29 15:49:33 UTC
The actual rescue kernel isn't created by the kernel package itself, pretty sure that comes from dracut

Comment 3 Harald Hoyer 2018-07-05 13:09:47 UTC
system-upgrade could remove the rescue files... neither the kernel nor dracut knows, that a system-upgrade is happening.

The problem is, that nobody knows, if any of the oldest installed kernels is any good, or the current installed dracut produces valid initramfs files (even with old kernels). So keeping the tested (from a working install) version around might be the only solution.

Comment 4 Warren Sturm 2018-07-14 16:23:38 UTC
Package dracut-config-rescue-47 still works but 48 is broken and doesn't produce the vmlinux-0-rescue-MACHINE-ID.  There are some errors produced when trying to run '/usr/lib/kernel/install.d/51-dracut-rescue.install add':

cp: cannot stat '': No such file or directory
Can't copy ' to '/boot/vmlinuz-0-rescue-528ae04402ec4f1691b944ed0dc8e8af'!
cp: cannot stat '/bls.conf': No such file or directory
sed: can't read /boot/loader/entries/528ae04402ec4f1691b944ed0dc8e8af-0-rescue.conf: No such file or directory

initramfs-0-rescue is created but vmlinuz-0-rescue does not exist nor is the grub entry created.

Comment 5 Harald Hoyer 2018-07-16 13:28:53 UTC
Javier Martinez Canillas, seems like you broke something

Comment 6 Javier Martinez Canillas 2018-07-16 15:58:04 UTC
(In reply to Warren Sturm from comment #4)
> Package dracut-config-rescue-47 still works but 48 is broken and doesn't
> produce the vmlinux-0-rescue-MACHINE-ID.  There are some errors produced
> when trying to run '/usr/lib/kernel/install.d/51-dracut-rescue.install add':
> 

I don't think the kernel/install.d scripts are meant to be called directly but rather these should be called by the kernel-install script.

So instead you should call these scripts with the correct options, i.e:

$ kernel-install add 4.17.5-200.fc28.x86_64 /lib/modules/4.17.5-200.fc28.x86_64/vmlinuz

> cp: cannot stat '': No such file or directory
> Can't copy ' to '/boot/vmlinuz-0-rescue-528ae04402ec4f1691b944ed0dc8e8af'!
> cp: cannot stat '/bls.conf': No such file or directory
> sed: can't read
> /boot/loader/entries/528ae04402ec4f1691b944ed0dc8e8af-0-rescue.conf: No such
> file or directory
> 
> initramfs-0-rescue is created but vmlinuz-0-rescue does not exist nor is the
> grub entry created.


Having said that, I think there is some confusion here. The /usr/lib/kernel/install.d/51-dracut-rescue.install script is executed when either systemd-boot is used or grub with a BootLoaderSpec entries configuration.

When using grub with a standard configuration (menu entries in grub.cfg), the /etc/kernel/postinst.d/51-dracut-rescue-postinst.sh script is used to generate the rescue kernel and initamfs images.

This script is shipped by the dracut-config-rescue package and I noticed that it's indeed missing in dracut-config-rescue-048:

$ rpm -qa | grep dracut-config-rescue
dracut-config-rescue-047-8.git20180305.fc28.x86_64

$ rpm -ql dracut-config-rescue
/etc/kernel/postinst.d/51-dracut-rescue-postinst.sh
/usr/lib/dracut/dracut.conf.d/02-rescue.conf
/usr/lib/kernel/install.d/51-dracut-rescue.install

$ rpm -qa | grep dracut-config-rescue
dracut-config-rescue-048-1.fc28.x86_64

$ rpm -ql dracut-config-rescue
/usr/lib/dracut/dracut.conf.d/02-rescue.conf
/usr/lib/kernel/install.d/51-dracut-rescue.install

The upstream dracut commit that removed this file is 0bb9a683d4d ("spec: drop support for legacy distributions"):

https://github.com/dracutdevs/dracut/commit/0bb9a683d4d

The author claims that the script is only used for RHEL <=7, but it is needed by Fedora when grubby / new-kernel-pkg is used. So I think that commit should either be reverted or the script included in the dracut-config-rescue package.

Now about the actual bug report that the rescue image isn't updated, as Harald said both /etc/kernel/postinst.d/51-dracut-rescue-postinst.sh and /usr/lib/kernel/install.d/51-dracut-rescue.install only attempt to create new kernel and initramfs images if these don't exist. The rationale as he explained is that the kernel used during installation is known to work (since the installation was possible) so it can be used to rescue the system.

I don't know if dnf system-upgrade removes the rescue images to force these to be created again on a system upgrade. But I don't think that's a good idea since it can leave the system with a rescue image that doesn't work and can't be used to rescue the system.

Comment 7 Javier Martinez Canillas 2018-07-24 09:03:56 UTC
Did a Rawhide installation today using the latest image (https://dl.fedoraproject.org/pub/fedora/linux/development/rawhide/Workstation/x86_64/iso/Fedora-Workstation-Live-x86_64-Rawhide-20180723.n.0.iso) and confirmed that the rescue kernel and initramfs images are not created anymore due the missing /etc/kernel/postinst.d/51-dracut-rescue-postinst.sh script.

Comment 8 Javier Martinez Canillas 2018-07-24 09:57:23 UTC
(In reply to Harald Hoyer from comment #5)
> Javier Martinez Canillas, seems like you broke something

As mentioned in Comment 6, the regression has nothing to do with my changes.

But since I already tracked down the issue, I've proposed fixes for both upstream dracut [0] and the Fedora package [1].

[0]: https://github.com/dracutdevs/dracut/pull/430
[1]: https://src.fedoraproject.org/rpms/dracut/pull-request/7

Comment 9 Harald Hoyer 2018-07-24 09:58:12 UTC
I thought, that is what 51-dracut-rescue.install is for, following the BLS

Comment 10 Harald Hoyer 2018-07-24 09:59:54 UTC
kernel-install 51-dracut-rescue.install obsoleting the grubby script 51-dracut-rescue-postinst.sh.

Comment 11 Javier Martinez Canillas 2018-07-24 10:07:01 UTC
(In reply to Harald Hoyer from comment #10)
> kernel-install 51-dracut-rescue.install obsoleting the grubby script
> 51-dracut-rescue-postinst.sh.

That's correct, but as you said it's only used when the grub configuration is using BLS. For non-BLS grub configuration the 51-dracut-rescue-postinst.sh is still used by new-kernel-pkg / grubby.

So we have to keep the script around until grubby is deprecated (the plan is for F30).

From F29, new installations will default to BLS so kernel-install 51-dracut-rescue.install will be used in that case (or for F28 / F29 users who choose to migrate to a BLS setup).

Comment 12 Javier Martinez Canillas 2018-07-24 10:09:07 UTC
(In reply to Javier Martinez Canillas from comment #11)

> 51-dracut-rescue.install will be used in that case (or for F28 / F29 users
> who choose to migrate to a BLS setup).

Sorry, this should had been "or for F28 users who choose to migrate".

Comment 13 Javier Martinez Canillas 2018-07-24 10:16:50 UTC
(In reply to Javier Martinez Canillas from comment #12)
> (In reply to Javier Martinez Canillas from comment #11)
> 
> > 51-dracut-rescue.install will be used in that case (or for F28 / F29 users
> > who choose to migrate to a BLS setup).
> 
> Sorry, this should had been "or for F28 users who choose to migrate".

Ah no, my original comment was since it's only for new installations. So a F28 user can upgrade to F29 and keep a non-BLS configuration. It will only be forced to change to a BLS setup when grubby is obsoleted in F30.

Anyway, point is that we should still keep that script until all users have migrated to a BLS configuration.

Comment 14 Fedora Update System 2018-07-26 11:29:11 UTC
dracut-048-14.git20180726.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-0b82983708

Comment 15 Harald Hoyer 2018-07-26 11:51:11 UTC
(In reply to Javier Martinez Canillas from comment #13)
> Anyway, point is that we should still keep that script until all users have
> migrated to a BLS configuration.

Will there be a migration script of some kind?

Comment 16 Javier Martinez Canillas 2018-07-26 12:18:00 UTC
(In reply to Harald Hoyer from comment #15)
> (In reply to Javier Martinez Canillas from comment #13)
> > Anyway, point is that we should still keep that script until all users have
> > migrated to a BLS configuration.
> 
> Will there be a migration script of some kind?

Yes, there's a grub2-switch-to-blscfg script for grub2 and petitboot (that also parses grub config files) and a zipl-switch-to-blscfg for zipl.

The migration script will be called from a %postun scriptlet once grubby is obsoleted and removed, but users might choose to migrate explicitly by executing the script.

Comment 17 Fedora Update System 2018-07-26 16:33:39 UTC
dracut-048-14.git20180726.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-0b82983708

Comment 18 Fedora Update System 2018-07-29 03:25:13 UTC
dracut-048-14.git20180726.fc28 has been pushed to the Fedora 28 stable repository. If problems still persist, please make note of it in this bug report.

Comment 19 Marko Bevc 2018-07-31 16:11:09 UTC
After removing rescue image it's not regenerated using this latest package in F28.

Using dracut-048-14.git20180726.fc28.x86_64.


Note You need to log in before you can comment on or make changes to this bug.