Bug 1601578

Summary: All x86 BIOS boots fail with 'invalid magic number' errors
Product: [Fedora] Fedora Reporter: Adam Williamson <awilliam>
Component: grub2Assignee: Peter Jones <pjones>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: rawhideCC: chepioq, filbranden, jwboyer, lkundrak, mcsontos, mpitt, pjones, ppisar, robatino
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: grub2-2.02-45.fc29 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1601815 (view as bug list) Environment:
Last Closed: 2018-07-20 17:58:43 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Bug Depends On:    
Bug Blocks: 1517011    
Attachments:
Description Flags
full grub.cfg file none

Description Adam Williamson 2018-07-16 18:15:27 UTC
Since the Fedora-20180714.n.0 compose (the previous compose, Fedora-Rawhide-20180711.n.0, was OK), all x86 BIOS installs fail to boot after installation. After an option is chosen from the bootloader menu, an "invalid magic number" error is displayed for several seconds, then the system either reboots, stops, or displays a kernel panic (seems to depend on the system).

Some further info from jwb:

<jwb> 06:12:18> pjones, so i just did a real install and got an error after boot.  i looked at the boot stanza that was written out and it has a linux16 line for the vmlinuz, then a linux16 line for the initramfs, then an initrd16 line for the initramfs
<jwb> 06:12:35> pjones, if i remove that linux16 line trying to load the initramfs, it boots fine

I'll do a local reproduction and attach an exact grub.cfg file for the record, though I believe pjones has already reproduced the issue and is working on it.

Between the working and the failed compose were these changes to grub2:

* Fri Jul 13 2018 Peter Jones <pjones@redhat.com> - 2.02-42
 - Revert broken moduledir fix *again*.

* Thu Jul 12 2018 Peter Jones <pjones@redhat.com> - 2.02-41
 - Fix our linuxefi/linux command reunion.

* Wed Jul 11 2018 Peter Jones <pjones@redhat.com> - 2.02-40
 - Port several fixes from the F28 tree and a WIP tree.

This is an obvious Beta blocker per Basic criterion "...A system installed with a release-blocking desktop must boot to a log in screen where it is possible to log in to a working desktop using a user account created during installation or a 'first boot' utility.
A system installed without a graphical package set must boot to a state where it is possible to log in through at least one of the default virtual consoles." on x86 BIOS, proposing.

Comment 1 Adam Williamson 2018-07-16 18:41:36 UTC
These are the exact lines I get in grub.cfg after an install from the Fedora-20180714.n.0 Everything netinst image:

	linux16	/vmlinuz-4.18.0-0.rc4.git4.1.fc29.x86_64 root=/dev/mapper/fedora-root ro resume=/dev/mapper/fedora-swap rd.lvm.lv=fedora/root rd.lvm.lv=fedora/swap rhgb quiet LANG=en_CA.UTF-8
	linux16	/initramfs-4.18.0-0.rc4.git4.1.fc29.x86_64.img
	initrd16 /initramfs-4.18.0-0.rc4.git4.1.fc29.x86_64.img

I'll attach the full file as well.

Comment 2 Adam Williamson 2018-07-16 18:42:17 UTC
Created attachment 1459248 [details]
full grub.cfg file

Comment 3 Andre Robatino 2018-07-17 03:00:31 UTC
Still broken with grub2-2.02-44 after installing from https://koji.fedoraproject.org/koji/buildinfo?buildID=1127924 using rescue mode.

Comment 4 Filipe Brandenburger 2018-07-17 07:25:49 UTC
Take a look at this pull request:
https://src.fedoraproject.org/rpms/grub2/pull-request/6

Cheers,
Filipe

Comment 5 Filipe Brandenburger 2018-07-17 07:29:06 UTC
Just rebased that PR on top of current specfile...

I'd appreciate it if someone could take a look at it and merge it.

Cheers,
Filipe

Comment 6 Martin Pitt 2018-07-17 07:55:53 UTC
In our Cockpit test VMs (for RHEL devel series) I'm getting the exact same broken config as in comment #1, i. e. initramfs gets loaded by *both* linux16 and initrd16. However, when I just run grub2-mkconfig in an otherwise unmodified system, I *only* get the

   linux16	/initramfs-4.18.0-0.rc4.git4.1.fc29.x86_64.img

line without the initrd16 one. So this is rather unpredictable. For the former the workaround is to simply drop the line, for the latter case it's s/linux16/initrd16/. So this is a bit brittle to work around in scripts.

But I confirm that dropping the linux16 /initramfs line fixes the boot.

Comment 7 Andre Robatino 2018-07-17 15:27:22 UTC
Confirmed fixed with grub2-2.02-45.fc29 ( https://koji.fedoraproject.org/koji/buildinfo?buildID=1128108 ).

Comment 8 Adam Williamson 2018-07-17 16:36:32 UTC
*** Bug 1601585 has been marked as a duplicate of this bug. ***

Comment 9 Marian Csontos 2018-07-17 18:18:16 UTC
Workaround (add to %post) which works for me:

# Workaround for installation Bug 1601578:
cp /boot/grub2/grub.cfg{,.0orig}
sed -i -e 's|^\s\+linux16\s\+/initramfs-|\tinitrd16\t/initramfs-|' /boot/grub2/grub.cfg
sed -i -e 's|^\s\+initrd16\s\+/initramfs-|\tinitrd16\t/initramfs-|' /boot/grub2/grub.cfg
mv /boot/grub2/grub.cfg{,.1sed}
uniq /boot/grub2/grub.cfg{.1sed,}

Seeing Comment 6, instead of simply deleting linux16 line, it changes linux16 to initrd16, then makes whitespace same, and delete identical lines :-)

Seems there are some problems with zram on shutdown too... :-(

Comment 10 dominique 2018-07-20 09:46:16 UTC
I confirm that grub2-2.02-45.fc29 fix problem.
Thank

Comment 11 Adam Williamson 2018-07-20 17:58:43 UTC
Indeed, openQA tests are working now.