Bug 964828

Created attachment 750274 [details]
grub.cfg

Version-Release number of selected component (if applicable):
Fedora-Live-Desktop-x86_64-19-Beta-1.iso (RC2)
grub2-2.00-16.fc19.x86_64

How reproducible:
Always


Proposing for freeze exception: it's a negative user experience, with a non-obvious work around for most users who aren't grub experts, but for development there may be a low risk fix possible.

Discussed at 2013-05-20 freeze exception review meeting: http://meetbot.fedoraproject.org/fedora-blocker-review/2013-05-20/f19beta-blocker-review-7.2013-05-20-16.07.log.txt . Rejected as a freeze exception issue: it would be nice to fix this, but we're quite late for beta now and we don't want to poke grub unless it's really really necessary.

*** Bug 1051632 has been marked as a duplicate of this bug. ***

This is still a bug with Fedora Rawhide, but it's a new outcome in that the grub menu lacks a Fedora 20 entry entirely. There is no "Advanced" submenu. If I boot Fedora Rawhide and manually run grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg, os-prober finds Fedora 20 and three indistinguishable menu entries for it are created, none of which work because they use linux/initrd instead of linuxefi/initrdefi commands.

Test system is VirtualBox UEFI VM. Fedora Rawhide installed to free space along side existing bootable Fedora 20 system. After Rawhide installation, Fedora 20 is not a boot option in the GRUB menu.

grub2-tools-2.02-0.3.fc21
grub2-2.02-0.3.fc21
grub2-efi-2.02-.0.3.fc21
grubby-8.35-1.fc21
anaconda-21.37.1.fc21
python-blivet-0.53-1.fc21

os-prober-1.58-6.fc21.x86_64

Booted in the live environment:

[root@localhost fedora]# os-prober
/dev/mapper/fedora00-root:Fedora release 21 (Rawhide):Fedora:linux
[root@localhost fedora]# lvs
  LV   VG       Attr       LSize  Pool Origin Data%  Move Log Cpy%Sync Convert
  home fedora   -wi------- 11.72g                                             
  root fedora   -wi------- 14.65g                                             
  swap fedora   -wi-ao----  2.52g                                             
  root fedora00 -wi-ao---- 47.43g                                             
  swap fedora00 -wi-ao----  2.50g  

os-prober in the live environment isn't finding Fedora 20 because fedora-root is not active.

[root@localhost fedora]# lvchange -a y /dev/fedora
[root@localhost fedora]# lvs
  LV   VG       Attr       LSize  Pool Origin Data%  Move Log Cpy%Sync Convert
  home fedora   -wi-a----- 11.72g                                             
  root fedora   -wi-a----- 14.65g                                             
  swap fedora   -wi-ao----  2.52g                                             
  root fedora00 -wi-ao---- 47.43g                                             
  swap fedora00 -wi-ao----  2.50g                                             

[root@localhost fedora]# os-prober
/dev/mapper/fedora-root:Fedora release 20 (Heisenbug):Fedora:linux
/dev/mapper/fedora00-root:Fedora release 21 (Rawhide):Fedora1:linux

So this is either an anaconda bug, not making the existing Fedora active so that os-prober can find it; or os-prober bug for not making all LV's active so it can locate all possible existing OS's.

I think I should file a new bug based on the new missing entry behavior. And this bug remains the problem with grub2-mkconfig using the wrong grub command and creating an icky and malformed grub.cfg.

The LVM issue is already reported here:
https://bugzilla.redhat.com/show_bug.cgi?id=825236

I really believe that os-prober should not try to change the environment it is being run in, so it shouldn't try to activate LVM partitions.

Re: comment 6, new bug 1110920.

Proposed as a Blocker for 21-final by Fedora user catanzaro using the blocker tracking app because:

 This bug does not appear to violate any current release criterion, which I find surprising. I propose a new blocker criterion:

"The installer must be able to detect all preexisting Linux installations and create working boot menu entries for those installations."

I'm sure that wording needs to be more precise, but the intent is that installing Fedora should not make previous Linux installations disappear. If that's too broad then we could restrict it to previous Fedora installations instead.

as noted in the other bug, bugzilla is not the place for proposing criteria; please do so on the test@ mailing list. thanks!

OK!

This is still a problem with Fedora 21 beta RC1. The problem is Fedora has patched /etc/grub.d/10_linux to use linuxefi and initrdefi, and yet 30_osprober still causes grub.cfg to use linux and initrd for existing Linux installs; so they fail because Fedora GRUB doesn't have linux/initrd commands. 

This bug is the UEFI twin for bug 1108296 and bug 1108344 (which is on BIOS systems where Fedora uses linux16/initrd16 but again other Linux's get entries with linux/initrd which commands do not exist in Fedora's GRUB).

Proposing as final blocker on the basis that rendering a previously bootable Linux installation unbootable is not acceptable; similar to the Windows and OS X dual boot criteria. At the time of this block proposal we don't yet have Linux dual boot criterion.

OK, this is fixed by the same patches that fix things for linux16/initrd16.  See RHBZ#1108296 and RHBZ#1108344

Discussed at 2014-11-05 blocker review meeting [1]. Rejected as a blocker. This doesn't violate any specific criteria. If you want to block on this in the future, please propose the criterion on the list first, let's have it discussed and agreed on. It's good to do this early in the cycle.

[1] http://meetbot.fedoraproject.org/fedora-blocker-review/2014-11-05/

In c12 when I said there's no criterion, this is misleading. What I meant is there's no criterion on the Final Release Criterion page.

However, it was proposed, discussed and agreed to on test@ 2 months ago in the following thread, in which *this* bug was explicitly mentioned by mcatanzaro.
https://lists.fedoraproject.org/pipermail/test/2014-August/122496.html

Revisions are broadly accepted without complaints here:
https://lists.fedoraproject.org/pipermail/test/2014-September/122738.html

The final language is:

The installer must be able to install into free space alongside an existing GNU/Linux installation, install and configure a bootloader that will boot both systems, within the limitations of the upstream bootloader.
Within the limitations? [show] Purpose of this clause is to not require us to fix upstream bootloader bugs or design limitations.

Fedora's grub2-mkconfig creates grub.cfg with the wrong GRUB command for all Linux distros other than the current Fedora install. Gene has a bug fix for this and for the BIOS equivalent bug which should also be a blocker.

The os-prober side of the bug is fixed (bug #1108344), and the grub side for BIOS mode is also fixed (bug #1108296). I'm not sure, but the fix might actually have fixed the grub side in EFI mode too. 

(note: this bug clearly shows that even if the other OS is installed in BIOS mode, EFI grub can't use linux/initrd commands to boot the other OS. So, either the other OS should/can be booted using linuxefi/initrdefi commands, or normal grub2 package should be also installed alongside grub2-efi, so that it can (?) use linux/initrd commands to boot the other OS). If the latter is the solution, then os-prober should tell how the OS is booted (EFI, BIOS32 or BIOS16)).

(In reply to Chris Murphy from comment #15)
> In c12 when I said there's no criterion, this is misleading. What I meant is
> there's no criterion on the Final Release Criterion page.
> 
> However, it was proposed, discussed and agreed to on test@ 2 months ago in
> the following thread, in which *this* bug was explicitly mentioned by
> mcatanzaro.
> https://lists.fedoraproject.org/pipermail/test/2014-August/122496.html

I have read through the thread, and the Windows and MacOS X part were truly incorporated into the criteria. But the dual-Linux part was not, and I see last relevant comments around Sept 29, when Adam says "This is the complex one we're still struggling with". Then the discussion died down. If you believe you can fine-tune the requirements so that they are broadly accepted, can you please revive that discussion? Thanks.

OK, I have looked at the code and here is my understanding of what happens.

1. os-prober returns info about a system if it finds a linux* (linux | linux16 | linuxefi) and initrd* (initrd | initrd16 | initrdefi).

2. /etc/grub.d/30_prober has been modified so that when it gets the infor from os-prober, it creates a linux16 (and initrd16) if the system is i?86 | x86_64.  If the system is efi "if [ -d /sys/firmware/efi ]" then linuxefi (and initrdefi) are used.

3.  These may or may not work and may or may not be what the other installation used.

4.  A comment in a grubby commit mentions that rhbz#1031192 and rhbz#1034743 were the cause for switching to linux16/initrd16.  Unfortunately, these are restricted report and I have no idea as the the reasoning.  However, if it is good reasoning, I would like to see more of an effort to get this accepted by grub upstream.

Neither of those bugs actually seems to be the justification for why anything started creating linux16 entries. They both seem to be issues that cropped up *after* the linux16 entries started appearing. #1031192 is a case where 'grubby --set-default' didn't work for an entry of the 'linux16' type, and #1034743 is a case where entries of the 'linux16' type were being created by 'grubby --add-kernel' on PPC64 (which apparently isn't desired) - the resolution was that the entries for PPC64 are now just 'linux' again, if I'm reading it right.

cmurf: my understanding was as kparal wrote, that we hadn't really reached consensus on the alongside-existing-Linux criterion. Though I think if we were to propose one which simply said "Fedora must be able to dual-boot with previous Fedoras" (in Criteria-ese) everyone would be happy with that for a start, and we could work on broadening it out to other distros later.

So I'm digging into this a bit, but this bug is very clearly for the fairly straightforward case of: do a UEFI install of a given Fedora, then do a UEFI install of a later Fedora and try to boot the older Fedora. Can we please separate discussion of other things:

* linux vs. linux16 for BIOS/CSM boots
* cross-boot scenarios where we have a UEFI install of Fedora trying to boot a BIOS/CSM install of Fedora or some other distro
* ...anything else

into other bugs? Thanks.

I am currently testing the straightforward scenario described: doing a Fedora 20 native UEFI install and then I will do a UEFI native install of Fedora 21 Final TC1 on top of it.

Adam, that not only works for me but is a very reasonable approach: Fedora bios multiboot OR Fedora efi multiboot plus multiboot for Windows/MacOS as best as can be done and certainly not involving any linux16 stuff.

IMHO, multibooting other distributions is still likely something a few folks will want to do but the qemu-kvm technology is (IMHO) a better approach to bring up such systems.  The few who want to multiboot other distributions on real hardware will likely also have the expertise to create something in 40_custom to do the booting.

So, my opinion is that the current changes for os-prober and grub2 will address the problem.

So here's my test results:

If your previous install is on LVM, do not pass Go, because you'll just hit https://bugzilla.redhat.com/show_bug.cgi?id=825236 - the existing install won't be detected at all.

Testing with a UEFI install of F20 to raw partitions, then a UEFI F21 Beta TC1 network install alongside it, the existing install is detected and an entry (and an 'Advanced options' menu) for it is written to the grub config. However, it doesn't boot, because it uses 'linux / initrd', not 'linuxefi / initrdefi'. i.e., you get exactly the result cmurf got way back when he filed this.

Both the installed system and the 21 Final TC1 tree have grub2-efi-2.02-0.10 , which is the build with the change:

"Try to emit linux16/initd16 and linuxefi/initrdefi when appropriate in 30_os-prober."

so it seems like this *should* have been fixed, but somehow the fix isn't working.

Ah. I see why.

30_os-prober.in tries to use $machine but never defines it. It looks like the test was cribbed from 10_linux.in or 20_linux_xen.in , but those both define machine before using it, with:

machine=`uname -m`

So this should fix it:

--- grub-efi-2.02~beta2/util/grub.d/30_os-prober.in	2014-11-07 12:14:12.125401566 -0800
+++ grub-efi-2.02~beta2/util/grub.d/30_os-prober.in.new	2014-11-07 12:21:45.039047781 -0800
@@ -138,6 +138,7 @@
   sixteenbit=""
   linuxefi="linux"
   initrdefi="initrd"
+  machine=`uname -m`
   case "$machine" in
     i?86|x86_64)
       sixteenbit="16"

Confirmed that if I edit that directly on the test VM and run 'grub2-mkconfig', the config file it produces has 'linuxefi' and 'initrdefi', not 'linux' and 'initrd'.

grub2-2.02-0.11.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/grub2-2.02-0.11.fc21

Package grub2-2.02-0.11.fc21:
* should fix your issue,
* was pushed to the Fedora 21 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing grub2-2.02-0.11.fc21'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2014-14556/grub2-2.02-0.11.fc21
then log in and leave karma (feedback).

grub2-2.02-0.11.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.