The problem is with lines 83-93 in http://pkgs.fedoraproject.org/cgit/grub2.git/tree/0073-blscfg-add-blscfg-module-to-parse-Boot-Loader-Specif.patch

+#ifdef GRUB_MACHINE_EFI
+#define GRUB_LINUX_CMD "linuxefi"
+#define GRUB_INITRD_CMD "initrdefi"
+#define GRUB_BLS_CONFIG_PATH "/EFI/fedora/loader/entries/"
+#define GRUB_BOOT_DEVICE "($boot)"
+#else
+#define GRUB_LINUX_CMD "linux"
+#define GRUB_INITRD_CMD "initrd"
+#define GRUB_BLS_CONFIG_PATH "/loader/entries/"
+#define GRUB_BOOT_DEVICE "($root)"
+#endif

Ostree doesn't distinguish between UEFI and BIOS, nor should it. I don't see the merit in having two locations for bootloader configuration files, but even if there is, the sensible thing to do is for bls_import to look in both locations. Only looking for configuration files on the ESP breaks valid workflows that currently work on BIOS (resilient multiple device booting).

OK no matter what both of these deviate from the BootLoaderSpec. On UEFI /loader is supposed to go at the root of the fs, not in /EFI. And on BIOS, /loader is supposed to go on an 0xEA FAT16/32 formatted $BOOT, not on the typical ext4 /boot. So if we're going to deviate willy nilly from the BLS, I think we should deviate in a simplifying manner by going with one /loader location regardless of the firmware type, and that would be on an ext4 /boot. 

See bug 1100938 for Btrfs. In that case it might be simpler for /loader to be a Btrfs subvolume in the top level of the file system. That way /loader is in the same location regardless of firmware and filesystems.

(In reply to Chris Murphy from comment #2)
> OK no matter what both of these deviate from the BootLoaderSpec. On UEFI
> /loader is supposed to go at the root of the fs, not in /EFI. And on BIOS,
> /loader is supposed to go on an 0xEA FAT16/32 formatted $BOOT, not on the
> typical ext4 /boot. 

For OSTree, /boot can't be FAT because the entire model is atomic updates via swapping a symlink.

See: https://bugzilla.gnome.org/show_bug.cgi?id=724246

I know for EFI I'll need to handle the FAT case, so we could view the current situation as transient - and attempt to converge on /boot being FAT.

> So if we're going to deviate willy nilly from the BLS, I
> think we should deviate in a simplifying manner by going with one /loader
> location regardless of the firmware type, and that would be on an ext4
> /boot. 

Where "ext4" here is a standin for "native Linux fs", or possibly "native Linux fs on LVM", right?

Maybe we need to support the entire matrix, which is UEFI, BIOS + FAT /boot, BIOS + Unix /boot, BIOS + BTRFS subvol /boot.

> See bug 1100938 for Btrfs. In that case it might be simpler for /loader to
> be a Btrfs subvolume in the top level of the file system. That way /loader
> is in the same location regardless of firmware and filesystems.

Hmm...let's discuss that in that bug.

(In reply to Colin Walters from comment #3)

> I know for EFI I'll need to handle the FAT case, so we could view >the current situation as transient - and attempt to converge on /boot >being FAT.

I'll argue you should instead look at either the bootloader or a single static bootloader configuration file on the ESP, pointing to the actual location for dynamic bootloader configuration files which are not on the ESP; rather than to enhance the bad idea of dynamically changing config files being on the ESP. No other OS does this, and for good reason: fragility, and it breaks a legit workflow that work today on BIOS systems: resilient multiple device booting.

Alternatively a Rube Goldberg method of rsyncing the mounted ESP with the (normally) unmounted ESPs.

I'd rather see all the dynamic configuration files in the same location regardless of firmware. There is a singular argument in favor of configuration files on the ESP, and it merely is to legitimize the world's only bootloader that doesn't understand any file systems, instead relying on what the firmware itself can read.

> Where "ext4" here is a standin for "native Linux fs", or possibly >"native Linux fs on LVM", right?

No I literally mean ext4 since all Linux bootloaders except one can read it. Whereas if you put /boot on ext4 on LVM, only one bootloader supports this, GRUB2, and only if it's conventional LVM LV's, it doesn't yet support thinp LVs, and I don't think it supports multiple device LVs (raid0,1,5,6).

XFS support I think is coming to syslinux or extlinux, I'm not sure where that work is at. Btrfs might be supportable because both GRUB2 and extlinux support it; although only GRUB2 supports multiple devices (btrfs raid0,1,10 and single).


> 
> Maybe we need to support the entire matrix, which is UEFI, BIOS + FAT /boot,
> BIOS + Unix /boot, BIOS + BTRFS subvol /boot.

Or simplify the matrix, and update the BootLoaderSpec to reflect reality, seeing as there's no good reason for this to be so different just because of firmware and file systems. We're already deviating from the spec for good reasons. I think the spec should be changed, rather than neither following it, nor updating it.

(In reply to Chris Murphy from comment #4)
> 
> I'll argue you should instead look at either the bootloader or a single
> static bootloader configuration file on the ESP, pointing to the actual
> location for dynamic bootloader configuration files which are not on the
> ESP; rather than to enhance the bad idea of dynamically changing config
> files being on the ESP. No other OS does this, and for good reason:
> fragility, and it breaks a legit workflow that work today on BIOS systems:
> resilient multiple device booting.

I don't understand your proposal exactly (bear with me due to my limited understanding of EFI) - but when a new kernel appears, I need to update the bootloader configuration for it, right? 

What you're suggesting is basically bridging the UEFI model "backwards" to the BIOS-like world where bootloaders parse filesystems, as I understand it.  I thought a major advantage of UEFI was getting away from this.

I don't have a UEFI machine handy, though I clearly need to get one...

> No I literally mean ext4 since all Linux bootloaders except one can read it.

Ok, I see that argument.  On the other hand, keep in mind RHEL7 will ship with XFS as default for /boot.

> Whereas if you put /boot on ext4 on LVM, only one bootloader supports this,
> GRUB2, and only if it's conventional LVM LV's, it doesn't yet support thinp
> LVs, and I don't think it supports multiple device LVs (raid0,1,5,6).

I think there is a valid use case for having /boot on raid 1, it seems easiest to support as well.  (I'm not an expert in this area though)

> XFS support I think is coming to syslinux or extlinux, I'm not sure where
> that work is at.

extlinux 6.x supports it supposedly.


> Btrfs might be supportable because both GRUB2 and extlinux
> support it; although only GRUB2 supports multiple devices (btrfs raid0,1,10
> and single).
> 
> 
> > 
> > Maybe we need to support the entire matrix, which is UEFI, BIOS + FAT /boot,
> > BIOS + Unix /boot, BIOS + BTRFS subvol /boot.
> 
> Or simplify the matrix, and update the BootLoaderSpec to reflect reality,
> seeing as there's no good reason for this to be so different just because of
> firmware and file systems. We're already deviating from the spec for good
> reasons. I think the spec should be changed, rather than neither following
> it, nor updating it.

all of this discussion is focused on x86, do not forget we need to ensure that we can support many other arches.

(In reply to Dennis Gilmore from comment #6)
> all of this discussion is focused on x86, do not forget we need to ensure
> that we can support many other arches.

that should be that we need to support many other arches.

(In reply to Colin Walters from comment #5) 
> I don't understand your proposal exactly (bear with me due to my
> limited understanding of EFI) - but when a new kernel appears, I
> need to update the bootloader configuration for it, right? 

Yes. No different than BIOS. And I'm arguing regardless of arch, firmware, and file system used, those conf files should go in one location. At the moment, /boot/loader/entries seems reasonable to me, it's effectively the same location as /boot/grub which has served us reasonably well for what, over a decade? Why move them to the ESP?

> What you're suggesting is basically bridging the UEFI model
> "backwards" to the BIOS-like world where bootloaders parse
> filesystems, as I understand it.

I'm suggesting today we have /boot on ext/xfs/btrfs, and moving /boot to FAT as the BLS proposes is actually moving backwards to 1984. I suggest using the EFI System partition, and FAT, as little as possible.

> I thought a major advantage of UEFI was getting away from this.

I don't know why. Step 1 for the Windows UEFI bootloader is learning to read NTFS, and the rest of the boot process is done from an NTFS volume. Step 0 for Apple was to bake HFS+ support directly in the firmware, so they don't use the EFI System partition for booting at all.

The question is, what's the advantage of /boot on FAT? You have more work to do in order to support that, is what you said in comment 3. 

Because Windows systems come with a 100MB ESP, the BLS mandates the Linux installer create a defacto 2nd ESP, an "Extended Boot Partition" which serves as /boot and is likewise FAT. Imagine explaining this to users. It's more complicated than Windows or OS X booting, for what? Why?

All Fedora supported bootloaders (grub2, yaboot, extlinux) now support at least ext4, if not also xfs and btrfs. Why should that work be discarded on UEFI by mandating /boot on FAT only?


> I don't have a UEFI machine handy, though I clearly need to get one...

*shrug* The same dollar value of Hendricks will have greater efficacy, and far less deletrious effects.


>... RHEL7 will ship with XFS as default for /boot.

Yes, it was a factor in making it default also for Fedora Server. So again, why are we seriously entertaining the BLS when it comes to regressing to /boot on FAT? Change the spec. Or fork it, the conf file format is really the only useful part of it that I can see.


> I think there is a valid use case for having /boot on raid 1,
> it seems easiest to support as well.

I agree there is a valid use case here. On BIOS systems it works today by default, automatically, without user intervention, including degraded boot. Best practices requires a bootloader like GRUB2 that understands mdraid 1.x metadata on the /boot partition. It assembles the array first, and then the /boot file system appears.

However this breaks on Fedora on UEFI because currently we put the dynamic (changing) grub.cfg on the one and only created ESP of a multiple disk install. So it's a single point of failure. The ESP can't be raid1, except hardware raid.

Conversely, a static configuration file installed to each ESP of a multiple disk install, which is then never updated and thus always in sync, would enable this workflow on UEFI systems. The static bootloader files each point to the dynamic configuration files located on /boot on mdraid1 (or btrfs raid1 for that matter), and therefore we have the ability to boot degraded. The BLS is incompatible with this too, not just OSTree.

So we have two valid use cases so far that the BLS is not compatible with. So what's the point of /boot on FAT? And what's the point of /loader/entries going on the ESP?

The BIOS equivalent is bug 1100938 comment 4.

This bug appears to have been reported against 'rawhide' during the Fedora 22 development cycle.
Changing version to '22'.

More information and reason for this action is here:
https://fedoraproject.org/wiki/Fedora_Program_Management/HouseKeeping/Fedora22

The discussion here and in bug 1100938 shows a few distinct issues. For now I'd like to get hands on one of them.

(In reply to Chris Murphy from comment #2)
…
> So if we're going to deviate willy nilly from the BLS, I
> think we should deviate in a simplifying manner by going with one /loader
> location regardless of the firmware type, and that would be on an ext4
> /boot. 

I'm fully for this proposal - let's start with aligning the search paths.
Is there anything blocking us from making this change, and have the same search path for BIOS and EFI?

Currently I don't see a reason which would prevent this change.
We probably do not need to be concerned to much about backwards compatibility, because the EFI search path is quite specific to Fedora - and how many users on Fedora are using EFI with bls?

My goal is to get BLS propperly working with anaconda's autopart logic on BIOS and EFI.

I'm not thinking of anything that would block this. 

However, it's confusing that upstream bootloaderspec requires FAT, whereas this implementation presumably supports anything the bootloader supports; and it's also confusing that the grub bootloader configuration file will still differ in path grub2/grub.cfg on BIOS and EFI/fedora/grub.cfg on UEFI (bug 1048999).

I don't know why Colin abandoned fixing the GRUB blscfg.module, and instead went with src/boot/grub2-15_ostree and ostree/src/libostree/ostree-bootloader-grub2.c to integrate bls snippets into the grub.cfg.

I think the Anaconda development list would be the best place to discuss high level architecture - then we can return to bugzilla for details.

See https://www.redhat.com/archives/anaconda-devel-list/2014-July/msg00002.html

This bug appears to have been reported against 'rawhide' during the Fedora 24 development cycle.
Changing version to '24'.

More information and reason for this action is here:
https://fedoraproject.org/wiki/Fedora_Program_Management/HouseKeeping/Fedora24#Rawhide_Rebase

This message is a reminder that Fedora 24 is nearing its end of life.
Approximately 2 (two) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 24. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '24'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 24 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Fedora 24 changed to end-of-life (EOL) status on 2017-08-08. Fedora 24 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.