Description of problem: Currently, grubby doesn't handle submenus well, which means that snapshot entries cannot be added by snapper in a way that won't have things break when new kernels are installed. Consequently, Fedora's import of the SUSE patches includes a patch to drop the boot to snapshot menu[1]. This functionality needs to be properly supported in grubby. [1]: https://github.com/vathpela/grub2-fedora/commit/64bc21fd172aaced86e283451037e378e410e9c4 Version-Release number of selected component (if applicable): 8.40-4.fc26
This bug appears to have been reported against 'rawhide' during the Fedora 27 development cycle. Changing version to '27'.
This message is a reminder that Fedora 27 is nearing its end of life. On 2018-Nov-30 Fedora will stop maintaining and issuing updates for Fedora 27. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '27'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 27 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
This bug appears to have been reported against 'rawhide' during the Fedora 30 development cycle. Changing version to '30.
*** Bug 579500 has been marked as a duplicate of this bug. ***
Changing to grub2 as grubby is deprecated; and also grub2 is what properly supports and must parse for Btrfs snapshots; and whatever solution should be compatible with the F30 bootloaderspec feature change. https://fedoraproject.org/wiki/Releases/30/ChangeSet#Make_BootLoaderSpec-style_configuration_files_the_default
This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle. Changing version to '31'.
This bug appears to have been reported against 'rawhide' during the Fedora 32 development cycle. Changing version to 32.
Neal, what else is needed for this bug?
I think the last time I talked to Peter about this, he said I need a good way to populate this with Bootloader Spec-based setup, but I don't know what I'm supposed to do here. Help, I guess?
I think the SUSE code for this in GRUB2 dynamically discovers the snapshots. Needing to make that BLS based I think is suboptimal. The difficulty with dynamic discovery is it requires standardized (predictable) layouts; where BLS can describe arbitrary layouts. Plus, there's a fork in BLS. There's the upstream spec and interface. And there's the Fedora macro variant. Two central features of the original BLS: shared $BOOT among distros; all relevant boot information is completely contained in BLS snippets, have been dropped from Fedora's macro variant. For example: The sysroot is defined by boot parameter 'root=UUID=<fsuuid>' and additionally for Btrfs 'rootflags=subvol=root'. But neither are found in Fedora BLS snippets, they are found in grubenv, and pulled in by each BLS snippets' $kernelopts macro. Since a btrfs snapshot is a (pre-populated) subvolume, something needs to specify the 'rootflag=subvol=' parameter somehow; masking the one pulled in from grubenv and substituting a dynamically determined one. Alternative is investigating unwinding the Fedora macro stuff, and adopting the original upstream approach; and then having e.g. snapper, write out BLS snippets. This is a superficial evaluation, I haven't really dug through the full extent of either one. It's kinda complicated. And I'm not really sure there's enough desire or commitment to have a shared $BOOT anyway. The various distros seem pretty hostile toward each other. And also the multiboot complexities are kinda crazy. But as a consequence, we can't even support Fedora n and Fedora n+1; or even Fedora n + CentOS n, on the same computer. So unfortunately the evaluation has a much bigger scope and consequence than just Btrfs snapshot support; and I think you'd wanna know the big picture plan before building something that might later get ripped up.
> > Two central features of the original BLS: shared $BOOT among distros; all > relevant boot information is completely contained in BLS snippets, have been > dropped from Fedora's macro variant. > Coincidentally we were discussing exactly this with Peter and Hans a few days ago. And we came to the conclusion that the shared kernel cmdline as a GRUB environment variable caused more harm than good. The goal was that it would help to: a) make adding a new boot entry just dropping a BLS fhile that's shipped by the kernel package b) change the cmdline for all the boot entries without needing to iterate over all the BLS snippets and update them. But as you said this also had disadvantages, since it was not compatible with sd-boot and also relied in grubenv that's quite fragile (there were bugs about grubenv getting corrupted, users deleting it, etc). And last but not least is the issue that you mentioned, that having it as an env var doesn't allow to have two Fedora installations on the same computer (on EFI or legacy BIOS if the boot partition is shared). At the end we came to the conclusion that having the cmdline as an env var is not the best trade-off and instead the cmdline should be in the BLS itself. The goal is to have this new behaviour from F33 onwards. About your other point that $BOOT should be shared among distros according to the spec, this change will also allow that but in practice I don't think matters that much since the other distros are not using the BLS spec anyways (I think Endless OS is the only non-Fedora derived distro that uses it). Would having the cmdline in the BLS and not in grubenv be enough to support $SUBJECT? The idea is for the grub2 kernel-install plugin to do the same than the plugin for sd-boot does and get the cmdline from the following paths (sorted by precedence): /etc/kernel/cmdline /usr/lib/kernel/cmdline /proc/cmdline So it just updating the cmdline with the rootflags=subvol= for the new subvolumes. For having /boot in a subvolume, that's already supported since the grub2 kernel-install plugin calls grub2-mkrelpath to get the real path of the kernel and initrd images and replaces that in the BLS snippets. Because GRUB can only access the files from the top level btrfs subvolume. I'll reach you both for more details on how this all should work since I'm not that familiar with btrfs.
I came here to file a bug related to this, and found this existing thread. This would seem to be even more important now that Btrfs is now the default filesystem. It is a problem that GRUB includes rootflags=subvol=root by default. When I use snapper to rollback to a prior snapshot, it will attempt to boot into the subvolume related to the rollback snapshot. GRUB overrides this with the latter argument, enforcing boot into the top-level root snapshot against the user's wishes. The strange thing is, snapper updates the default Btrfs subvolume and GRUB is aware of default Btrfs subvolume. Simply removing the above argument fixes this issue. This makes me wonder - why is it there in the first place? It creates problems for normal use cases (snapshot rollback) without seeming to be necessary. Having said that, in my experience of a fresh Fedora 32 install with Btrfs, the default subvolume had not been set during install. I removed the above argument, rebooted and ran into problems. I had to run the 'btrfs sub set-default' command before it worked. This would be easy to fix though, by simply adding that command to the install process.
The installer doesn't currently support setting default subvolumes, and GRUB2 doesn't yet either. Since things have worked this way for many years reliably the decision was to keep things the same until there's a specific plan, and it's not certain how the intended sysroot subvolume will be discovered. There's no analog for default subvolume elsewhere. So all of this involves many pieces: BootLoaderSpec, Discoverable Partitions Spec, whether /boot goes on Btrfs, substitute for grubenv for GRUB hidden menu feature, snapshot and rollback design, etc. Significant consideration goes to having the least amount of churn for both users and developers. Thanks.
Submenus are supported with BLS stuff, when using the Boom snapshot boot configuration manager. From this perspective, I consider this ticket resolved.