Sorry for the limited precision, but as discussed in the Matrix chat in #devel, I cannot do a sophisticated report due to lack of time but its worth to report this given that it can have a noteworthy impact in production environments, potentially not reversible in some cases: web-ui breaks /boot and/or grub when other operating systems are preinstalled. Install CentOS in a VM, put / on an LVM with 10G (the LVM has 30GB), encrypt the LVM (encrypt the LVM, not the / file system) add 3 GB /boot, no swap, but it creates also a small biosboot due to the VM environment (this one is shared among all OS later too). Then test (works fine). Install Alma on the same VM, use the same /boot, and add its / partition with another 10GB in the same LVM of centos. no swap. Then test (works fine). Install Rocky on the same VM, use the same /boot, and add its / partition with another 10GB in the same LVM of centos (which then contains centos, rocky and alma in one LVM). no swap. Then test (works fine) -> all three operating systems can be booted, and all work fine. Then, I would refer to the reports of BZ#2439682 and [1] to explain why the result is that I can do only prepare everything in advance and then let anaconda do the "mountpoints only", or let anaconda use all remaining space of two devices and setup things in a "one size fits all way". In most cases, anaconda web-ui crashes while or after setting up the bootloader (a comment about this and the error message at the end of this report). With regards to the report of [1], I found out the "anaconda does mount points only"-way works if I prepare the btrfs in advance, do not allow anaconda to format it, do not split / and /var/ and keep / on one disk (another disk than the rest of the system though: so all OS and /boot partitions are on one disk, and Fedora's / partition on another). This ends up with a successful installation. However, when then booting, all other operationg systems are gone: only Fedora is remaining. Unfortunately, I reverted the VM through a snapshot and forgot in advance to test if only the grub configuration ignores the other systems so that they could be re-added by other means, or if it broke/removed their files. What I have done in details is, when I tried to install Fedora in the final round (that did not crash but lead to a Fedora-only grub/boot): 1) add another 10GB disk to the VM because the remaining 8.2 GB of the original disk were not sufficient for anaconda web-ui 2) create a partition table and then a 10GB partition through gparted. To ensure a default btrfs, I did another `mkfs.btrfs -f` on the resulting partition (I did that from within the Fedora KDE live system in which I later installed everything) 3) Go through web-ui, choose both disks, unlock LUKS, and choose to set mount points manually ("mount point assignment") 4) set the /boot all OS share as /boot, but do NOT format (the same way I did it in the other OS on their installations) 5) set the 10GB btrfs on the second disk as / and also do not format it 6) install with a root account enabled -> successful -> reboot 7) Only Fedora is remaining to choose in grub Test 2: I have done reverted the VM to its earlier state (I had a snapshot in KVM/QEMU after each OS installation), to retest the setup in the exact same constellation (beginning at the time after setting up and testing Rocky): I again repeated step 1) [but skipped 2], but then in 3) I did not set mount points manually, but set anaconda to do all automatically: I unlocked LUKS at first, and I did NOT reclaim further space, but set anaconda to use remaining space and setup stuff itself. What anaconda has done (superficial information imho) can be seen in the screenshot "anaconda-multi-OS-variant_auto-created-partitions.png" attached. It indicates that anaconda web-ui can identify all three other OS. A note of the environment to put in context what web-ui is doing: there was 8.2 GB empty in the first disk, and the 10 GB of the second disk: it put them together into one btrfs with 18.2 GB, which itself makes sense (although it is a strong restriction for users: keep in mind that users cannot even choose to use another file system like xfs or ext4, not to talk of a custom partitioning). However, it creates an additional /boot for Fedora (not touching the older /boot but effectively obsoleted the older /boot given that it is no longer in any use). That said, the outcome of test 2 is the same: There is only the grub with Fedora entries remaining... It does not consider the other three OS. ----- I marked this for now as urgent, as it can break things in a potentially un-revertable way, especially for users with limited knowledge of grub & /boot (as I could not verify if /boot files of other OS were removed or otherwise broke in the outcome of test 1, I cannot say if experts could have reverted the issue in test 1). I know best practices and such, but the fact is many users take for granted that such functions work and do not do the appropriate backups in advance, and some lack the knowledge. Feel free to change the priority though. ----- With regards to the issues elaborated in [1] and partly above (the test in which /var/ and / were split and located on different disks, etc.), the error of the elaborated crash in the first test of using mountpoints took place at the final stages of the installation (the lat time I reviewed the panel it was setting up the bootloader) and the actual error message was: ``` org.fedoraproject.Anaconda.Error: 'NoneType' object has no attribute 'path' ``` -> it might be added that the means to report the detailed information from anaconda crashes to you is time intensive and complex (and the conditions in which it is possible at all are not always given). E.g., something like the KDE crash reports could be implemented: the easier and quicker it is, the more people will report detailed information to you. ----- Another thing I saw in this installation attempt: I cannot remove the user from wheel: old installers allowed me to configure the user account to be not sudo capable. Sorry for squashing again several things in one, but I thought any report is better than none in such cases. [1] https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/FTR6TLSMNVMPMUUODQV2ARXLACC2OODB/ (this does not intend to replace anaconda, but to get an overview of the situation and identify ideas to find any way to tackle or mitigate noteworthy issues around Fedora installation over time: this can mean to consider alternatives to anaconda, backup solutions, but also to find better ways to work on anaconda and improve it, or to increase contribution to it) Reproducible: Always Actual Results: Anaconda either crashes or destroys /boot or grub entries (in one case maybe other systems' /boot files at all), and it leads users in many cases either in limitations or crashes, making Fedora not usable for several use cases Expected Results: None of the actual results :) Additional Information: x86_64. The tests here took place within a KVM/QEMU environment. I used a F43 KDE Spin for Fedora installations. Also, I have used the KDE live variants of CentOS, Rocky and Alma in order to install them respectively!
Created attachment 2130923 [details] anaconda-multi-OS-variant_auto-created-partitions.png See text
Haven't needed a dual/multi boot for a long time (at least 15 years, maybe up to 20:). There was a time in which separate /boot did not work, but it seems now its intended, and while it seems to work with the other OS, my feeling is that today this is not the expected default but to have /boot separated (correct me if I'm wrong). Not sure if I have time to re-test with separated /boot partitions for each OS with Fedora considering multiple OS on installation (at the moment I evaluate mostly with Fedora being the first OS to be installed before the other OS), but if so, I report if and how it differs. I could imagine that is the issue/origin that might be not considered in usual tests.
When I'm curious, I cannot stop :) I tested it (separated /boot partitions for every OS), but this time with the F43 Workstation Web-UI: Unfortunately, the outcome is the same as before -> web-ui does not consider other OS when configuring grub. In this test, I had subsequently installed F43 KDE Spin (web-ui), RockyLinux, AlmaLinux, and now for this test another F43 Workstation (web-ui) as 4th OS. Each OS with its own /boot partition, a common bootbios, and each its own / and /home within a large encrypted LVM, whereas I had to create a second LVM for the 4th OS. All worked out fine, up to the point I installed the F43 Workstation (I always tested all installed OS after each installation). I just skimmed roughly over the files of all the /boot partitions, they seem to be not touched by web-ui at first glance. But once Web-UI is done, only Fedora GRUB entries are left. So the user has to repair that in the aftermath. I'll reset the F43 Workstation installation tomorrow, and beginning from the KVM/QEMU snapshot of before the F43 Workstation web-ui multi-boot installation attempt (so after the AlmaLinux installation was done), I'll try again, then with CentOS as 4th OS. Assuming the latter works out as AlmaLinux and RockyLinux, I will not report again but assume it is a web-ui issue as reported. If CentOS breaks the same as F43 Workstation, I'll add another post here tomorrow, as that would allow the possibility its not web-ui specific. But so far, the non-web-ui installations worked in such a multiboot setting. Hope that information is somehow helpful.
It seems CentOS KDE ISO has the same issues as Fedora web-ui. I wanted to ensure the issue is not the 4th OS (being a too high number or so) or the second disk. So I set back the snapshot to the time of the Rocky installation (which already included Fedora) and instead of repeating the Alma test as earlier, I used the CentOS KDE ISO to exclude the issue was the second disk or being the 4th OS. But the issue remains: CentOS KDE breaks grub's multiboot options the same way as Fedora web-ui. As I used CentOS as first system in the 1st test (see above), this fact remained hidden. So I can use Fedora and CentOS both only as the first OS, which means I cannot use both at once. I did not repeat this test with CentOS/Fedora being the second OS in this constellation. I tested the constellations with a second (virtual) disk both with having the 4th OS /boot in sda and sdb. I wonder about the different behavior between CentOS and Rocky/Alma. I might do some more testing about that, and report if something turns out to be relevant for the Fedora web-ui issue. Let me know if you have any questions or so.
I forgot, to avoid any issue/impact of any of the OS being not able to mount BTRFS, I did not use BTRFS for any partition in any of the above tests, but used only ext4 and xfs as file systems.
As per the reporter's validation, this issue is reproducible on CentOS systems that do not use the WebUI. Therefore, I am moving this to the Anaconda project for further investigation.
I could also reproduce it with Fedora 43 Kinoite, which also does not use web-ui (I tested only the last condition with Kinoite though). The wheel issue, and the `org.fedoraproject.Anaconda.Error: 'NoneType' object has no attribute 'path'` error, both apply only to web-ui though.
The ` 'Nonetype' object `crash has been reported multiple dozens of times for F43 [1]. This bug was very unfortunate that slipped in, it affected a big amount of users. [1] https://bugzilla.redhat.com/show_bug.cgi?id=2394554
I repeated the test with a new VM, but setting up the VM with UEFI and not BIOS in order to ensure this is not a legacy-like issue. I repeated the same test as before, installing subsequently: F43 KDE (with web-ui), rocky (no web-ui), alma (no web-ui), centos (no-webui). Again, it worked fine until the last one: centos broke the grub entries again. I reverted the VM to the time after rocky was installed but before alma was installed: I installed then again centos (beginning at the very same condition in which alma worked out fine) -> again, centos broke grub. I then reverted again to the time after rocky but before alma was installed, and repeated the test once again but not with centos but with Fedora 43 Workstation (web-ui). Again, it broke grub. I used separate /boot for every OS, but a joint EFI partition mounted on each OS as /boot/efi. All on one virtual disk (fwiw, gpt). I used again a pre-setup LVM (reasons elaborated earlier), which was encrypted with cryptsetup, with a / and separated /home for every OS as logical volume each, but with all /boot partitions (and the one efi partition) being outside the LVM. I used this time ext4 for all /boot (all /boot as ext4 being a difference to earlier tests which I consider not relevant for the test results) and for Fedora's / and /home also ext4, but xfs for the other OS / and /home partitions. I am thinking if that is even worth to be worked on (in terms of: can we get this stable?), or if it makes more sense for the time being to add a warning that multi-Linux-boot is not supported?
I can verify that the UEFI choices (once enabling them to be shown on boot or triggering them through the "UEFI settings" choice in respective grub options) of all OS work fine in all cases: even in constellations in which CentOS or so break grub entries (obviously there are many grub configs on different /boot each OS creating for themselves), if I go through the UEFI choices, it always shows all OS and reliably triggers their respective grub, even if many are installed. The UEFI choices seem therefore reliable. I also tried a test with a Kinoite as final OS (non-web-ui, but did not add other OS to its grub boot entries like CentOS in earlier test), but anaconda always crashed* when installing Kinoite (based on an earlier experience, I assume this is related to the constellation of partitions*) and when I tried to submit a report with a bugzilla API key, that crashed as well at the moment I clicked on submit after entering the api key (no time for another project, sorry:). ----- Since anaconda also wants to attract users with limited experience, this is not straightforward because many (if not most) UEFI do not ask by default on boot, and not every user can link such an issue to the option "UEFI settings": this needs to be triggered or permanently enabled by the user. So people might end up in GRUB and think their other OS is/are gone. Adjusting grub is also possible but the users need to know and get this customized themselves. So a possibility remains to not do technical adjustments but to only add warnings/messages to the user: if bios + multi-OS boot = "issues with grub likely, it's your responsibility to get the different grub configs of different /boot partitions aligned after each update" / if uefi + multi-OS boot = "don't rely on grub but ensure to make choices with UEFI and enable its options on boot or through UEFI settings option" -> something like that. ----- Further: I maybe found a correlation between CentOS and Fedora that COULD be the cause of these two breaking/ignoring other OS in their grub entries: both are configured by default to not show the grub menu at all but only if the system did not boot/shutdown properly. Given the reasoning of their use case of grub menus (= only in emergencies / to mitigate failure), I assume no one considered the use case of the grub menu in a multi-OS setting. I am actually not sure if the CentOS / Fedora variant is more useful given that the UEFI menu is the only reliable one for choosing different OS, and then it is less confusing for users if the very grub configuration instance the UEFI is triggering is focused on only the one OS that has been chosen rather then offering multiple OS a second time. ----- I end my testing with this. Feel free to ask questions if there are some or if there is something specific I shall test. Feel free to close the ticket if you want :) ----- * I tested the kinoite in the UEFI variant as the other OS with a common EFI partition as /boot/efi in each OS, and a separated /boot for each OS. I tried the Kinoite installation with this three times, always crashing. Beyond the boot-related partitions that were constant among the three tests, these are the different partition constellations of the three tests: 1) / and /home (on the given dm-crypt-encrypted LVM) 2) / and /var/home (on the given dm-crypt-encrypted LVM) 3) / and /var/home (on the given dm-crypt-encrypted LVM) and on a new device vdb with 5 GB a /var/ outside the LVM (setup with partition table and formatted by anaconda itself) No btrfs used, only ext4.