Bug 1022316 - RFE: always create required bootloader partitions in custom partitioning
RFE: always create required bootloader partitions in custom partitioning
Status: CLOSED NOTABUG
Product: Fedora
Classification: Fedora
Component: anaconda (Show other bugs)
rawhide
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Anaconda Maintenance Team
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-10-22 23:41 EDT by Chris Murphy
Modified: 2015-06-30 11:14 EDT (History)
11 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-11-10 21:12:22 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Chris Murphy 2013-10-22 23:41:24 EDT
Description of problem: Users routinely forget, or don't know, that BIOS boot or EFI System paritions are required on GPT disks. (Let alone the proposed 0xEA boot partition for MBR disks.) We should always create the required partition, using sane size defaults, if they aren't already created.

Optionally, allow the user to still create them in the UI, and only automatically create them if they don't.

Expected results:

For BIOSboot of 1MB, it doesn't matter where this space is pilfered from. Take it from /boot or /.

For EFI System Partition, it probably needs to be pilfered from /.


Additional info:
No, I do not think the user needs to be informed this is going to be done for them.
Comment 1 Gene Czarcinski 2013-10-27 20:16:57 EDT
I agree. Yes, I know that in kickstart there is the part biosboot --fsbiosboot but why not do this "automagically" if there is a clearpart --initlabel ?
Comment 2 Chris Murphy 2014-02-02 14:13:58 EST
Related bug 1060576, lack of ESPs on each chosen physical device breaks bootable md raid1 on UEFI.
Comment 3 Andy Lutomirski 2014-02-04 14:00:20 EST
IMO we should create the ESP and BIOSboot partitions even if they're *not* required.  That is, a BIOS install should still create ESP and a UEFI install should still create BIOSboot.  This costs very little and will make it much easier to switch back and forth if needed (ugh!).
Comment 4 Chris Murphy 2014-02-04 15:40:43 EST
(In reply to Andy Lutomirski from comment #3)
I think that pollutes this bug with something completely different and off topic, please file a new RFE for that.
Comment 5 Adam Williamson 2014-02-04 16:28:01 EST
I don't like this idea precisely as stated - I think trying to create the partitions silently without bugging the user, as I think Chris' proposal is, would be complex and fragile.

But, how about this? When entering custom partitioning we could pre-configure the appropriate special partitions into the 'new Fedora installation' layout. We don't try and do anything silently, or on *exit* of the spoke, and we don't try to 'protect' the partitions or something. But I don't think it adds too much complexity to current paths to just pre-configure the partitions in the layout when you enter custom partitioning. We already even have logic somewhere for re-using an existing ESP when doing autopart, so we could do that too.

Basically this would be automatically running a restricted version of the autopart algorithm immediately upon entering custom part, which only created partitions required for stage1 bootloader purposes, not /boot and swap and / and /home.
Comment 6 Chris Murphy 2014-02-04 17:21:22 EST
I don't see how silently always creating (or reuse) an ESP/BiosBoot on every chosen drive is complex. Since no UI or notification or translation is needed, it simplifies code significantly.

In your proposal, we now need some coherent UI to represent the ESPs for each drive with bootable raid, seeing as only one of them will be mounted.

The additional code implied is copying the contents of /boot/efi to the other ESPs so that they are in fact bootable disks.
Comment 7 Adam Williamson 2014-02-04 17:24:40 EST
well, this bug is kind of being pulled in two directions. You're thinking of one particular complex case - 'make anaconda capable of automagically configuring redundant UEFI boot on a RAID-1 set' - but the bug description is much more generic - 'always create required bootloader partitions in custom partitioning' - and I'm honestly rather more interested in the simpler specific cases of that generic idea.

perhaps you could split the RAID-1 stuff off into another report?
Comment 8 Andy Lutomirski 2014-02-04 17:26:05 EST
I like Adam's approach here.

I split out my suggestion into bug 1061478.  Please disregard comment #3 :)
Comment 9 Chris Murphy 2014-02-04 17:41:34 EST
(In reply to Adam Williamson from comment #7)
> perhaps you could split the RAID-1 stuff off into another report?

See comment 2.

As mentioned on devel@ - Windows and OS X always create an EFI System Partition for the user without their say so. Windows does it for any boot drive. And OS X does it on every GPT partitioned drive whether it's for booting or not. There is no user choice, and it's brain dead simple code as a result.

What's making this complex, annoying, and prone to breakage regardless of the number of chosen disks, is giving users choice and feedback where none is needed. Not even an expert user benefits from this verbosity.

It is, however, reasonable to leave the ESP in the partition summary dialog for the sake of completeness, but even if they were MIA I wouldn't complain about it.
Comment 10 Andy Lutomirski 2014-02-04 17:43:21 EST
I think it's not that simple for Fedora.  Either EFI/redhat/grub.conf would have to be automatically synced or it would have to be configured once and never changed.
Comment 11 Chris Murphy 2014-02-04 17:59:02 EST
(In reply to Andy Lutomirski from comment #10)
The latter. See bug 1048999.
Comment 12 Gene Czarcinski 2014-05-11 08:06:03 EDT
NOTE:  Grub2 will not boot off a multi-device BTRFS filesystem if one or more devices is missing.

See https://bugzilla.redhat.com/show_bug.cgi?id=989644#c8
Comment 13 Chris Murphy 2014-11-10 19:43:38 EST
(In reply to bcl@redhat.com from 
https://bugzilla.redhat.com/show_bug.cgi?id=1010495#c57


> Chris, please stop referring to things as 'user hostile'. 

Request denied.
a.) The error message "you have not created a bootloader stage1 target device" is nonsense. It uses archaic and deprecated terminology.

b.) Since the dawn of the BIOS+MBR era, the creation of a bootloader region has never been user domain. The UEFI spec language doesn't make it user domain now either just because there's an explicit partition location for it.

c.) Domain conflation tends to piss off users, and pissing off users is happily and most diplomatically referred to as 'being user hostile'.


>Custom is custom,
> you're expected to know what you are doing.

You're wrong to expect it.

The most recent user having a problem with this did, in fact, know what he was doing. He tried to use the existing EFI System partition. This is, itself, esoteric knowledge that most users of even custom partitioning do not have and cannot be expected to know. The very fact so many non-Mac users have custom installation problems because the didn't create an ESP or BIOSBoot demonstrates your expectations are misaligned with reality.

But this esoteric knowledge wasn't good enough, in the recent case. Because he was also on a Mac. And you're tacitly arguing that, oooh well he should have known that his Mac's firmware doesn't use EFI System partitions on internal drives for booting OS's. Further he should have known that Fedora uses an HFS+ volume as an ESP; that Fedora has come up with a clever yet very non-standard work around for a non-standard firmware behavior. Yes, he should have known this. That's why he got the error message. It's his fault.

And blaming the user is a huge fat fail for a GUI installer. Southpark, you get an F minus, fail.

Who never has a problem? The user of a BIOS+MBR system. Why? Because they don't need to know esoteric things, because they can't make a mistake, they're not allowed to e.g. create partition 1 with a starting LBA of 63. But you're proposing, oh BIOS+GPT or UEFI, and now the burden is on the user, he should know esoteric USELESS information. It's his fault for getting error messages and then not understanding them.

Well you're wrong. This is a domain violation. The ESP, BIOSBoot, HFS+ ESP, these things are not user domain. The user has no business creating them, not in his domain he has a very good chance of doing it wrong anyway.

And BTW, Windows and OS X custom partitioning and installation always create the ESP for the user. So fix it, or don't fix it, but the current behavior is grossly deficient in helping the user with this, and yes I call that user hostile.
Comment 14 Chris Murphy 2014-11-10 22:42:01 EST
Never was a bug, it's flawed design. That's why it was opened as an RFE from the outset.
Comment 15 Nick Coghlan 2015-01-29 00:12:27 EST
I'd like to request that this RFE be reconsidered. From a Beaker perspective, the lack of any ability to tell Anaconda "create all the partitions this particular hardware requires" means that we have to duplicate all this logic in Beaker itself whenever Anaconda starts supporting new hardware. Which means we have to repeat the Anaconda teams research to figure out what those conditions *are*.

It doesn't need to be implicit for our use case though - a "do whatever needs to be done" opt-in feature would suffice. In the interactive installer, that could be represented as an "Automatically create all required partitions" option that is checked by default.
Comment 17 Nick Coghlan 2015-01-29 00:19:23 EST
Additional details on the Beaker use case:

As an automated hardware integration testing system, Beaker is regularly running unattended kickstart installations over a wide range of systems, some of which need particular custom partitions in order to boot correctly.

These systems are handled correctly by autopart, as Anaconda just creates the partitions it needs.

However, as soon as *any* custom partitions are needed for a test, all of the associated platform specific logic (including the knowledge of which kinds of systems need which custom partitions) needs to be duplicated in Beaker's kickstart generation system, entirely breaking the hardware abstraction that Anaconda should be able to provide.

When using custom partitioning, Beaker always creates some standard partitions:

===============
part /boot --size 200 --recommended --asprimary
part / --size 1024 --grow
part swap --recommended
===============

With "/" being configured to use any remaining space not consumed by custom partitions, it seems reasonable that Anaconda could either:

1. Just *make* any required partitions, shrinking "/" accordingly (i.e. the "create required partitions" request would be implicit in the use of the "--grow" option)
2. Provide a separate option or command to request the creation of any required partitions, without having to resort to full automatic partitioning
Comment 18 Nick Coghlan 2015-01-29 00:22:58 EST
The Beaker use case also provides an argument for making it possible to deliberately leave out the required partitions: it means we can test how the installer behaves when they're missing.

So if adding them was made implicit in the use of "--grow", we'd still want a way to opt out for testing purposes.

That suggests that a separate command or option is likely a good approach.
Comment 19 James Beale 2015-04-07 08:44:03 EDT
I think I'm reasonably experienced with ICT. 

However, the auto-install allowed me to create a broken raid 1 in CentOS/RHEL 7 due to not getting the BIOSboot mirrored. The install finished and I had a working OS, but with strange behaviours - hangs, disappearing files, etc. 

I went round and round trying to see what was wrong, eventually deciding the install must have been the culprit.  So I reinstalled, but this time install failed on creating boot.  That led me to the CentOS forum, where I found the answer, and followed the easy how-to and this time I manually created the partitions.

I now have a working raid 1.

I request that this "feature" be reconsidered.  It may not strictly be a "bug", but it is, at best, a very poorly documented part of the install.

I'd like the auto-install to be much more proactive in this aspect, or simply not allow software raid to be set up with it, pointing the user to further documentation and how-tos elsewhere.  This way I would not have been led "up the garden path", and would have saved 3 days of my valuable life!

Thanks for listening!
Comment 20 Adam Williamson 2015-04-07 10:56:55 EDT
Um. I'm not sure how a BIOS boot partition not being mirrored could possibly result in 'disappearing files'. Do you think you could elaborate a bit?
Comment 21 James Beale 2015-04-07 13:19:17 EDT
Thank you very much for your reply. I'm afraid I can't. All I can say is an hour or so after being downloaded into the "Downloads" folder, Google's install .rpm had vanished! And I did check trash. As I said, strange behaviour.  :-)
Comment 22 Chris Murphy 2015-04-07 20:57:48 EDT
(In reply to James Beale from comment #21)
Disappearing files problem isn't bootloader location or code related. The bootloader has long been vacated by the time you're running a DE. The 2nd problem, 3 days of screwing around because of bootloader partitions is the valid part of the complaint. 3 minutes is excessive. There's no good, valid reason to dump the creation of required partitions onto the user.
Comment 23 James Beale 2015-04-08 12:23:42 EDT
Chris - point taken!  The installer should not have allowed me to install a broken raid, let alone then permit me to boot into the OS. I was completely led astray.  No messages in the log meant 3 days of messing around chasing red herrings.  Luckily my years of ICT experience has taught me to question everything when unexpected results appear. Hey-ho!
Comment 24 Adam Williamson 2015-04-08 14:01:31 EDT
You still haven't said anything to suggest there was anything 'broken' about the RAID set. System partitions on a RAID set with a separate non-RAID /boot is a perfectly valid and working configuration; it's what I'm using on this very system. There's no obvious way the issues discussed in this bug could have anything to do with 'hangs' or 'disappearing files'. The symptom of a broken bootloader is that the system doesn't boot (or boots the wrong kernel, or the wrong parameters, or something like that).
Comment 25 James Beale 2015-04-08 16:25:50 EDT
Thanks again for the reply. I may be naïve, but in my book, when something doesn't work, it's broken!

I install: system is unstable = "broken" :-) 

I install again: installer falls over installing bootloader with "unknown bug" error message = "broken" :-) 

I research. Find this: https://www.centos.org/forums/viewtopic.php?f=47&t=49541&p=219909#p210685

It now works = not broken!

Nowhere did installer make any warning or comment or refer me to more information or RedHat how-to.

Q.E.D.
Comment 26 Adam Williamson 2015-04-08 16:41:19 EDT
There are about sixty bajillion moving parts in any given install. Clearly *something* was broken, but nothing about your description suggests that whatever the problem was had anything to do with this bug.

A BIOS boot partition is only of any use when doing BIOS boot on a GPT-labelled disk, and there's only any need for a BIOS boot partition on the disk which gets the stage1 bootloader written to its MBR, and the symptom of not having a BIOS boot partition when it's needed is that the installed system will not boot. The symptom is not 'files disappear' or 'the system hangs after booting'. Once you see the grub menu, the BIOS boot partition has already served its purpose and is of exactly zero relevance to anything else that happens.

The forum thread you linked to is trying to do a relatively specific thing: have /boot mirrored so that the system can boot from *either* drive (i.e. make the system entirely redundant against one disk failing). The stuff it talks about is only necessary if you're trying to do that *and* you're using disks >2TB in size.
Comment 27 James Beale 2015-04-09 13:31:28 EDT
Many thanks for the reply. 

I agree with all you say!  Therefore the installer needs to point the user towards the information they need. Currently it is silent, and allowed me to set up a broken raid 1 on the first install; and on the second attempt it simply fell over.

"Bug" or "feature", or what you will, this "gotcha" exists in the install, and I think it shouldn't.

And in my opinion, raid 1 should be about complete redundancy: the ability to boot from either disk.

Note You need to log in before you can comment on or make changes to this bug.