Bug 1010495

*** Bug 979205 has been marked as a duplicate of this bug. ***

Proposing as beta blocker: When using the guided partitioning flow, the installer must be able to cleanly install to a disk with a valid ms-dos or gpt disk label and partition table which contains existing data and sufficient unpartitioned space for a Fedora installation, and complete an installation using any combination of disk configuration options it allows the user to select.

Created attachment 800749 [details]
program.log

Created attachment 800750 [details]
anaconda.log

Created attachment 800751 [details]
storage.log

Chris, why did you close the previous bug?

As it stands, this will not be fixed in F20.

The solution involves setting a new GUID on the HFS+ ESP that Anaconda creates so that we can differentiate between it and the Apple HFS+ root partition. I have code for the parted part of this in rawhide, but given the level of change it would involve I don't want to change things for F20 at this point. I haven't done any work on the Anaconda/blivet part of things yet.

I closed it as a dupe of this one a.) because the one for 19 obviously isn't ever going to be fixed, and b.) I wasn't sure about the effect of flipping it from 19 to 20 with the whiteboard filled in.

What GUID are you proposing? It seems doubtful either the firmware or Startup disk is going to display a volume for a partition type GUID it doesn't recognize. And if it's going to be the GUID for Apple Boot, then how is this differentiated from the existing Apple Boot already on every Mac?

On the Mac side things already work, mactel-boot sets up the HFS+ ESP to look like a fake boot volume so that the icons show up. This is what you get when you start from scratch.

The problem is when we do another install Anaconda has no way to tell if the HFS+ volume is the Apple disk or the one it created. So I made up a new partition type GUID (47CB5633-7E3E-408B-B7B8-2D915B7B21B1) so that we can test for it. You can experiment with the parted side of things using  the version from rawhide and setting the hfs_esp flag on a partition.

On a working Fedora 19 install, I installed parted-3.1-16.fc21.x86_64.rpm, and used parted to set the hfs_esp flag on the Fedora ESP, and rebooted. The system is no longer booting by default. When I reboot using the option key, Fedora is not a listed option. When I reboot OS X, in the Startup disk panel, Fedora is not an option. 

When I use gdisk to restore the Fedora ESP to a partition type GUID of 48465300-0000-11AA-AA11-00306543ECAC (AF00 in gdisk), the Fedora system is once again bootable by default, visible with the option key boot manager, and the OS X startup disk panel.

So the proposal doesn't seem like it's going to work, despite a test sample size of one, there's every reason to believe other Apple EFI firmwares behave the same way and will only show volume with partition type codes of:

426F6F74-0000-11AA-AA11-00306543ECAC
48465300-0000-11AA-AA11-00306543ECAC
EBD0A0A2-B9E5-4433-87C0-68B6B72699C7

Although that last one, for all I know, will cause the firmware to assume it's Windows and thus subject to booting via the CSM.

The Fedora HFS+ ESP is consumed by the firmware if the type code is either:
426F6F74-0000-11AA-AA11-00306543ECAC
48465300-0000-11AA-AA11-00306543ECAC

Meaning if it's set to either of those, Fedora boots. If it's set to anything else, it doesn't, including EBD0A0A2-B9E5-4433-87C0-68B6B72699C7. But it's a momentary brain lapse on my part forgetting this is expected. The firmware trigger for CSM booting is finding code in the MBR bootstrap region, and a hybrid MBR (2 or more entries), with any partition other than 1 with the active flag set. Not a partition type GUID.

The test machine is a MacbookPro 4,1 circa 2008.

Another test machine, MacBookPro8,2 circa 2011, also with working Fedora 19, is broken upon using parted-3.1-16.fc21 to set the hfs_esp flag on the Fedora ESP. Bootablity is restored by resorting the former partition type GUID.

Crap. Thanks for testing. That's going to make it hard to detect previous installs then. We could write something to our EFS that we can check for, but that means we have to mount them first instead of just checking GUID.

Yes, and to look for likely suspects you could discount >20GB partitions from being mounted and parsed, because those are unlikely to be what you're looking for, most likely to be an installed OS X system or user data volume.

I'm fundamentally opposed, even in Manual Partitioning, to the idea of users being responsible for creating required partitions like BIOSBoot and the ESP. I think the installer should just do that, and use a fixed size that would further allow you to use that as a check for likely, rather than unlikely, suspects. It would also eliminate hundreds of these really unhelpful "you have not created" error messages.

Another idea is to just set a label on it. Something like 'Fedora ESP'

Another thing you can do is set the partition name field in the GPT, and use that as an early search method. If still present (user hasn't renamed it), the search is faster. If nothing is found, then more searching is required, up to possibly mounting HFS+ file systems and searching for a key of some sort (e.g. a text file with hfs_esp GUID in it). So in order, something like this:

1. Search by partition type GUID + partition name.
2. Search by partition type GUID + fixed size.
3. Search by partition type GUID + size range (less than 20G).

When a search method gets a hit, mount that. If the mount is forced RO, which happens if it's a journaled HFS+ file system, then you know it's not what you're looking for, since you're creating non-journaled HFS+ for the ESP as journaled HFS+ is still not supported on linux for RW.

(In reply to Brian C. Lane from comment #14)
Yes but technically the partition name is user domain, so it's fair for it to be changed by the user, meaning you still need a search fallback if it's not found.

You could use the Apple Boot partition type GUID:
426F6F74-0000-11AA-AA11-00306543ECAC. 

Rather than the general HFS+/HFSJ/HFSX partitino type GUID:
48465300-0000-11AA-AA11-00306543ECAC

The occurrence of Apple Boot partition type GUID will be less common than the general partition type, and in a sense it's more appropriate anyway. If that seems reasonable let me know and I'll explore possible negative side effects of using it.

Discussed in 2013-09-25 Blocker Review Meeting [1]. There's a question as to whether this affects only systems with two existing ESPs, or any system with an existing EFI OS installation. We will establish this and re-visit the bug next time.

[1]

Here is the link for [1] in the ^^^ comment: http://meetbot.fedoraproject.org/fedora-blocker-review/2013-09-25/

(In reply to Mike Ruckman from comment #17)
It always occurs using Guided Partitioning regardless of whether there are 0, 1 or 2 or more ESPs. The only way to use Guided Partitioning on Macs is:
a.) There is no partition scheme on the disk, or
b.) All existing partitions are deleted either individually one at a time, or using Delete All button, or clicking on the disk device and clicking Delete (in reclaim space partition listing).

If the existing GPT is preserved with 1+ partitions, this bug is triggered.

FWIW, this is an autopartitioning bug which occurs pretty much always in the Guided path, but is avoidable in the Manual partitioning path if the user does NOT click on the blue link to automatically create Fedora partitions.

The gotcha is that there's no assistance for the user to create the required /boot/efi mount point. If they don't do that, they likewise get the unhelpful message that's the summary of this bug. (See bug 1012132)

OK I think I know what's going on that explains all of the failure cases: On Macs, anaconda's check for existing ESPs is triggered by two partition type codes, the legit FAT32 ESP:
C12A7328-F81F-11D2-BA4B-00A0C93EC93B
And the faux HFS+ ESP:
48465300-0000-11AA-AA11-00306543ECAC

On a clean OS X only system, both are present. On a Fedora only system, the 2nd one is present. And even if I delete both the Fedora and OS X ESPs, but leave OS X there, it still fails because OS X's partition type GUID is likewise 48465300-0000-11AA-AA11-00306543ECAC.

<adamw> there's a significant difference between 'affects re-install over an >existing fedora install, can be worked around by deleting the old fedora ESP' and >'affects any attempt to auto-partition alongside an existing EFI OS'

The latter does fail.

<bcl> If it blocks installing to a clean system I'm +1

If clean means a GPT partition layout of: ESP, OS X, OS X Boot, 400GB free space, yes it fails to install to that. Let me know if you want logs.

Here's a fairly simple patch. It:
 * makes sure it is a hfs+ partition and meets other criteria for stage1 on Mac
 * Is smaller than 501.0MB
 * Is NOT labeled "OSX"

http://bcl.fedorapeople.org/updates/1010495.img

For me it works fine with both a clean install (shrunk OSX partition) and re-install where you reclaim space by deleting the previous install's partitions.

Thanks for the patch. Works for me.

Discussed this in the 2013-10-02 Blocker Review Meeting [1]. This has been voted an AcceptedBlocker. This volates the following F20 beta release criterion for EFI system partitions on apple hardware: "Cleanly install to a disk with a valid ms-dos or gpt disk label and partition table which contains existing data and sufficient unpartitioned space for a Fedora installation" [2].


[1] http://meetbot.fedoraproject.org/fedora-blocker-review/2013-10-02/
[2] https://fedoraproject.org/wiki/Fedora_20_Beta_Release_Criteria#Guided_partitioning

After hitting this bug, I tested Brian's updates img from Comment #22 with Fedora 19 live installation on a Macbook, and it worked great.

Discussed this in 2013-10-09 Blocker Review Meeting [1]. Waiting on more information. Will revisit next meeting.

[1] http://meetbot.fedoraproject.org/fedora-blocker-review/2013-10-09/

For the record - bcl informs me that the patch in its current form isn't complete and needs more refinement, so that's why this one isn't in POST or progressed further. He does have it on his todo list to improve and submit the patch.

I wasn't able to reproduce this bug with Beta TC5.

I had Mac with OsX. I shrinked main partition by gparted. Then I run anaconda and chose to use all available space (default with LVM).

Installation went well. Everything runs as it should.

Only one thing is wrong. During installation is created new system EFI HFS+ partition.

(In reply to Petr Schindler from comment #28)
>I shrinked main partition by gparted.
The ability to resize OS X volumes outside of OS X is unexpected.

> Only one thing is wrong. During installation is created new system EFI HFS+
> partition.

That's normal for Macs.

That is likely related to bug 1021258, it stopped reusing existing UEFI partitions on non-mac as well.

With that patch this should start to fail again.

Petr, Chris - could you retry it with anaconda-20.25.4-1 as referenced in Brian's comment?

Confirmed. It fails now again :)

Mixed results with Desktop betaTC6 which has 20.25.4-1.

1. Bug is alive for Guided partitioning, reclaim space, delete prior Fedora partitions retaining the three OS X partitions: FAT32-ESP, HFSJ-OS X, HFSJ-RecoveryHD. I get "failed to find a suitable stage1 device" which is a different message than as originally reported.

2. Bug is NOT alive for custom partitioning when removing the same OS X partitions, then clicking the blue link "create them for me automatically". /boot/efi is created for me without error in this spoke, or upon returning to the hub.

Ok, give this a try (against TC6)

http://bcl.fedorapeople.org/updates/1010495.img

It uses a hfs+ partition with a name of 'Linux HFS+ ESP' so it will not detect current installs unless you add this name to your current Fedora ESP.

c34 updates image fixes c33 item 1, no regression on item 2. I did not click Begin Installation, I'm not ready to blow away this system just yet. But in all previous cases the bug has occurred by now.

I'll give it full install testing with the next TC or RC.

(note that fix also requires python-blivet-0.23.3-1)

python-blivet-0.23.3-1.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/python-blivet-0.23.3-1.fc20

Package python-blivet-0.23.3-1.fc20, anaconda-20.25.5-1.fc20:
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing python-blivet-0.23.3-1.fc20 anaconda-20.25.5-1.fc20'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-20387/anaconda-20.25.5-1.fc20,python-blivet-0.23.3-1.fc20
then log in and leave karma (feedback).

This bug is fricaceed. 

Tested with anaconda-20.25.5.boot.iso, using guided and custom installs to free space, along side OS X to completion. Install & reboot to prompt/shell successful.

Also tested installer behavior, without committing using "Begin Installation", install to:

"empty" 2nd HFSJ partition, along side OS X.
existing Fedora install, along side OS X.

Tested with and without preserving "macefi" in both paths.

No errors.

(In reply to Chris Murphy from comment #39)
> This bug is fricaceed. 

What?

I tested with F20 Beta RC2. I shrunk Apple's partition and installed using guided part into the free space. The installation went fine, but Anaconda ignored Mac's ESP and created its own.

# parted -l
Model: ATA TOSHIBA MK5065GS (scsi)
Disk /dev/sda: 500GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 

Number  Start   End    Size    File system  Name                  Flags
 1      20.5kB  210MB  210MB   fat32        EFI system partition  boot
 2      210MB   452GB  451GB   hfs+         Customer
 4      452GB   452GB  210MB   hfs+         Linux HFS+ ESP
 5      452GB   452GB  524MB   ext4
 6      452GB   499GB  47.2GB                                     lvm
 3      499GB   500GB  650MB   hfs+         Recovery HD

sda[1-3] were available prior to installation, sda[4-6] were created by anaconda.

Is this considered "fixed"?

>(In reply to Kamil Páral from comment #40)
>The installation went fine, but Anacondaignored Mac's ESP and created its own.
>  4      452GB   452GB  210MB   hfs+         Linux HFS+ ESP
> Is this considered "fixed"?

Yes.

python-blivet-0.23.3-1.fc20, anaconda-20.25.6-1.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Hello,

I've tried install Fedora 21 Beta on my macbook but i'm getting this bug about the efi partition.

This is my partition table:

#diskutil list
/dev/disk0
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *240.1 GB   disk0
   1:                        EFI EFI                     209.7 MB   disk0s1
   2:          Apple_CoreStorage                         179.1 GB   disk0s2
   3:                 Apple_Boot Recovery HD             650.0 MB   disk0s3
   4:                 Linux Swap                         511.7 MB   disk0s4
   5: 0FC63DAF-8483-4772-8E79-3D69D8477DE4               59.5 GB    disk0s5
/dev/disk1
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:                 Apple_HFSX Mac                    *178.8 GB   disk1
                                 Logical Volume on disk0s2
                                 7C79EC16-0578-4A80-B09E-81D564C1AB14
                                 Unencrypted

On disk0s5 i've installed opensuse with efi using disk0s1 as efi boot.
But on fedora 21 installer it says that i've not specified a efi boot partition even if I tell it to mount disk0s1 as /boot/efi.

Can you please attach /tmp/program.log and /tmp/storage.log ? Thanks.

Created attachment 955434 [details]
fedora 21 beta logs

On anaconda.log there's this insteresting messages:

09:26:05,727 DEBUG anaconda: MacEFIGRUB.is_valid_stage1_device(sda5) returning False
09:26:05,727 ERR anaconda: storage configuration failed: failed to find a suitable stage1 device
09:26:05,729 ERR anaconda: No valid bootloader target device found. See below for details.
09:26:05,730 ERR anaconda: For a UEFI installation, you must include an EFI System Partition on a GPT-formatted disk, mounted at /boot/efi.

I ran into the same issue on my macbook air. Should I open a new bug since this one is closed?

Created attachment 955500 [details]
pedro's anaconda.log

Created attachment 955501 [details]
pedro's storage.log

Created attachment 955502 [details]
pedro's program.log

09:25:23,016 DEBUG anaconda: _is_valid_format(sda1) returning False
09:25:23,016 DEBUG anaconda: is_valid_stage1_device(sda1) returning False
09:25:23,016 DEBUG anaconda: device.format.name is 'EFI System Partition'

class MacEFI(EFI):
    _boot_stage1_format_types = ["macefi"]
    _boot_efi_description = N_("Apple EFI Boot Partition")

MacEFIGRUB class modifies is_valid_stage1_device() thus:

    def is_valid_stage1_device(self, device, early=False):
        valid = super(MacEFIGRUB, self).is_valid_stage1_device(device, early)

        # Make sure we don't pick the OSX root partition
        if valid and getattr(device.format, "name", "") != "Linux HFS+ ESP":
            valid = False

        if hasattr(device.format, "name"):
            log.debug("device.format.name is '%s'", device.format.name)

looks like that's what we're hitting. It expects the ESP's device.format.name to be "Linux HFS+ ESP", in your case it's "EFI System Partition".

Per #c34 that may not be considered a bug exactly, but I'll leave it up to bcl to decide. If that's the case we may want to look at improving the error message (which I wrote, I know :>) for the Mac case.

Does it work if you create a new ESP as part of your Fedora layout? Does it work if you use automatic / 'guided' configuration? (You can 'delete' some existing partition or something to test the non-custom partitioning, it won't actually do it until you click Begin Install or whatever it's called on the hub, so it's safe as long as you don't do that).

Yes it works if I create a new layout or use the automatic configuration.
Just don't understand why we can't use that EFI partition since it's a simple FAT32 with ESP flag partition.

In order to see Fedora as a boot option in OS X's Startup disk panel, and the firmware option+bootchime boot manager menu, Fedora's bootloader (GRUB) needs to be on an HFS+ volume, which Apple's firmware can read directly unlike non-Apple UEFI systems. This is part of mactel-boot.

So the required "Linux HFS+ ESP" is what this error message is referring to. You need to create a standard partition, ~100-200MB in size, and set the type to Linux HFS+ ESP. That's where GRUB will go, and the installer will stop complaining.

This is why I filed bug 1022316. Users can't be expected to know this things, it's user hostile to require them to manually create mandatory partitions, even in manual partitioning. FAT ESP, HFS+ ESP, and BIOSBoot, should all be automatically created expressly because of all the confusion we continue to get around these things.

(In reply to Pedro Saraiva from comment #43)
> Hello,
> 
> I've tried install Fedora 21 Beta on my macbook but i'm getting this bug
> about the efi partition.

Trying to triple boot a Mac is a situation we really don't support. Does it work if you create /boot/efi/ and set the type to 'Linux HFS+ ESP'?



(In reply to Chris Murphy from comment #56)
> This is why I filed bug 1022316. Users can't be expected to know this
> things, it's user hostile to require them to manually create mandatory
> partitions, even in manual partitioning. FAT ESP, HFS+ ESP, and BIOSBoot,
> should all be automatically created expressly because of all the confusion
> we continue to get around these things.

Chris, please stop referring to things as 'user hostile'. Custom is custom, you're expected to know what you are doing. Autopart should do the right thing, if it has the space. If it doesn't let's file a bug about that.



(In reply to Roman Dubtsov from comment #47)
> I ran into the same issue on my macbook air. Should I open a new bug since
> this one is closed?

If you aren't triple booting and you are using autopart or custom + correct /boot/efi/ type then yes. With individually attached logs.

(In reply to Pedro Saraiva from comment #55)
> Yes it works if I create a new layout or use the automatic configuration.
> Just don't understand why we can't use that EFI partition since it's a
> simple FAT32 with ESP flag partition.

Yes, you could as Ubuntu can do it with two or one OS. As reference https://help.ubuntu.com/community/MacBookPro11-1/Saucy.
The Fedora method is better in my opinion.

bcl: I'll see if I can come up with something to post a better error message for the Mac case (have to re-familiarize myself with the way the errors there work).

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days