Bug 1156380

Summary: 21 Beta RC1 live images cannot be written with 'livecd-iso-to-disk --format --reset-mbr --efi'
Product: [Fedora] Fedora Reporter: Adam Williamson <awilliam>
Component: livecd-toolsAssignee: Brian Lane <bcl>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 21CC: adam.stokes, awilliam, bcl, bruno, dhuff, Jasper.Hartline, katzj, kevin, kparal, lars.s.jensen, robatino, satellitgo, znmeb
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: All   
OS: Unspecified   
Whiteboard: AcceptedBlocker
Fixed In Version: livecd-tools-21.4-1.fc21 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-11-02 10:49:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1043124    
Attachments:
Description Flags
Fixed upper case extension on BOOT*.EFI none

Description Adam Williamson 2014-10-24 10:57:59 UTC
Trying to write the Fedora 21 Beta RC1 Workstation x86_64 live image to a USB stick with:

livecd-iso-to-disk --format --reset-mbr --efi Fedora-Live-Workstation-x86_64-21_Beta-1.iso /dev/sdc

(or /dev/sdc1, doesn't seem to make a difference) results in some strange output and an error at the end of the writing process - see output below. The same occurs whether you try and write the image from Fedora 20 or Fedora 21. The resulting stick does not boot at all - not even in BIOS mode, never mind UEFI. However, writing the Beta TC4 Workstation x86-64 live works successfully with the same command line, and the resulting stick boots in both BIOS and UEFI modes. So it seems the bug is in the image, not in livecd-iso-to-disk.

As this broke between TC4 and RC1 I suspect it's broken by the new livecd-tools we pulled into RC1 to fix https://bugzilla.redhat.com/show_bug.cgi?id=1154138 - that was livecd-tools-21.3-1.fc21 .

Nominating as a Beta blocker per criterion "All release-blocking images must boot in their supported configurations." - https://fedoraproject.org/wiki/Fedora_21_Beta_Release_Criteria#Release-blocking_images_must_boot . This criterion is explicitly intended to require all supported USB writing methods to work; there is a footnote which specifically states this, "This criterion differs from the similar Alpha criterion only in that it requires all supported methods of writing a Fedora USB stick to work, not just any single one."



Setting up /EFI/BOOT
Validation Entry:
        Header Indicator: 0x01 (Validation Entry)
        PlatformId: 0x00 (80x86)
        ID: ""
        Checksum: 0x55aa
        Key bytes: 0x55aa
Boot Catalog Default Entry:
        Entry is bootable
        Boot Media emulation type: no emulation
        Media load segment: 0x0 (0000:7c00)
        System type: 0 (0x00)
        Load Sectors: 4 (0x0004)
        Load LBA: 2584 (0x00000a18)
Dumping boot image to "/share/data/isos/21/Beta_RC1/Fedora-Live-Workstation-x86_64-21_Beta-1.iso.0"
Section Header Entry:
        Header Indicator: 0x90 (Section Header Entry)
        PlatformId: 0xef (EFI)
        Section Entries: 1
        ID: ""
Boot Catalog Section Entry:
        Entry is bootable
        Boot Media emulation type: no emulation
        Media load address: 0 (0x0000)
        System type: 0 (0x00)
        Load Sectors: 10164 (0x27b4)
        Load LBA: 43 (0x0000002b)
Dumping boot image to "/share/data/isos/21/Beta_RC1/Fedora-Live-Workstation-x86_64-21_Beta-1.iso.1"
Section Header Entry:
        Header Indicator: 0x91 (Final Section Header Entry)
        PlatformId: 0xef (EFI)
        Section Entries: 1
        ID: ""
Boot Catalog Section Entry:
        Entry is bootable
        Boot Media emulation type: no emulation
        Media load address: 0 (0x0000)
        System type: 0 (0x00)
        Load Sectors: 40704 (0x9f00)
        Load LBA: 2604 (0x00000a2c)
Dumping boot image to "/share/data/isos/21/Beta_RC1/Fedora-Live-Workstation-x86_64-21_Beta-1.iso.2"
No BOOT*.efi found in eltorito image. EFI will not boot

Comment 1 Kamil Páral 2014-10-24 11:04:42 UTC
The same error occurs with Server DVD, when used with --efi. Without --efi, it works fine.

Comment 2 Lars S. Jensen 2014-10-24 12:45:20 UTC
It also created/leaves 3 files beside the ISO file: .../F21-Live-3.17.1-303.iso.[012] 
So the clean up is not correct:
umount: /media/srctmp.C7QbK3: target is busy
        (In some cases useful info about processes that
         use the device is found by lsof(8) or fuser(1).)
Cleaning up to exit...

Comment 3 Adam Williamson 2014-10-24 17:47:35 UTC
Discussed at 2014-10-24 Go/No-Go meeting: http://meetbot.fedoraproject.org/fedora-meeting-2/2014-10-24/f21_beta_gono-go_meeting.2014-10-24-17.01.log.txt . Accepted as a blocker per criterion cited in description.

Comment 4 Lars S. Jensen 2014-10-25 12:11:36 UTC
Created attachment 950649 [details]
Fixed upper case extension on BOOT*.EFI

Correct the check and copy code in 
# test for presence of *.efi grub binary
by replacing +(BOOT|boot)?*.efi with +(BOOT|boot)?*.+(EFI|efi)

The patch is done  livecd-tools-20.5-1.fc20.x86_64 but should also work on livecd-tools-21.3-1.fc21.x86_64.

Comment 5 Lars S. Jensen 2014-10-26 10:15:03 UTC
(In reply to Lars S. Jensen from comment #4)
> Created attachment 950649 [details]
> Fixed upper case extension on BOOT*.EFI

And it boots on a UEFI laptop (ASUS UX32VD, Secure Boot OFF) with FC21-live image created with livecd-tools-21.3-1.fc21 and lorax-21.26-1.fc21.
The USB-stick was created with the patched livecd-iso-to-disk from livecd-tools-20.5-1.fc20.x86_64

files in /EFI/BOOT/
-rwxr-xr-x. 1 root root 1387744 Oct 25 15:03 BOOTX64.EFI <-- "shim.efi"
drwxr-xr-x. 2 root root    8192 Oct 25 15:03 fonts
-rwxr-xr-x. 1 root root    1138 Oct 25 15:17 grub.cfg
-rwxr-xr-x. 1 root root  984424 Oct 25 15:03 grubx64.efi

Comment 6 Lars S. Jensen 2014-10-26 11:44:05 UTC
Can the (BOOT|boot)?.conf also have the extension in uppercase (lines 1139/1140)?

There is general issue with the use of glob in combination with "[ -f" (file test) in the if-statements. The glob expression can return more that one argument and the script will fail the test.
This is only an issue if there are more that one boot?*.efi/boot?*.conf file in the ISO image. 
lines: 1139, 1151, 1162

Comment 7 Brian Lane 2014-10-27 18:32:41 UTC
Thanks for the help. I've fixed it in a slightly different way, using nocaseglob and a helper function. There will only be 1 matching file so we don't need to handle more than one.

Comment 8 Adam Williamson 2014-10-27 18:46:21 UTC
For blocker purposes, this bug now follows the informal 'special blocker' convention - though the change was to image generation, we consider that change was correct not a bug, and the fix is to adjust livecd-iso-to-disk to cope with it. That means the fix falls outside the image build process and therefore we don't block image composes on this bug.

The requirement is that the fix should be made available as an update for supported Fedoras (19 and 20) by the time of Beta's public release. It'd also be good to have the fix available for F21, but not as critical.

Comment 9 Fedora Update System 2014-10-27 19:25:58 UTC
livecd-tools-21.4-1.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/livecd-tools-21.4-1.fc21

Comment 10 Fedora Update System 2014-10-27 19:53:27 UTC
livecd-tools-20.6-1.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/livecd-tools-20.6-1.fc20

Comment 11 Fedora Update System 2014-10-27 20:26:55 UTC
livecd-tools-19.10-1.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/livecd-tools-19.10-1.fc19

Comment 12 Brian Lane 2014-10-27 21:14:54 UTC
*** Bug 1156324 has been marked as a duplicate of this bug. ***

Comment 13 Fedora Update System 2014-10-28 10:06:42 UTC
Package livecd-tools-21.4-1.fc21:
* should fix your issue,
* was pushed to the Fedora 21 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing livecd-tools-21.4-1.fc21'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2014-13801/livecd-tools-21.4-1.fc21
then log in and leave karma (feedback).

Comment 14 Kamil Páral 2014-10-29 14:12:21 UTC
This is fixed with livecd-tools-20.6-1.fc20

Comment 15 Fedora Update System 2014-11-01 01:44:30 UTC
livecd-tools-20.6-1.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 16 Fedora Update System 2014-11-01 17:07:00 UTC
livecd-tools-19.10-1.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 17 Fedora Update System 2014-11-01 17:10:44 UTC
livecd-tools-21.4-1.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 18 Kevin Kofler 2014-11-02 10:42:42 UTC
Huh? What was the point of pushing this update to Fedora 19 and 20, when Fedora 19 and 20's livecd-creator has not been able to create Fedora 21 live images that actually boot for months? Has anybody even tested the Fedora 19 and 20 updates that were pushed here?

This commit is needed to create bootable Fedora 21 live images:
https://git.fedorahosted.org/cgit/livecd/commit/imgcreate/live.py?id=a267c4ab89ff97bcbad550b9ec331d5a0631d444

Comment 19 Kevin Kofler 2014-11-02 10:49:38 UTC
Ah, I see, the point is to make livecd-iso-to-disk work, not livecd-creator. That's a weird state, where we support SOME of the livecd-creator tools for F21 images on F19/F20 to the point of making it a blocker, but not others. The net effect is still that I now have to update my patched specs (http://svn.calcforge.org/viewvc/kannolo/trunk/build-packages/livecd-tools/) that make livecd-creator actually work. :-( Sigh…

Comment 20 DO NOT USE account not monitored (old adamwill) 2014-11-02 18:06:13 UTC
"That's a weird state, where we support SOME of the livecd-creator tools for F21 images on F19/F20 to the point of making it a blocker, but not others."

Er, not really. They're totally different tools for different use cases. It would be absurd to block a release on people being able to build custom live images of it from a different release; that's a very niche function that I don't think we've ever even claimed to support. On the contrary, writing USB sticks of F-N+1 from F-N is one of the most common things people *do* when a release comes out. Which is why we have a release criterion which says:

"Release-blocking live and dedicated installer images must boot when written to optical media of an appropriate size (if applicable) and when written to a USB stick with any of the officially supported methods."

But we do not have one which says, er, "It must be possible to build a custom live image of the release from the previous release".