This service will be undergoing maintenance at 00:00 UTC, 2017-10-23 It is expected to last about 30 minutes
Bug 1260799 - pyanaconda.bootloader.BootLoaderError: failed to set new efi boot target. This is most likely a kernel or firmware bug.
pyanaconda.bootloader.BootLoaderError: failed to set new efi boot target. Thi...
Status: CLOSED NEXTRELEASE
Product: Fedora
Classification: Fedora
Component: python-blivet (Show other bugs)
23
x86_64 Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Brian Lane
Fedora Extras Quality Assurance
abrt_hash:73c7db4ec9ca100060d33db9284...
:
: 1260999 1262071 (view as bug list)
Depends On:
Blocks: F23BetaBlocker
  Show dependency treegraph
 
Reported: 2015-09-07 17:11 EDT by Dale Turner
Modified: 2015-09-16 14:35 EDT (History)
18 users (show)

See Also:
Fixed In Version: 23.19.4-1.fc23
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-09-16 14:35:52 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
File: anaconda-tb (1.82 MB, text/plain)
2015-09-07 17:12 EDT, Dale Turner
no flags Details
File: anaconda.log (29.96 KB, text/plain)
2015-09-07 17:12 EDT, Dale Turner
no flags Details
File: environ (538 bytes, text/plain)
2015-09-07 17:12 EDT, Dale Turner
no flags Details
File: journalctl (799.83 KB, text/plain)
2015-09-07 17:12 EDT, Dale Turner
no flags Details
File: lsblk_output (4.55 KB, text/plain)
2015-09-07 17:12 EDT, Dale Turner
no flags Details
File: nmcli_dev_list (2.20 KB, text/plain)
2015-09-07 17:12 EDT, Dale Turner
no flags Details
File: os_info (518 bytes, text/plain)
2015-09-07 17:12 EDT, Dale Turner
no flags Details
File: program.log (404.07 KB, text/plain)
2015-09-07 17:12 EDT, Dale Turner
no flags Details
File: storage.log (506.16 KB, text/plain)
2015-09-07 17:12 EDT, Dale Turner
no flags Details
File: ifcfg.log (4.39 KB, text/plain)
2015-09-07 17:12 EDT, Dale Turner
no flags Details
File: packaging.log (132 bytes, text/plain)
2015-09-07 17:12 EDT, Dale Turner
no flags Details
journal from boot with efi=debug on affected VM (154.65 KB, text/plain)
2015-09-08 14:27 EDT, Adam Williamson
no flags Details
Possible fix (10.45 KB, application/x-gzip)
2015-09-08 17:23 EDT, Brian Lane
no flags Details

  None (edit)
Description Dale Turner 2015-09-07 17:11:54 EDT
Description of problem:
Trying to install F23

Version-Release number of selected component:
anaconda-core-23.19.2-1.fc23.x86_64

The following was filed automatically by anaconda:
anaconda 23.19.2-1 exception report
Traceback (most recent call first):
  File "/usr/lib64/python3.4/site-packages/pyanaconda/bootloader.py", line 1750, in _add_single_efi_boot_target
    raise BootLoaderError("failed to set new efi boot target. This is most likely a kernel or firmware bug.")
  File "/usr/lib64/python3.4/site-packages/pyanaconda/bootloader.py", line 1754, in add_efi_boot_target
    self._add_single_efi_boot_target(self.stage1_device)
  File "/usr/lib64/python3.4/site-packages/pyanaconda/bootloader.py", line 1762, in install
    self.add_efi_boot_target()
  File "/usr/lib64/python3.4/site-packages/pyanaconda/bootloader.py", line 1782, in write
    self.install()
  File "/usr/lib64/python3.4/site-packages/pyanaconda/bootloader.py", line 2375, in writeBootLoaderFinal
    storage.bootloader.write()
  File "/usr/lib64/python3.4/site-packages/pyanaconda/bootloader.py", line 2453, in writeBootLoader
    writeBootLoaderFinal(storage, payload, instClass, ksdata)
  File "/usr/lib64/python3.4/site-packages/pyanaconda/install.py", line 240, in doInstall
    writeBootLoader(storage, payload, instClass, ksdata)
  File "/usr/lib64/python3.4/threading.py", line 868, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib64/python3.4/site-packages/pyanaconda/threads.py", line 253, in run
    threading.Thread.run(self, *args, **kwargs)
pyanaconda.bootloader.BootLoaderError: failed to set new efi boot target. This is most likely a kernel or firmware bug.

Additional info:
cmdline:        /usr/bin/python3  /sbin/anaconda --liveinst --method=livecd:///dev/mapper/live-base
cmdline_file:   BOOT_IMAGE=/isolinux/vmlinuz0 root=live:LABEL=Fedora-Live-WS-x86_64-23-2015090 ro rd.live.image quiet rhgb
executable:     /sbin/anaconda
hashmarkername: anaconda
kernel:         4.2.0-1.fc23.x86_64
other involved packages: python3-libs-3.4.3-5.fc23.x86_64
product:        Fedora
release:        Fedora release 23 (Twenty Three)
type:           anaconda
version:        23
Comment 1 Dale Turner 2015-09-07 17:12:01 EDT
Created attachment 1071131 [details]
File: anaconda-tb
Comment 2 Dale Turner 2015-09-07 17:12:03 EDT
Created attachment 1071132 [details]
File: anaconda.log
Comment 3 Dale Turner 2015-09-07 17:12:04 EDT
Created attachment 1071133 [details]
File: environ
Comment 4 Dale Turner 2015-09-07 17:12:07 EDT
Created attachment 1071134 [details]
File: journalctl
Comment 5 Dale Turner 2015-09-07 17:12:08 EDT
Created attachment 1071135 [details]
File: lsblk_output
Comment 6 Dale Turner 2015-09-07 17:12:08 EDT
Created attachment 1071136 [details]
File: nmcli_dev_list
Comment 7 Dale Turner 2015-09-07 17:12:09 EDT
Created attachment 1071137 [details]
File: os_info
Comment 8 Dale Turner 2015-09-07 17:12:11 EDT
Created attachment 1071138 [details]
File: program.log
Comment 9 Dale Turner 2015-09-07 17:12:13 EDT
Created attachment 1071139 [details]
File: storage.log
Comment 10 Dale Turner 2015-09-07 17:12:14 EDT
Created attachment 1071140 [details]
File: ifcfg.log
Comment 11 Dale Turner 2015-09-07 17:12:15 EDT
Created attachment 1071141 [details]
File: packaging.log
Comment 12 Peter Jones 2015-09-08 10:50:13 EDT
Okay, so in the kernel log we see:
Sep 07 16:32:20 localhost kernel: Initializing cgroup subsys cpuset
Sep 07 16:32:20 localhost kernel: Initializing cgroup subsys cpu
Sep 07 16:32:20 localhost kernel: Initializing cgroup subsys cpuacct
Sep 07 16:32:20 localhost kernel: Linux version 4.2.0-1.fc23.x86_64 (mockbuild@bkernel02.phx2.fedoraproject.org) (gcc version 5.1.1 20150618 (Red Hat 5.1.1-4) (GCC) ) #1 SMP Mon Aug 31 15:57:27 UTC 2015
Sep 07 16:32:20 localhost kernel: Command line: BOOT_IMAGE=/isolinux/vmlinuz0 root=live:LABEL=Fedora-Live-WS-x86_64-23-2015090 ro rd.live.image quiet rhgb
Sep 07 16:32:20 localhost kernel: tseg: 00bff00000
Sep 07 16:32:20 localhost kernel: x86/fpu: Legacy x87 FPU detected.
Sep 07 16:32:20 localhost kernel: x86/fpu: Using 'lazy' FPU context switches.
Sep 07 16:32:20 localhost kernel: e820: BIOS-provided physical RAM map:
Sep 07 16:32:20 localhost kernel: BIOS-e820: [mem 0x0000000000000000-0x000000000009ffff] usable
Sep 07 16:32:20 localhost kernel: BIOS-e820: [mem 0x0000000000100000-0x00000000bf68efff] usable
Sep 07 16:32:20 localhost kernel: BIOS-e820: [mem 0x00000000bf68f000-0x00000000bf907fff] reserved
Sep 07 16:32:20 localhost kernel: BIOS-e820: [mem 0x00000000bf908000-0x00000000bfb0afff] ACPI NVS
Sep 07 16:32:20 localhost kernel: BIOS-e820: [mem 0x00000000bfb0b000-0x00000000bfd0efff] usable
Sep 07 16:32:20 localhost kernel: BIOS-e820: [mem 0x00000000bfd0f000-0x00000000bfd5cfff] ACPI NVS
Sep 07 16:32:20 localhost kernel: BIOS-e820: [mem 0x00000000bfd5d000-0x00000000bfd65fff] ACPI data
Sep 07 16:32:20 localhost kernel: BIOS-e820: [mem 0x00000000bfd66000-0x00000000bfd68fff] ACPI NVS
Sep 07 16:32:20 localhost kernel: BIOS-e820: [mem 0x00000000bfd69000-0x00000000bfd69fff] usable
...
Sep 07 16:32:20 localhost kernel: efi: EFI v2.00 by American Megatrends
Sep 07 16:32:20 localhost kernel: efi:  SMBIOS=0xbf906a18  ACPI=0xbfd5d000  ACPI 2.0=0xbfd5d000  MPS=0xfcaf0
...
Sep 07 16:32:20 localhost kernel: efifb: probing for efifb
Sep 07 16:32:20 localhost kernel: efifb: framebuffer at 0xc0000000, mapped to 0xffffc90001000000, using 1876k, total 1875k
Sep 07 16:32:20 localhost kernel: efifb: mode is 800x600x32, linelength=3200, pages=1
Sep 07 16:32:20 localhost kernel: efifb: scrolling: redraw
Sep 07 16:32:20 localhost kernel: efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0
Sep 07 16:32:20 localhost kernel: Console: switching to colour frame buffer device 100x37
Sep 07 16:32:20 localhost kernel: fb0: EFI VGA frame buffer device
...
Sep 07 16:32:20 localhost kernel: fb: switching to radeondrmfb from EFI VGA
...

What's happening here is this: the kernel is being called from the EFI entry point, and it's finding the EFI system table, so the EFI stub can do things like set the video mode parameters and such, but for whatever reason, it never actually finds the EFI memory map.  So it creates /sys/firmware/efi , and rightly so, but efi_enabled(EFI_RUNTIME_SERVICES) inside the kernel is never set to true, so /sys/firmware/efi/efivars is never mounted.

So in this case anaconda has done the right thing - this is an EFI machine, but also efibootmgr is correct - the machine, as exposed to userland, does not support EFI variables.

I think this is either a kernel or firmware bug; either the memory map isn't being exposed correctly, or the kernel is messing it up somehow.
Comment 13 Peter Jones 2015-09-08 12:52:36 EDT
Can you boot with "efi=debug" on the kernel command line and attach a copy of /var/log/messages that results from it?
Comment 14 Adam Williamson 2015-09-08 14:15:55 EDT
This and https://bugzilla.redhat.com/show_bug.cgi?id=1260999 look like dupes, and as it seems like everyone who tries a UEFI install of Beta TC4 is hitting this (on systems which previously installed fine), it's not a firmware issue.

Also note this from that bug:

"However, Lukas forgot to mention that if you switch to VT2 and run "efibootmgr", it prints current boot options just fine. So there seems to be some difference between running it in a VT and anaconda running it inside its process (in a chroot?)."
Comment 15 David Shea 2015-09-08 14:25:05 EDT
*** Bug 1260999 has been marked as a duplicate of this bug. ***
Comment 16 Adam Williamson 2015-09-08 14:27:29 EDT
Created attachment 1071450 [details]
journal from boot with efi=debug on affected VM

Here's the journal contents after booting with 'efi=debug' on an affected VM.
Comment 17 Peter Jones 2015-09-08 14:51:46 EDT
(In reply to awilliam@redhat.com from comment #16)
> Created attachment 1071450 [details]
> journal from boot with efi=debug on affected VM
> 
> Here's the journal contents after booting with 'efi=debug' on an affected VM.

This doesn't look like the same issue at all; the original has:

Sep 07 16:32:20 localhost kernel: efi: EFI v2.00 by American Megatrends
Sep 07 16:32:20 localhost kernel: efi:  SMBIOS=0xbf906a18  ACPI=0xbfd5d000  ACPI 2.0=0xbfd5d000  MPS=0xfcaf0
Sep 07 16:32:20 localhost kernel: SMBIOS 2.6 present.
Sep 07 16:32:20 localhost kernel: DMI: System manufacturer System Product Name/F1A75-V PRO, BIOS 2302 09/26/2012

(i.e. no EFI memory map), and your log has:

Sep 08 18:16:33 localhost kernel: reserve setup_data: [mem 0x000000007fff0000-0x000000007fffffff] usable
Sep 08 18:16:33 localhost kernel: efi: EFI v2.50 by EDK II
Sep 08 18:16:33 localhost kernel: efi:  SMBIOS=0x7fec3000  ACPI=0x7fef2000  ACPI 2.0=0x7fef2014 
Sep 08 18:16:33 localhost kernel: efi: mem00: [Boot Code          |   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000000000000-0x0000000000001000) (0MB)
...
Comment 18 Peter Jones 2015-09-08 15:00:32 EDT
(Hrm, might be the same after all - jwb tells me the dumping of the efi memory map has moved to efi=debug only, which surprised me because it's much more useful than the e820 map...)
Comment 19 Adam Williamson 2015-09-08 15:05:11 EDT
pjones: in the dupe bug you can find the output *without* efi=debug from my system, for comparison, if it helps - https://bugzilla.redhat.com/attachment.cgi?id=1071442
Comment 20 Josh Boyer 2015-09-08 15:19:34 EDT
Not particularly helpful collapsing these two bugs into one yet, but I guess we'll work through it.

Can everyone see if efivarfs is mounted on their afflicted machine?

'mount | grep efivarfs' in a vt will do (or terminal).  It should be mounted on /sys/firmware/efi/efivars/
Comment 21 Adam Williamson 2015-09-08 16:30:16 EDT
Josh: it really seems pretty certain that we're all hitting the same thing. This isn't some system-specific firmware issue that just came up - it's something that changed circa Beta TC4 which somehow makes efibootmgr when called by the installer not think efivar support is available. Every report I've seen so far has the "

Did you note the thing from #c14? efibootmgr *does* work apparently OK from tty2. It's only when anaconda calls it that it's not working right.

Yes, efivarfs is mounted on /sys/firmware/efi/efivars .
Comment 22 Adam Williamson 2015-09-08 16:36:40 EDT
Looking at the code, anaconda does apparently run efibootmgr from within a chroot to the installed system.
Comment 23 Adam Williamson 2015-09-08 16:44:19 EDT
Yeah, so if I do this (after install has run):

chroot /mnt/sysimage
efibootmgr

I get the same error. As mentioned before, running 'efibootmgr' outside the chroot runs fine.

Inside the chroot, /sys/firmware/efi/efivars is not populated. How new is this efivarfs thing?

I guess I'd better double-check exactly when this broke, too. I'll check Beta TC1 and Alpha.
Comment 24 Josh Boyer 2015-09-08 16:53:50 EDT
(In reply to awilliam@redhat.com from comment #23)
> Yeah, so if I do this (after install has run):
> 
> chroot /mnt/sysimage
> efibootmgr
> 
> I get the same error. As mentioned before, running 'efibootmgr' outside the
> chroot runs fine.
> 
> Inside the chroot, /sys/firmware/efi/efivars is not populated. How new is
> this efivarfs thing?

It isn't new at all.  If it isn't available in the chroot, perhaps it is because something did not bind mount it into the chroot before chroot was done.

> I guess I'd better double-check exactly when this broke, too. I'll check
> Beta TC1 and Alpha.

My best guess is the following:

The kernel used to be built with both EFI_VARS (old interface) and EFIVARFS (new).  A couple weeks ago, we disabled EFI_VARS because 1) it's broken, and 2) it's extra broken when both of them are available.

If the theory about the efivarfs mount not being in the chroot is accurate, then perhaps what is happening is that efibootmgr tries to use efivarfs and it fails.  Then it falls back to trying the old efivars interface, and that now also fails because we disabled it in the kernel.

(Which is somewhat frightening, since that means the old interface has been used for a long time when the efivarfs interface was intended to be used for quite a while.)
Comment 25 Adam Williamson 2015-09-08 17:01:31 EDT
That sounds very plausible, because I just tested Beta TC1 - and indeed install works, and running 'efibootmgr' manually in the sysimage chroot works. But /sys/firmware/efi/efivars is definitely not present in the chroot, and neither efibootmgr nor efivar-libs has changed at all between TC1 and TC4, the versions are identical.

So that sounds like it's almost certainly what's going on, yes.
Comment 26 Adam Williamson 2015-09-08 17:18:34 EDT
"If it isn't available in the chroot, perhaps it is because something did not bind mount it into the chroot before chroot was done."

I don't think anything even *tries* to do that, unless I'm missing it. If you dig through all the anaconda utility functions you wind up here:

https://github.com/rhinstaller/anaconda/blob/master/pyanaconda/iutil.py#L126

here is where it sets up a chroot, if needed:

https://github.com/rhinstaller/anaconda/blob/master/pyanaconda/iutil.py#L164

it just runs chroot, there's no bind mounting or anything else clever done. I poked around other bits of anaconda but can't find anything else that does any bind mounts into /mnt/sysimage except some stuff in the exotic ostree payload (which we're not using here).
Comment 27 Brian Lane 2015-09-08 17:23:02 EDT
Created attachment 1071514 [details]
Possible fix

https://github.com/rhinstaller/blivet/pull/219

Give this a try and see if it works.
Comment 28 Adam Williamson 2015-09-08 19:20:48 EDT
Can affected folks please test https://www.happyassassin.net/updates/1260799.1.img ? It's a slightly tweaked version of bcl's patch (see discussion on the github issue), seems to work for me.
Comment 29 Lukas Brabec 2015-09-09 03:54:54 EDT
(In reply to awilliam@redhat.com from comment #28)
> Can affected folks please test
> https://www.happyassassin.net/updates/1260799.1.img ? It's a slightly
> tweaked version of bcl's patch (see discussion on the github issue), seems
> to work for me.

The update fixed the problem for me.
Comment 30 David Shea 2015-09-10 14:45:49 EDT
*** Bug 1262071 has been marked as a duplicate of this bug. ***
Comment 31 Adam Williamson 2015-09-10 15:23:11 EDT
Discussed at 2015-09-10 blocker review meeting: https://meetbot-raw.fedoraproject.org/fedora-blocker-review/2015-09-10/f23-blocker-review.2015-09-10-16.00.log.txt . Accepted as a blocker: this violates "A system installed with a release-blocking desktop must boot to a log in screen where it is possible to log in to a working desktop using a user account created during installation or a 'first boot' utility." in the case of a UEFI install where the fallback mechanism doesn't work (we know of several scenarios where that would be the case), which we consider a broad enough impact to make this a Beta blocker.
Comment 32 Fedora Update System 2015-09-10 16:27:54 EDT
python-blivet-1.12.3-1.fc23 anaconda-23.19.3-1.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2015-15562
Comment 33 Fedora Update System 2015-09-10 23:49:17 EDT
anaconda-23.19.3-1.fc23, python-blivet-1.12.3-1.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.\nIf you want to test the update, you can install it with \n su -c 'yum --enablerepo=updates-testing update anaconda python-blivet'. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-15562
Comment 34 Fedora Update System 2015-09-14 13:22:05 EDT
python-blivet-1.12.4-1.fc23 anaconda-23.19.4-1.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2015-15846
Comment 35 Dale Turner 2015-09-14 13:58:35 EDT
When would install images be available with this update?
Comment 36 Adam Williamson 2015-09-14 14:00:34 EDT
Beta RC1 will include it. You should still be able to use the updates.img linked above for now, as well.
Comment 37 Fedora Update System 2015-09-15 12:50:53 EDT
anaconda-23.19.4-1.fc23, python-blivet-1.12.4-1.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.\nIf you want to test the update, you can install it with \n su -c 'yum --enablerepo=updates-testing update anaconda python-blivet'. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-15846
Comment 38 Adam Williamson 2015-09-15 21:51:57 EDT
Verified in openQA testing.
Comment 39 Fedora Update System 2015-09-16 14:35:39 EDT
anaconda-23.19.4-1.fc23, python-blivet-1.12.4-1.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.