Bug 947142 - write to /sys/firmware/efi/vars/new_var results in ENOSPC
Summary: write to /sys/firmware/efi/vars/new_var results in ENOSPC
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: 18
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Kernel Maintainer List
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: AcceptedBlocker
: 949122 949786 950022 (view as bug list)
Depends On:
Blocks: F19Alpha, F19AlphaBlocker 972945 973833
TreeView+ depends on / blocked
 
Reported: 2013-04-01 18:10 UTC by Rolf Fokkens
Modified: 2014-09-13 19:00 UTC (History)
27 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 972945 (view as bug list)
Environment:
Last Closed: 2013-06-10 21:21:15 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
mac grub screen (282.47 KB, image/jpeg)
2013-04-09 12:19 UTC, Kamil Páral
no flags Details
console.log (15.13 KB, text/plain)
2013-04-16 04:25 UTC, Lingzhu Xiang
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 949721 0 high CLOSED Convert UEFI boot failure data from /sys/fs/pstore to abrt problem data 2021-02-22 00:41:40 UTC
Red Hat Bugzilla 949786 0 unspecified CLOSED BootLoaderError: failed to set new efi boot target 2021-02-22 00:41:40 UTC
Red Hat Bugzilla 950709 0 unspecified CLOSED Clean pstore at install time 2021-02-22 00:41:40 UTC
Red Hat Bugzilla 1006304 0 unspecified CLOSED BootLoaderError: failed to set new efi boot target 2021-02-22 00:41:40 UTC
Red Hat Bugzilla 1049552 0 unspecified CLOSED efibootmgr called too soon, if it fails then post install scripts don't run and resulting system is really messy to clea... 2021-02-22 00:41:40 UTC

Internal Links: 949721 949786 950709 1006304 1049552

Description Rolf Fokkens 2013-04-01 18:10:58 UTC
Description of problem:
After upgrading to F18 (using kernel-3.8.4-202.fc18) efibootmgr is no longer able to write to "/sys/firmware/efi/vars/new_var" because it results in ENOSPC.

After rebooting to the prevous (F17) kernel (kernel-3.7.9-104.fc17.x86_64) efbootmgr is able to write to "/sys/firmware/efi/vars/new_var".

Version-Release number of selected component (if applicable):
kernel-3.8.4-202.fc18.x86_64

How reproducible:
100% (on my Asus C60M1-I BIOS 0205)

Steps to Reproduce:
1. upgrace F17 tot F18 using yum
2. reboot
3. try to use grub2-efi instead of grub by (also) using efibootmgr
  
Actual results:
Nothing changes in efivars/NFVRAM

Expected results:
Modified efivars/NVRAM

Additional info:
Bug 922275

Comment 1 Josh Boyer 2013-04-01 18:16:37 UTC
The kernel now limits efivars from using more than 50% of the available firmware space.  This has some known deficiencies if the firmware on the machine doesn't do garbage collection to free up space.  This is being worked on upstream.

Comment 2 Matthew Garrett 2013-04-01 18:35:57 UTC
Also, some Asus boards return 0 for the maximum variable size.

Comment 3 Rolf Fokkens 2013-04-01 19:36:28 UTC
For informational purposes: I upgraded the C60M1-I bios to 0305, that did not help.

Comment 4 Adam Williamson 2013-04-08 17:48:50 UTC
*** Bug 949122 has been marked as a duplicate of this bug. ***

Comment 5 Adam Williamson 2013-04-08 17:49:18 UTC
This could be a candidate for F19 Alpha blocker, but we'd kinda need to guess how many systems it's likely to affect; anyone have a WAG?

Comment 6 Jaroslav Reznik 2013-04-09 11:18:07 UTC
(In reply to comment #1)
> The kernel now limits efivars from using more than 50% of the available
> firmware space.  This has some known deficiencies if the firmware on the
> machine doesn't do garbage collection to free up space.  This is being
> worked on upstream.

Josh, do you know if the fix is already available upstream/could be backported? Also it would be nice to have guess how wide spread this bug is as Adam asked - forh Alpha, we could be tolerant (but it really depends). Thanks.

Comment 7 Josh Boyer 2013-04-09 11:39:01 UTC
(In reply to comment #6)
> (In reply to comment #1)
> > The kernel now limits efivars from using more than 50% of the available
> > firmware space.  This has some known deficiencies if the firmware on the
> > machine doesn't do garbage collection to free up space.  This is being
> > worked on upstream.
> 
> Josh, do you know if the fix is already available upstream/could be
> backported? 

Matthew Garrett has patches submitted, but the upstream EFI maintainer has some concerns and would like a few changes.

> Also it would be nice to have guess how wide spread this bug is
> as Adam asked - forh Alpha, we could be tolerant (but it really depends).
> Thanks.

I honestly don't know.  I would think the number of machines that run out of space while doing an OS install would be fairly low, as we only add a boot variable for shim.  However, it really depends on the state of the NVRAM before we install so it's hard to say.  If Peter or Matthew have a better idea on how widespread of a problem this would be, I'm sure they'll speak up.

(Note, I'm focusing on the default install here.  Not people using gummiboot or manually changing bootloaders, or doing other stuff that isn't part of getting Fedora booting.)

Comment 8 Kamil Páral 2013-04-09 12:19:22 UTC
Created attachment 733186 [details]
mac grub screen

It seems that Mac Mini is affected. See screenshot. You can't do anything. F19 Alpha RC1 UEFI (dd'd DVD to USB).

Comment 9 Matthew Garrett 2013-04-09 12:22:09 UTC
No, if grub is starting at all then you're not affected by this.

Comment 10 Kamil Páral 2013-04-09 13:33:18 UTC
I tried to reproduce on an ASUS UEFI motherboard and I received bug 950022. I don't know whether it is related, it could be.

Comment 11 Alexander Volovics 2013-04-10 09:46:34 UTC
As far as I can judge something like this seems to be happening
after install of Fedora 19 Alpha TC 1 on Lenovo Thinkpad T520 using UEFI.
(latest firmware).

The install appears to complete but at reboot grub does not
start, at least it 'tries to' but fails, leaving black screen.

I see nothing in the logs that appears related using rescue mode.

Comment 12 Matthew Garrett 2013-04-10 16:52:52 UTC
*** Bug 950022 has been marked as a duplicate of this bug. ***

Comment 13 Adam Williamson 2013-04-10 17:23:58 UTC
To explain the "See also: 949721":

#949721 is kind of a complementary bug here. Apparently, F17 and F18 have code to write kernel dumps to the UEFI firmware (so they can be recovered and debugged if the crash is particularly hard), but there was no corresponding mechanism for cleaning them out after a certain time, or after they're reported, or after a certain amount have accumulated. So if you have an F17 or F18 install that you've actually been using for a while, it's possible that mechanism will have completely filled your NVRAM with crash dumps. If you get affected by *that*, then you still won't be able to do an install even after this bug is fixed.

So what Josh and Matthew are looking at here is to implement both the upstream kernel patches for this bug  (once they're accepted), and a mechanism in anaconda to clean out kernel dumps from the NVRAM before attempting to do its efibootmgr runs.

Comment 14 Jaroslav Reznik 2013-04-10 17:43:23 UTC
(In reply to comment #13)
> mechanism in anaconda to clean out kernel dumps from the NVRAM before
> attempting to do its efibootmgr runs.

https://bugzilla.redhat.com/show_bug.cgi?id=950709

Comment 15 John Reiser 2013-04-10 17:55:56 UTC
Similar problem: bug 949786 "failed to set new efi boot target".  EFI install to bare metal ASUS P8Z68-V/GEN3, BIOS version 3402 (May 2012) using Fedora-19-Alpha-RC2 (and RC1)

Comment 16 Adam Williamson 2013-04-10 23:34:30 UTC
Yup, and smooge also had a failure that libreport considered a dupe of 949786.

Comment 17 Adam Williamson 2013-04-11 00:04:12 UTC
Description of problem:
Trying a UEFI install of F19 Alpha RC2; this will be the ENOSPC bug, probably.

Version-Release number of selected component:
anaconda-19.17-1

Additional info:
cmdline:        /usr/bin/python  /sbin/anaconda
cmdline_file:   BOOT_IMAGE=/syslinux/vmlinuz inst.stage2=hd:UUID=76E6-7761 quiet slub_debug=-x
executable:     /sbin/anaconda
hashmarkername: anaconda
kernel:         3.9.0-0.rc4.git0.1.fc19.x86_64
product:        Fedora
release:        Cannot get release name.
type:           anaconda
version:        19-Alpha

Truncated backtrace:
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/pyanaconda/threads.py", line 141, in run
    threading.Thread.run(self, *args, **kwargs)
  File "/usr/lib64/python2.7/threading.py", line 504, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib64/python2.7/site-packages/pyanaconda/install.py", line 141, in doInstall
    writeBootLoader(storage, payload, instClass, ksdata)
  File "/usr/lib64/python2.7/site-packages/pyanaconda/bootloader.py", line 2246, in writeBootLoader
    storage.bootloader.write()
  File "/usr/lib64/python2.7/site-packages/pyanaconda/bootloader.py", line 1690, in write
    self.install()
  File "/usr/lib64/python2.7/site-packages/pyanaconda/bootloader.py", line 1670, in install
    self.remove_efi_boot_target()
  File "/usr/lib64/python2.7/site-packages/pyanaconda/bootloader.py", line 1642, in remove_efi_boot_target
    raise BootLoaderError("failed to remove old efi boot entry")
BootLoaderError: failed to remove old efi boot entry

Comment 18 Charles R. Anderson 2013-04-11 05:08:32 UTC
Description of problem:
Fedora 19 Alpha RC2 install in UEFI mode booted from netinst.iso dd'ed to USB stick.
Minimal install on reclaimed disk /dev/sdb, default partitioning on whole disk (GPT).
Crash during "Installing bootloader".
Lenovo ThinkPad T520.

Version-Release number of selected component:
anaconda-19.17-1

Additional info:
cmdline:        /usr/bin/python  /sbin/anaconda
cmdline_file:   BOOT_IMAGE=/images/pxeboot/vmlinuz inst.stage2=hd:LABEL=Fedora\x2019-Alpha\x20x86_64 inst.updates=http://dlehman.fedorapeople.org/updates/updates-950510.0.img
executable:     /sbin/anaconda
hashmarkername: anaconda
kernel:         3.9.0-0.rc4.git0.1.fc19.x86_64
product:        Fedora
release:        Cannot get release name.
type:           anaconda
version:        19-Alpha

Truncated backtrace:
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/pyanaconda/threads.py", line 141, in run
    threading.Thread.run(self, *args, **kwargs)
  File "/usr/lib64/python2.7/threading.py", line 504, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib64/python2.7/site-packages/pyanaconda/install.py", line 141, in doInstall
    writeBootLoader(storage, payload, instClass, ksdata)
  File "/usr/lib64/python2.7/site-packages/pyanaconda/bootloader.py", line 2246, in writeBootLoader
    storage.bootloader.write()
  File "/usr/lib64/python2.7/site-packages/pyanaconda/bootloader.py", line 1690, in write
    self.install()
  File "/usr/lib64/python2.7/site-packages/pyanaconda/bootloader.py", line 1670, in install
    self.remove_efi_boot_target()
  File "/usr/lib64/python2.7/site-packages/pyanaconda/bootloader.py", line 1642, in remove_efi_boot_target
    raise BootLoaderError("failed to remove old efi boot entry")
BootLoaderError: failed to remove old efi boot entry

Comment 19 Charles R. Anderson 2013-04-11 07:48:25 UTC
After updating Lenovo ThinkPad T520 to the newest BIOS (going from version 1.31 to version 1.42) I no longer see this error and can successfully complete an EFI install of 19 Alpha RC2.

Comment 20 Jaroslav Reznik 2013-04-11 10:06:45 UTC
Could you try to wipe out pstore as described in Mathew's blog - http://mjg59.dreamwidth.org/23554.html - and try to reproduce this bug again? If reproducible - it would be probably this bug (so another option could be updating UEFI as it seems it makes difference too based on comment #19). Otherwise I'd say it would be more issue of not clearing pstore - and #950709 and we could document it in release notes in case we would be ok with workaround. Correct me, if I'm wrong ;-).

Comment 21 Kamil Páral 2013-04-11 10:30:21 UTC
I tested F19 Alpha RC2 on my production UEFI system. I ended up with "BootLoaderError: failed to remove old efi boot entry". However, it _did_ remove my old UEFI entry (pointing to an existing Fedora 18 installation), and it didn't create a new one, so I ended up with no boot entry at all. The system also didn't boot using a BIOS boot (that's another bug I suppose, the fallback should have worked), so I had no way to boot my previous system. I had a bloody hard time to figure out how to fix this. Eventually I find out that removing  /sys/fs/pstore/* will free up the NVRAM space and allow me to manually create a new UEFI boot entry using efibootmgr.

I don't think there is more than a few percent of our user base who would be able to resolve this.

Comment 22 Jaroslav Reznik 2013-04-11 11:16:23 UTC
(In reply to comment #21)
> Eventually I find out
> that removing  /sys/fs/pstore/* will free up the NVRAM space and allow me to
> manually create a new UEFI boot entry using efibootmgr.

Seems like this confirms it's really more about #950709 - if we clean up pstore, we have enough space there, at least reported for some systems.

> I don't think there is more than a few percent of our user base who would be
> able to resolve this.

Agree, it could be difficult for many users to resolve the issue. Question is if Release Notes and Common Bugs with guide what to do before installing UEFI would be sufficient or not.

Comment 23 Charles R. Anderson 2013-04-11 14:16:14 UTC
(In reply to comment #20)
> Could you try to wipe out pstore as described in Mathew's blog -
> http://mjg59.dreamwidth.org/23554.html - and try to reproduce this bug
> again? If reproducible - it would be probably this bug (so another option
> could be updating UEFI as it seems it makes difference too based on comment
> #19). Otherwise I'd say it would be more issue of not clearing pstore - and
> #950709 and we could document it in release notes in case we would be ok
> with workaround. Correct me, if I'm wrong ;-).

In my case where updating the BIOS "fixed" the problem, I still have the same contents of pstore that I had before updating the BIOS, so I don't think it cleaned up anything:

[root@localhost pstore]# find /sys/fs/pstore/ -ls
  8232    0 drwxr-xr-x   2 root     root            0 Apr 11 04:24 /sys/fs/pstore/
  8243    0 -r--r--r--   1 root     root         1024 Nov  2  2011 /sys/fs/pstore/dmesg-efi-1
  8242    0 -r--r--r--   1 root     root         1024 Nov  2  2011 /sys/fs/pstore/dmesg-efi-2
  8241    0 -r--r--r--   1 root     root         1024 Nov  2  2011 /sys/fs/pstore/dmesg-efi-3
  8240    0 -r--r--r--   1 root     root         1024 Nov  2  2011 /sys/fs/pstore/dmesg-efi-4
  8239    0 -r--r--r--   1 root     root         1024 Nov  2  2011 /sys/fs/pstore/dmesg-efi-5
  8238    0 -r--r--r--   1 root     root         1024 Nov  2  2011 /sys/fs/pstore/dmesg-efi-6
  8237    0 -r--r--r--   1 root     root         1024 Nov  2  2011 /sys/fs/pstore/dmesg-efi-7
  8236    0 -r--r--r--   1 root     root         1024 Nov  2  2011 /sys/fs/pstore/dmesg-efi-8
  8235    0 -r--r--r--   1 root     root         1024 Nov  2  2011 /sys/fs/pstore/dmesg-efi-9
  8234    0 -r--r--r--   1 root     root         1024 Nov  2  2011 /sys/fs/pstore/dmesg-efi-10
  8233    0 -r--r--r--   1 root     root         1024 Nov  2  2011 /sys/fs/pstore/dmesg-efi-11

Comment 24 John Reiser 2013-04-11 14:59:10 UTC
In my failing case (ASUS P8Z68-V/GEN3 with BIOS version 3402 from May 2012), I never had a kernel oops or crash, or anything else in pstore.  After "mount -t pstore /sys/fs/pstore /sys/fs/pstore" the directory /sys/fs/pstore was empty.  So pstore probably isn't the only problem.

In particular, multiple attempts at installing via UEFI can create enough garbage to get in the way.  See the output from "efibootmgr --verbose" in 
https://bugzilla.redhat.com/show_bug.cgi?id=949761#c5

Comment 25 Adam Williamson 2013-04-11 19:53:21 UTC
Discussed at 2013-04-11 F19 Alpha go/no-go meeting, acting as a blocker review meeting: http://meetbot.fedoraproject.org/fedora-meeting-1/2013-04-11/f19_alpha_gono-go_meeting.2013-04-11-17.00.log.html . Accepted as a blocker as testing so far indicates a lot of UEFI systems are susceptible to this bug, and it can leave the system in a somewhat messy state.

Comment 26 Alexander Volovics 2013-04-12 12:24:56 UTC
The same result as in Comment 24: /sys/fs/pstore was empty
(and also no kernel oops or crash on this machine ever)

I also cannot find any references to a 'bootloader error' of any kind.
So it seems there might be more ways problems can arise when doing an UEFI install.

Comment 27 Felix Kaechele 2013-04-14 00:33:10 UTC
On my ThinkPad X230 (latest EFI) the pstore was empty too.
However in /sys/firmware/efi/efivars I had lots of Files named dump-type0*
After deleting those I could use efibootmgr again.

Comment 28 Josh Boyer 2013-04-15 14:27:53 UTC
(In reply to comment #27)
> On my ThinkPad X230 (latest EFI) the pstore was empty too.
> However in /sys/firmware/efi/efivars I had lots of Files named dump-type0*
> After deleting those I could use efibootmgr again.

Was that with F19 or F18?  If it was with F19, was pstore mounted?

Comment 29 Josh Boyer 2013-04-15 14:34:45 UTC
(In reply to comment #24)
> In my failing case (ASUS P8Z68-V/GEN3 with BIOS version 3402 from May 2012),
> I never had a kernel oops or crash, or anything else in pstore.  After
> "mount -t pstore /sys/fs/pstore /sys/fs/pstore" the directory /sys/fs/pstore
> was empty.  So pstore probably isn't the only problem.

If you're using F19, systemd should be mounting pstore by default for you.  A manual mount should not be required.

Comment 30 Felix Kaechele 2013-04-15 14:47:28 UTC
(In reply to comment #28)
> (In reply to comment #27)
> > On my ThinkPad X230 (latest EFI) the pstore was empty too.
> > However in /sys/firmware/efi/efivars I had lots of Files named dump-type0*
> > After deleting those I could use efibootmgr again.
> 
> Was that with F19 or F18?  If it was with F19, was pstore mounted?

This was on F19 (Fedora-Live-Desktop-x86_64-19-Alpha-TC6-1.iso).
pstore was mounted. At least I could not mount it manually because it said something was already mounted there.

Comment 31 Josh Boyer 2013-04-15 20:30:19 UTC
I committed Matthew's V6 patches after they got an informal review from upstream.

Comment 32 Fedora Update System 2013-04-15 21:57:15 UTC
kernel-3.9.0-0.rc6.git2.3.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/FEDORA-2013-5224/kernel-3.9.0-0.rc6.git2.3.fc19

Comment 33 Lingzhu Xiang 2013-04-16 04:25:21 UTC
Created attachment 736131 [details]
console.log

efibootmgr can create entry with .git2.3 now. Other problems happened:

BUG: sleeping function called from invalid context at mm/slub.c:925
pstore: failed to load 2 record(s) from 'efi'
when mounting pstore. Looks like remnant of bug 877366.

After removing files in pstore, you can't rm dump-* again in efivarfs. Otherwise general protection fault happens in do_raw_spin_lock, the same pattern with bug 918422.

Upstream report is here http://thread.gmane.org/gmane.linux.kernel.efi/1075

Comment 34 Lingzhu Xiang 2013-04-16 04:39:47 UTC
(In reply to comment #29)
> If you're using F19, systemd should be mounting pstore by default for you. 
> A manual mount should not be required.

Note that I've been booting kernel with efivars.pstore_disable=N in case efi pstore isn't enabled by default. Is it what cause pstore to be empty?

Comment 35 Fedora Update System 2013-04-16 16:09:41 UTC
Package kernel-3.9.0-0.rc6.git2.3.fc19:
* should fix your issue,
* was pushed to the Fedora 19 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing kernel-3.9.0-0.rc6.git2.3.fc19'
as soon as you are able to, then reboot.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-5224/kernel-3.9.0-0.rc6.git2.3.fc19
then log in and leave karma (feedback).

Comment 36 Kamil Páral 2013-04-16 17:39:09 UTC
Petr Schindler did a UEFI installation today with F19 Alpha RC3, but he failed to add a comment here, so I'll add it: The installer fails during bootloader installation with "BootLoaderError: failed to remove old efi boot entry". /sys/fs/pstore is mounted and empty, so nothing to prune there. We deleted /sys/firmware/efi/efivars/dump* (10 files or so), but it didn't fix the issue, the following installation failed again with the same message. efibootmgr -v shows correct output, no binary mess. efibootmgr -c fails with exit code 1 and no error message.

Comment 37 Lingzhu Xiang 2013-04-17 12:30:59 UTC
On a different machine (Dell XPS 8500) I reproduced this with RC4 (.git2.3). 

A workaround is to reboot 30 times (each reboot generates some garbage; my nvram is 128K; YMMV) and trigger firmware garbage collection.

Comment 38 Kamil Páral 2013-04-17 12:38:32 UTC
(In reply to comment #36)
Martin Holec had exactly the same problem with ThinkPad T520 and F19 Alpha RC4. The only difference is that he had some old UEFI entries he was able to delete. If he deleted one of these old entries, he was then able to create a new UEFI entry.

Comment 39 Josh Boyer 2013-04-17 14:35:21 UTC
In the RC3/RC4 kernel, the EFI pstore backend is disabled.  That means that it doesn't register with the pstore filesystem and the dump files don't show up under /sys/fs/pstore/ so there is nothing for anaconda to remove.  That was inadvertent, however it also means that any kernel crashes done with those kernels won't write out to EFI NVRAM space either.

The other issues is that if you enable the option manually, or via kernel parameters, the pstore code itself seems fairly broken when trying to gather records stored in EFI.  The mount succeeds, but the record collection fails to complete as it should.  That is most likely what Lingzhu hit in comment #33.

At this point we avoid the pstore issue by default because the option is disabled by default in the Alpha kernel, but the dump-* files stick around.  Other than instructing people to remove them from /sys/firmware/efi/efivarfs/ manually, I'm not sure there's much that can be done.

Comment 40 Martin 2013-04-17 14:52:31 UTC
(In reply to comment #38)
> (In reply to comment #36)
> Martin Holec had exactly the same problem with ThinkPad T520 and F19 Alpha
> RC4. The only difference is that he had some old UEFI entries he was able to
> delete. If he deleted one of these old entries, he was then able to create a
> new UEFI entry.

Then I tried install RC4 again and it was successful.

Comment 41 Jaroslav Reznik 2013-04-17 15:26:03 UTC
(In reply to comment #39)
> In the RC3/RC4 kernel, the EFI pstore backend is disabled.  That means that
> it doesn't register with the pstore filesystem and the dump files don't show
> up under /sys/fs/pstore/ so there is nothing for anaconda to remove.  That
> was inadvertent, however it also means that any kernel crashes done with
> those kernels won't write out to EFI NVRAM space either.

Is EFI pstore backend enabled in current F17/F18 kernels (if I understood it correctly it is)? Shouldn't we disable it too in that case too?

> The other issues is that if you enable the option manually, or via kernel
> parameters, the pstore code itself seems fairly broken when trying to gather
> records stored in EFI.  The mount succeeds, but the record collection fails
> to complete as it should.  That is most likely what Lingzhu hit in comment
> #33.
> 
> At this point we avoid the pstore issue by default because the option is
> disabled by default in the Alpha kernel, but the dump-* files stick around. 
> Other than instructing people to remove them from
> /sys/firmware/efi/efivarfs/ manually, I'm not sure there's much that can be
> done.

Reading this bug, #919485 and Googling - it seems like we have to document/recommend manual efivars cleanup (with up to 30 restarts? wow), for some systems firmware update or resetting BIOS settings but that's probably everything we can do for people with entries in efi vars and hitting variants/combinations of efivars related bugs.

Comment 42 Kamil Páral 2013-04-17 16:35:59 UTC
Josh, can you please make a guess why our ASUS board (comment 36) can't be cleaned up? We have removed all dump* files, we don't see anything extraordinary, but we still can't create a new UEFI boot entry.

$ efibootmgr -v
BootCurrent: 0007
Timeout: 3 seconds
BootOrder: 0001,0003,0002,0007
Boot0001* Hard Drive	BIOS(2,0,00)
Boot0002* Network Card	BIOS(6,0,00)Realtek PXE B02 D00.
Boot0003* CD/DVD Drive	BIOS(3,0,00)P3: ATAPI   DVD D  DH16DYS    .
Boot0007* UEFI: KingstonDT Elite 3.0 1.01	ACPI(a0341d0,0)PCI(7,0)PCI(0,0)USB(2,0)HD(1,800,1d6e800,646bcc0d-6df7-4774-9b53-3e55ac381dd0)

$ ls -l /sys/firmware/efi/efivars
total 0
-rw-r--r--. 1 root root   36 Apr 17  2013 AMIMemInfo-43387991-1223-7645-b5bb-aa7675c5c8ef
-rw-r--r--. 1 root root   85 Apr 17  2013 AMITSESetup-c811fa38-42c8-4579-a9bb-60e94eddfb34
-rw-r--r--. 1 root root    8 Apr 17  2013 AcpiGlobalVariable-af9ffd67-ec10-488a-9dfc-6cbf5ee22c2e
-rw-r--r--. 1 root root   45 Apr 17  2013 Boot0001-8be4df61-93ca-11d2-aa0d-00e098032b8c
-rw-r--r--. 1 root root   69 Apr 17  2013 Boot0002-8be4df61-93ca-11d2-aa0d-00e098032b8c
-rw-r--r--. 1 root root   80 Apr 17  2013 Boot0003-8be4df61-93ca-11d2-aa0d-00e098032b8c
-rw-r--r--. 1 root root  150 Apr 17  2013 Boot0007-8be4df61-93ca-11d2-aa0d-00e098032b8c
-rw-r--r--. 1 root root    6 Apr 17  2013 BootCurrent-8be4df61-93ca-11d2-aa0d-00e098032b8c
-rw-r--r--. 1 root root    8 Apr 17  2013 BootFlow-ef152fb4-7b2f-427d-bdb4-7e0a05826e64
-rw-r--r--. 1 root root   12 Apr 17  2013 BootOrder-8be4df61-93ca-11d2-aa0d-00e098032b8c
-rw-r--r--. 1 root root    6 Apr 17  2013 CMOSfailflag-c89dc9c7-5105-472c-a743-b1621e142b41
-rw-r--r--. 1 root root   40 Apr 17  2013 ConErrDev-8be4df61-93ca-11d2-aa0d-00e098032b8c
-rw-r--r--. 1 root root   38 Apr 17  2013 ConIn-8be4df61-93ca-11d2-aa0d-00e098032b8c
-rw-r--r--. 1 root root   38 Apr 17  2013 ConInDev-8be4df61-93ca-11d2-aa0d-00e098032b8c
-rw-r--r--. 1 root root   40 Apr 17  2013 ConOut-8be4df61-93ca-11d2-aa0d-00e098032b8c
-rw-r--r--. 1 root root   40 Apr 17  2013 ConOutDev-8be4df61-93ca-11d2-aa0d-00e098032b8c
-rw-r--r--. 1 root root    8 Apr 17  2013 CpuS3Resume-30b98b95-dfa3-4501-a3ce-e38c186384a0
-rw-r--r--. 1 root root    7 Apr 17  2013 EfiTime-9d0da369-540b-46f8-85a0-2b5f2c301e15
-rw-r--r--. 1 root root    5 Apr 17  2013 HpcModeBU-8be4df61-93ca-11d2-aa0d-00e098032b8c
-rw-r--r--. 1 root root    7 Apr 17  2013 Lang-8be4df61-93ca-11d2-aa0d-00e098032b8c
-rw-r--r--. 1 root root   28 Apr 17  2013 LangCodes-8be4df61-93ca-11d2-aa0d-00e098032b8c
-rw-r--r--. 1 root root   12 Apr 17  2013 LegacyDevChecksum-a56074db-65fe-45f7-bd21-2d2bdd8e9652
-rw-r--r--. 1 root root   30 Apr 17  2013 LegacyDevOrder-a56074db-65fe-45f7-bd21-2d2bdd8e9652
-rw-r--r--. 1 root root  588 Apr 17  2013 MemoryS3SaveNv-b1cfc482-4cb2-4cee-9b00-ce2579ec7186
-rw-r--r--. 1 root root   12 Apr 17  2013 MemoryS3SaveVol-0a51b41d-de21-43fe-be27-d6dbc9efd104
-rw-r--r--. 1 root root   12 Apr 17  2013 MemoryS3SaveVolLength-0a51b41d-de21-43fe-be27-d6dbc9efd104
-rw-r--r--. 1 root root    8 Apr 17  2013 MonotonicCounter-8be4df61-93ca-11d2-aa0d-00e098032b8c
-rw-r--r--. 1 root root   20 Apr 17  2013 NBMemoryLength-490216c0-076a-44d3-a536-ace05c90e386
-rw-r--r--. 1 root root    7 Apr 17  2013 PNP0400_0_NV-560bf58a-1e0d-4d7e-953f-2980a261e031
-rw-r--r--. 1 root root   13 Apr 17  2013 PNP0400_0_VV-560bf58a-1e0d-4d7e-953f-2980a261e031
-rw-r--r--. 1 root root    7 Apr 17  2013 PNP0501_0_NV-560bf58a-1e0d-4d7e-953f-2980a261e031
-rw-r--r--. 1 root root   13 Apr 17  2013 PNP0501_0_VV-560bf58a-1e0d-4d7e-953f-2980a261e031
-rw-r--r--. 1 root root    7 Apr 17  2013 PNP0501_1_NV-560bf58a-1e0d-4d7e-953f-2980a261e031
-rw-r--r--. 1 root root   13 Apr 17  2013 PNP0501_1_VV-560bf58a-1e0d-4d7e-953f-2980a261e031
-rw-r--r--. 1 root root    7 Apr 17  2013 PNP0510_0_NV-560bf58a-1e0d-4d7e-953f-2980a261e031
-rw-r--r--. 1 root root   13 Apr 17  2013 PNP0510_0_VV-560bf58a-1e0d-4d7e-953f-2980a261e031
-rw-r--r--. 1 root root    7 Apr 17  2013 PNP0604_0_NV-560bf58a-1e0d-4d7e-953f-2980a261e031
-rw-r--r--. 1 root root   13 Apr 17  2013 PNP0604_0_VV-560bf58a-1e0d-4d7e-953f-2980a261e031
-rw-r--r--. 1 root root    8 Apr 17  2013 PlatformLang-8be4df61-93ca-11d2-aa0d-00e098032b8c
-rw-r--r--. 1 root root   36 Apr 17  2013 PlatformLangCodes-8be4df61-93ca-11d2-aa0d-00e098032b8c
-rw-r--r--. 1 root root  486 Apr 17  2013 Setup-ec87d643-eba4-4bb5-a1e5-3f3e36b20da9
-rw-r--r--. 1 root root    6 Apr 17  2013 SetupAccFeatures-ec87d643-eba4-4bb5-a1e5-3f3e36b20da9
-rw-r--r--. 1 root root   13 Apr 17  2013 SetupCpuFeatures-ec87d643-eba4-4bb5-a1e5-3f3e36b20da9
-rw-r--r--. 1 root root    5 Apr 17  2013 SetupHWMFeatures-ec87d643-eba4-4bb5-a1e5-3f3e36b20da9
-rw-r--r--. 1 root root 3009 Apr 17  2013 StdDefaults-4599d26f-1a11-49b8-b91f-858745cff824
-rw-r--r--. 1 root root    6 Apr 17  2013 Timeout-8be4df61-93ca-11d2-aa0d-00e098032b8c
-rw-r--r--. 1 root root   12 Apr 17  2013 UMAVar.-8be4df61-93ca-11d2-aa0d-00e098032b8c
-rw-r--r--. 1 root root   12 Apr 17  2013 UMAVarB.-8be4df61-93ca-11d2-aa0d-00e098032b8c
-rw-r--r--. 1 root root    6 Apr 17  2013 UsbMassDevNum-ec87d643-eba4-4bb5-a1e5-3f3e36b20da9
-rw-r--r--. 1 root root  712 Apr 17  2013 VARSTORE_OCMR_SETTINGS_NAME-c05fba7d-7a92-49e0-bcee-233b14dca803
-rw-r--r--. 1 root root 1044 Apr 17  2013 VARSTORE_OCMR_TIMING_SETTINGS_NAME-93c483a1-d3fa-4e92-b437-733b2a346e23

$ efibootmgr -c -L foo; echo $?
1
(and the 'foo' item is not added)

Anaconda installations therefore fail. Any ideas?

Comment 43 Josh Boyer 2013-04-17 16:52:34 UTC
My only guess is that the firmware did not do garbage collection after you removed the dump vars.

Comment 44 Kamil Páral 2013-04-17 17:17:13 UTC
A firmware update "fixed" the issue, we can create new UEFI boot entries now.

Comment 45 Peter Jones 2013-04-17 17:22:32 UTC
*** Bug 949786 has been marked as a duplicate of this bug. ***

Comment 46 Fedora Update System 2013-04-19 05:47:27 UTC
kernel-3.9.0-0.rc6.git2.3.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 47 Um Nontasuwan 2013-04-25 11:10:30 UTC
Update BIOS to ver. 218 fixed this problem for me. 
Fedora 19 alpha - Asus Zenbook Prime UX31A.

Comment 48 Japplo 2013-05-30 09:39:41 UTC
Same Problem in F19 Beta, the solution was for me to delete the dump-type0* files mentioned in Comment 27, Thanks

Comment 49 Adam Williamson 2013-06-05 19:29:14 UTC
Heads up, folks: there's a substantial change to how the kernel handles this whole mess in 19 Final TC1, available at https://dl.fedoraproject.org/pub/alt/stage/19-TC1/ . We're *relatively* confident this should be rather better than what was in 18, Alpha and Beta. If folks could try it out - especially those of you who have systems known to run into this issue a lot - and let us know how it goes, that'd be great.

http://mjg59.dreamwidth.org/25091.html explains the technical details of the change that went into TC1.

Comment 50 John Reiser 2013-06-06 01:56:16 UTC
(In reply to Adam Williamson from comment #49)
> Heads up, folks: there's a substantial change to how the kernel handles this
> whole mess in 19 Final TC1 ...

It still fails on ASUS P8Z68-V/GEN3; original report in bug 949786 "BootLoaderError: failed to set new efi boot target", marked as duplicate of this bug 947142.

Default Gnome desktop fresh install from Fedora-19-TC1-x86_64-DVD.iso after "livecd-iso-to-disk --efi" onto USB flash memory.

anaconda-19.30.2-1
kernel-3.9.4-300.fc19.x86_64

Comment 51 John Reiser 2013-06-06 02:51:22 UTC
TC1 was not composed according to the request.

The compose request ticket
  https://fedorahosted.org/rel-eng/ticket/5623
asks for
  https://koji.fedoraproject.org/koji/buildinfo?buildID=424421
which is
  kernel-3.9.4-301.fc19
TC1 contains
  Packages/k/kernel-3.9.4-300.fc19.x86_64.rpm

Note the mismatch between requested "-301" and actual "-300".

Comment 52 Adam Williamson 2013-06-06 04:27:28 UTC
Right, looks like releng missed the kernel build :( Indeed, it needs to be -301 to have the fix. So, wait for TC2 I guess.

Comment 53 João Carlos Mendes Luís 2013-06-07 04:25:13 UTC
This can also be reproduced by using a VirtualBox VM with EFI activated.  Hope this make easier for everybody to test.

Comment 54 Adam Williamson 2013-06-08 00:08:11 UTC
I'm kinda reluctant to use EFI virtualization for testing, as it's still pretty early and can differ quite a lot from the implementations on real systems. But it sure beats nothing.

Comment 55 John Reiser 2013-06-08 04:16:32 UTC
Success!  Final TC2 of Fedora-19-x86_64-DVD.iso installs default Gnome desktop successfully after "livecd-iso-to-disk --efi" onto USB flash memory, then booting the install from USB flash memory.   The installed system boots successfully.  The UEFI boot even inter-operates with an existing UEFI boot of Windows Vista on the same drive.  Both systems share the same 100MB EFI System Partition, using different directories 'fedora' and 'Microsoft'.

anaconda-19.30.3-1
kernel-3.9.4-301.fc19.x86_64

Comment 56 Adam Williamson 2013-06-08 04:44:45 UTC
A UEFI install of Vista?! Isn't that banned under the Geneva Convention? :P

Good news, thanks for testing!

Comment 57 John Reiser 2013-06-08 13:53:47 UTC
(In reply to Adam Williamson from comment #56)
> A UEFI install of Vista?!
http://www.techpowerup.com/forums/showthread.php?t=167245  "Installing Windows Vista/7 on a GUID Partition Table"
(The problems may come later, when "Repair your system" via MS DVD refuses.)

Comment 58 Adam Williamson 2013-06-08 15:42:03 UTC
jreiser: I wasn't expressing surprise that such a thing is possible, just that it has not been outlawed by all right-thinking nations ;)

Comment 59 Russ Anderson 2013-06-10 17:10:20 UTC
Apparently I just hit this problem trying to install FC19-beta.
The reason I say apparently is the anaconda install failed trying to write
EFI variables.  Clicked on the buttons to open a BZ and it flagged it as
a duplicate of this BZ (and dropped all the failing data).

This was using Fedora-19-Beta-x86_64-netinst.iso downloaded today from the
fedora website and installing on an SGI UV2 system.

If what I am hitting is not the same as this BZ, then I need some way to trick
anaconda into opening a new BZ with the failing info.

Comment 60 Matthew Garrett 2013-06-10 17:22:39 UTC
The beta did not have this fix. Please try the TC2 image at https://dl.fedoraproject.org/pub/alt/stage/19-TC2/

Comment 61 Russ Anderson 2013-06-10 20:34:46 UTC
FC19-TC2 failed the same way.  Same backtrace as BZ 949786.  Anaconda refuses
to open a new BZ, indicating it is a dup of this one.

Note this should not be a case of nvram being full.

FWIW, we have a backdoor way of printing EFI variables.
These are the variables set after the failure.

---------------------------------------
name: Boot0000
name: Boot0001
name: Boot0002
name: Boot0003
name: Boot0004
name: BootOrder
name: ConIn
name: ConOut
name: EDD30
name: GigNVData
name: Lang
name: LastEnumLang
name: LegacyDevOrder
name: MTC
name: MemoryTypeInformation
name: PlatformLang
name: RTC
name: Setup
name: SetupApicMode
name: SetupAutoSelectMNOff
name: SetupMIoValue
name: SetupMvalue
name: SetupNIoValue
name: SetupNodeType
name: SetupNvalue
name: copy
name: cr
name: del
name: dir
name: dump-type0-1-1370814147
name: dump-type0-10-1370814147
name: dump-type0-11-1370814147
name: dump-type0-2-1370814147
name: dump-type0-3-1370814147
name: dump-type0-4-1370814147
name: dump-type0-5-1370814147
name: dump-type0-6-1370814147
name: dump-type0-7-1370814147
name: dump-type0-8-1370814147
name: dump-type0-9-1370814147
name: md
name: rd

Comment 62 George Beshers 2013-06-10 21:07:27 UTC
Reopening as we are seeing this failure on UV2000

The machine is semi-under beaker
   ssh root.bos.redhat.com
   ssh root.eng.bos.redhat.com

This breakage is part of what is inhibiting making
the machine generally available.

This is the point of failure in Anaconda:

1638                        continue
1639                                
1640                    rc = self.efibootmgr("-b", slot_id, "-B",
1641                                         root=ROOT_PATH)     
1642                    if rc:                                   
1643 ->                     raise BootLoaderError("failed to remove old efi boot entry")
1644                                                                                    
1645        @property                                                                   
1646        def efi_dir_as_efifs_dir(self):                                             
1647            ret = self._config_dir.replace('efi/', '')                              
1648            return "\\" + ret.replace('/', '\\')                                    

(Pdb) p slot_id                                                                         
'0003'
(Pdb) p slot
'Boot0003*'
(Pdb) p _product
'Red Hat Enterprise Linux'
(Pdb)

Comment 63 Matthew Garrett 2013-06-10 21:21:15 UTC
Closing this again - we'll track the SGI issue in #972945.


Note You need to log in before you can comment on or make changes to this bug.