Bug 1334075 - TPM prevents grub menu, drops to grub rescue; BIOS settings no help
Summary: TPM prevents grub menu, drops to grub rescue; BIOS settings no help
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: grub2
Version: 24
Hardware: i686
OS: Linux
unspecified
urgent
Target Milestone: ---
Assignee: Peter Jones
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: AcceptedFreezeException
Depends On:
Blocks: F24FinalFreezeException
TreeView+ depends on / blocked
 
Reported: 2016-05-08 02:44 UTC by Peter Gückel
Modified: 2016-06-14 08:40 UTC (History)
13 users (show)

Fixed In Version: grub2-2.02-0.34.fc24
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-06-14 08:40:24 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
anaconda.log (177.36 KB, text/plain)
2016-05-11 20:45 UTC, Peter Gückel
no flags Details
journal.log (1.79 MB, text/x-vhdl)
2016-05-11 20:46 UTC, Peter Gückel
no flags Details
program.log (96.95 KB, text/plain)
2016-05-12 03:27 UTC, Peter Gückel
no flags Details
Don't fail on TPM errors (735 bytes, patch)
2016-05-20 16:16 UTC, Matthew Garrett
no flags Details | Diff

Description Peter Gückel 2016-05-08 02:44:29 UTC
Description of problem:
Clean install of Fedora Workstation 24 20160506.n.0 onto laptop. Went fine, rebooted, no grub menu! Instead, TPM error 1 and the grub rescue prompt.

Version-Release number of selected component (if applicable):
grub2 from Fedora 24 beta (I don't know the number, but it's the one on the Workstation live disk)

How reproducible:
Install disc.

Steps to Reproduce:
1.
2.
3. reinstall this OS and it's the same thing

Actual results:
no grub menu; instead a nasty TPM error 1 and a grub rescue prompt

Expected results:
grub menu

Additional info:
I checked the BIOS and verified that TPM is disabled. I searched the web, found forum discussion, primarily older Uuntu, that suggested either enable or disable it, so I tried both. I also tried resetting to factory default. Also, I tried the 2-3 other available settings (one about a certificate, or something). None of the many permitations and combinations I was able to come up with made a difference. I was unable to get a grub menu or to boot the system. I tried getting to another virtual terminal, the one with the shell, to see if I could see anything, but everything is frozen. The grub rescue prompt is all that will take any input and it isn't allowing me access to any o the files on the disc. My access to my own computer appears to be blocked. I have used this computer with Fedora exclusively since acquiring it in the mid-00s without issue... and now this!

Comment 1 Peter Gückel 2016-05-08 03:30:39 UTC
Suggested was that I add the output of dmesg|grep DMI. Unfortuntely, I have been unable to keep the firmware up-to-date, since Windows is required to do so. There is a LinuxBIOS project I scanned one in a blue moon, but I never found anything I felt sure enough about to try out, so I've got pretty much what was on the computer when I got it (second hand, but quite new at the time and still pretty much in the shape I got it in).

So, here's the output:

[    0.000000] DMI: Hewlett-Packard HP Compaq nc6220 (PU982AW#ABA)/308A, BIOS 68DTU Ver. F.13 02/27/2007

Comment 2 sixpack13 2016-05-10 23:47:02 UTC
cit: "I have been unable to keep the firmware up-to-date, since Windows is required to do so."


you can download hiren's boot CD
http://www.hiren.info/pages/bootcd
or virus free from here
http://www.heise.de/download/hirens-bootcd-1186184.html


boot from CD, start the "Mini Windows XP" from it and *try*[1] to update your bios.

BIOS (double check this !!!):
http://h20564.www2.hp.com/hpsc/swd/public/readIndex?sp4ts.oid=447304&swLangOid=8&swEnvOid=1098


HINT: 
with my acer NB I need to keep my power cord plugged in during bios update !
if not it's prevent to start the BIOS Update without any message.

[1]
I do this with an WinPE, so I never tried BIOS update via Hiren ...

Comment 3 Peter Gückel 2016-05-11 05:38:28 UTC
I've been fiddling with FreeDOS, trying to get it onto a USB stick, but it wouldn't boot, then another hour with Hiren's and, finally, with the help of mini WinXP, I got the BIOS flashed to the most recent version: 07/24/2009 F.16 !!!

I am so encouraged, I might try to flash my desktop one day soon, too :-O I wonder how I could flash the firmware of my amplifier, though?

Anyway, back to the TPM. Unfortunately, nothing has changed :-( The status is as indicated in Comment 1.

Comment 4 Brian Lane 2016-05-11 17:03:44 UTC
Please attach the logs from the install, either from /tmp/*log before rebooting or from the failed install in /var/log/anaconda if you can get to it using a rescue disk.

Comment 5 Peter Gückel 2016-05-11 20:45:38 UTC
Created attachment 1156306 [details]
anaconda.log

Comment 6 Peter Gückel 2016-05-11 20:46:36 UTC
Created attachment 1156307 [details]
journal.log

Comment 7 Peter Gückel 2016-05-11 20:54:40 UTC
There was nothing called *log in /tmp, just x and unix sockets and stuff.

I scoured /var and /var/tmp and /var/log and the only things that seemed at all relevant (and that you asked for) are: anaconda.log and journal.log.

In case it is significant, I still have not been able to boot the computer or get past the grub rescue prompt, despite having tried numerous times. The attached files should therefore be unaltered from their state at the time of installation.

Comment 8 Brian Lane 2016-05-11 23:50:12 UTC
There should be a program.log next to the anaconda.log, that'll have the output  of the grub2 install in it and may show something useful.

Comment 9 Peter Gückel 2016-05-12 03:27:09 UTC
Created attachment 1156416 [details]
program.log

Comment 10 Peter Gückel 2016-05-12 03:36:18 UTC
I just noticed at the end of the program.log that it shows me creating a user account for myself. I sure hope it doesn't show my user or root passwords in any of these files :-O

Comment 11 Brian Lane 2016-05-12 15:19:05 UTC
Darn, there's no indication of what's going wrong in the logs.

No, passwords aren't logged. Even with the anaconda-tb-* object dump we filter them out.

Comment 12 Peter Gückel 2016-05-15 15:41:36 UTC
I still cannot get past the TPM error and the subsequent display of the grub rescue prompt.

Comment 13 Peter Gückel 2016-05-20 15:09:40 UTC
Is there a fix? How do I get my computer booted and running?

Comment 14 Chris Murphy 2016-05-20 15:46:22 UTC
There is no fix, but there is a work around which is to downgrade to the Fedora 23 grub package and grub2-install. I posted a draft of the steps on the test@ list.

Comment 15 Matthew Garrett 2016-05-20 16:16:22 UTC
Created attachment 1160008 [details]
Don't fail on TPM errors

This should work around the issue

Comment 16 Peter Gückel 2016-05-20 17:20:36 UTC
(In reply to Matthew Garrett from comment #15)
> Created attachment 1160008 [details]
> Don't fail on TPM errors
> 
> This should work around the issue

I used to build my own custom kernels back in the late '90s, due to the need for special hardware drivers/modules/whatever, but I haven't done so for aeons. I was told a few years ago that the whole kernel building procedure has totally changed.

I don't know if this is affected, but...

How do I apply this patch?

And, I suppose that means—assuming that I do finally get the machine running—that I must exclude grub from dnf updates... until a fix is found (or is this the actual fix?), for Fedora 24 or for as long as I continue to use this old, but still good, computer?

Comment 17 Chris Murphy 2016-05-20 17:27:00 UTC
Peter it's a grub patch, not a kernel patch. If you can't apply the patch to the source rpm and compile it yourself and test it, it may be possible to get a scratch build for you to test.

Comment 18 Peter Gückel 2016-05-20 18:22:12 UTC
(In reply to Chris Murphy from comment #17)
> Peter it's a grub patch, not a kernel patch. If you can't apply the patch to
> the source rpm and compile it yourself and test it, it may be possible to
> get a scratch build for you to test.

Chris, I chrooted like you said and I now have a running computer, with the F23 grub2 installed :-)

Yeah, it would be a lot easier to get a scratch rpm ;-)

Comment 19 Adam Williamson 2016-05-20 20:56:16 UTC
matthew: you've got a tabs/spaces mix in that patch (your change uses spaces, the original uses tabs)

Comment 20 Matthew Garrett 2016-05-20 21:04:18 UTC
The original was incorrect

Comment 21 Adam Williamson 2016-05-20 21:16:56 UTC
well fine, but your patch winds up with the whitespace on the relevant line being one tab followed by two spaces. that's never right. ;)

Comment 22 Adam Williamson 2016-05-20 21:17:24 UTC
Scratch build: http://koji.fedoraproject.org/koji/taskinfo?taskID=14193080

Comment 23 Chris Murphy 2016-05-20 21:26:30 UTC
Peter, download these:
https://kojipkgs.fedoraproject.org//work/tasks/3082/14193082/grub2-2.02-0.30.1.awb.i686.rpm
https://kojipkgs.fedoraproject.org//work/tasks/3082/14193082/grub2-tools-2.02-0.30.1.awb.i686.rpm

Then 'dnf upgrade *rpm' and then 'grub2-install <dev>' where <dev> is your boot drive, presumably /dev/sda. Then reboot. My expectation is you may briefly see the TPM error but it won't be fatal now.

Comment 24 Matthew Garrett 2016-05-20 21:42:54 UTC
Oops, I see what you mean. Yup, will fix that before submitting it properly.

Comment 25 Peter Gückel 2016-05-20 23:28:07 UTC
OK, guys, I installed grub2-2.02-0.1.awb*.rpm and I catch a glimpse of something, so brief that I can barely register whether it really is there (the TPM error Chris mentions?), before the grub menu appears. It works!!!

So, is this patch now live and all future grub2 updates will have it? I'm not going t end up with a non-patched grub2 from updates or updates-testing, am I?

Comment 26 Adam Williamson 2016-05-20 23:39:04 UTC
"So, is this patch now live and all future grub2 updates will have it?"

No. I don't usually send grub changes to production, I leave that to pjones. He's the expert, I'd prefer he look over this before it gets pushed out properly.

"I'm not going t end up with a non-patched grub2 from updates or updates-testing, am I?"

If Peter does a 0.31 build of grub2 without including this patch, you would get it (I actually intentionally version my scratch builds to be superseded by the next 'normal' build, so you don't get stuck out on a non-supported limb).

Comment 27 Peter Gückel 2016-05-21 00:14:49 UTC
(In reply to Adam Williamson from comment #26)
> If Peter does a 0.31 build of grub2 without including this patch, you would
> get it (I actually intentionally version my scratch builds to be superseded
> by the next 'normal' build, so you don't get stuck out on a non-supported
> limb).

Uhhh...? So, you mean that the next update will not have the patch unless Peter (Jones) builds with the patch? So, I will need to be very cautious when upgrading, unless I receive notification here that a package numbered such and such will be okay, right?

Comment 28 Samuel Sieb 2016-05-21 03:15:20 UTC
I'm pretty sure that upgrading the grub package doesn't do a grub2-install, so you're fine.

Comment 29 Peter Gückel 2016-05-23 17:35:54 UTC
Well, it's working, but it's not perfect.

Due to another problem, now resolved, I had to reinstall grub. Since then, things have changed, mysteriously. I checked and there have been no grub2 updates and I that I still have this 'awb' scratch version installed.

What now happens is that after the BIOS screen, an intermediate screen appears that is filled from top to bottom with TPM error messages. Without me doing anything (or did I hit return once? no, I think not), it switches to the grub menu.

Well, that would be ok, but...

When I select a kernel from the menu and type return, the screen blanks and the TPM error message appears with the word More underneath. I click return and another line of TPM error with more ad infinitum. This goes on until I reach the end of the screen or even further (the message is always exactly the same, so it is not possible to determine whether scrolling is occuring), and then, all of a sudden, it boots!

I presume all of those error messages actually do nothing and grub simply reaches its timeout and boots.

It's not pretty. Other than that, everything seems to be working.

Comment 30 Adam Williamson 2016-05-23 17:38:56 UTC
well, the patch basically tells grub that when it gets a TPM error it should just print it and carry on, instead of blowing up.

so yeah, if the error occurs multiple times, it'll get printed each time, presumably.

Comment 31 Adam Williamson 2016-05-31 18:19:02 UTC
per IRC discussion this morning, pjones is planning to drop the whole TPM patch set for F24 as it's clearly not ready.

Comment 32 Adam Williamson 2016-05-31 18:19:29 UTC
Nominating as a Final FE, obviously fixing non-booting systems is a good thing.

Comment 33 Chris Murphy 2016-05-31 18:51:13 UTC
+1FE it's affecting some x86_64 as well it seems so it could possibly even be a conditional blocker

Comment 34 Petr Schindler 2016-06-06 12:21:11 UTC
+1 FE.

Comment 35 Geoffrey Marr 2016-06-06 19:36:27 UTC
This bug was discussed at the 2016-06-06 blocker review meeting [1] and determined to be an AcceptedFreezeException as it currently breaks boot. QA approval was given to fix this bug or drop the TPM patchset as part of the freeze-breaking grub2 update.

[1] https://meetbot.fedoraproject.org/fedora-blocker-review/2016-06-06/f24-blocker-review.2016-06-06-16.00.txt

Comment 36 Fedora Update System 2016-06-09 20:18:21 UTC
grub2-2.02-0.33.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-c4d43baacc

Comment 37 Chris Murphy 2016-06-10 00:09:40 UTC
Hey so if people could grab this update and install it, then do a 'grub2-install /dev/sdX' pointed at your boot drive, reboot, and report back if it's fixed, that'd be great!

Comment 38 Peter Gückel 2016-06-10 02:08:39 UTC
I installed the packages and installed grub2 and rebooted.

After the BIOS screen, there was a momentary transitional screen that went by fairly fast. It appeared to have a period in the upper left corner. Then, the grub menu appeared. After I selected the uppermost (default) kernel, there was another transitional screen with a flashing cursor in the upper left corner. This lingered for a bit, before transitioning to the plymouth boot sequence.

I'm not sure I ever saw these two transitional screens before (the period or whatever it was immediately before and the flashing cursor immediately after the grub menu), but everything progressed as it should and the computer appeared to work normally... and no unsightly, albeit inactivated, TPM error messages. Yay!

Comment 39 Peter Gückel 2016-06-10 02:09:59 UTC
I should add that I only tried the update on my i686 laptop that had problems. The x86_64 desktop was working just fine and I have not tried the update there (here).

Comment 40 Tomasz Torcz 2016-06-10 05:49:27 UTC
Build 2.0.2-0.33 fixes problem for me, on HP model ProLiant DL585 G5 BIOS A07.

Comment 41 Fedora Update System 2016-06-10 18:00:08 UTC
grub2-2.02-0.33.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-c4d43baacc

Comment 42 Fedora Update System 2016-06-10 19:24:17 UTC
grub2-2.02-0.33.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-c4d43baacc

Comment 43 Fedora Update System 2016-06-11 11:51:09 UTC
grub2-2.02-0.34.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-c4d43baacc

Comment 44 Fedora Update System 2016-06-13 14:37:57 UTC
grub2-2.02-0.33.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-c4d43baacc

Comment 45 Fedora Update System 2016-06-13 14:41:08 UTC
grub2-2.02-0.33.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-c4d43baacc

Comment 46 Fedora Update System 2016-06-13 14:44:27 UTC
grub2-2.02-0.33.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-c4d43baacc

Comment 47 Fedora Update System 2016-06-13 15:58:02 UTC
grub2-2.02-0.33.fc24, grub2-2.02-0.34.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-c4d43baacc

Comment 48 Fedora Update System 2016-06-14 08:39:51 UTC
grub2-2.02-0.34.fc24 has been pushed to the Fedora 24 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.