Bug 1164937

Summary: ThinkPad T540p does not resume from suspend when TPM is enabled but no driver is loaded
Product: [Fedora] Fedora Reporter: rh
Component: kernelAssignee: Kernel Maintainer List <kernel-maint>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 21CC: gansalmon, itamar, jonathan, kernel-maint, luto, madhu.chinakonda, mchehab, pbrobinson, sb, valent.turkovic
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: kernel-3.17.4-200.fc20 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-11-25 22:36:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description rh 2014-11-17 21:48:12 UTC
Description of problem:

The Lenovo ThinkPad T540p notebook (and possibly other models) cannot resume from suspend to RAM when the onboard TPM chip is enabled in the BIOS.

Version-Release number of selected component (if applicable):
kernel 3.17.3-300.fc21

How reproducible:
Every time

Steps to Reproduce:
1. Close the lid
2. Open the lid

Actual results:
Screen is black, backlight is off, nothing happens.
Only pressing the power button for a few seconds can make the laptop power off and boot again.

Expected results:
Computer resumes from suspend.

Additional info:
This affects at least one other user: 
https://ask.fedoraproject.org/en/question/55343/thinkpad-t540p-wont-wake-from-sleep/

Installing kernel-modules-extra (which contains the tpm_tis module) fixes this completely. It should maybe be integrated into the core kernel package or be installed automatically on thinkpads.
In my case, the chip has to be enabled for BitLocker to work on Windows, so disabling it is not an option. But I remember that suspend worked before the TPM was enabled by Windows.

Comment 1 Valent Turkovic 2014-11-18 19:47:29 UTC
Thanks for tracking this issue down!
I have exact same issue with suspend and resume on Lenovo T440s

I can confirm that after installing kernel-modules-extra now resume also works.

I can also report that suspend now takes almost twice long, and screen blinks quite few more times like this: on-pressed_suspend-off-on-off-on-off

But main thing is that both suspend and resume work now as expected.

Here is my bugreport: https://bugzilla.redhat.com/show_bug.cgi?id=1162793

Comment 2 Josh Boyer 2014-11-18 21:22:18 UTC
(In reply to rh from comment #0)
> Description of problem:
> 
> The Lenovo ThinkPad T540p notebook (and possibly other models) cannot resume
> from suspend to RAM when the onboard TPM chip is enabled in the BIOS.
> 
> Version-Release number of selected component (if applicable):
> kernel 3.17.3-300.fc21
> 
> How reproducible:
> Every time
> 
> Steps to Reproduce:
> 1. Close the lid
> 2. Open the lid
> 
> Actual results:
> Screen is black, backlight is off, nothing happens.
> Only pressing the power button for a few seconds can make the laptop power
> off and boot again.
> 
> Expected results:
> Computer resumes from suspend.
> 
> Additional info:
> This affects at least one other user: 
> https://ask.fedoraproject.org/en/question/55343/thinkpad-t540p-wont-wake-
> from-sleep/
> 
> Installing kernel-modules-extra (which contains the tpm_tis module) fixes
> this completely. It should maybe be integrated into the core kernel package
> or be installed automatically on thinkpads.
> In my case, the chip has to be enabled for BitLocker to work on Windows, so
> disabling it is not an option. But I remember that suspend worked before the
> TPM was enabled by Windows.

Um, just so I understand things:

- A cold boot works regardless
- The machine suspends correctly without the TPM drivers installed
- The machine does not resume without the TPM drivers installed

Is that accurate?  If so, then that is really really sad.  If the TPM isn't in use by the OS (which it shouldn't be if the drivers are not installed) at suspend time, then something other than the kernel seems broken if the machine won't resume without them there.

Comment 3 Andy Lutomirski 2014-11-18 22:31:57 UTC
BIOS bug?  At least if TXT is involved, the TPM is likely to be involved in suspend and resume.

It makes no sense for resume to need the TPM, but I can imagine BIOS getting confused if the TPM isn't initialized at suspend time.

Comment 4 rh 2014-11-18 22:47:32 UTC
(In reply to Josh Boyer from comment #2)
> Um, just so I understand things:
> 
> - A cold boot works regardless
> - The machine suspends correctly without the TPM drivers installed
> - The machine does not resume without the TPM drivers installed
> 
> Is that accurate?
Yes, except I can't say whether the underlying problem technically occurs on suspend or on resume. But the machine appears correctly suspended.

> If so, then that is really really sad.  If the TPM isn't
> in use by the OS (which it shouldn't be if the drivers are not installed) at
> suspend time, then something other than the kernel seems broken if the
> machine won't resume without them there.
I agree, and my Fedora is definitely not actively the TPM, only BitLocker is. Maybe Lenovo should be contacted here, but I don't know whether they have the motivation to look into this and fix their BIOS.
Regardless, loading the kernel module seems to be a good workaround at the moment.

Comment 5 Josh Boyer 2014-11-18 23:31:06 UTC
(In reply to Andy Lutomirski from comment #3)
> BIOS bug?  At least if TXT is involved, the TPM is likely to be involved in
> suspend and resume.
> 
> It makes no sense for resume to need the TPM, but I can imagine BIOS getting
> confused if the TPM isn't initialized at suspend time.

Yes, probably a firmware bug.  Which really kind of sucks.

(In reply to rh from comment #4)
> > If so, then that is really really sad.  If the TPM isn't
> > in use by the OS (which it shouldn't be if the drivers are not installed) at
> > suspend time, then something other than the kernel seems broken if the
> > machine won't resume without them there.
> I agree, and my Fedora is definitely not actively the TPM, only BitLocker
> is. Maybe Lenovo should be contacted here, but I don't know whether they
> have the motivation to look into this and fix their BIOS.
> Regardless, loading the kernel module seems to be a good workaround at the
> moment.

The issue we have with including them in the main kernel package is that they actually break boot and S/R on various other machines.  Since they aren't widely used, it was convenient to just throw them in modules-extra.

Sigh.  I foresee in-driver blacklists coming.  This is going to be painful.

Comment 6 Andy Lutomirski 2014-11-18 23:36:34 UTC
I'll see if I can debug this.  My regular laptop is an X220, for better or for worse...

Comment 7 Josh Boyer 2014-11-19 00:07:56 UTC
(In reply to Andy Lutomirski from comment #6)
> I'll see if I can debug this.  My regular laptop is an X220, for better or
> for worse...

Does your machine exhibit the resume issue?  I've been told several times that the X220 doesn't suffer from this and my own X220 (which I no longer have) didn't have this issue either.  If I had a machine that did this I'd be able to debug it as well, but I don't.

Comment 8 Andy Lutomirski 2014-11-19 00:30:54 UTC
(In reply to Josh Boyer from comment #7)
> (In reply to Andy Lutomirski from comment #6)
> > I'll see if I can debug this.  My regular laptop is an X220, for better or
> > for worse...
> 
> Does your machine exhibit the resume issue?  I've been told several times
> that the X220 doesn't suffer from this and my own X220 (which I no longer
> have) didn't have this issue either.  If I had a machine that did this I'd
> be able to debug it as well, but I don't.

I have no idea, but I know that my X220 has the TPM driver installed.  I'll play with it when I get home.

Comment 9 rh 2014-11-19 07:26:29 UTC
Just tell me if I can help you with anything. I might try to disable BitLocker on Windows again later and see if Windows shows the same behaviour with BIOS TPM enabled, but the driver disabled.

Comment 10 rh 2014-11-19 11:30:26 UTC
Two new insights:
1. If you load tpm_tis and then unload it, suspend is broken again.
2. I disabled the TPM, renamed tpm.sys in Windows (it's impossible to deactivate or remove the TPM device driver normally once it's active) and enabled the TPM again - guess what, Windows can't resume from suspend either then.

1 is interesting since the tpm_tis code does not seem to actively deactivate the TPM other than disabling interrupts. Maybe the problem really lies in the suspending process? Specifically, tpm_pm_suspend (which sends a SAVESTATE command) would not get called once tpm_tis is unloaded.

Comment 11 Peter Robinson 2014-11-19 12:06:55 UTC
> Does your machine exhibit the resume issue?  I've been told several times
> that the X220 doesn't suffer from this and my own X220 (which I no longer
> have) didn't have this issue either.  If I had a machine that did this I'd
> be able to debug it as well, but I don't.

My x220 doesn't have resume problems, I don't have modules-extras installed. No idea if the TPM is enable, can I check that from userspace?

Is moving TPM modules into the mainmodules package an option?

Comment 12 rh 2014-11-19 13:40:52 UTC
I've been digging through the specifications, and the behaviour is consistent with not issuing a SAVESTATE command before going to suspend. The BIOS will then try to restore the state on resume (issue TPM_Startup(TPM_ST_STATE)), which fails because there is no state to restore.
Failing to resume is actually allowed by the TCG spec then:

From http://www.trustedcomputinggroup.org/files/resource_files/CB0B2BFA-1A4B-B294-D0C3B9075B5AFF17/TCG_PCClientImplementation_1-21_1_00.pdf Page 86:
"When issuing the TPM_Startup(TPM_ST_STATE), the S-CRTM SHOULD ignore an error
resulting from the TPM entering failure mode (TPM_FAILEDSELFTEST). Note: This will happen when the TPM’s state was not saved before entering S3."

It's SHOULD, not MUST.

Additionally, on page 40:
"If the TPM interface is accessible and one of the following situations occur:
[...]
2. the S-CRTM issues the TPM_Startup command and the return result does not equal
TPM_SUCCESS or TPM_FAILEDSELFTEST
then the S-CRTM MUST either
1. make the TPM interface inaccessible via hardware for the remainder of the power cycle;
2. reboot the Host Platform;
3. disable the Host Platform; OR
4. perform a vendor-specific action that is equivalent to one of the options above."

Seems like Lenovo opted for #3.

Comment 13 Josh Boyer 2014-11-19 13:48:15 UTC
(In reply to Peter Robinson from comment #11) 
> Is moving TPM modules into the mainmodules package an option?

It is, at the risk of having them break S/R or cause very long boots on a number of other platforms.  See comment #5

Comment 14 Andy Lutomirski 2014-11-19 19:26:00 UTC
My X220 resumes fine with tpm_tis unloaded.

FWIW, I think that there have been a ton of tpm-related bugfixes lately.  It might not be so bad these days to include the tpm modules be default.

Comment 15 poma 2014-11-19 21:05:14 UTC
So this is still 'in'

tpm_tis in 2.6.35-rcX: suspend problems
http://sourceforge.net/p/tpmdd/mailman/message/25479401

TPM bug fix causes hang on suspend (2.6.35-rc4)
http://sourceforge.net/p/tpmdd/mailman/message/25838402

[REGRESSION] Suspend fails because of TPM modules
http://sourceforge.net/p/tpmdd/mailman/message/26670949

TPM chip prevents machine from suspending
http://sourceforge.net/p/tpmdd/mailman/message/27270734

etc.

Maintained by: <tpmdd-devel.net>
Tpm Device Driver maintainance

Comment 16 poma 2014-11-19 23:18:31 UTC
- ThinkPad X1 Carbon (Machine types: 34xx)
  http://download.lenovo.com/ibmdl/pub/pc/pccbbs/mobiles/g6uj17uc.txt

  <2.59>
   UEFI: 2.59 / ECP: 1.04
  ...
  - (Fix) Fixed an issue where the computer might not resume normal operation from
          sleep state.


- ThinkPad X1 Carbon (Machine types: 20A7, 20A8) - Gen 2
  http://download.lenovo.com/ibmdl/pub/pc/pccbbs/mobiles/gruj14uc.txt

  <1.13>
   UEFI: 1.13 / ECP: 1.12
  ...
  - (Fix) Fixed an issue where the computer might not resume normal operation from
          sleep state on Linux.


For the 2nd gen explicitly is mentioned the Linux, not for the 1st, so is this really platform agnostic.

Comment 17 rh 2014-11-20 11:32:06 UTC
> FWIW, I think that there have been a ton of tpm-related bugfixes lately.
> It might not be so bad these days to include the tpm modules be default.
Shouldn't there be blacklists in place for the cases where loading tpm breaks something anyway? If I understand correctly, just installing kernel-modules-extra on affected machines would break boot and/or S/R otherwise, wouldn't it?

(In reply to poma from comment #16)
> - ThinkPad X1 Carbon (Machine types: 34xx)
>   http://download.lenovo.com/ibmdl/pub/pc/pccbbs/mobiles/g6uj17uc.txt
> 
>   <2.59>
>    UEFI: 2.59 / ECP: 1.04
>   ...
>   - (Fix) Fixed an issue where the computer might not resume normal
> operation from
>           sleep state.
Seems unrelated: http://mattoncloud.org/2014/05/15/fedora-20-on-a-thinkpad-x1-carbon/ says it's an USB3.0 issue.

This issue seems to be caused by the firmware, so you could call it platform agnostic, but it will never occur on Windows. Since Vista the TIS TPM driver is shipped by default and always loaded when a TPM is detected. Windows XP would probably fail to resume, though.

Comment 18 poma 2014-11-21 15:56:47 UTC
Folks surprised by this problem, whether they discussed with the firmware providers?

Comment 19 Josh Boyer 2014-11-21 18:44:16 UTC
I've moved the TPM drivers into the respective main kernel packages in F20 - rawhide.  The next build of each release will reflect that change.  We'll deal with further fallout in individual bugs if they occur.

Comment 20 rh 2014-11-21 21:46:21 UTC
Thanks! Hope it won't be too bad.

Comment 21 Fedora Update System 2014-11-22 20:28:23 UTC
kernel-3.17.4-300.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/kernel-3.17.4-300.fc21

Comment 22 Fedora Update System 2014-11-24 14:22:34 UTC
kernel-3.17.4-200.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/kernel-3.17.4-200.fc20

Comment 23 Fedora Update System 2014-11-24 20:59:32 UTC
Package kernel-3.17.4-300.fc21:
* should fix your issue,
* was pushed to the Fedora 21 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing kernel-3.17.4-300.fc21'
as soon as you are able to, then reboot.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2014-15668/kernel-3.17.4-300.fc21
then log in and leave karma (feedback).

Comment 24 Fedora Update System 2014-11-25 22:36:54 UTC
kernel-3.17.4-300.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 25 Fedora Update System 2014-12-01 19:05:45 UTC
kernel-3.17.4-200.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 26 Josh Boyer 2014-12-10 17:19:17 UTC
*** Bug 1084742 has been marked as a duplicate of this bug. ***