Bug 2012758 - clevis - thread 'main' panicked at 'called `Option::unwrap()
Summary: clevis - thread 'main' panicked at 'called `Option::unwrap()
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora
Classification: Fedora
Component: clevis-pin-tpm2
Version: 35
Hardware: x86_64
OS: Linux
unspecified
low
Target Milestone: ---
Assignee: Peter Robinson
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: RejectedBlocker
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-10-11 08:39 UTC by lejeczek
Modified: 2022-01-08 12:43 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-12-08 11:09:27 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description lejeczek 2021-10-11 08:39:33 UTC
Description of problem:

[root@whale ~]# export RUST_BACKTRACE=1; clevis luks bind -d /dev/nvme0n1p3 tpm2 '{"hash":"sha1","key":"rsa","pcr_bank":"sha1","pcr_ids":"0,1"}'
Enter existing LUKS password: 
Warning: Value 512 is outside of the allowed entropy range, adjusting it.
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', /usr/share/cargo/registry/tpm2-policy-0.4.0/src/lib.rs:144:30
stack backtrace:
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
Invalid input!
Usage: jose jwe fmt -i JWE [-I CT] [-o JWE] [-O CT] [-c]

Converts a JWE between serialization formats

  -i JSON --input=JSON     Parse JWE from JSON
  -i FILE --input=FILE     Read JWE from FILE
  -i -    --input=-        Read JWE from standard input

  -I FILE --detached=FILE  Read decoded ciphertext from FILE
  -I -    --detached=-     Read decoded ciphertext from standard input

  -o JSON --output=JSON    Parse JWE from JSON
  -o FILE --output=FILE    Read JWE from FILE
  -o -    --output=-       Read JWE from standard input
                           Default: "-"

  -O JSON --detach=JSON    Parse JWE from JSON
  -O FILE --detach=FILE    Read JWE from FILE
  -O -    --detach=-       Read JWE from standard input

  -c      --compact        Output JWE using compact serialization

Failed to import token from file.
Error saving metadata to LUKS2 header in device /dev/nvme0n1p3
Unable to update metadata; operation cancelled
Error adding new binding to /dev/nvme0n1p3

Version-Release number of selected component (if applicable):

clevis-pin-tpm2-0.3.0-2.fc35.x86_64
clevis-18-3.fc35.x86_64
jose-11-3.fc35.x86_64

How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Peter Robinson 2021-10-12 10:14:37 UTC
Crash verified, due to changes is tss-esapi rust crate

Comment 2 Fedora Blocker Bugs Application 2021-10-13 19:51:55 UTC
Proposed as a Blocker for 35-final by Fedora user pbrobinson using the blocker tracking app because:

 This breaks TPM2 storage unlock for Fedora IoT and this is some of the key functionality around security for IoT

Comment 3 Adam Williamson 2021-10-13 19:55:36 UTC
The clevis test is passing on F35 IoT nightly tests, it's only failing on Rawhide. Is the bug actually in F35?

Comment 4 lejeczek 2021-10-14 08:26:41 UTC
-> $ hostnamectl 
 Static hostname: whale
       Icon name: computer-desktop
         Chassis: desktop
      Machine ID: e3aa16a69b154548917a5a838257afa4
         Boot ID: 9337fe82a89f4402b9c7f48cb56f16e1
Operating System: Fedora Linux 35 (Server Edition)
     CPE OS Name: cpe:/o:fedoraproject:fedora:35
          Kernel: Linux 5.14.11-300.fc35.x86_64
    Architecture: x86-64
 Hardware Vendor: Gigabyte Technology Co., Ltd.
  Hardware Model: B550M DS3H

Comment 5 Patrick Uiterwijk 2021-10-16 13:01:18 UTC
Could you provide the output to "tpm2_pcrread" (package tpm2-tools) and/or "ls /sys/class/tpm/tpm*/"?
The only case where this issue should trigger is when you are trying to seal against PCRs that just do not exist on your TPM.
And for some reason, on my test system, that is missing "sha256", which I know it did before. So I'm wondering if there's a regression in the kernel with regards to reading TPM PCRs.

I would expect to see both a SHA1 and SHA256 bank listed (at least) in tpm2_pcrread, and the corresponding "pcr-sha1" and "pcr-sha256" in /sys/class/tpm.

Comment 6 Patrick Uiterwijk 2021-10-16 13:29:41 UTC
An additional way to determine the supported PCRs would be "tpm2_getcap pcrs" (also tpm2-tools).

Comment 7 Geoffrey Marr 2021-10-18 21:05:25 UTC
Discussed during the 2021-10-18 blocker review meeting: [0]

The decision to classify this bug as an "AcceptedBlocker (Final)" was made as it violates the following criterion:

"When configured on hardware with a Trusted Platform Module (TPM), it must be possible for the Clevis utility to automatically decrypt any encrypted partitions on boot without any user intervention".

If further information suggests this is not the case, we may change the decision.

[0] https://meetbot.fedoraproject.org/fedora-blocker-review/2021-10-18/f35-blocker-review.2021-10-18-16.00.txt

Comment 8 Adam Williamson 2021-10-18 21:35:50 UTC
BTW, Geraldo said in the blocker review meeting:

"it seems if one remove the clevis-pin-tpm2 all work fine"

but I can't find any foundation for that here, can anyone confirm/deny it?

Comment 9 Sergio Correia 2021-10-19 01:28:11 UTC
Following @puiterwijk's hint in comment #5, I was able to reproduce this scenario using swtpm + qemu and making only the sha256 bank available.
When trying to use the configuration in the OP, '{"hash":"sha1","key":"rsa","pcr_bank":"sha1","pcr_ids":"0,1"}', then I got the exact same result.

In this case, trying to use PCR IDs from sha1 bank would be a configuration error, since they are not available, so we do expect to see a failure here, although clevis error handling in this particular case can be improved to at least stop after getting an error from the encryption procedure, instead of going ahead and spitting out additional errors -- I submitted [1] to handle this.

The OP should confirm whether this is also his case, and if so, I would say this is not FB material, probably not even FE one.

[1] https://github.com/latchset/clevis/pull/337/files

Comment 10 Patrick Uiterwijk 2021-10-19 05:52:01 UTC
I fully agree with Sergio: if the TPM just does not support the requested PCR, that is a configuration bug.
I did make a change to rust-tpm2-policy (the part that actually crashed) to make it not have a rust backtrace, but instead just print an error message and still exit with exit code 1 ("PCR value not returned by TPM, bank: {:?}, slot: {:?}"): https://github.com/fedora-iot/rust-tpm2-policy/pull/3.
When that'll get released, the output will be a bit cleaner, but it will still fail to seal (encrypt) in exactly the same cases.

Comment 11 Adam Williamson 2021-10-19 06:19:53 UTC
Thanks for the input, Patrick, Sergio. lejeczek , can you clarify your case based on the above? Are you sure the sha1 bank should actually be available in your case?

Comment 12 lejeczek 2021-10-20 11:58:59 UTC
In my case 'sha1' seems absent, in 'tpm2_pcrread' too.

-> $ ls /sys/class/tpm/tpm*/
dev  device  pcr-sha256  power  ppi  subsystem  tpm_version_major  uevent

Comment 13 Geraldo Simião 2021-10-20 12:31:25 UTC
So, taking that into consideration I agree with Sergio this is not really a FB, and not a FE too.

Comment 14 Adam Williamson 2021-10-20 16:54:20 UTC
lejeczek: thanks! so, just to take that a bit further: do you think it *ought* to be present? how exactly did you construct the command that caused the crash? did you craft it yourself, expecting the sha1 PCR to be present, or just copy/paste it from somewhere without knowing the details about these PCRs? Did it work on some previous version of Fedora, or other distribution, or anything?

Comment 15 lejeczek 2021-10-20 19:51:17 UTC
cmd line executed I included in my original report and it was taken from rhel/fedora docs somewhere, sorry I cannot link to nor quote that verbatim.
I presume it boils down to the hardware platform, does it not? Whether which platform can be tweaked and/or you might need more info on that then -  I've already included a snippet - then please just ask.

Comment 16 Adam Williamson 2021-10-20 19:58:16 UTC
Thanks a lot!

It does boil down to the hardware platform, but also the kernel. Basically there are three possibilities - your hardware just doesn't do sha1, your hardware does support it but the kernel has never exposed it, or your hardware does support it and the kernel used to expose it, but doesn't now. The third would be the *worst* of those, but it sounds like that's probably not the case here. If you said you knew your hardware should offer sha1, or you'd used the same command on an older Fedora and it had worked, I'd be worried. But it sounds like that's not the case, so likely everything is more or less working as intended here, and we should just find the document you got the command from, and fix that up to not assume sha1 will be available :)

Comment 17 Adam Williamson 2021-10-20 20:01:38 UTC
I do see similar commands as examples here:

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/security_hardening/configuring-automated-unlocking-of-encrypted-volumes-using-policy-based-decryption_security-hardening

but they don't just tell you to use sha1 regardless, they're more given as examples and do explain how to check for other possibilities. So I think that doc is probably OK. Was that the one you were looking at, do you remember?

Comment 18 Adam Williamson 2021-10-20 20:05:07 UTC
Per above discussion, I reckon we should re-consider blocker status on this one. Clearing AcceptedBlocker for a re-vote.

Comment 19 Adam Williamson 2021-10-20 22:54:45 UTC
With the new info, there's a clear -4 vote in https://pagure.io/fedora-qa/blocker-review/issue/543 (and I'm also -1, so that's -5). Rejecting.

Comment 20 davestux 2021-10-25 18:46:29 UTC
Experiencing the issue on fedora 34, fresh install on a new x1 yoga gen 6. 
ls shows I have both sha1 and sha256, so is tpm2_pcread. 
Removing clevis-pin-tpm2 didn't solve the issue, nor did downgrading to 16-2.fc34.

Comment 21 Sergio Correia 2021-10-25 18:57:53 UTC
(In reply to davestux from comment #20)
> Experiencing the issue on fedora 34, fresh install on a new x1 yoga gen 6. 
> ls shows I have both sha1 and sha256, so is tpm2_pcread. 
> Removing clevis-pin-tpm2 didn't solve the issue, nor did downgrading to
> 16-2.fc34.

Could you post the command you used and the output you are getting, please? What does it say after you remove clevis-pin-tpm2, for instance?

Comment 22 davestux 2021-10-25 20:14:15 UTC
After clevis-pin-tpm2 removal:
sudo clevis luks bind -d /dev/nvme0n1p6 tpm2 '{"hash":"sha256","key":"ecc","pcr_bank":"sha256","pcr_ids":"0,1,2,3,4,5,6,7"}'
Enter existing LUKS password: 
Warning: Value 512 is outside of the allowed entropy range, adjusting it.
Failed to import token from file.
Error saving metadata to LUKS2 header in device /dev/nvme0n1p6
Unable to update metadata; operation cancelled
Error adding new binding to /dev/nvme0n1p6

Comment 23 davestux 2021-10-25 20:15:01 UTC
 sudo tpm2_pcrread
sha1:
  0 : 0xEFC6CCBBFE24CE28C7638226BA47A5DA1EEC09E0
  1 : 0xB4ED85A3FBC3100E877B726F3D841CC98FADB51A
  2 : 0x4EB3F931BABA5DE91E2002FED6BBDC47519C82BA
  3 : 0xB2A83B0EBF2F8374299A5B2BDFC31EA955AD7236
  4 : 0x4293330222FF049C2113D8F3FBEC70731C0A6EA6
  5 : 0x290FE625DD73D23F6108426C161A9C4E6C85AAA5
  6 : 0xB2A83B0EBF2F8374299A5B2BDFC31EA955AD7236
  7 : 0x40A2328DA0499B9F5B215D4BBDFDBE0134476234
  8 : 0xBA8B9967617490A12332FF3EDD9A3CD97458EE69
  9 : 0xA25F0ECF5BAE84EA1A198256A212FF880702B49A
  10: 0x1B5276A21D4A9A2FA6655746E35E877FB06247CF
  11: 0x0000000000000000000000000000000000000000
  12: 0x0000000000000000000000000000000000000000
  13: 0x0000000000000000000000000000000000000000
  14: 0xD93FF8609F4C858F0000E25E016C820AA1EA2C31
  15: 0x0000000000000000000000000000000000000000
  16: 0x0000000000000000000000000000000000000000
  17: 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
  18: 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
  19: 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
  20: 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
  21: 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
  22: 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
  23: 0x0000000000000000000000000000000000000000
sha256:
  0 : 0xABD3C026FA31E7A68B76321F4E9768D5234D1F53B0F132D2FCE9BA81DD33C730
  1 : 0xECB66D3C18EEBE543C8FF80F66A0F452E908138E38A6D52B874F7ABBEFCE0283
  2 : 0xF36CD85DA45E0B9F34E86FF6368A77829AC30401C8068CD03902D057F7857621
  3 : 0x3D458CFE55CC03EA1F443F1562BEEC8DF51C75E14A9FCF9A7234A13F198E7969
  4 : 0x6AAC1B4A6BDC67E097BDB4DC077DB4F5A1BE62AAEE877B80A54E34DB1D9DEE1C
  5 : 0xF99F7271AA35887C8254545ACDDAE5E34149E4A316740756F636B3480C5D3E4E
  6 : 0x3D458CFE55CC03EA1F443F1562BEEC8DF51C75E14A9FCF9A7234A13F198E7969
  7 : 0x7C5BF8C842DEE9B1214923A66EC069F01EB4CF0A23CE49311D070EC99C1AE6D5
  8 : 0x1CE253487802FD53FA022A269E028141DE9C72D76F242A8F58088BA76908E9CF
  9 : 0x6B8629C1E566B3B8C54B95E3725F6C2811CFB49490E7B46372DBE806E86EA591
  10: 0x1CB64349AA8B0279913F44BBD59CD149718FA13EC2B7286F69A18137B5954944
  11: 0x0000000000000000000000000000000000000000000000000000000000000000
  12: 0x0000000000000000000000000000000000000000000000000000000000000000
  13: 0x0000000000000000000000000000000000000000000000000000000000000000
  14: 0xE2D3F225DE80E60D3D29DB95DFE9D66500A21331C08914F90EE96602A96FA595
  15: 0x0000000000000000000000000000000000000000000000000000000000000000
  16: 0x0000000000000000000000000000000000000000000000000000000000000000
  17: 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
  18: 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
  19: 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
  20: 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
  21: 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
  22: 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
  23: 0x0000000000000000000000000000000000000000000000000000000000000000
[d@fedora ~]$

Comment 24 davestux 2021-10-25 20:16:15 UTC
sudo clevis-luks-list  -d  /dev/nvme0n1p6
1: tpm2 '{"hash":"sha256","key":"ecc","pcr_bank":"sha256","pcr_ids":"0,1,2,3,4,5,6,7"}'
3: tpm2 '{"hash":"sha256","key":"ecc","pcr_bank":"sha256","pcr_ids":"0,1,2,3,4,5,6,7"}'
4: tpm2 '{"hash":"sha256","key":"ecc","pcr_bank":"sha256","pcr_ids":"0,1,2,3,4,5,6,7"}'
5: tpm2 '{"hash":"sha256","key":"ecc","pcr_bank":"sha256","pcr_ids":"0,1,2,3,4,5,6,7"}'
6: tpm2 '{"hash":"sha256","key":"ecc","pcr_bank":"sha256","pcr_ids":"0,1,2,3,4,5,6"}'
7: tpm2 '{"hash":"sha256","key":"ecc","pcr_bank":"sha1","pcr_ids":"0,1,2,3,4,5,6"}'
8: tpm2 '{"hash":"sha256","key":"rsa","pcr_bank":"sha1","pcr_ids":"0,1,2,3,4,5,6"}'
9: tpm2 '{"hash":"sha256","key":"ecc","pcr_bank":"sha256","pcr_ids":"0,1,2,3,4,5,6"}'

Comment 25 davestux 2021-10-25 20:17:17 UTC
sudo ls /sys/class/tpm/tpm0/pcr-*
/sys/class/tpm/tpm0/pcr-sha1:
0  1  10  11  12  13  14  15  16  17  18  19  2  20  21  22  23  3  4  5  6  7  8  9

/sys/class/tpm/tpm0/pcr-sha256:
0  1  10  11  12  13  14  15  16  17  18  19  2  20  21  22  23  3  4  5  6  7  8  9

Comment 26 davestux 2021-10-25 20:21:25 UTC
hostnamectl
---snip---
  Operating System: Fedora 34 (Workstation Edition) 
       CPE OS Name: cpe:/o:fedoraproject:fedora:34
            Kernel: Linux 5.14.13-200.fc34.x86_64
      Architecture: x86-64
   Hardware Vendor: Lenovo
    Hardware Model: ThinkPad X1 Yoga Gen 6

Comment 27 davestux 2021-10-25 20:23:05 UTC
Please let me know if there's anything more you want to know or any other way I can help.
Thank you.

Comment 28 Sergio Correia 2021-10-25 21:09:04 UTC
(In reply to davestux from comment #24)
> sudo clevis-luks-list  -d  /dev/nvme0n1p6
> 1: tpm2
> '{"hash":"sha256","key":"ecc","pcr_bank":"sha256","pcr_ids":"0,1,2,3,4,5,6,
> 7"}'
> 3: tpm2
> '{"hash":"sha256","key":"ecc","pcr_bank":"sha256","pcr_ids":"0,1,2,3,4,5,6,
> 7"}'
> 4: tpm2
> '{"hash":"sha256","key":"ecc","pcr_bank":"sha256","pcr_ids":"0,1,2,3,4,5,6,
> 7"}'
> 5: tpm2
> '{"hash":"sha256","key":"ecc","pcr_bank":"sha256","pcr_ids":"0,1,2,3,4,5,6,
> 7"}'
> 6: tpm2
> '{"hash":"sha256","key":"ecc","pcr_bank":"sha256","pcr_ids":"0,1,2,3,4,5,6"}'
> 7: tpm2
> '{"hash":"sha256","key":"ecc","pcr_bank":"sha1","pcr_ids":"0,1,2,3,4,5,6"}'
> 8: tpm2
> '{"hash":"sha256","key":"rsa","pcr_bank":"sha1","pcr_ids":"0,1,2,3,4,5,6"}'
> 9: tpm2
> '{"hash":"sha256","key":"ecc","pcr_bank":"sha256","pcr_ids":"0,1,2,3,4,5,6"}'

From this output, you have 9 slots filled with clevis metadata. What seems to be happening here is that you have ran out of space in the LUKS header for more metadata, which then causes clevis luks bind to fail. Try removing these bindings first (one by one) with "clevis luks unbind -d /dev/nvme0n1p6 -s <SLOT>", if you intend to add new bindings, but a single binding should be enough.

In any case, doesn't the unlock work? Did you run "dracut -f --regenerate-all" to update your initramfs? Also, did you enable clevis-luks-askpass unit with "systemctl enable clevis-luks-askpass.path"?

Comment 29 lejeczek 2021-10-25 21:58:27 UTC
I see something similar to #22 in CentOS Stream 9 - I filed it here: https://bugzilla.redhat.com/show_bug.cgi?id=2017173
Only one slot used up in this case.

Comment 30 Sergio Correia 2021-10-25 22:14:26 UTC
(In reply to lejeczek from comment #29)
> I see something similar to #22 in CentOS Stream 9 - I filed it here:
> https://bugzilla.redhat.com/show_bug.cgi?id=2017173
> Only one slot used up in this case.

That's a different issue. I commented there.

Comment 31 davestux 2021-10-25 22:20:15 UTC
(In reply to Sergio Correia from comment #28)

OK, deleted them all, did single bind. ran dracut, enabled the unit. 
I started writing back that it doesn't work, but then I found out that if I wait enough time it finally succeeded and gets straight to the gdm greeter.
The password prompt just doesn't go, tried changing plymouth theme to details but it still won't show.
This is minor cosmetics issue, just confusing.
Thank you very much for your help and your work.

Comment 32 Sergio Correia 2021-10-25 22:33:34 UTC
(In reply to davestux from comment #31)
> (In reply to Sergio Correia from comment #28)
> 
> OK, deleted them all, did single bind. ran dracut, enabled the unit. 
> I started writing back that it doesn't work, but then I found out that if I
> wait enough time it finally succeeded and gets straight to the gdm greeter.
> The password prompt just doesn't go, tried changing plymouth theme to
> details but it still won't show.
> This is minor cosmetics issue, just confusing.
> Thank you very much for your help and your work.

Glad to hear it worked. As for the password prompt in plymouth, I reported it upstream in the past, but it still hasn't been resolved.

[1] https://gitlab.freedesktop.org/plymouth/plymouth/-/issues/126

Comment 33 Vincent S. Cojot 2022-01-06 21:38:41 UTC
I was encountering something similar on F34 (had been using clevis for auto-unlock on a Lenovo P50 in Intel PTT TPM2 model since F32).

my disk stopped autounlocking a few weeks ago.. (I was already on F34).

I was getting this:

# clevis luks bind -s 0 -d /dev/nvme0n1p2 tpm2 '{"pcr_ids":"7"}'
Enter existing LUKS password:
Warning: Value 512 is outside of the allowed entropy range, adjusting it.
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', /usr/share/cargo/registry/tpm2-policy-0.4.0/src/lib.rs:144:30
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Unable to save/update key slot; operation cancelled
Error adding new binding to /dev/nvme0n1p2

Thanks to this BZ, I removed that clevis-pin-tpm2 rpm and that time it didn't give an error:

# rpm -e clevis-pin-tpm2
# clevis luks bind -d /dev/nvme0n1p2 tpm2 '{"pcr_ids":"7"}'
Enter existing LUKS password:
Warning: Value 512 is outside of the allowed entropy range, adjusting it.
#

I'm dracut'ing -f and will see how it goes after reboot.

Comment 34 Peter Robinson 2022-01-08 12:43:29 UTC
> my disk stopped autounlocking a few weeks ago.. (I was already on F34).

See details in comment 28/31


Note You need to log in before you can comment on or make changes to this bug.