Bug 1688283 - Clevis fails to unlock encrypted partition with iot (ostree) installation
Summary: Clevis fails to unlock encrypted partition with iot (ostree) installation
Keywords:
Status: NEW
Alias: None
Product: Fedora
Classification: Fedora
Component: clevis
Version: 30
Hardware: Unspecified
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Sergio Correia
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: IoT
TreeView+ depends on / blocked
 
Reported: 2019-03-13 12:58 UTC by Paul Whalen
Modified: 2019-08-01 07:25 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
If this bug requires documentation, please select an appropriate Doc Type value.http://iot.stg.fedoraproject.org/
Clone Of:
Environment:
Last Closed: 2019-03-31 00:03:59 UTC


Attachments (Terms of Use)
tpm2-luks1-ks.cfg (1.91 KB, text/plain)
2019-04-16 15:09 UTC, Javier Martinez Canillas
no flags Details
tpm2-luks2-ks.cfg (1.91 KB, text/plain)
2019-04-16 15:11 UTC, Javier Martinez Canillas
no flags Details

Description Paul Whalen 2019-03-13 12:58:13 UTC
Description of problem:
Attempting to use clevis/tpm2 with Fedora IoT edition (ostree) to unlock an encrypted partition at boot. The installation is completed without error but requires user input to unlock the root partition.

Version-Release number of selected component (if applicable):
clevis-11-5.fc30.x86_64
clevis-luks-11-5.fc30.x86_64
clevis-systemd-11-5.fc30.x86_64
clevis-dracut-11-5.fc30.x86_64

tpm2-tools-3.1.3-4.fc30.x86_64
tpm2-tss-2.2.0-1.fc30.x86_64



How reproducible:
Always when using an ostree based installation.

Steps to Reproduce:
1. Encrypted install using the latest IoT installation media and tpm2 hardware
2. In the kickstart (or manually enter the following commands after installation) use:


    autopart --encrypted --type=lvm --luks-version=luks1 --passphrase=fedora

    %post --erroronfail
    # Find the LUKS volume
    UUID=$(lsblk | grep luks | sed 's/^.*luks-//' | cut -d ' ' -f1)
    DEV=$(blkid --uuid $UUID)
    # Bind the LUKS volume to a TPM2 for automatic unlocking on boot
    clevis luks bind -f -k- -d $DEV tpm2 '{}' <<< fedora

Additional info:

Works when using a standard Fedora 30 installation with the same package versions and kickstart snippets.

Clevis is included in the initramfs for IoT:

lsinitrd initramfs-5.0.0-0.rc8.git0.1.fc30.x86_64.img | grep clevis
clevis
-rw-r--r--   1 root     root            1 Feb 14 14:03 etc/cmdline.d/99clevis.conf
-rwxr-xr-x   1 root     root         1563 Aug 13  2018 usr/bin/clevis
-rwxr-xr-x   1 root     root         1614 Aug 13  2018 usr/bin/clevis-decrypt
-rwxr-xr-x   1 root     root        33304 Jan 31 09:46 usr/bin/clevis-decrypt-sss
-rwxr-xr-x   1 root     root         2721 Aug 13  2018 usr/bin/clevis-decrypt-tang
-rwxr-xr-x   1 root     root         3794 Aug 13  2018 usr/bin/clevis-decrypt-tpm2
-rwxr-xr-x   1 root     root           49 Jan 31 09:46 usr/lib/dracut/hooks/initqueue/online/60-clevis-hook.sh
-rwxr-xr-x   1 root     root           49 Jan 31 09:46 usr/lib/dracut/hooks/initqueue/settled/60-clevis-hook.sh
-rwxr-xr-x   1 root     root         2873 Aug 13  2018 usr/libexec/clevis-luks-askpass

Comment 1 Paul Whalen 2019-03-13 13:10:42 UTC
After booting the system to check:

[root@fitlet2 ~]# luksmeta show -d /dev/sda3
0   active empty
1   active cb6e8904-81ff-40da-a84a-07ab9ab5715e
2 inactive empty
3 inactive empty
4 inactive empty
5 inactive empty
6 inactive empty
7 inactive empty

Password is correct:

[root@fitlet2 ~]# cryptsetup luksOpen --test-passphrase --key-slot 0 /dev/sda3 && echo correct
Enter passphrase for /dev/sda3:
correct

Remove the password and try again:

[root@fitlet2 ~]# clevis luks unbind -d /dev/sda3 -s 1
The unbind operation will wipe a slot. This operation is unrecoverable.
Do you wish to erase LUKS slot 1 on /dev/sda3? [ynYN] y
Enter any remaining passphrase:

[root@fitlet2 ~]# luksmeta show -d /dev/sda3
0   active empty
1 inactive empty
2 inactive empty
3 inactive empty
4 inactive empty
5 inactive empty
6 inactive empty
7 inactive empty

Try to use the clevis commands again:

[root@fitlet2 ~]# UUID=$(lsblk | grep luks | sed 's/^.*luks-//' | cut -d ' ' -f1)
[root@fitlet2 ~]# DEV=$(blkid --uuid $UUID)
[root@fitlet2 ~]# echo $DEV
/dev/sda3
[root@fitlet2 ~]# clevis luks bind -f -k- -d $DEV tpm2 '{}' <<< fedora
[root@fitlet2 ~]# luksmeta show -d /dev/sda3
0   active empty
1   active cb6e8904-81ff-40da-a84a-07ab9ab5715e
2 inactive empty
3 inactive empty
4 inactive empty
5 inactive empty
6 inactive empty
7 inactive empty

Reboot and it again fails to unlock. The same commands work on a standard F29/30 installation.

Comment 2 Javier Martinez Canillas 2019-03-13 14:47:39 UTC
(In reply to Paul Whalen from comment #0)

[snip]

> 
> Works when using a standard Fedora 30 installation with the same package
> versions and kickstart snippets.
> 
> Clevis is included in the initramfs for IoT:
> 
> lsinitrd initramfs-5.0.0-0.rc8.git0.1.fc30.x86_64.img | grep clevis
> clevis

Are the tpm2-tools and libraries also included in the initramfs? i.e:

lsinitrd /boot/initramfs-$(uname -r).img | grep -E 'clevis|tpm2|tss2'

Comment 3 Paul Whalen 2019-03-13 18:48:54 UTC
lsinitrd initramfs-5.0.0-300.fc30.x86_64.img | grep -E 'clevis|tpm2|tss2'

clevis
-rw-r--r--   1 root     root            1 Feb 14 14:03 etc/cmdline.d/99clevis.conf
-rwxr-xr-x   1 root     root         1563 Aug 13  2018 usr/bin/clevis
-rwxr-xr-x   1 root     root         1614 Aug 13  2018 usr/bin/clevis-decrypt
-rwxr-xr-x   1 root     root        33304 Jan 31 09:46 usr/bin/clevis-decrypt-sss
-rwxr-xr-x   1 root     root         2721 Aug 13  2018 usr/bin/clevis-decrypt-tang
-rwxr-xr-x   1 root     root         3794 Aug 13  2018 usr/bin/clevis-decrypt-tpm2
-rwxr-xr-x   1 root     root        95408 Feb  3 04:10 usr/bin/tpm2_createprimary
-rwxr-xr-x   1 root     root        92712 Feb  3 04:10 usr/bin/tpm2_load
-rwxr-xr-x   1 root     root        87968 Feb  3 04:10 usr/bin/tpm2_pcrlist
-rwxr-xr-x   1 root     root       108424 Feb  3 04:10 usr/bin/tpm2_unseal
-rwxr-xr-x   1 root     root       303880 Feb 14 14:03 usr/lib64/libtss2-mu.so.0.0.0
lrwxrwxrwx   1 root     root           19 Feb 14 14:03 usr/lib64/libtss2-mu.so.0 -> libtss2-mu.so.0.0.0
-rwxr-xr-x   1 root     root       403216 Feb 14 14:03 usr/lib64/libtss2-sys.so.0.0.0
lrwxrwxrwx   1 root     root           20 Feb 14 14:03 usr/lib64/libtss2-sys.so.0 -> libtss2-sys.so.0.0.0
-rwxr-xr-x   1 root     root        36416 Feb 14 14:03 usr/lib64/libtss2-tcti-device.so.0.0.0
lrwxrwxrwx   1 root     root           28 Feb 14 14:03 usr/lib64/libtss2-tcti-device.so.0 -> libtss2-tcti-device.so.0.0.0
-rwxr-xr-x   1 root     root           49 Jan 31 09:46 usr/lib/dracut/hooks/initqueue/online/60-clevis-hook.sh
-rwxr-xr-x   1 root     root           49 Jan 31 09:46 usr/lib/dracut/hooks/initqueue/settled/60-clevis-hook.sh
-rwxr-xr-x   1 root     root         2873 Aug 13  2018 usr/libexec/clevis-luks-askpass

Comment 4 Javier Martinez Canillas 2019-03-14 09:25:18 UTC
With Fedora-IoT-ostree-x86_64-30-20190215.0.iso I get:

# echo foobar | clevis encrypt tpm2 '{}' | clevis decrypt
WARNING:tcti:src/tss2-tcti/tcti-device.c:254:tcti_device_receive() Got EOF instead of response. 
ERROR: 
Create Object Failed ! ErrorCode: 0xa0008

ERROR: Unable to run tpm2_create
Creating TPM2 object for jwk failed!

# tpm2_rc_decode 0xa0008
error layer
  hex: 0xa0000
  identifier: TSS2_TCTI_RC_LAYER
  description: Error from the TCTI
base error code
  identifier: TSS2_BASE_RC_NO_CONNECTION
  description: Fails to connect to next lower layer

# uname -r
5.0.0-0.rc6.git1.1.fc30.x86_64

Which is reported to be a regression in the kernel:

https://github.com/tpm2-software/tpm2-tools/issues/1356

Comment 5 Javier Martinez Canillas 2019-03-14 09:32:36 UTC
Following is the thread in the linux-integrity mailing list where the problem and possible solutions are being discussed:

https://www.spinics.net/lists/linux-integrity/msg06386.html

Comment 6 Javier Martinez Canillas 2019-03-22 09:37:40 UTC
Upstream patch that solves this bug:

https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1959715.html

Comment 7 Fedora Update System 2019-03-26 16:14:54 UTC
kernel-tools-5.0.4-300.fc30 kernel-headers-5.0.4-300.fc30 kernel-5.0.4-300.fc30 has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2019-4e7590b99c

Comment 8 Fedora Update System 2019-03-26 16:16:32 UTC
kernel-tools-5.0.4-200.fc29 kernel-headers-5.0.4-200.fc29 kernel-5.0.4-200.fc29 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2019-0ba1e6642f

Comment 9 Fedora Update System 2019-03-27 00:45:26 UTC
kernel-5.0.4-300.fc30, kernel-headers-5.0.4-300.fc30, kernel-tools-5.0.4-300.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-4e7590b99c

Comment 10 Fedora Update System 2019-03-27 04:34:42 UTC
kernel-5.0.4-200.fc29, kernel-headers-5.0.4-200.fc29, kernel-tools-5.0.4-200.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-0ba1e6642f

Comment 11 Fedora Update System 2019-03-29 02:59:06 UTC
kernel-5.0.4-200.fc29, kernel-headers-5.0.4-200.fc29, kernel-tools-5.0.4-200.fc29 has been pushed to the Fedora 29 stable repository. If problems still persist, please make note of it in this bug report.

Comment 12 Fedora Update System 2019-03-31 00:03:59 UTC
kernel-5.0.4-300.fc30, kernel-headers-5.0.4-300.fc30, kernel-tools-5.0.4-300.fc30 has been pushed to the Fedora 30 stable repository. If problems still persist, please make note of it in this bug report.

Comment 13 Paul Whalen 2019-04-02 14:26:47 UTC
Using compose Fedora-IoT-30-20190327.0

[root@fitlet2 ~]# echo foo | clevis encrypt tpm2 '{}' | clevis decrypt
foo
[root@fitlet2 ~]# uname -r
5.0.0-300.fc30.x86_64

But the automatic decryption still fails on boot. If I add 'rd.break=initqueue' to the kernel args to get a shell prior to decryption, then 'exit' the system will decrypt the volume and boot to log in. It appears to be working to some degree but perhaps a timing issue?

This also happens with a F29 compose which includes kernel-4.20.16-200.fc29.

Comment 14 Javier Martinez Canillas 2019-04-16 15:09:19 UTC
Created attachment 1555593 [details]
tpm2-luks1-ks.cfg

Comment 15 Javier Martinez Canillas 2019-04-16 15:11:01 UTC
Created attachment 1555594 [details]
tpm2-luks2-ks.cfg

Comment 16 Javier Martinez Canillas 2019-04-16 15:29:50 UTC
I tried to reproduce this issue on the same hardware than Paul used and couldn't reproduce the exact issue but had a different one that could be related.

Using the kickstart tpm2-luks1-ks.cfg file attached in Comment 14, the system was installed correctly but the LUKS volume was not unlocked automatically by clevis. Looking at the LUKSv1 and LUKSMeta headers, I noticed that a LUKSMeta slot was added for the bound pin but no a LUKS slot:

$ sudo luksmeta show -d /dev/sda3 | grep "^[0-9][ ]*active"
0   active empty
1   active cb6e8904-81ff-40da-a84a-07ab9ab5715e

$ sudo cryptsetup luksDump /dev/sda3 | grep ENABLED
Key Slot 0: ENABLED

But if I bind a pin to a LUKS volume after the installation, it did work correctly:

$ sudo luksmeta wipe -d /dev/sda3 -s 1

$ sudo clevis luks bind -d /dev/sda3 tpm2 '{}'

$ sudo luksmeta show -d /dev/sda3 | grep "^[0-9][ ]*active"
0   active empty
1   active cb6e8904-81ff-40da-a84a-07ab9ab5715e

$ sudo cryptsetup luksDump /dev/sda3 | grep ENABLED
Key Slot 0: ENABLED
Key Slot 1: ENABLED

And also the LUKS volume was automatically unlocked on boot.

It also worked correctly if I used LUKSv2 instead of LUKSv1 like in the tpm2-luks2-ks.cfg kickstart file attached in Comment 15.

So the problem seems to be only when storing the clevis pin on a LUKSv1 volume and when the clevis luks bind command is executed in the kickstart file.

Comment 17 pk.eskse 2019-07-16 18:03:20 UTC
I've been having a similar issue on Fedora Silverblue, as this seems to still be open I thought I'd post it here rather than open a new bug report.

It seems to be that the required initramfs modules are not loaded by the layer provided by what I assume would be the "clevis-dracut" package (if that's not the correct terminology let me know as I only installed Silverblue yesterday to play around with).

Enabling client side generation of the initramfs via "rpm-ostree initramfs --enable" resolves this.


Note You need to log in before you can comment on or make changes to this bug.