Bug 1784524

Summary: Under heavy load (due to many LUKS devices to decrypt), clevis-luks-askpass doesn't decrypt devices
Product: Red Hat Enterprise Linux 8 Reporter: Renaud Métrich <rmetrich>
Component: clevisAssignee: Sergio Correia <scorreia>
Status: CLOSED ERRATA QA Contact: Martin Zelený <mzeleny>
Severity: medium Docs Contact: Mirek Jahoda <mjahoda>
Priority: medium    
Version: 8.1CC: dapospis, mjahoda, mzeleny, scorreia
Target Milestone: rcKeywords: Triaged
Target Release: 8.0   
Hardware: All   
OS: Linux   
Fixed In Version: clevis-11-8.el8 Doc Type: Enhancement
Doc Text:
.Clevis now provides improved support for decrypting multiple LUKS devices on boot The `clevis` packages have been updated to provide better support for decrypting multiple LUKS-encrypted devices on boot. Prior to this improvement, the administrator had to perform complicated changes to the system configuration to enable the proper decryption of multiple devices by Clevis on boot. With this release, you can set up the decryption by using the `clevis luks bind` command and updating the initramfs through the `dracut -fv --regenerate-all` command. For more details, see the link:https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/security_hardening/configuring-automated-unlocking-of-encrypted-volumes-using-policy-based-decryption_security-hardening[Configuring automated unlocking of encrypted volumes using policy-based decryption] section.
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-04-28 15:37:11 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Description Flags
Proposed patch to use one clevis-luks-askpass per device none

Description Renaud Métrich 2019-12-17 16:04:18 UTC
Description of problem:

Booting a VM with many LUKS devices (9 devices, one per file system: / /usr /var /var/log /var/log/audit /tmp /opt /home and swap), clevis-luks-askpass exits after unlocking 1 device only, but systemd doesn't restart it because there is no change in the /run/systemd/ask-password directory.

/ /usr and swap were unlocked in the initramfs phase.
We talk here about unlocking the rest: /var /var/log ...

Currently, with actual code, it's not possible to unlock all these local file systems without modifying heavily the configuration through following Article https://access.redhat.com/articles/4500491
We suppose here that BZ 1784084 was implemented.

Debugging shows the following race condition due to contention on the CPU (my VM has only 2 CPUs, which slows down the boot while setting up LUKS devices):

1. 1 systemd-cryptsetup@ instance sets a path in /run/systemd/ask-password
2. clevis-luks-asspass.service executes "/usr/libexec/clevis-luks-askpass -l"
3. the tool unlocks the device then finds out no other device has to be unlocked, hence starts exiting
4. at that moment other systemd-cryptsetup@ instances set their path in /run/systemd/ask-password
5. due to clevis-luks-asspass.service still being active from systemd's point of view (but currently exiting), systemd doesn't start a new clevis-luks-asspass.service instance later

This leads to a hang of the boot. The only way to un-hang the boot is to "touch /run/systemd/ask-password" directory (done because I had a debug-shell ...).

There is a design issue here: the script /usr/libexec/clevis-luks-askpass should probably run *forever* and get killed by systemd once cryptsetup.target is reached.

Version-Release number of selected component (if applicable):


How reproducible:

Always on my VM having 2 CPUs and 9 LUKS devices (6 to unlock early after switching root)

Comment 2 Martin Zelený 2020-01-10 13:10:34 UTC

System with multiple LUKS encrypted partitions (included /) bound to tang server will boot without need of typing passwords.

Comment 3 Sergio Correia 2020-01-14 04:03:37 UTC
Created attachment 1652060 [details]
Proposed patch to use one clevis-luks-askpass per device

Comment 6 Sergio Correia 2020-02-13 11:02:42 UTC
*** Bug 1751912 has been marked as a duplicate of this bug. ***

Comment 7 Sergio Correia 2020-02-13 11:06:37 UTC
*** Bug 1784084 has been marked as a duplicate of this bug. ***

Comment 9 errata-xmlrpc 2020-04-28 15:37:11 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.