Thanks for the report, will update the documentation, and perhaps also add some "best effort" logic for virt-sysprep not to choke on LUKS-on-LV.

(Background: https://bugzilla.redhat.com/show_bug.cgi?id=1809453#c34 and onwards.)

(In reply to Laszlo Ersek from comment #1)
> Thanks for the report, will update the documentation, and perhaps also add
> some "best effort" logic for virt-sysprep not to choke on LUKS-on-LV.
> 
> (Background: https://bugzilla.redhat.com/show_bug.cgi?id=1809453#c34 and
> onwards.)

Got it. Adding "best effort" logic will be wonderful. Thanks.

Argh, this is nasty. Beyond "lvm-uuids", we have another sysprep operation that works on block devices, and not on filesystems: it's called "fs-uuids". It does basically the same (UUID regeneration), just for filesystems, not logical volumes.

Here's why it is a problem: if we close the decrypted devices between unmounting all filesystems and running the device-level operations, then filesystems on LUKS-on-LV devices will no longer be accessible at the block device level, and "fs-uuids" will not see them.

Basically the way filesystems, LUKS block devices, and logical volumes are stacked, virt-sysprep's current lumping-together of all device-level operations is not flexible enough. There is a particular order in which (1) fs unmounting, (2) fs-uuid regen, (3) LUKS closing, and (4) lvm-uuid regen need to occur. The problem is that (1) and (3) are (supposed to be) in "sysprep/main.ml", whereas (2) and (4) are supposed to be "pluggable" and at most reorderable between each other -- but not possible to order relative to (1) and (3). So with this we're hitting an infrastructure / design limit of virt-sysprep.

With LV-on-LUKS this was no problem; we'd never try to close LUKS -- step (3) from above falls away --, so (1) fs unmounting in "main" would occur, then separately the device-level operations (2) fs-uuid and (4) lvm-uuid, would execute successfully in any (unspecified) order between each other.

Rich, any ideas? Thanks.

As a workaround we could try ordering lvm-uuid strictly after fs-uuid (I think we have infrastructure for that), *plus* hack lvm-uuid such that, if the initial LVM deactivation attempt fails, we internally try to close all LUKS devices (again, internally to the lvm-uuid operation). This looks super ugly though.

[guestfs-tools PATCH 0/2] sysprep: full disk encryption improvements
Message-Id: <20220714104005.8334-1-lersek>
https://listman.redhat.com/archives/libguestfs/2022-July/029450.html

(In reply to Laszlo Ersek from comment #5)
> [guestfs-tools PATCH 0/2] sysprep: full disk encryption improvements
> Message-Id: <20220714104005.8334-1-lersek>
> https://listman.redhat.com/archives/libguestfs/2022-July/029450.html

Upstream commit range a7ffb717306f..b49ee909f5d1.

Tested with package:
guestfs-tools-1.48.2-5.el9.x86_64

Steps:

1.
# virt-sysprep -a /var/lib/libvirt/images/rhel9-luks-on-lv.qcow2 
[   0.0] Examining the guest ...
Enter key or passphrase ("/dev/rhel/root"): 
Enter key or passphrase ("/dev/rhel/swap"): 
[  38.4] Performing "abrt-data" ...
[  38.4] Performing "backup-files" ...
[  38.9] Performing "bash-history" ...
[  38.9] Performing "blkid-tab" ...
[  38.9] Performing "crash-data" ...
[  38.9] Performing "cron-spool" ...
[  38.9] Performing "dhcp-client-state" ...
[  38.9] Performing "dhcp-server-state" ...
[  38.9] Performing "dovecot-data" ...
[  39.0] Performing "ipa-client" ...
[  39.0] Performing "kerberos-hostkeytab" ...
[  39.0] Performing "logfiles" ...
[  39.0] Performing "lvm-system-devices" ...
[  39.0] Performing "machine-id" ...
[  39.0] Performing "mail-spool" ...
[  39.0] Performing "net-hostname" ...
[  39.0] Performing "net-hwaddr" ...
[  39.1] Performing "net-nmconn" ...
[  39.1] Performing "pacct-log" ...
[  39.1] Performing "package-manager-cache" ...
[  39.1] Performing "pam-data" ...
[  39.1] Performing "passwd-backups" ...
[  39.2] Performing "puppet-data-log" ...
[  39.2] Performing "rh-subscription-manager" ...
[  39.2] Performing "rhn-systemid" ...
[  39.2] Performing "rpm-db" ...
[  39.2] Performing "samba-db-log" ...
[  39.2] Performing "script" ...
[  39.2] Performing "smolt-uuid" ...
[  39.2] Performing "ssh-hostkeys" ...
[  39.2] Performing "ssh-userdir" ...
[  39.3] Performing "sssd-db-log" ...
[  39.3] Performing "tmp-files" ...
[  39.3] Performing "udev-persistent-net" ...
[  39.3] Performing "utmp" ...
[  39.3] Performing "yum-uuid" ...
[  39.3] Performing "customize" ...
[  39.3] Setting a random seed
[  39.3] Setting the machine ID in /etc/machine-id
[  39.3] SELinux relabelling
[  54.6] Performing "lvm-uuids" ...

No error is printed.

2.
# man virt-sysprep
...
Virtual machines that employ full disk encryption internally to the guest should not be considered for cloning and distribution, as it provides multiple parties with the same internal volume key, enabling any one such party to decrypt all the other clones.  Refer to the LUKS FAQ for details.
...

The above lines are added to the man page.

Hi,lersek

The Verified:Tested flag has been set, but I don't know why this bug is not added into the erratum automatically. Could you help with that manually? Thanks.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Low: guestfs-tools security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:7959