Created attachment 1223667 [details]
rhel7.3_2disks_debug

Let's track this upstream until a customer reports a bug.

Description of problem:
Virt-v2v can't convert encrypted guest

Version-Release number of selected component (if applicable):
virt-v2v-1.36.3-3.el7.x86_64
libguestfs-1.36.3-3.el7.x86_64
libvirt-3.2.0-5.el7.x86_64
qemu-kvm-1.5.3-137.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1.Prepare a linux guest which has encrypted partition on xen server and using virt-v2v can convert the encrypted guest but the conversion is failed. pls refer to log
# virt-v2v -ic xen+ssh://root.3.21 Auto-xen-rhel6.9-case13063 -of qcow2 -b default
[   0.0] Opening the source -i libvirt -ic xen+ssh://root.3.21 Auto-xen-rhel6.9-case13063
[   0.7] Creating an overlay to protect the source from being modified
[   7.2] Initializing the target -o libvirt -os default
[   7.2] Opening the overlay
[ 237.4] Inspecting the overlay
virt-v2v: error: inspection could not detect the source guest (or physical 
machine).

Assuming that you are running virt-v2v/virt-p2v on a source which is 
supported (and not, for example, a blank disk), then this should not 
happen.

No root device found in this operating system image.

If reporting bugs, run virt-v2v with debugging enabled and include the 
complete output:

  virt-v2v -v -x [...]


Hi rjones,

  Could you help to check whether above problem is same with this bug?

Created attachment 1279529 [details]
virt-v2v-encrypted-guest.log

(In reply to mxie from comment #4)
>   Could you help to check whether above problem is same with this bug?

Sort of -- there are different cases of "LUKS encryption":

a) scenario 1 in comment 0 is a guest with the whole disk image encrypted as LUKS, so directly at QEMU level

b) scenario 2 in comment 0 seems a different error, mostly related to the fact that the second disk contains a whole partition (with no partition table).
zhoujunqin, how was it created? What does it contain?

c) the scenario in comment 4 is a guest with in-guest full disk LUKS encryption, i.e. what you would get by installing RHEL/Fedora and telling the installer to encrypt the whole disk

So IMHO these cases should be split in own bugs (leaving this for just one, I'd say (c)), as they will need very different fixes/implementations.

Hi Pino,

  Below is details steps for scenario 2, I can reproduce scenario 1 of comment0  with latest virt-v2v, but I can't reproduce scenario 2 of commment0 now ,so you could ignore scenario 2 for the moment

Packages:
virt-v2v-1.36.3-4.el7.x86_64
libguestfs-1.36.3-4.el7.x86_64
qemu-kvm-rhev-2.9.0-8.el7.x86_64
libvirt-3.2.0-6.el7.x86_64


Steps for scenario 2
1.Prepare secret xml file 
# cat luks-secret.xml 
      <secret ephemeral='no' private='yes'>
         <description>LUKS Sample Secret</description>
         <uuid>f981dd17-143f-45bc-88e6-ed1fe20ce9da</uuid>
         <usage type='volume'>
            <volume>/var/lib/libvirt/images/luks/luks_1.img</volume>
         </usage>
      </secret>

2.Define luks secret
# virsh secret-define luks-secret.xml
Secret f981dd17-143f-45bc-88e6-ed1fe20ce9da created

3 Set value for the luks secret.
# MYSECRET=`printf %s "redhat" | base64`
# virsh secret-set-value f981dd17-143f-45bc-88e6-ed1fe20ce9da $MYSECRET
Secret value set

4 Prepare a dir pool and name as luks

5.Prepare a luks volume xml
# cat luks_1.vol
<volume>
    <name>luks_1.img</name>
      <capacity unit='G'>6</capacity>
       <target>
         <path>/var/lib/libvirt/images/luks/luks_1.img</path>
          <format type='raw'/>
          <encryption format='luks'>
             <secret type='passphrase' uuid='f981dd17-143f-45bc-88e6-ed1fe20ce9da'/>
         <cipher name='twofish' size='128' mode='cbc' hash='sha256'/>
             <ivgen name='plain64' hash='sha1'/>        
          </encryption>        
         </target>
</volume>

6.Create a luks volume in luks pool according to luks_1.vol
# virsh vol-create luks luks_1.vol
Vol luks_1.img created from luks_1.vol


7.Prepare a guest which has installed os on non-luks disk and add the luks disk which is created at step6 to guest
*********
 <disk type='volume' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source pool='default' volume='esx6.0-rhel6.9-x86_64-sda'/>
      <target dev='vda' bus='virtio'/>
      <boot order='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
 </disk>
 </disk>
    <disk type='file' device='disk'>
     <driver name='qemu' type='raw'/>
    <source file='/var/lib/libvirt/images/luks/luks_1.img'/>
    <target dev='vdb' bus='virtio'/>
      <encryption format='luks'>
      <secret type='passphrase' uuid='f981dd17-143f-45bc-88e6-ed1fe20ce9da'/>
     </encryption>
    <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
</disk> 
*********

8.Using virt-v2v to convert this guest and the conversion could be finished without error

9.On host, use cryptsetup to decrypt the image, then hehe (block device) will be created at /dev/mapper 
#cryptsetup luksOpen /var/lib/libvirt/images/luks/luks_1.img hehe

10.Add the /dev/mapper/hehe device to the guest which has installed os in none-luks disk
**********
 <disk type='volume' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source pool='default' volume='esx6.0-rhel6.9-x86_64-sda'/>
      <target dev='vda' bus='virtio'/>
      <boot order='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='none' io='native'/>
      <source dev='/dev/mapper/hehe'/>
      <target dev='vdb' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </disk>
**********
11.Using virt-v2v to convert this guest and the conversion could be finished without error

(In reply to Pino Toscano from comment #6)
> c) the scenario in comment 4 is a guest with in-guest full disk LUKS
> encryption, i.e. what you would get by installing RHEL/Fedora and telling
> the installer to encrypt the whole disk

This was fixed in the meanwhile, see bug 1451665.

I wonder if we need this bug.  Virt-v2v has supported LUKS for
a while.  We also have:
https://bugzilla.redhat.com/show_bug.cgi?id=1809453
"[RFE] Add support for LUKS encrypted disks with Clevis & Tang"

(In reply to Richard W.M. Jones from comment #9)
> I wonder if we need this bug.  Virt-v2v has supported LUKS for
> a while.

Yes, we do need this bug, as libguestfs supports only few layouts of LUKS partitions.

>  We also have:
> https://bugzilla.redhat.com/show_bug.cgi?id=1809453
> "[RFE] Add support for LUKS encrypted disks with Clevis & Tang"

This is something completely different, and unrelated, from this bug.

- Bug 1658126 is about the guest internally using LUKS-on-LV, that is; formatting an LVM Logical Volume with cryptsetup (LUKS), and then placing a filesystem (incl. swap) on that. So we get something like:

# lsblk
NAME                                        MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
sda                                           8:0    0    9G  0 disk  
├─sda1                                        8:1    0    1G  0 part  /boot
└─sda2                                        8:2    0    8G  0 part  
  ├─rhel-root                               253:0    0  7.1G  0 lvm   
  │ └─luks-07934cf9-3cef-4677-8ce3-0eb047d641f0
                                            253:3    0  7.1G  0 crypt /
  └─rhel-swap                               253:1    0  924M  0 lvm   
    └─luks-262fb950-61a0-47f7-a1f6-bb23d8a5fddf
                                            253:2    0  922M  0 crypt [SWAP]

- Bug 1451665 (solved) is about the guest internally using PV-on-LUKS, that is; formatting a traditional partition  with cryptsetup (LUKS), then using the encrypted block device as an LVM Physical Volume, then creating LVs in the resultant volume group, and then placing swap and filesystems on those LVs. So we get something like:

# lsblk
NAME                          MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
xvda                          202:0    0 14.7G  0 disk  
├─xvda1                       202:1    0  500M  0 part  /boot
└─xvda2                       202:2    0 14.2G  0 part  
  └─luks-6b952cdc-9d5b-426d-bd0c-1ba16e00b4eb (dm-0)
                              253:0    0 14.2G  0 crypt 
    ├─VolGroup-lv_root (dm-1) 253:1    0 12.7G  0 lvm   /
    └─VolGroup-lv_swap (dm-2) 253:2    0  1.5G  0 lvm   [SWAP]
sr0                            11:0    1 1024M  0 rom

(I vaguely recall Anaconda switching between the two layouts at some point in the Fedora release history!)

- Bug 1398191 is about the guest NOT using cryptsetup internally at all; instead, the storage volume on the HOST side -- a regular file or a Logical Volume -- is encrypted, where QEMU performs LUKS encryption/decryption in userspace, and libvirtd manages the setup <https://libvirt.org/formatstorageencryption.html>. (In bug 1398191, only "scenario 1" matters.)

- Bug 1809453 is a feature request that applies to disk encryption *internal* to the guest. It applies to both LUKS-on-LV and PV-on-LUKS in Linux guests, and applies to BitLocker in Windows guests. The idea is that the secret (for decrypting the disks internally to the libguestfs appliance) is retrieved over the network <https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/security_hardening/configuring-automated-unlocking-of-encrypted-volumes-using-policy-based-decryption_security-hardening>, and not specified by the end-user (cf. --key, --keys-from-stdin for virt-v2v).