RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2168506 - RFE: Virt-v2v should recognize partition names like '/dev/mapper/rhel boot--73--75--123-root' in related keys option
Summary: RFE: Virt-v2v should recognize partition names like '/dev/mapper/rhel boot--7...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: virt-v2v
Version: 9.2
Hardware: x86_64
OS: Unspecified
medium
medium
Target Milestone: rc
: ---
Assignee: Laszlo Ersek
QA Contact: mxie@redhat.com
URL:
Whiteboard:
Depends On: 2216425
Blocks: 2209279 2209280
TreeView+ depends on / blocked
 
Reported: 2023-02-09 09:11 UTC by mxie@redhat.com
Modified: 2023-11-07 09:28 UTC (History)
9 users (show)

Fixed In Version: virt-v2v-2.3.4-3.el9
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 2209279 2209280 (view as bug list)
Environment:
Last Closed: 2023-11-07 08:28:57 UTC
Type: Feature Request
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Gitlab redhat/centos-stream/rpms virt-v2v merge_requests 60 0 None opened recognize "--key /dev/mapper/VG-LV:key:password"; fix %check phase 2023-06-20 15:25:19 UTC
Red Hat Issue Tracker RHELPLAN-148112 0 None None None 2023-02-09 09:12:58 UTC
Red Hat Product Errata RHBA-2023:6376 0 None None None 2023-11-07 08:29:15 UTC

Internal Links: 2216496

Description mxie@redhat.com 2023-02-09 09:11:22 UTC
Description of problem:
Virt-v2v should recognize partition names like '/dev/mapper/rhel boot--73--75--123-root' in related keys option


Version-Release number of selected component (if applicable):
virt-v2v-2.2.0-5.el9.x86_64

How reproducible:
100%

Steps to Reproduce:
1.Prepare a guest with luks encrypted LV on VMware
# lsblk
NAME                                            MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINTS
sda                                               8:0    0   16G  0 disk  
├─sda1                                            8:1    0    1G  0 part  /boot
└─sda2                                            8:2    0   15G  0 part  
  ├─rhel_bootp--73--75--123-root                253:0    0 13.4G  0 lvm   
  │ └─luks-cf19e4ac-3f54-450b-b801-72e539a8f870 253:2    0 13.4G  0 crypt /
  └─rhel_bootp--73--75--123-swap                253:1    0  1.6G  0 lvm   [SWAP]
sr0     
                     
2.Convert the guest from VMware to rhv by v2v
#  virt-v2v -ic vpx://administrator%40vsphere.local.213.207/data/10.73.212.38/?no_verify=1 -it vddk -io vddk-libdir=/home/vddk8.0.0 -io  vddk-thumbprint=09:9E:54:CF:C3:36:11:9D:7D:B6:45:E0:85:74:4D:DB:CE:24:7B:46  -ip /home/passwd  -o rhv-upload -oc https://dell-per740-22.lab.eng.pek2.redhat.com/ovirt-engine/api  -op /home/rhvpasswd -os nfs_data Auto-esx7.0-rhel9.1-only-root-luks-redhat123 --key "/dev/mapper/rhel_bootp--73--75--123-root":key:redhat123
[   0.0] Setting up the source: -i libvirt -ic vpx://administrator%40vsphere.local.213.207/data/10.73.212.38/?no_verify=1 -it vddk Auto-esx7.0-rhel9.1-only-root-luks-redhat123
[   1.8] Opening the source
Enter key or passphrase ("/dev/rhel_bootp-73-75-123/root"): 


Actual results:
As step2 shown,  use --key option to input password for luks partition '/dev/mapper/rhel_bootp--73--75--123-root' , but v2v still asks password for the partition, from v2v point of view, '/dev/mapper/rhel_bootp--73--75--123-root'  and '/dev/rhel_bootp-73-75-123/root' are not the same, but actually they are different names for the same thing.

Expected results:
As above description

Additional info:

Comment 2 Richard W.M. Jones 2023-02-09 09:15:46 UTC
Let's look at this for RHEL 9.3.  The current option is
not optimal but perfectly usable.

Comment 4 Laszlo Ersek 2023-02-09 10:03:31 UTC
And remember UUIDs -- you can use UUIDs with "--key", and (as UUID says in the name) those should uniquely identify the device.

Comment 5 Laszlo Ersek 2023-02-10 05:38:10 UTC
from: /dev/mapper/rhel_bootp--73--75--123-root
to: /dev/rhel_bootp-73-75-123/root

Rich wrote:

'In general /dev/VG/LV is the same as /dev/mapper/VG-LV.  If either "VG" or "LV" contain single "-" characters, they get doubled ("--").'

I thought we'd need a regex with capturing parens for translating the /dev/mapper/VG-LV format to /dev/VG/LV, but it seems that collapsing "--" to "-", plus ensuring there is precisely one standalone "-" before the collapsing, should suffice. AFAICT this should be possible to do with a single loop pass over VG-LV, with some tracking pointers maintained.

Comment 6 Laszlo Ersek 2023-05-15 17:55:55 UTC
[libguestfs-common PATCH 0/2] recognize "/dev/mapper/VG-LV" with "--key"
Message-Id: <20230515174924.290409-1-lersek>
https://listman.redhat.com/archives/libguestfs/2023-May/031497.html

[v2v PATCH 0/2] test "/dev/mapper/VG-LV" with "--key"
Message-Id: <20230515175529.290724-1-lersek>
https://listman.redhat.com/archives/libguestfs/2023-May/031501.html

Comment 7 Laszlo Ersek 2023-05-18 13:10:12 UTC
[libguestfs-common PATCH v2 0/2] recognize "/dev/mapper/VG-LV" with "--key"
Message-Id: <20230518130942.288152-1-lersek>
https://listman.redhat.com/archives/libguestfs/2023-May/031571.html

Comment 8 Laszlo Ersek 2023-05-19 09:31:45 UTC
(In reply to Laszlo Ersek from comment #7)
> [libguestfs-common PATCH v2 0/2] recognize "/dev/mapper/VG-LV" with "--key"
> Message-Id: <20230518130942.288152-1-lersek>
> https://listman.redhat.com/archives/libguestfs/2023-May/031571.html

Commit range 38e6988c1864..b636c3f20a1b.

Comment 9 Laszlo Ersek 2023-05-19 09:47:45 UTC
(In reply to Laszlo Ersek from comment #6)
> [v2v PATCH 0/2] test "/dev/mapper/VG-LV" with "--key"
> Message-Id: <20230515175529.290724-1-lersek>
> https://listman.redhat.com/archives/libguestfs/2023-May/031501.html

Commit range e83de8abe6c5..3060af01e87f.

Comment 10 Laszlo Ersek 2023-05-19 14:09:42 UTC
[libguestfs PATCH 0/3] test "/dev/mapper/VG-LV" with "--key"
Message-Id: <20230519140849.310774-1-lersek>
https://listman.redhat.com/archives/libguestfs/2023-May/031584.html

Comment 11 Laszlo Ersek 2023-05-19 15:56:21 UTC
[guestfs-tools PATCH 0/3] test "/dev/mapper/VG-LV" with "--key"
Message-Id: <20230519155507.369494-1-lersek>
https://listman.redhat.com/archives/libguestfs/2023-May/031589.html

Comment 12 Laszlo Ersek 2023-05-22 12:25:45 UTC
(In reply to Laszlo Ersek from comment #10)
> [libguestfs PATCH 0/3] test "/dev/mapper/VG-LV" with "--key"
> Message-Id: <20230519140849.310774-1-lersek>
> https://listman.redhat.com/archives/libguestfs/2023-May/031584.html

Commit range 692802bf96e0..32408a9c3616.

Comment 13 Laszlo Ersek 2023-05-22 12:47:13 UTC
(In reply to Laszlo Ersek from comment #11)
> [guestfs-tools PATCH 0/3] test "/dev/mapper/VG-LV" with "--key"
> Message-Id: <20230519155507.369494-1-lersek>
> https://listman.redhat.com/archives/libguestfs/2023-May/031589.html

Commit range 67647b883e13..569bd1dd29da.

Comment 14 Laszlo Ersek 2023-05-23 05:36:20 UTC
Rich, for backporting purposes, should I clone this BZ for libguestfs (guestfish -i) and guestfs-tools (virt-inspector) as well? Thanks!

Comment 15 Laszlo Ersek 2023-05-23 05:47:33 UTC
Whoops, setting needinfo as well now :)

Comment 16 Richard W.M. Jones 2023-05-23 07:45:00 UTC
Yes please, we will need separate bugs for every RHEL component that has to be changed.

Comment 18 Laszlo Ersek 2023-06-19 16:28:11 UTC
[v2v PATCH] test-data/phony-guests: fix prerequisite list of "fedora-luks-on-lvm.img"
Message-Id: <20230619162729.153334-1-lersek>
https://listman.redhat.com/archives/libguestfs/2023-June/031856.html

Comment 21 Laszlo Ersek 2023-06-20 14:25:49 UTC
(In reply to Laszlo Ersek from comment #18)
> [v2v PATCH] test-data/phony-guests: fix prerequisite list of "fedora-luks-on-lvm.img"
> Message-Id: <20230619162729.153334-1-lersek>
> https://listman.redhat.com/archives/libguestfs/2023-June/031856.html

Commit 13a6f4b9686e.

Comment 29 RHEL Program Management 2023-06-21 10:37:41 UTC
Development Management has reviewed and declined this request. You may appeal this decision by reopening this request.

Comment 30 Laszlo Ersek 2023-06-21 10:38:41 UTC
I've pushed the following commits to the upstream rhel-9.3 branch (all cherry-picks from the master branch):

     1  6dea82d823c3 Update common submodule
     2  1d69132b7b72 update common submodule
     3  2558084d081c LUKS-on-LVM conversion test: rename VGs and LVs
     4  c8902c551014 LUKS-on-LVM conversion test: test /dev/mapper/VG-LV translation
     5  10192f8ee3a7 test-data/phony-guests: fix prerequisite list of "fedora-luks-on-lvm.img"

Comment 31 Laszlo Ersek 2023-06-21 10:40:04 UTC
This bug is not stale. It is blocked, just on something that's not another RHBZ (at this point).

Comment 33 Richard W.M. Jones 2023-06-21 10:47:14 UTC
I'd be surprised if resubmitting the build works.  It seems something is going
very wrong when patching the paravirt instructions in the kernel.  Kernel or TCG bug?

Comment 35 Laszlo Ersek 2023-06-21 11:42:52 UTC
(In reply to Richard W.M. Jones from comment #33)
> I'd be surprised if resubmitting the build works.  It seems something
> is going very wrong when patching the paravirt instructions in the
> kernel.  Kernel or TCG bug?

Yes, either guest kernel or TCG bug (or both!)

I expect there's a fair chance for the resubmitted build to succeed, for
the following reason:

- In c9s Koji task 2378814 (comment#24) and Brew task 53460798
  (comment#25), the same QEMU version was used (17:8.0.0-5.el9), and the
  same guest kernel version was used (5.14.0-330.el9.x86_64).

- In the c9s Koji task, the appliance kernel ran 6 times successfully.
  The first two runs are for image preparations (windows.img,
  fedora.img), the third run is a virt-v2v null conversion open-coded in
  the SPEC file, the fourth run is another image preparation
  (fedora-luks-on-lvm.img). The fifth and sixth runs are not logged as
  verbosely; they are from "test-v2v-fedora-luks-on-lvm-conversion.sh",
  from the test suite -- those correspond to two virt-v2v null
  conversions.

- In the Brew task, the appliance kernel ran 2 times successfully
  (windows.img, fedora.img) , it is the virt-v2v null conversion that's
  open-coded in the SPEC file where the appliance kernel crashes.

So it does not look deterministic. We can file a new bug later if
needed, but for now I want to complete the backport here.

Comment 37 Laszlo Ersek 2023-06-21 11:54:11 UTC
Ah, the friendly bots have filed a new RHBZ about the build failure for us:

https://bugzilla.redhat.com/show_bug.cgi?id=2216425

That allows for some better dependency tracking.

Comment 38 Laszlo Ersek 2023-06-21 12:29:06 UTC
Definitely a QEMU (TCG) or guest kernel bug: in the latest Brew build (comment 36), the first *four* appliance kernel boots succeeded, but then one of the last two (= 5th / 6th) ones hung -- the Brew build has been running for ~50 minutes now, and the log is stuck in "test-v2v-fedora-luks-on-lvm-conversion.sh".

Comment 51 Laszlo Ersek 2023-06-21 15:37:49 UTC
(In reply to Laszlo Ersek from comment #38)
> Definitely a QEMU (TCG) or guest kernel bug: in the latest Brew build
> (comment 36), the first *four* appliance kernel boots succeeded, but then
> one of the last two (= 5th / 6th) ones hung -- the Brew build has been
> running for ~50 minutes now, and the log is stuck in
> "test-v2v-fedora-luks-on-lvm-conversion.sh".

--> https://bugzilla.redhat.com/show_bug.cgi?id=2216496

Comment 52 Richard W.M. Jones 2023-06-22 08:16:23 UTC
I had a loop running this all day yesterday, on RHEL 9, with all the same
packages installed, and it didn't fail at all, not that this proves anything much:

while LIBGUESTFS_BACKEND_SETTINGS=force_tcg libguestfs-test-tool -t 60 >& /tmp/1 ; do echo -n . ; done

Comment 54 mxie@redhat.com 2023-06-26 05:05:42 UTC
Verify the bug with below builds:
virt-v2v-2.3.4-3.el9.x86_64
libguestfs-1.50.1-6.el9.x86_64
libvirt-libs-9.4.0-1.el9.x86_64
qemu-img-8.0.0-5.el9.x86_64
nbdkit-server-1.34.1-1.el9.x86_64
libnbd-1.16.0-1.el9.x86_64


Steps:
1.Prepare a guest with luks encrypted lvm on VMware
# lsblk
NAME                                            MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINTS
sda                                               8:0    0   16G  0 disk  
├─sda1                                            8:1    0    1G  0 part  /boot
└─sda2                                            8:2    0   15G  0 part  
  ├─rhel_bootp--73--75--123-root                253:0    0 13.4G  0 lvm   
  │ └─luks-cf19e4ac-3f54-450b-b801-72e539a8f870 253:2    0 13.4G  0 crypt /
  └─rhel_bootp--73--75--123-swap                253:1    0  1.6G  0 lvm   [SWAP]
sr0     
                     
2.Convert the guest from VMware by v2v
#  virt-v2v -ic vpx://administrator%40vsphere.local.213.93/data/10.73.212.38/?no_verify=1 -it vddk -io vddk-libdir=/home/vddk8.0.1 -io  vddk-thumbprint=1B:83:D8:5A:33:31:62:DB:BA:9E:73:6D:A8:29:14:48:3F:82:F6:FD  -ip /home/passwd Auto-esx7.0-rhel9.1-only-root-luks-redhat123 --key "/dev/mapper/rhel_bootp--73--75--123-root":key:redhat123
[   0.2] Setting up the source: -i libvirt -ic vpx://administrator%40vsphere.local.213.93/data/10.73.212.38/?no_verify=1 -it vddk Auto-esx7.0-rhel9.1-only-root-luks-redhat123
[   2.0] Opening the source
[  10.1] Inspecting the source
[  23.0] Checking for sufficient free disk space in the guest
[  23.0] Converting Red Hat Enterprise Linux 9.1 Beta (Plow) to run on KVM
virt-v2v: This guest has virtio drivers installed.
[ 148.6] Mapping filesystem data to avoid copying unused and blank areas
virt-v2v: warning: fstrim on guest filesystem 
/dev/mapper/luks-cf19e4ac-3f54-450b-b801-72e539a8f870 failed.  Usually you 
can ignore this message.  To find out more read "Trimming" in virt-v2v(1).

Original message: fstrim: fstrim: /sysroot/: the discard operation is not 
supported
[ 150.1] Closing the overlay
[ 150.4] Assigning disks to buses
[ 150.4] Checking if the guest needs BIOS or UEFI to boot
[ 150.4] Setting up the destination: -o libvirt
[ 152.0] Copying disk 1/1
█ 100% [****************************************]
[ 517.9] Creating output metadata
[ 518.0] Finishing off

3.Check the guest after v2v conversion, checkpoints of guest are passed

Result:

  The bug has been fixed, move the bug from ON_QA to VERIFIED

Comment 59 errata-xmlrpc 2023-11-07 08:28:57 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (virt-v2v bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:6376


Note You need to log in before you can comment on or make changes to this bug.