(1) forked from <https://bugzilla.redhat.com/show_bug.cgi?id=2124193#c26> and onwards)

(2) Gerd said in <https://bugzilla.redhat.com/show_bug.cgi?id=2124193#c32>:

> Note the module has been renamed to just 'bochs' in recent kernels,
> so you probably want add both names to the check list.

(3) My (much) earlier statement that this issue was a RHEL-7.6 guest kernel bug, fixed in RHEL-7.9, was incorrect. I was misled by the fact that installing the RHEL-7.9 GA kernel in the *converted* (output) domain had restored "rhgb" to functional state. In fact what happened was that the RHEL-7.9 guest kernel installation triggered an initrd rebuild (just like "plymouth-set-default-theme -R charge" in <https://bugzilla.redhat.com/show_bug.cgi?id=2124193#c28>), and that masked the actual problem. It's great that mxie went on to install the RHEL-7.9 kernel on the *source* (vmware) guest side, and that she then redid the conversion; that disproved my theory.

Here's the plymouthd (plymouth-0.8.9-0.34.20140113.el7.x86_64) crash backtrace:

#0  0x000055ce0f191540 in ?? ()
#1  0x00007f76214015f9 in ply_renderer_open_input_source (renderer=0x55ce0f1943e0, input_source=<optimized out>) at ply-renderer.c:386
#2  0x00007f76213fb408 in ply_keyboard_watch_for_renderer_input (keyboard=0x55ce0f19e980) at ply-keyboard.c:311
#3  ply_keyboard_watch_for_input (keyboard=keyboard@entry=0x55ce0f19e980) at ply-keyboard.c:394
#4  0x00007f76213fa97b in ply_device_manager_activate_keyboards (manager=0x55ce0f192950) at ply-device-manager.c:1035
#5  0x000055ce0ede3828 in show_theme (state=0x7ffc72096930, theme_path=<optimized out>) at main.c:1761
#6  0x000055ce0ede3eb8 in show_default_splash (state=0x7ffc72096930) at main.c:470
#7  0x000055ce0ede4fe8 in show_default_splash (state=0x7ffc72096930) at main.c:910
#8  show_splash (state=0x7ffc72096930) at main.c:921
#9  0x00007f7621610612 in ply_event_loop_handle_timeouts (loop=0x55ce0f18d150) at ply-event-loop.c:1234
#10 ply_event_loop_process_pending_events (loop=loop@entry=0x55ce0f18d150) at ply-event-loop.c:1303
#11 0x00007f7621610f70 in ply_event_loop_run (loop=0x55ce0f18d150) at ply-event-loop.c:1368
#12 0x000055ce0eddd455 in main (argc=<optimized out>, argv=<optimized out>) at main.c:2390

In ply_renderer_open_input_source(), we have

  renderer->input_source_is_open = renderer->plugin_interface->open_input_source (renderer->backend,

And the function pointer "renderer->plugin_interface->open_input_source" is 0x000055ce0f191540, which apparently points to outer space.

There are three "open_input_source" implementations (more precisely, three plugin interfaces):

  plugins/renderers/frame-buffer/plugin.c
  plugins/renderers/x11/plugin.c
  plugins/renderers/drm/plugin.c

I *think* the problem is that one of these (the DRM one?) gets initially set up, but then potentially unloaded (?), due to the bochs_drm kernel driver not being present in the initrd. Then plymouth leaves some invalid pointers around, and we get the crash.

I can reproduce this locally. In fact in my case it seems to be worse -- I don't get a converted guest that boots to a blank screen but otherwise fine; I get a stuck guest (with spinning CPU) that doesn't seem to quiesce at all.

Yup, including "bochs-drm" (NOTE: *NOT* "bochs_drm"!!!) in the initrd solves the problem for me.

[v2v PATCH] convert_linux: include the BOCHS DRM driver in the initial ram disk
Message-Id: <20220930120444.11883-1-lersek>
https://listman.redhat.com/archives/libguestfs/2022-September/030092.html

(In reply to Laszlo Ersek from comment #6)
> [v2v PATCH] convert_linux: include the BOCHS DRM driver in the initial ram disk
> Message-Id: <20220930120444.11883-1-lersek>
> https://listman.redhat.com/archives/libguestfs/2022-September/030092.html

Upstream commit aa69d64cd452.

Available in virt-v2v-2.0.7-7.el9

Test the bug with below builds:
virt-v2v-2.0.7-7.el9.x86_64
libguestfs-1.48.4-4.el9.x86_64
guestfs-tools-1.48.2-8.el9.x86_64
nbdkit-server-1.30.8-2.el9_1.x86_64
libnbd-1.12.6-1.el9.x86_64
libvirt-libs-8.10.0-1.el9.x86_64
qemu-img-7.1.0-6.el9.x86_64


Steps:
1.Convert a rhel7 uefi guest from VMware to local libvirt by v2v
# virt-v2v -ic vpx://administrator%40vsphere.local.213.207/data/10.73.212.38/?no_verify=1 -it vddk -io vddk-libdir=/home/vddk8.0.0 -io  vddk-thumbprint=09:9E:54:CF:C3:36:11:9D:7D:B6:45:E0:85:74:4D:DB:CE:24:7B:46  -ip /home/passwd esx7.0-rhel7.9-uefi-enable-secureboot -of qcow2
[   0.9] Setting up the source: -i libvirt -ic vpx://administrator%40vsphere.local.213.207/data/10.73.212.38/?no_verify=1 -it vddk esx7.0-rhel7.9-uefi-enable-secureboot
[   2.6] Opening the source
[   9.2] Inspecting the source
[  29.9] Checking for sufficient free disk space in the guest
[  29.9] Converting Red Hat Enterprise Linux Server 7.9 (Maipo) to run on KVM
virt-v2v: This guest has virtio drivers installed.
[ 236.3] Mapping filesystem data to avoid copying unused and blank areas
[ 237.5] Closing the overlay
[ 237.8] Assigning disks to buses
[ 237.8] Checking if the guest needs BIOS or UEFI to boot
virt-v2v: This guest requires UEFI on the target to boot.
[ 237.8] Setting up the destination: -o libvirt
[ 241.0] Copying disk 1/1
█ 100% [****************************************]
[ 417.8] Creating output metadata
[ 417.9] Finishing off

2.Check guest after v2v finishing conversion, checkpoints of the guest are passed

3.Convert a rhel7 uefi guest from VMware to rhv by v2v
# virt-v2v -ic vpx://administrator%40vsphere.local.213.207/data/10.73.212.38/?no_verify=1 -it vddk -io vddk-libdir=/home/vddk8.0.0 -io  vddk-thumbprint=09:9E:54:CF:C3:36:11:9D:7D:B6:45:E0:85:74:4D:DB:CE:24:7B:46  -ip /home/passwd esx7.0-rhel7.9-uefi-enable-secureboot -of qcow2  -o rhv-upload -oc https://dell-per740-22.lab.eng.pek2.redhat.com/ovirt-engine/api -os nfs_data  -op /home/rhvpasswd 
[   0.1] Setting up the source: -i libvirt -ic vpx://administrator%40vsphere.local.213.207/data/10.73.212.38/?no_verify=1 -it vddk esx7.0-rhel7.9-uefi-enable-secureboot
[   1.9] Opening the source
[   8.8] Inspecting the source
[  43.8] Checking for sufficient free disk space in the guest
[  43.8] Converting Red Hat Enterprise Linux Server 7.9 (Maipo) to run on KVM
virt-v2v: This guest has virtio drivers installed.
[ 347.9] Mapping filesystem data to avoid copying unused and blank areas
[ 349.0] Closing the overlay
[ 349.3] Assigning disks to buses
[ 349.3] Checking if the guest needs BIOS or UEFI to boot
virt-v2v: This guest requires UEFI on the target to boot.
[ 349.3] Setting up the destination: -o rhv-upload -oc https://dell-per740-22.lab.eng.pek2.redhat.com/ovirt-engine/api -os nfs_data
[ 370.6] Copying disk 1/1
█ 100% [****************************************]
[1260.6] Creating output metadata
[1284.2] Finishing off


4.Check guest after v2v finishing conversion, checkpoints of the guest are passed

Result:
  The bug has been fixed

Verify the bug with below builds:
virt-v2v-2.0.7-7.el9.x86_64
libguestfs-1.48.4-4.el9.x86_64
guestfs-tools-1.48.2-8.el9.x86_64
nbdkit-server-1.30.8-2.el9_1.x86_64
libnbd-1.12.6-1.el9.x86_64
libvirt-libs-8.10.0-2.el9.x86_64
qemu-img-7.1.0-6.el9.x86_64

Steps:
1.Convert a rhel7 uefi guest from VMware to local libvirt by v2v
# virt-v2v -ic vpx://administrator%40vsphere.local.213.207/data/10.73.212.38/?no_verify=1 -it vddk -io vddk-libdir=/home/vddk8.0.0 -io  vddk-thumbprint=09:9E:54:CF:C3:36:11:9D:7D:B6:45:E0:85:74:4D:DB:CE:24:7B:46  -ip /home/passwd esx7.0-rhel7.9-uefi-enable-secureboot -of qcow2
[   0.1] Setting up the source: -i libvirt -ic vpx://administrator%40vsphere.local.213.207/data/10.73.212.38/?no_verify=1 -it vddk esx7.0-rhel7.9-uefi-enable-secureboot
[   2.0] Opening the source
[  11.2] Inspecting the source
[  74.4] Checking for sufficient free disk space in the guest
[  74.4] Converting Red Hat Enterprise Linux Server 7.9 (Maipo) to run on KVM
virt-v2v: This guest has virtio drivers installed.
[ 330.9] Mapping filesystem data to avoid copying unused and blank areas
[ 331.9] Closing the overlay
[ 332.2] Assigning disks to buses
[ 332.2] Checking if the guest needs BIOS or UEFI to boot
virt-v2v: This guest requires UEFI on the target to boot.
[ 332.2] Setting up the destination: -o libvirt
[ 336.7] Copying disk 1/1
█ 100% [****************************************]
[ 546.1] Creating output metadata
[ 546.1] Finishing off

2.Check guest after v2v finishing conversion, checkpoints of the guest are passed

3.Convert a rhel7 uefi guest from VMware to rhv by v2v
# virt-v2v -ic vpx://administrator%40vsphere.local.213.207/data/10.73.212.38/?no_verify=1 -it vddk -io vddk-libdir=/home/vddk8.0.0 -io  vddk-thumbprint=09:9E:54:CF:C3:36:11:9D:7D:B6:45:E0:85:74:4D:DB:CE:24:7B:46  -ip /home/passwd esx7.0-rhel7.9-uefi-enable-secureboot -of qcow2  -o rhv-upload -oc https://dell-per740-22.lab.eng.pek2.redhat.com/ovirt-engine/api -os nfs_data  -op /home/rhvpasswd 
[   0.0] Setting up the source: -i libvirt -ic vpx://administrator%40vsphere.local.213.207/data/10.73.212.38/?no_verify=1 -it vddk esx7.0-rhel7.9-uefi-enable-secureboot
[   1.8] Opening the source
[   7.6] Inspecting the source
[  24.1] Checking for sufficient free disk space in the guest
[  24.1] Converting Red Hat Enterprise Linux Server 7.9 (Maipo) to run on KVM
virt-v2v: This guest has virtio drivers installed.
[ 181.2] Mapping filesystem data to avoid copying unused and blank areas
[ 182.1] Closing the overlay
[ 182.4] Assigning disks to buses
[ 182.4] Checking if the guest needs BIOS or UEFI to boot
virt-v2v: This guest requires UEFI on the target to boot.
[ 182.4] Setting up the destination: -o rhv-upload -oc https://dell-per740-22.lab.eng.pek2.redhat.com/ovirt-engine/api -os nfs_data
[ 204.3] Copying disk 1/1
█ 100% [****************************************]
[ 652.2] Creating output metadata
[ 674.1] Finishing off

4.Check guest after v2v finishing conversion, checkpoints of the guest are passed

Result:
   No new problem was found, move the bug from ON_QA to VERIFIED

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (virt-v2v bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:2313