Bug 2131123
| Summary: | RHEL7 UEFI guest turns into black after v2v conversion | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 9 | Reporter: | mxie <mxie> | ||||
| Component: | virt-v2v | Assignee: | Laszlo Ersek <lersek> | ||||
| Status: | CLOSED ERRATA | QA Contact: | mxie <mxie> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | high | ||||||
| Version: | 9.1 | CC: | chhu, hongzliu, juzhou, lersek, mzhan, rjones, tyan, tzheng, vwu, xiaodwan, ymankad | ||||
| Target Milestone: | rc | Keywords: | Triaged | ||||
| Target Release: | --- | ||||||
| Hardware: | x86_64 | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | virt-v2v-2.2.0-5.el9 | Doc Type: | If docs needed, set a value | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2023-05-09 07:45:47 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | 2135762 | ||||||
| Bug Blocks: | |||||||
| Attachments: |
|
||||||
(1) forked from <https://bugzilla.redhat.com/show_bug.cgi?id=2124193#c26> and onwards) (2) Gerd said in <https://bugzilla.redhat.com/show_bug.cgi?id=2124193#c32>: > Note the module has been renamed to just 'bochs' in recent kernels, > so you probably want add both names to the check list. (3) My (much) earlier statement that this issue was a RHEL-7.6 guest kernel bug, fixed in RHEL-7.9, was incorrect. I was misled by the fact that installing the RHEL-7.9 GA kernel in the *converted* (output) domain had restored "rhgb" to functional state. In fact what happened was that the RHEL-7.9 guest kernel installation triggered an initrd rebuild (just like "plymouth-set-default-theme -R charge" in <https://bugzilla.redhat.com/show_bug.cgi?id=2124193#c28>), and that masked the actual problem. It's great that mxie went on to install the RHEL-7.9 kernel on the *source* (vmware) guest side, and that she then redid the conversion; that disproved my theory. Here's the plymouthd (plymouth-0.8.9-0.34.20140113.el7.x86_64) crash backtrace: #0 0x000055ce0f191540 in ?? () #1 0x00007f76214015f9 in ply_renderer_open_input_source (renderer=0x55ce0f1943e0, input_source=<optimized out>) at ply-renderer.c:386 #2 0x00007f76213fb408 in ply_keyboard_watch_for_renderer_input (keyboard=0x55ce0f19e980) at ply-keyboard.c:311 #3 ply_keyboard_watch_for_input (keyboard=keyboard@entry=0x55ce0f19e980) at ply-keyboard.c:394 #4 0x00007f76213fa97b in ply_device_manager_activate_keyboards (manager=0x55ce0f192950) at ply-device-manager.c:1035 #5 0x000055ce0ede3828 in show_theme (state=0x7ffc72096930, theme_path=<optimized out>) at main.c:1761 #6 0x000055ce0ede3eb8 in show_default_splash (state=0x7ffc72096930) at main.c:470 #7 0x000055ce0ede4fe8 in show_default_splash (state=0x7ffc72096930) at main.c:910 #8 show_splash (state=0x7ffc72096930) at main.c:921 #9 0x00007f7621610612 in ply_event_loop_handle_timeouts (loop=0x55ce0f18d150) at ply-event-loop.c:1234 #10 ply_event_loop_process_pending_events (loop=loop@entry=0x55ce0f18d150) at ply-event-loop.c:1303 #11 0x00007f7621610f70 in ply_event_loop_run (loop=0x55ce0f18d150) at ply-event-loop.c:1368 #12 0x000055ce0eddd455 in main (argc=<optimized out>, argv=<optimized out>) at main.c:2390 In ply_renderer_open_input_source(), we have renderer->input_source_is_open = renderer->plugin_interface->open_input_source (renderer->backend, And the function pointer "renderer->plugin_interface->open_input_source" is 0x000055ce0f191540, which apparently points to outer space. There are three "open_input_source" implementations (more precisely, three plugin interfaces): plugins/renderers/frame-buffer/plugin.c plugins/renderers/x11/plugin.c plugins/renderers/drm/plugin.c I *think* the problem is that one of these (the DRM one?) gets initially set up, but then potentially unloaded (?), due to the bochs_drm kernel driver not being present in the initrd. Then plymouth leaves some invalid pointers around, and we get the crash. I can reproduce this locally. In fact in my case it seems to be worse -- I don't get a converted guest that boots to a blank screen but otherwise fine; I get a stuck guest (with spinning CPU) that doesn't seem to quiesce at all. Yup, including "bochs-drm" (NOTE: *NOT* "bochs_drm"!!!) in the initrd solves the problem for me. [v2v PATCH] convert_linux: include the BOCHS DRM driver in the initial ram disk Message-Id: <20220930120444.11883-1-lersek> https://listman.redhat.com/archives/libguestfs/2022-September/030092.html (In reply to Laszlo Ersek from comment #6) > [v2v PATCH] convert_linux: include the BOCHS DRM driver in the initial ram disk > Message-Id: <20220930120444.11883-1-lersek> > https://listman.redhat.com/archives/libguestfs/2022-September/030092.html Upstream commit aa69d64cd452. Available in virt-v2v-2.0.7-7.el9 Test the bug with below builds: virt-v2v-2.0.7-7.el9.x86_64 libguestfs-1.48.4-4.el9.x86_64 guestfs-tools-1.48.2-8.el9.x86_64 nbdkit-server-1.30.8-2.el9_1.x86_64 libnbd-1.12.6-1.el9.x86_64 libvirt-libs-8.10.0-1.el9.x86_64 qemu-img-7.1.0-6.el9.x86_64 Steps: 1.Convert a rhel7 uefi guest from VMware to local libvirt by v2v # virt-v2v -ic vpx://administrator%40vsphere.local.213.207/data/10.73.212.38/?no_verify=1 -it vddk -io vddk-libdir=/home/vddk8.0.0 -io vddk-thumbprint=09:9E:54:CF:C3:36:11:9D:7D:B6:45:E0:85:74:4D:DB:CE:24:7B:46 -ip /home/passwd esx7.0-rhel7.9-uefi-enable-secureboot -of qcow2 [ 0.9] Setting up the source: -i libvirt -ic vpx://administrator%40vsphere.local.213.207/data/10.73.212.38/?no_verify=1 -it vddk esx7.0-rhel7.9-uefi-enable-secureboot [ 2.6] Opening the source [ 9.2] Inspecting the source [ 29.9] Checking for sufficient free disk space in the guest [ 29.9] Converting Red Hat Enterprise Linux Server 7.9 (Maipo) to run on KVM virt-v2v: This guest has virtio drivers installed. [ 236.3] Mapping filesystem data to avoid copying unused and blank areas [ 237.5] Closing the overlay [ 237.8] Assigning disks to buses [ 237.8] Checking if the guest needs BIOS or UEFI to boot virt-v2v: This guest requires UEFI on the target to boot. [ 237.8] Setting up the destination: -o libvirt [ 241.0] Copying disk 1/1 █ 100% [****************************************] [ 417.8] Creating output metadata [ 417.9] Finishing off 2.Check guest after v2v finishing conversion, checkpoints of the guest are passed 3.Convert a rhel7 uefi guest from VMware to rhv by v2v # virt-v2v -ic vpx://administrator%40vsphere.local.213.207/data/10.73.212.38/?no_verify=1 -it vddk -io vddk-libdir=/home/vddk8.0.0 -io vddk-thumbprint=09:9E:54:CF:C3:36:11:9D:7D:B6:45:E0:85:74:4D:DB:CE:24:7B:46 -ip /home/passwd esx7.0-rhel7.9-uefi-enable-secureboot -of qcow2 -o rhv-upload -oc https://dell-per740-22.lab.eng.pek2.redhat.com/ovirt-engine/api -os nfs_data -op /home/rhvpasswd [ 0.1] Setting up the source: -i libvirt -ic vpx://administrator%40vsphere.local.213.207/data/10.73.212.38/?no_verify=1 -it vddk esx7.0-rhel7.9-uefi-enable-secureboot [ 1.9] Opening the source [ 8.8] Inspecting the source [ 43.8] Checking for sufficient free disk space in the guest [ 43.8] Converting Red Hat Enterprise Linux Server 7.9 (Maipo) to run on KVM virt-v2v: This guest has virtio drivers installed. [ 347.9] Mapping filesystem data to avoid copying unused and blank areas [ 349.0] Closing the overlay [ 349.3] Assigning disks to buses [ 349.3] Checking if the guest needs BIOS or UEFI to boot virt-v2v: This guest requires UEFI on the target to boot. [ 349.3] Setting up the destination: -o rhv-upload -oc https://dell-per740-22.lab.eng.pek2.redhat.com/ovirt-engine/api -os nfs_data [ 370.6] Copying disk 1/1 █ 100% [****************************************] [1260.6] Creating output metadata [1284.2] Finishing off 4.Check guest after v2v finishing conversion, checkpoints of the guest are passed Result: The bug has been fixed Verify the bug with below builds: virt-v2v-2.0.7-7.el9.x86_64 libguestfs-1.48.4-4.el9.x86_64 guestfs-tools-1.48.2-8.el9.x86_64 nbdkit-server-1.30.8-2.el9_1.x86_64 libnbd-1.12.6-1.el9.x86_64 libvirt-libs-8.10.0-2.el9.x86_64 qemu-img-7.1.0-6.el9.x86_64 Steps: 1.Convert a rhel7 uefi guest from VMware to local libvirt by v2v # virt-v2v -ic vpx://administrator%40vsphere.local.213.207/data/10.73.212.38/?no_verify=1 -it vddk -io vddk-libdir=/home/vddk8.0.0 -io vddk-thumbprint=09:9E:54:CF:C3:36:11:9D:7D:B6:45:E0:85:74:4D:DB:CE:24:7B:46 -ip /home/passwd esx7.0-rhel7.9-uefi-enable-secureboot -of qcow2 [ 0.1] Setting up the source: -i libvirt -ic vpx://administrator%40vsphere.local.213.207/data/10.73.212.38/?no_verify=1 -it vddk esx7.0-rhel7.9-uefi-enable-secureboot [ 2.0] Opening the source [ 11.2] Inspecting the source [ 74.4] Checking for sufficient free disk space in the guest [ 74.4] Converting Red Hat Enterprise Linux Server 7.9 (Maipo) to run on KVM virt-v2v: This guest has virtio drivers installed. [ 330.9] Mapping filesystem data to avoid copying unused and blank areas [ 331.9] Closing the overlay [ 332.2] Assigning disks to buses [ 332.2] Checking if the guest needs BIOS or UEFI to boot virt-v2v: This guest requires UEFI on the target to boot. [ 332.2] Setting up the destination: -o libvirt [ 336.7] Copying disk 1/1 █ 100% [****************************************] [ 546.1] Creating output metadata [ 546.1] Finishing off 2.Check guest after v2v finishing conversion, checkpoints of the guest are passed 3.Convert a rhel7 uefi guest from VMware to rhv by v2v # virt-v2v -ic vpx://administrator%40vsphere.local.213.207/data/10.73.212.38/?no_verify=1 -it vddk -io vddk-libdir=/home/vddk8.0.0 -io vddk-thumbprint=09:9E:54:CF:C3:36:11:9D:7D:B6:45:E0:85:74:4D:DB:CE:24:7B:46 -ip /home/passwd esx7.0-rhel7.9-uefi-enable-secureboot -of qcow2 -o rhv-upload -oc https://dell-per740-22.lab.eng.pek2.redhat.com/ovirt-engine/api -os nfs_data -op /home/rhvpasswd [ 0.0] Setting up the source: -i libvirt -ic vpx://administrator%40vsphere.local.213.207/data/10.73.212.38/?no_verify=1 -it vddk esx7.0-rhel7.9-uefi-enable-secureboot [ 1.8] Opening the source [ 7.6] Inspecting the source [ 24.1] Checking for sufficient free disk space in the guest [ 24.1] Converting Red Hat Enterprise Linux Server 7.9 (Maipo) to run on KVM virt-v2v: This guest has virtio drivers installed. [ 181.2] Mapping filesystem data to avoid copying unused and blank areas [ 182.1] Closing the overlay [ 182.4] Assigning disks to buses [ 182.4] Checking if the guest needs BIOS or UEFI to boot virt-v2v: This guest requires UEFI on the target to boot. [ 182.4] Setting up the destination: -o rhv-upload -oc https://dell-per740-22.lab.eng.pek2.redhat.com/ovirt-engine/api -os nfs_data [ 204.3] Copying disk 1/1 █ 100% [****************************************] [ 652.2] Creating output metadata [ 674.1] Finishing off 4.Check guest after v2v finishing conversion, checkpoints of the guest are passed Result: No new problem was found, move the bug from ON_QA to VERIFIED Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (virt-v2v bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2023:2313 |
Created attachment 1915140 [details] rhel7-uefi-guest-console-black.png Description of problem: RHEL7 UEFI guest turns into black after v2v conversion Version-Release number of selected component (if applicable): virt-v2v-2.0.7-6.el9.x86_64 libguestfs-1.48.4-2.el9.x86_64 guestfs-tools-1.48.2-5.el9.x86_64 libvirt-libs-8.7.0-1.el9.x86_64 qemu-img-7.1.0-1.el9.x86_64 nbdkit-server-1.30.8-1.el9.x86_64 libnbd-1.12.6-1.el9.x86_64 How reproducible: 100% Steps to Reproduce: 1.Convert a RHEL7 UEFI guest from VMware to libvirt by v2v # virt-v2v -ic vpx://root.213.107/data/10.73.212.38/?no_verify=1 -it vddk -io vddk-libdir=/home/vddk7.0.3 -io vddk-thumbprint=87:F9:29:E7:33:DE:34:68:74:3F:6A:C9:61:96:C3:51:E2:1E:EA:2B -ip /home/passwd -of qcow2 esx7.0-rhel7.9-uefi-enable-secureboot -v -x |& tee > v2v-convert-rhel7-uefi-guest-to-libvirt.log █ 100% [****************************************] 2.Check guest after v2v finishing conversion, but guest turns into black after powering on, pls refer to screenshot 'rhel7-uefi-guest-console-black.png' Actual results: As above description Expected results: Checkpoints of RHEL7 UEFI guest are passed after v2v conversion Additional info: