Bug 1519204 - v2v should improve the result when convert a rhel7.4 guest with no available kernels found in the bootloader
Summary: v2v should improve the result when convert a rhel7.4 guest with no available ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libguestfs
Version: 7.5
Hardware: x86_64
OS: Linux
low
medium
Target Milestone: rc
: ---
Assignee: Richard W.M. Jones
QA Contact: Virtualization Bugs
URL:
Whiteboard: V2V
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-11-30 11:39 UTC by kuwei@redhat.com
Modified: 2018-04-10 09:21 UTC (History)
7 users (show)

Fixed In Version: libguestfs-1.36.10-3.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-04-10 09:20:40 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
bootloader_image (21.59 KB, image/png)
2017-11-30 11:39 UTC, kuwei@redhat.com
no flags Details
v2v.log (1.95 MB, text/plain)
2017-11-30 11:40 UTC, kuwei@redhat.com
no flags Details
grub1 (2.47 KB, text/plain)
2017-12-07 13:00 UTC, kuwei@redhat.com
no flags Details
grub2 (2.70 KB, text/plain)
2017-12-07 13:01 UTC, kuwei@redhat.com
no flags Details
v2v-scenario-1 (1.96 MB, text/plain)
2017-12-08 10:21 UTC, kuwei@redhat.com
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:0677 0 None None None 2018-04-10 09:21:36 UTC

Description kuwei@redhat.com 2017-11-30 11:39:21 UTC
Created attachment 1360868 [details]
bootloader_image

Description of problem:
v2v should improve the result when convert a rhel7.4 guest with no available kernels found in the bootloader

Version-Release number of selected component (if applicable):
qemu-kvm-rhev-2.10.0-9.el7.x86_64
libvirt-3.9.0-3.el7.x86_64
libguestfs-1.36.10-2.el7.x86_64
virt-v2v-1.36.10-2.el7.x86_64
How reproducible:
100%

Steps to Reproduce:
1.Prepare a rhel7.4 guest on kvm server

2.Edit the guest's "boot/grub2/grub.cfg/" file,remove all the available kernel message,and add a bogus kernel.Then restart the guest,we could see the bootloader as the attachment picture.(As there is on available kernel,so the guest can't boot normally)

3.Using virt-v2v to convert the guest
# virt-v2v rhel7.4_no-kernel -o null
[   0.0] Opening the source -i libvirt rhel7.4_no-kernel
[   0.0] Creating an overlay to protect the source from being modified
[   0.2] Initializing the target -o null
[   0.2] Opening the overlay
[   1.1] Inspecting the overlay
[  13.3] Checking for sufficient free disk space in the guest
[  13.3] Estimating space required on target for each disk
[  13.3] Converting Red Hat Enterprise Linux Server 7.4 (Maipo) to run on KVM
virt-v2v: error: libguestfs error: statns: statns_stub: path must start 
with a / character

If reporting bugs, run virt-v2v with debugging enabled and include the 
complete output:

  virt-v2v -v -x [...]


Actual results:
As above

Expected results:
Because "no kernels were found in the bootloader configuration"
is a fatal error,we should get a clear info from conversion.

Additional info:

Comment 2 kuwei@redhat.com 2017-11-30 11:40:54 UTC
Created attachment 1360869 [details]
v2v.log

Comment 3 kuwei@redhat.com 2017-11-30 11:42:22 UTC
When convert a rhel6.9 guest with no available kernels found in the bootloader,we will get the result:
# virt-v2v rhel6.9-no-kernel -o null 
[   0.0] Opening the source -i libvirt rhel6.9-no-kernel
[   0.0] Creating an overlay to protect the source from being modified
[   0.2] Initializing the target -o null
[   0.2] Opening the overlay
[   1.1] Inspecting the overlay
[   9.3] Checking for sufficient free disk space in the guest
[   9.3] Estimating space required on target for each disk
[   9.3] Converting Red Hat Enterprise Linux Server release 6.9 (Santiago) to run on KVM
virt-v2v: warning: ignoring kernel /boot/vmlinuz-2.6.32-66.el6.x86_64 in 
bootloader, as it does not exist.
virt-v2v: warning: ignoring kernel /boot/vmlinuz-2.6.32-66.el6.x86_64 in 
bootloader, as it does not exist.
virt-v2v: error: no kernels were found in the bootloader configuration.

This probably indicates that virt-v2v was unable to parse the bootloader 
configuration of this guest.

If reporting bugs, run virt-v2v with debugging enabled and include the 
complete output:

  virt-v2v -v -x [...]

Comment 4 Pino Toscano 2017-12-01 13:51:01 UTC
Easy patch sent:
https://www.redhat.com/archives/libguestfs/2017-December/msg00000.html

Comment 5 Pino Toscano 2017-12-04 09:18:53 UTC
Fixed with
https://github.com/libguestfs/libguestfs/commit/1b2020ba04e3a986a931acce7465d0a99a213ade
which is in libguestfs >= 1.37.35.

Comment 7 kuwei@redhat.com 2017-12-07 11:36:23 UTC
Hi.I get some different results with the new patch.
Update the packages to the latest version as below:
virt-v2v-1.36.10-3.el7.x86_64
libguestfs-1.36.10-3.el7.x86_64
libvirt-3.9.0-5.el7.x86_64
qemu-kvm-rhev-2.10.0-11.el7.x86_64
libguestfs-winsupport-7.2-2.el7.x86_64
virtio-win-1.9.3-1.el7.noarch

Scenario 1:
1.Prepare a rhel7.4 guest on kvm server,and edit the guest's "boot/grub2/grub.cfg/" file,remove all the available kernel message.
2.Using virt-v2v to convert the guest:
[root@localhost ~]# virt-v2v rhel7.4-multi-kernel-1 -o null
[   0.0] Opening the source -i libvirt rhel7.4-multi-kernel-1
*********
[  14.2] Converting Red Hat Enterprise Linux Server 7.4 (Maipo) to run on KVM
virt-v2v: error: libguestfs error: command: grubby: doing this would leave 
no kernel entries. Not writing out new config.

If reporting bugs, run virt-v2v with debugging enabled and include the 
complete output:

  virt-v2v -v -x [...]
3.From -v -x log could find the command ["grubby --default-kernel"], I think the above conversion looks good.

Scenario 2:
1.Prepare a rhel7.4 guest on kvm server,and edit the guest's "boot/grub2/grub.cfg/" file, add a bogus kernel info and delete or disable the default kernel info of the guest as attachment picture.
2.Using virt-v2v to convert the guest:
# virt-v2v rhel7.4-1-clone  -o null 
[   0.0] Opening the source -i libvirt rhel7.4-1-clone
[   0.0] Creating an overlay to protect the source from being modified
[   0.2] Initializing the target -o null
[   0.2] Opening the overlay
[   1.1] Inspecting the overlay
[  10.9] Checking for sufficient free disk space in the guest
[  10.9] Estimating space required on target for each disk
[  10.9] Converting Red Hat Enterprise Linux Server 7.4 (Maipo) to run on KVM
virt-v2v: This guest has virtio drivers installed.
[  56.4] Mapping filesystem data to avoid copying unused and blank areas
[  56.5] Closing the overlay
[  56.6] Checking if the guest needs BIOS or UEFI to boot
[  56.6] Assigning disks to buses
[  56.6] Copying disk 1/1 to /var/tmp/null.CFVRNE/sda (raw)
    (100.00/100%)
[  78.3] Creating output metadata
[  78.3] Finishing off

3.I think the above conversion should fail but it completed successfully.

Is there any problem with my steps? From my result the bug maybe also has problem.

Comment 8 Pino Toscano 2017-12-07 11:53:52 UTC
For each of the two scenarios, please provide:

- the output of `rpm -qa | grep kernel`
- the output of `ls -l /boot`
- the exact content of the /boot/grub2/grub.cfg

Please also note that:
- v2v considers only kernels installed by packages, so any custom kernel is ignored
- v2v regenerates the grub2 configuration at the end of the conversion step, so any custom change to /boot/grub2/grub.cfg (which is a file generated by grub2-mkconfig) will be lost

Comment 9 kuwei@redhat.com 2017-12-07 13:00:23 UTC
(In reply to Pino Toscano from comment #8)
> For each of the two scenarios, please provide:
> 
> - the output of `rpm -qa | grep kernel`
> - the output of `ls -l /boot`
> - the exact content of the /boot/grub2/grub.cfg
 The two scenario same parts :
# rpm -qa | grep kernel
kernel-3.10.0-693.el7.x86_64
kernel-tools-3.10.0-693.el7.x86_64
abrt-addon-kerneloops-2.1.11-48.el7.x86_64
kernel-tools-libs-3.10.0-693.el7.x86_64
# ls -l /boot
total 122736
-rw-r--r--. 1 root root   140894 Jul  7 08:01 config-3.10.0-693.el7.x86_64
drwxr-xr-x. 3 root root       17 Dec  1 17:07 efi
drwx------. 5 root root       97 Dec  1 17:27 grub2
-rw-------. 1 root root 69251085 Dec  1 17:24 initramfs-0-rescue-53d8f8088f674dc4a5b0d9ea1c66903d.img
-rw-------. 1 root root 30814909 Dec  1 17:27 initramfs-3.10.0-693.el7.x86_64.img
-rw-r--r--. 1 root root 10182072 Dec  1 17:24 initrd-plymouth.img
-rw-r--r--. 1 root root   293027 Jul  7 08:02 symvers-3.10.0-693.el7.x86_64.gz
-rw-------. 1 root root  3228420 Jul  7 08:01 System.map-3.10.0-693.el7.x86_64
-rwxr-xr-x. 1 root root  5875184 Dec  1 17:24 vmlinuz-0-rescue-53d8f8088f674dc4a5b0d9ea1c66903d
-rwxr-xr-x. 1 root root  5875184 Jul  7 08:01 vmlinuz-3.10.0-693.el7.x86_64

Scenario 1:   #cat /boot/grub2/grub.cfg  as attachment grub1
Scenario 2:   #cat /boot/grub2/grub.cfg  as attachment grub2

> Please also note that:
> - v2v considers only kernels installed by packages, so any custom kernel is
> ignored
Yes,you are right.But I just modify grub.cfg file,from scenario 1 and scenario 2 i get different result.

Comment 10 kuwei@redhat.com 2017-12-07 13:00:52 UTC
Created attachment 1364311 [details]
grub1

Comment 11 kuwei@redhat.com 2017-12-07 13:01:16 UTC
Created attachment 1364312 [details]
grub2

Comment 12 Pino Toscano 2017-12-08 09:43:57 UTC
Hm I cannot reproduce the scenario 1, can you please attach the -v -x logs of virt-v2v?

Comment 13 kuwei@redhat.com 2017-12-08 10:20:35 UTC
HI,i am sorry for my comment9, kernel from scenario 1 is different from scenario 2.
Scenario 1:
# rpm -qa | grep kernel
kernel-3.10.0-799.el7.x86_64
kernel-3.10.0-693.el7.x86_64
kernel-debug-3.10.0-799.el7.x86_64
kernel-tools-3.10.0-693.el7.x86_64
abrt-addon-kerneloops-2.1.11-48.el7.x86_64
kernel-tools-libs-3.10.0-693.el7.x86_64

## ls -l /boot/
total 202516
-rw-r--r--. 1 root root   140894 Jul  7 08:01 config-3.10.0-693.el7.x86_64
-rw-r--r--. 1 root root   143458 Nov 27 20:08 config-3.10.0-799.el7.x86_64
-rw-r--r--. 1 root root   143732 Nov 27 20:15 config-3.10.0-799.el7.x86_64.debug
drwxr-xr-x. 3 root root       17 Dec  1 17:07 efi
drwx------. 5 root root       97 Dec  1 19:00 grub2
-rw-------. 1 root root 69251085 Dec  1 17:24 initramfs-0-rescue-53d8f8088f674dc4a5b0d9ea1c66903d.img
-rw-------. 1 root root 30814909 Dec  1 17:27 initramfs-3.10.0-693.el7.x86_64.img
-rw-------. 1 root root 30541739 Dec  1 19:00 initramfs-3.10.0-799.el7.x86_64.debug.img
-rw-------. 1 root root 30414504 Dec  1 18:59 initramfs-3.10.0-799.el7.x86_64.img
-rw-r--r--. 1 root root 10182072 Dec  1 17:24 initrd-plymouth.img
-rw-r--r--. 1 root root   293027 Jul  7 08:02 symvers-3.10.0-693.el7.x86_64.gz
-rw-r--r--. 1 root root   303042 Nov 27 20:16 symvers-3.10.0-799.el7.x86_64.debug.gz
-rw-r--r--. 1 root root   300928 Nov 27 20:09 symvers-3.10.0-799.el7.x86_64.gz
-rw-------. 1 root root  3228420 Jul  7 08:01 System.map-3.10.0-693.el7.x86_64
-rw-------. 1 root root  3339986 Nov 27 20:08 System.map-3.10.0-799.el7.x86_64
-rw-------. 1 root root  3498611 Nov 27 20:15 System.map-3.10.0-799.el7.x86_64.debug
-rwxr-xr-x. 1 root root  5875184 Dec  1 17:24 vmlinuz-0-rescue-53d8f8088f674dc4a5b0d9ea1c66903d
-rwxr-xr-x. 1 root root  5875184 Jul  7 08:01 vmlinuz-3.10.0-693.el7.x86_64
-rwxr-xr-x. 1 root root  6221920 Nov 27 20:08 vmlinuz-3.10.0-799.el7.x86_64
-rwxr-xr-x. 1 root root  6758496 Nov 27 20:15 vmlinuz-3.10.0-799.el7.x86_64.debug

The scenario 1 has multi kernels.The v2v log as attachment: v2v-scenario-1.

Another testing with the multi kernel guest, edit the guest's "boot/grub2/grub.cfg/" file, add a bogus kernel info and delete available kernel info of the guest.The v2v conversion also can be finished successfully.


Thanks

Comment 14 kuwei@redhat.com 2017-12-08 10:21:29 UTC
Created attachment 1364730 [details]
v2v-scenario-1

Comment 16 Pino Toscano 2017-12-20 11:40:45 UTC
The failure in scenario 1 happens because:
a) the grub configuration has no entries (they were manually removed)
b) v2v detects the installed kernels, and calculates the best one among them
c) because of (a), grubby returns no default kernel
d) because of (c), v2v tries to set the best kernel as new default
e) grubby does not like that the kernel requested to be default is not in the configuration, and thus fails

This situation is on the borderline of "not going to happen": the configuration of grub2 is generated by grub2-mkconfig when installing/updating/removing kernels, so any manual editing will be lost.

That said, maybe v2v could always update the grub2 configuration, and set the default kernel after that: this should give the guest a valid grub2 configuration again, and grubby should happily do its job after that.  Please open a new bug for it, although it will not be an high priority (since, as mentioned above, the situation is not something that will be usually found in guests).

Comment 17 kuwei@redhat.com 2017-12-22 03:30:08 UTC
From comment 16,and Regarding RHEL 7: the method we use to list the kernels is to get all the paths that match the globs "/boot/kernel-*", "/boot/vmlinuz-*" and
"/vmlinuz-*".  This is also what grub2-mkconfig does (in particular,the /etc/grub.d/10_linux script) to list kernels.

I have file another bug 1528523 to track the grub2 problem.

Test another scenario:
Steps:
1.Prepare a guest and there are no "/boot/kernel-*", "/boot/vmlinuz-*" and "/vmlinuz-*" 
2.Use virt-v2v to convert the guest.
]# virt-v2v rhel7.4 -o null
[   0.0] Opening the source -i libvirt rhel7.4
[   0.0] Creating an overlay to protect the source from being modified
[   0.2] Initializing the target -o null
[   0.2] Opening the overlay
[   1.2] Inspecting the overlay
[  11.1] Checking for sufficient free disk space in the guest
[  11.1] Estimating space required on target for each disk
[  11.1] Converting Red Hat Enterprise Linux Server 7.4 Beta (Maipo) to run on KVM
virt-v2v: error: no installed kernel packages were found.

This probably indicates that virt-v2v was unable to inspect this guest 
properly.

If reporting bugs, run virt-v2v with debugging enabled and include the 
complete output:

  virt-v2v -v -x [...]

Move the bug to VERIFIED, because the grub2 problem has been tracked by the new bug 1528523.

Comment 20 errata-xmlrpc 2018-04-10 09:20:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0677


Note You need to log in before you can comment on or make changes to this bug.