Bug 1492633

Created attachment 1327331 [details]
virt-v2v-rhel7.4-kernel-scenario2.log

Hi Pino,

   I found v2v will not run "grub2-mkconfig -o /boot/grub2/grub.cfg" to adjust the kernel order of grub.cfg for rhel7 guest if guest has same version debug kernel and normal kernel during conversion, but why v2v will run that command when converting rhel7 guest which has different version kernels like comment0?

Before converting:
   # awk -F\' '$1=="menuentry " {print i++ " : " $2}' /etc/grub2.cfg
     0 : Red Hat Enterprise Linux Server (3.10.0-693.el7.x86_64.debug) 7.4 (Maipo)
     1 : Red Hat Enterprise Linux Server (3.10.0-693.el7.x86_64) 7.4 (Maipo)
     2 : Red Hat Enterprise Linux Server (0-rescue-acd9b2fd1f8f4842b7b98344a756997d) 7.4 (Maipo)

   # grub2-editenv list
     saved_entry=0

   # grubby --default-kernel
    /boot/vmlinuz-3.10.0-693.el7.x86_64.debug
  
   # grubby --default-index
     0



After converting:
   # awk -F\' '$1=="menuentry " {print i++ " : " $2}' /etc/grub2.cfg
     0 : Red Hat Enterprise Linux Server (3.10.0-693.el7.x86_64.debug) 7.4 (Maipo)
     1 : Red Hat Enterprise Linux Server (3.10.0-693.el7.x86_64) 7.4 (Maipo)
     2 : Red Hat Enterprise Linux Server (0-rescue-acd9b2fd1f8f4842b7b98344a756997d) 7.4 (Maipo)

   # grub2-editenv list
     saved_entry=Red Hat Enterprise Linux Server (3.10.0-693.el7.x86_64) 7.4 (Maipo)

   # grubby --default-kernel
    /boot/vmlinuz-3.10.0-693.el7.x86_64
  
   # grubby --default-index
     1

In scenario 1, virt-v2v correctly lists the installed kernels, the
bootloader kernels, and the default kernel.  It also correctly sees
that the best kernel is the current default kernel, and so setting/changing
the default kernel is not necessary.

However because it updates paths in /boot/grub2/device.map, it then has
to run ‘/sbin/grub2-mkconfig -o /boot/grub2/grub.cfg’.

This command appears to cause the default kernel to change, which is
apparently a bug in grub2.  However I need to come up with some kind of
independent reproducer first before reassigning.

(In reply to mxie from comment #3)
>    I found v2v will not run "grub2-mkconfig -o /boot/grub2/grub.cfg" to
> adjust the kernel order of grub.cfg for rhel7 guest if guest has same
> version debug kernel and normal kernel during conversion, but why v2v will
> run that command when converting rhel7 guest which has different version
> kernels like comment0?

It has nothing to do with the kernel versions.  virt-v2v runs grub2-mkconfig
only if it changes a device name in either /etc/fstab, device.map, or one
of the other grub configuration files like /etc/sysconfig/grub.  See:

https://github.com/libguestfs/libguestfs/blob/93a2f9d21354807a44b7c71a0ab88a7063d91b87/v2v/convert_linux.ml#L913-L999

My colleague haizhao found another scenario about this bug

   Install kernel debug in original rhel7 guest.Set booted kernel as 0 although the first one is not a real kernel, so guest will boot into debug kernel. Use v2v to convert guest to rhv, but guest will not use the newest kernel to boot after v2vconversion(v2v didn't do anything to kernel during conversion)

Before converting:
   # awk -F\' '$1=="menuentry " {print i++ " : " $2}' /etc/grub2.cfg
    0:Red Hat Enterprise Linux Server 7.4 Rescue 01dfb37532da4bbfa52fe7814d12b900 (3.10.0-693.el7.x86_64.debug)
    1:Red Hat Enterprise Linux Server (3.10.0-693.el7.x86_64.debug) 7.4 (Maipo)
    2:Red Hat Enterprise Linux Server (3.10.0-693.el7.x86_64) 7.4 (Maipo)
    3:Red Hat Enterprise Linux Server (0-rescue-6998121f7a5b4649ad5d223c52da1e40) 7.4 (Maipo)
   
   # grub2-editenv list
     saved_entry=0

   # grubby --default-kernel
    /boot/vmlinuz-rescue-01dfb37532da4bbfa52fe7814d12b900 

   # grubby --default-index
     0

   #uname -a
    Linux localhost.localdomain 3.10.0-693.el7.x86_64.debug #1 SMP Thu Jul 6 19:56:57 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux



After converting:
   # awk -F\' '$1=="menuentry " {print i++ " : " $2}' /etc/grub2.cfg
    0:Red Hat Enterprise Linux Server 7.4 Rescue 01dfb37532da4bbfa52fe7814d12b900 (3.10.0-693.el7.x86_64.debug)
    1:Red Hat Enterprise Linux Server (3.10.0-693.el7.x86_64.debug) 7.4 (Maipo)
    2:Red Hat Enterprise Linux Server (3.10.0-693.el7.x86_64) 7.4 (Maipo)
    3:Red Hat Enterprise Linux Server (0-rescue-6998121f7a5b4649ad5d223c52da1e40) 7.4 (Maipo)
   
   # grub2-editenv list
     saved_entry=0

   # grubby --default-kernel
    /boot/vmlinuz-rescue-01dfb37532da4bbfa52fe7814d12b900 

   # grubby --default-index
     0

   #uname -a
    Linux dhcp-66-145-69.nay,redhat.com 3.10.0-693.el7.x86_64.debug #1 SMP Thu Jul 6 20:03:17 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux

I'm hitting this with freshly downloaded RHEL8 KVM images.
In my case I'm getting the host kernel instead of the guest. 

$ uname -r
3.10.0-957.21.3.el7.x86_64
$ guestfish --verbose --rw -a rhel-8.0-update-1-x86_64-kvm.qcow2 

Welcome to guestfish, the guest filesystem shell for
editing virtual machine filesystems and disk images.

Type: ‘help’ for help on commands
      ‘man’ to read the manual
      ‘quit’ to quit the shell

><fs> run
...
[    0.000000] Linux version 3.10.0-957.21.3.el7.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-36) (GCC) ) #1 SMP Fri Jun 14 02:54:29 EDT 2019
...

One problem with this is that XFS in RHEL8 uses features that are only read-only compatible with RHEL7, so the guest image cant be edited. 

><fs> mount /dev/sda1 /
...
command: mount: stderr:
mount: wrong fs type, bad option, bad superblock on /dev/sda1,
       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try
       dmesg | tail or so.
...
[    0.810152] alg: No test for __gcm-aes-aesni (__driver-gcm-aes-aesni)
[    0.811011] alg: No test for __generic-gcm-aes-aesni (__driver-generic-gcm-aes-aesni)
[    0.859302] device-mapper: uevent: version 1.0.3
[    0.860266] device-mapper: ioctl: 4.37.1-ioctl (2018-04-03) initialised: dm-devel
[   13.835874] SGI XFS with ACLs, security attributes, no debug enabled
[   13.840082] XFS (sda1): Superblock has unknown read-only compatible features (0x4) enabled.
[   13.841146] XFS (sda1): Attempted to mount read-only compatible filesystem read-write.
[   13.842127] XFS (sda1): Filesystem can only be safely mounted read only.
[   13.842964] XFS (sda1): SB validate failed with error -22.

(In reply to Donald Douwsma from comment #6)
> I'm hitting this with freshly downloaded RHEL8 KVM images.
> In my case I'm getting the host kernel instead of the guest. 

What you hit is the "RHEL 7 kernel cannot mount a RHEL 8 XFS filesystem" issue, which is unrelated to this bug.
See also bug 1667478.

Thanks Pinto, That's what I was searching for when I found this one.

After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.

Sorry, this bug should not have been closed.

I think this is genuinely a stale bug that we are not going to 
be able to solve (since the best case scenario for a fix involves
fixing grub in RHEL 7 inside existing guests).  The guest still
boots, albeit with a slightly older kernel.