Bug 1392904

Summary: Unable to v2v Vmware ESX guests due supermin issue
Product: Red Hat Enterprise Virtualization Manager Reporter: Alessio <alessio.dini>
Component: imgbasedAssignee: Ryan Barry <rbarry>
Status: CLOSED ERRATA QA Contact: Qin Yuan <qiyuan>
Severity: high Docs Contact:
Priority: high    
Version: 4.0.0CC: cshao, dfediuck, dguo, fsun, huzhao, juzhou, kuwei, leiwang, lsurette, mxie, nashok, pstehlik, qiyuan, rhodain, srevivo, tzheng, weiwang, yaniwang, ycui, ykaul
Target Milestone: ovirt-4.1.0-beta   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: imgbased-0.9.3-0.1.el7ev Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Redhat Enterprise Virtualization 4.0
Last Closed: 2017-04-20 19:01:17 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Node RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
virt-v2v debug log none

Description Alessio 2016-11-08 12:54:08 UTC
Created attachment 1218517 [details]
virt-v2v debug log

Description of problem:
Hello all,
our infrastructure is composited by:
2x RHEV-H 4.0 hypervisor  ( recently updated having kernel 3.10.0-327.36.1.el7.x86_64 ) 
1x RHEV-M hosted engine vm

We are trying to move multiple guests from Vmware ESX to RHEV 4.
Unfortunately each v2v failed with supermin5 "failed to find a suitable kernel" error.
I attach the complete v2v log file, virt-v2v.log

I still don't know if the issue is LVM thin installation related, but supermin is looking for 3.10.0-327.36.1.el7.x86_64 kernel under /boot

[root@rhvmi2almerc01 ~]# ls -lrt /boot
total 56484
-rw-r--r--. 1 root root   176500 Sep  5  2014 memtest86+-4.20
-rw-r--r--. 1 root root   178176 Sep  5  2014 elf-memtest86+-4.20
-rw-r--r--. 1 root root    12620 Nov 11  2014 tboot-syms
-rw-r--r--. 1 root root   326628 Nov 11  2014 tboot.gz
-rw-------. 1 root root  2964948 Jun 27 20:52 System.map-3.10.0-327.28.2.el7.x86_64
-rw-r--r--. 1 root root   126431 Jun 27 20:52 config-3.10.0-327.28.2.el7.x86_64
-rwxr-xr-x. 1 root root  5157728 Jun 27 20:52 vmlinuz-3.10.0-327.28.2.el7.x86_64
-rw-r--r--. 1 root root   252632 Jun 27 20:54 symvers-3.10.0-327.28.2.el7.x86_64.gz
drwxr-xr-x. 3 root root     4096 Aug 17 22:07 efi
drwxr-xr-x. 2 root root     4096 Aug 17 22:11 extlinux
-rw-r--r--. 1 root root   603547 Aug 17 22:20 initrd-plymouth.img
drwx------. 2 root root    16384 Sep 23 13:09 lost+found
-rw-r--r--. 1 root root 47978564 Sep 23 13:13 initramfs-3.10.0-327.28.2.el7.x86_64.img
drwxr-xr-x. 2 root root     4096 Sep 23 13:14 rhvh-4.0-0.20160817.0+1
drwxr-xr-x. 2 root root     4096 Oct 19 17:49 rhvh-4.0-0.20160919.0+1
drwxr-xr-x. 2 root root     4096 Nov  3 14:33 rhvh-4.0-0.20161012.0+1
drwx------. 6 root root     4096 Nov  3 14:34 grub2


The correct file is present under /boot/rhvh-4.0-0.20161012.0+1
rhvh-4.0-0.20161012.0+1 is made from vg+lvs:

[root@rhvmi2almerc01 ~]# vgs
  VG                                   #PV #LV #SN Attr   VSize    VFree
  08ac0374-5c4d-4259-ac35-eb625addbfaf   1  13   0 wz--n- 1023.75g  948.38g
  573a4372-7ab2-4daa-b850-397754b7836b   1   8   0 wz--n- 1023.75g 1019.62g
  rhvh                                   1  10   0 wz--n-   67.33g    5.95g

[root@rhvmi2almerc01 ~]# lvs | grep rhvh
  pool00                               rhvh                                 twi-aotz--  29.82g                              37.72  21.51                        
  rhvh-4.0-0.20160817.0                rhvh                                 Vwi---tz-k  14.78g pool00 root                                                      
  rhvh-4.0-0.20160817.0+1              rhvh                                 Vwi---tz--  14.78g pool00 rhvh-4.0-0.20160817.0                                     
  rhvh-4.0-0.20160919.0                rhvh                                 Vri---tz-k   3.81g pool00                                                           
  rhvh-4.0-0.20160919.0+1              rhvh                                 Vwi---tz--   3.81g pool00 rhvh-4.0-0.20160919.0                                     
  rhvh-4.0-0.20161012.0                rhvh                                 Vri---tz-k   3.81g pool00                                                           
  rhvh-4.0-0.20161012.0+1              rhvh                                 Vwi-aotz--   3.81g pool00 rhvh-4.0-0.20161012.0 47.05                               
  root                                 rhvh                                 Vwi---tz--  14.78g pool00                                                           
  swap                                 rhvh                                 -wi-ao----  31.50g                                                                  
  var                                  rhvh                                 Vwi-aotz--  15.00g pool00                       25.87 


We solved using two methods:

1)
export SUPERMIN_KERNEL_VERSION=3.10.0-327.36.1.el7.x86_64
export SUPERMIN_KERNEL=/boot/rhvh-4.0-0.20161012.0+1/vmlinuz-3.10.0-327.36.1.el7.x86_64
export SUPERMIN_MODULES=/lib/modules/$SUPERMIN_KERNEL_VERSION
rm -rf /var/tmp/.guestfs-*

2)
ln -s /boot/rhvh-4.0-0.20161012.0+1/vmlinuz-3.10.0-327.36.1.el7.x86_64 /boot/vmlinuz-3.10.0-327.36.1.el7.x86_64

Both workarounds are temporary because each time the hypervisor are updated we must update the workaround again.


supermin package version is supermin5-5.1.10-1.2.el7.x86_64


Version-Release number of selected component (if applicable):
supermin5-5.1.10-1.2.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Install RHEV-H 4.0 hypervisor and deploy RHEV-M hosted engine vm
2. Update the hypervisor
3. run export LIBGUESTFS_BACKEND=direct on the hypervisor
4. run libguestfs-test-tool on the hypervisor

Actual results:
supermin: failed to find a suitable kernel (host_cpu=x86_64).

Expected results:
TEST OK

Additional info:

Comment 1 Fabian Deutsch 2016-11-10 01:29:53 UTC
We actually new that there might be issues with the ernel not residing in the usual place, but somewhere else.

I tend to think that it might make sense to have a little systemd service which is adding a (sym)link during boot to link the currently used kernel into /boot - for compatability reasons.

But I'd avoid linking the initrd, because the initrd could be changed by i.e. dracut, and that raises other issues we'd then need to cover by such a service.

Comment 2 Qin Yuan 2016-11-21 06:47:41 UTC
Reproduced this issue by steps in the Description part, but didn't depoly RHEV-M hosted engin vm in step1.

Tested the two workarounds given in the Description part, both can work well.


Version-Release number of selected component (if applicable):
Build1:
redhat-virtualization-host-4.0-20160919.0
imgbased-0.8.5-0.1.el7ev.noarch
Build2:
redhat-virtualization-host-4.0-20161116.1
imgbased-0.8.10-0.1.el7ev.noarch
supermin5-5.1.16-4.el7.x86_64

Comment 3 Ryan Barry 2016-12-15 20:48:50 UTC
I'm not able to reproduce this -- the appropriate kernel is present in /boot 

We'll still want a service to copy the initrd over (in case of initrd rebuilds), but can you please give me exact steps to reproduce?

Comment 4 Ryan Barry 2016-12-15 20:49:20 UTC
Tested version:

redhat-virtualization-host-4.0-20161012.0

Comment 5 Qin Yuan 2016-12-16 07:17:07 UTC
Test versions:
Build1:
redhat-virtualization-host-4.0-20161012.0

Build2(the latest build):
redhat-virtualization-host-4.0-20161214.0


Test steps:
1. Install Build1.
2. Reboot and log in to Build1.
3. Upgrade to Build2, using "yum update".
4. Reboot and log in to Build2.
5. run "export LIBGUESTFS_BACKEND=direct"
6. run "libguestfs-test-tool"

Test results:
After step6, there is "supermin: failed to find a suitable kernel (host_cpu=x86_64)." printed on the screen.


Tips:
Don't execute libguestfs-test-tool under Build1, but only execute it after upgrade and login to Build2.

Comment 6 Ryan Barry 2016-12-16 17:10:34 UTC
I'm still not able to reproduce this.

Test steps:
1. Install redhat-virtualization-host-4.0-20161012.0 over PXE
2. Reboot and log in to redhat-virtualization-host-4.0-20161012.0
3. Upgrade to redhat-virtualization-host-4.0-20161214.0 using "yum update"
4. Reboot and log in to redhat-virtualization-host-4.0-20161214.0.
5. run "export LIBGUESTFS_BACKEND=direct"
6. run "libguestfs-test-tool"
7. libguestfs-test-tool completes successfully

Can you please provide a test system?

What method was used for installation?

Comment 7 Qin Yuan 2016-12-19 06:20:59 UTC
Have sent test system infos by email.

Comment 8 Sandro Bonazzola 2017-01-10 07:48:21 UTC
Referenced patch is still under review, moving back to POST

Comment 9 Qin Yuan 2017-01-19 10:10:15 UTC
Test Versions:
Build1:
redhat-virtualization-host-4.0-20161116.1
imgbased-0.8.10-0.1.el7ev.noarch

Build2:
redhat-virtualization-host-4.1-20170116.0
imgbased-0.9.4-0.1.el7ev.noarch

Test Steps:
The same as steps in Comment #5

Test Results:
Run libguestfs-test-tool successfully without supermin issue.

So, the bug is fixed, change status to VERIFIED.

Comment 10 Ryan Barry 2017-02-22 13:06:31 UTC
*** Bug 1425744 has been marked as a duplicate of this bug. ***

Comment 11 Ryan Barry 2017-03-27 23:39:48 UTC
*** Bug 1436133 has been marked as a duplicate of this bug. ***

Comment 12 errata-xmlrpc 2017-04-20 19:01:17 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1114