Bug 1854380

Summary: Virt-v2v cannot convert guest from ESXI7.0 server without vddk
Product: Red Hat Enterprise Linux 8 Reporter: Pino Toscano <ptoscano>
Component: libguestfsAssignee: Pino Toscano <ptoscano>
Status: CLOSED ERRATA QA Contact: mxie <mxie>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 8.2CC: jsuchane, juzhou, mxie, mzhan, ptoscano, rjones, tyan, tzheng, virt-maint, xiaodwan, zc640618, zili
Target Milestone: rcKeywords: Automation
Target Release: 8.3   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: V2V
Fixed In Version: libguestfs-1.40.2-25.module+el8.3.0+7421+642fe24f Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1837328 Environment:
Last Closed: 2020-11-04 02:54:04 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1837328, 1841038    
Bug Blocks:    

Description Pino Toscano 2020-07-07 10:49:17 UTC
+++ This bug was initially created as a clone of Bug #1837328 +++

Description of problem:
Virt-v2v cannot convert guest from ESXI7.0 server without vddk

Version-Release number of selected component (if applicable):
virt-v2v-1.40.2-22.module+el8.3.0+6423+e4cb6418.x86_64
libguestfs-1.40.2-22.module+el8.3.0+6423+e4cb6418.x86_64
VMware-vix-disklib-6.7.3-14389676.x86_64.tar.gz

How reproducible:
100%

Steps to Reproduce:
1.Convert a guest from ESXI7.0 server
# virt-v2v -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1  esx7.0-rhel8.2-x86_64 -o rhv -os 10.66.144.40:/home/nfs_export -ip /home/passwd
[   0.0] Opening the source -i libvirt -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1 esx7.0-rhel8.2-x86_64
'curl' -q --max-redirs '5' --globoff --head --silent --url 'https://10.73.198.169/folder/esx7.0-rhel8.2-x86%5f64/esx7.0-rhel8.2-x86%5f64-flat.vmdk?dcPath=data&dsName=esx7.0-matrix' --user <hidden> --insecure
HTTP/2 200 
date: Tue, 19 May 2020 09:07:26 GMT
set-cookie: vmware_soap_session="518c23795096a6ea558373f20d8ccc905e0a1ee1"; Path=/; HttpOnly; Secure; 
accept-ranges: bytes
content-security-policy: block-all-mixed-content
content-type: application/octet-stream
strict-transport-security: max-age=31536000
x-content-type-options: nosniff
x-frame-options: DENY
x-xss-protection: 1
content-length: 12884901888
x-envoy-upstream-service-time: 420
server: envoy

virt-v2v: error: vcenter: invalid response from server

If reporting bugs, run virt-v2v with debugging enabled and include the 
complete output:

  virt-v2v -v -x [...]


Actual results:
As above

Expected results:
Virt-v2v can convert guest from ESXI7.0 server without vddk 

Additional info:
virt-v2v can convert guest from ESXI7.0 server with vddk.

--- Additional comment from Richard W.M. Jones on 2020-05-19 11:48:15 CEST ---

Looks real.  I bet they've changed/broken the URLs ...

--- Additional comment from Pino Toscano on 2020-05-19 11:57:39 CEST ---

(In reply to liuzi from comment #0)
> 'curl' -q --max-redirs '5' --globoff --head --silent --url
> 'https://10.73.198.169/folder/esx7.0-rhel8.2-x86%5f64/esx7.0-rhel8.2-
> x86%5f64-flat.vmdk?dcPath=data&dsName=esx7.0-matrix' --user <hidden>
> --insecure
> HTTP/2 200 
> date: Tue, 19 May 2020 09:07:26 GMT
> set-cookie: vmware_soap_session="518c23795096a6ea558373f20d8ccc905e0a1ee1";
> Path=/; HttpOnly; Secure; 
> accept-ranges: bytes
> content-security-policy: block-all-mixed-content
> content-type: application/octet-stream
> strict-transport-security: max-age=31536000
> x-content-type-options: nosniff
> x-frame-options: DENY
> x-xss-protection: 1
> content-length: 12884901888
> x-envoy-upstream-service-time: 420
> server: envoy

What I see above is that vCenter replies HTTP/2, and that status string is what virt-v2v tries to parse as "HTTP/x.y nnn" string, which obviously fail.

I will have a try to improve the parsing of the HTTP status string.

--- Additional comment from Pino Toscano on 2020-05-19 12:21:21 CEST ---

Hopefully simple patch posted:
https://www.redhat.com/archives/libguestfs/2020-May/msg00036.html

Note that also the fix for bug 1837337 will be needed to test this, as otherwise nbdkit will not be able to parse HTTP response headers in a case-insensitive way.

--- Additional comment from Pino Toscano on 2020-05-19 13:35:23 CEST ---

Fixed upstream with
https://github.com/libguestfs/virt-v2v/commit/d2aa82317964d62fcc8dc7b6737773003d04b998

------------------------------------------------------

Moving it back (by cloning) to RHEL as it affects QA testing...

Comment 3 mxie@redhat.com 2020-07-29 08:06:58 UTC
Verify the bug with below builds:
virt-v2v-1.40.2-25.module+el8.3.0+7421+642fe24f.x86_64
libguestfs-1.40.2-25.module+el8.3.0+7421+642fe24f.x86_64
libvirt-6.0.0-26.module+el8.3.0+7455+623d0f4c.x86_64
qemu-kvm-4.2.0-31.module+el8.3.0+7437+4bb96e0d.x86_64

Steps:
1.Use virt-v2v to convert a rhel guest from ESXi7.0 to rhv without vddk
#  virt-v2v -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1 -ip /home/passwd   -o rhv-upload -oo rhv-cafile=/root/ca.pem -oo rhv-direct -oc https://dell-per740-22.lab.eng.pek2.redhat.com/ovirt-engine/api -op /home/rhvpasswd -of raw -oo rhv-cluster=Default -os nfs_data -b ovirtmgmt esx7.0-rhel8.2-x86_64
[   0.6] Opening the source -i libvirt -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1 esx7.0-rhel8.2-x86_64
[   2.3] Creating an overlay to protect the source from being modified
[   2.7] Opening the overlay
[  22.0] Inspecting the overlay
[ 142.5] Checking for sufficient free disk space in the guest
[ 142.5] Estimating space required on target for each disk
[ 142.5] Converting Red Hat Enterprise Linux 8.2 (Ootpa) to run on KVM
virt-v2v: warning: guest tools directory ‘linux/el8’ is missing from 
the virtio-win directory or ISO.

Guest tools are only provided in the RHV Guest Tools ISO, so this can 
happen if you are using the version of virtio-win which contains just the 
virtio drivers.  In this case only virtio drivers can be installed in the 
guest, and installation of Guest Tools will be skipped.
virt-v2v: This guest has virtio drivers installed.
[1629.7] Mapping filesystem data to avoid copying unused and blank areas
[1632.8] Closing the overlay
[1633.1] Assigning disks to buses
[1633.1] Checking if the guest needs BIOS or UEFI to boot
[1633.1] Initializing the target -o rhv-upload -oc https://dell-per740-22.lab.eng.pek2.redhat.com/ovirt-engine/api -op /home/rhvpasswd -os nfs_data
[1634.4] Copying disk 1/1 to qemu URI json:{ "file.driver": "nbd", "file.path": "/var/tmp/rhvupload.uFt08S/nbdkit0.sock", "file.export": "/" } (raw)
    (100.00/100%)
[3073.0] Creating output metadata
[3074.9] Finishing off

2.Use virt-v2v to convert a windows guest from ESXi7.0 to rhv without vddk
#  virt-v2v -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1 -ip /home/passwd -o rhv -os 10.73.224.195:/home/nfs_export -of raw esx7.0-win2019-x86_64
[   0.0] Opening the source -i libvirt -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1 esx7.0-win2019-x86_64
[   1.6] Creating an overlay to protect the source from being modified
[   2.1] Opening the overlay
[  16.6] Inspecting the overlay
[ 134.4] Checking for sufficient free disk space in the guest
[ 134.4] Estimating space required on target for each disk
[ 134.4] Converting Windows Server 2019 Standard to run on KVM
virt-v2v: warning: /usr/share/virt-tools/pnp_wait.exe is missing.  
Firstboot scripts may conflict with PnP.
virt-v2v: warning: there is no QXL driver for this version of Windows (10.0 
x86_64).  virt-v2v looks for this driver in 
/usr/share/virtio-win/virtio-win.iso

The guest will be configured to use a basic VGA display driver.
virt-v2v: This guest has virtio drivers installed.
[ 169.6] Mapping filesystem data to avoid copying unused and blank areas
[ 170.5] Closing the overlay
[ 170.8] Assigning disks to buses
[ 170.8] Checking if the guest needs BIOS or UEFI to boot
[ 170.8] Initializing the target -o rhv -os 10.73.224.195:/home/nfs_export
[ 171.5] Copying disk 1/1 to /tmp/v2v.CahJqd/f868d3b1-beee-43be-8895-0181012a7fd0/images/35ab9eea-5905-4982-95c7-6e6c8d1c5107/5ffff5ab-55e3-4bc8-85f9-a128ba9ee51a (raw)
    (100.00/100%)
[2104.0] Creating output metadata
[2104.2] Finishing off


3.Check the guest on rhv after v2v conversion, the checkpoints of two guests are passed


Result:
    But has been fixed with virt-v2v-1.40.2-25.module+el8.3.0+7421, so move the bug from ON_QA to VERIFIED

Comment 6 errata-xmlrpc 2020-11-04 02:54:04 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: virt:rhel and virt-devel:rhel security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:4676