Bug 1280501

Summary: Error converting any VMware vCloud VM
Product: Red Hat Enterprise Linux 7 Reporter: Alessandro <alfonsi.alex>
Component: libguestfsAssignee: Richard W.M. Jones <rjones>
Status: CLOSED DUPLICATE QA Contact: Virtualization Bugs <virt-bugs>
Severity: high Docs Contact:
Priority: unspecified    
Version: 7.3CC: alfonsi.alex, ptoscano
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-11-13 16:55:45 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1277738    
Bug Blocks:    

Description Alessandro 2015-11-11 22:36:31 UTC
Description of problem:
When Running the following command:
virt-v2v -ic vpx://Domain%5cUSERNAME@vCenterServer/Data%20Center%20Name/Logical%20Folder/ClusterESXi/ESXiHostName/?no_verify=1 -o libvirt -os virtimages "conversion-test-vm2 (f97c63bc-e711-4281-bb57-627c86e06b18)" --password-file /tmp/passwordfile

ERROR:
[   0.0] Opening the source -i libvirt -ic vpx://Domain%5cUSERNAME@vCenterServer/Data%20Center%20Name/Logical%20Folder/ClusterESXi/ESXiHostName/?no_verify=1 conversion-test-vm2 (f97c63bc-e711-4281-bb57-627c86e06b18)
curl -q --insecure --user '<hidden>' --head --silent --url 'https://vCenterServer/folder/conversion-test-vm2%20%28f97c63bc-e711-4281-bb57-627c86e06b18%29/conversion-test-vm2%20%28f97c63bc-e711-4281-bb57-627c86e06b18%29-flat.vmdk?dcPath=Data%20Center%20Name/Logical%20Folder/ClusterESXi&dsName=virtual%5fccf0444%5ft1%5fsnap07%5f01'
HTTP/1.0 200 Connection established

HTTP/1.1 404 Not Found
Date: Wed, 11 Nov 2015 21:40:06 GMT
Set-Cookie: vmware_soap_session="523f6e5e-3087-bf80-671c-907575a82ed4"; Path=/; HttpOnly; Secure; 
Connection: close
Content-Type: text; charset=plain
Content-Length: 0

virt-v2v: error: vcenter: URL not found: 
https://vCenterServer/folder/conversion-test-vm2%20%28f97c63bc-e711-4281-bb57-627c86e06b18%29/conversion-test-vm2%20%28f97c63bc-e711-4281-bb57-627c86e06b18%29-flat.vmdk?dcPath=Data%20Center%20Name/Logical%20Folder/ClusterESXi&dsName=virtual%5fccf0444%5ft1%5fsnap07%5f01

The '--dcpath' parameter may be useful.  See the explanation in the 
virt-v2v(1) man page OPTIONS section.

If reporting bugs, run virt-v2v with debugging enabled and include the 
complete output:

  virt-v2v -v -x [...]

This exact same command on the exact same VM works fine with older version of Virt-v2v.

Version-Release number of selected component (if applicable):
Libvirt version:
libvirt-client-1.2.17-3.el7.x86_64

Virt-v2v version:
virt-v2v-1.31.7-2.el7.x86_64

How reproducible:
Convert a vCloud Director VM using Virt-v2v latest version (7.3)

Actual results:
[   0.0] Opening the source -i libvirt -ic vpx://Domain%5cUSERNAME@vCenterServer/Data%20Center%20Name/Logical%20Folder/ClusterESXi/ESXiHostName/?no_verify=1 conversion-test-vm2 (f97c63bc-e711-4281-bb57-627c86e06b18)
curl -q --insecure --user '<hidden>' --head --silent --url 'https://vCenterServer/folder/conversion-test-vm2%20%28f97c63bc-e711-4281-bb57-627c86e06b18%29/conversion-test-vm2%20%28f97c63bc-e711-4281-bb57-627c86e06b18%29-flat.vmdk?dcPath=Data%20Center%20Name/Logical%20Folder/ClusterESXi&dsName=virtual%5fccf0444%5ft1%5fsnap07%5f01'
HTTP/1.0 200 Connection established

HTTP/1.1 404 Not Found
Date: Wed, 11 Nov 2015 21:40:06 GMT
Set-Cookie: vmware_soap_session="523f6e5e-3087-bf80-671c-907575a82ed4"; Path=/; HttpOnly; Secure; 
Connection: close
Content-Type: text; charset=plain
Content-Length: 0

virt-v2v: error: vcenter: URL not found: 
https://vCenterServer/folder/conversion-test-vm2%20%28f97c63bc-e711-4281-bb57-627c86e06b18%29/conversion-test-vm2%20%28f97c63bc-e711-4281-bb57-627c86e06b18%29-flat.vmdk?dcPath=Data%20Center%20Name/Logical%20Folder/ClusterESXi&dsName=virtual%5fccf0444%5ft1%5fsnap07%5f01

The '--dcpath' parameter may be useful.  See the explanation in the 
virt-v2v(1) man page OPTIONS section.

If reporting bugs, run virt-v2v with debugging enabled and include the 
complete output:

  virt-v2v -v -x [...]

Expected results:
[   0.0] Opening the source -i libvirt -ic vpx://Domain%5cUSERNAME@vCenterServer/Data%20Center%20Name/Logical%20Folder/ClusterESXi/ESXiHostName/?no_verify=1 conversion-test-vm2 (f97c63bc-e711-4281-bb57-627c86e06b18)
[   2.0] Creating an overlay to protect the source from being modified
[   3.0] Opening the overlay
ecc..

Additional info:
The same command works using the following tools versions:
Libvirt version:
libvirt-client-1.2.17-1.el7.x86_64

Virt-v2v version:
virt-v2v-1.28.1-1.48.el7.x86_64

Comment 2 Richard W.M. Jones 2015-11-12 14:57:39 UTC
Suggest going to:

https://vCenterServer/folder/

and comparing the URLs against the ones that virt-v2v is
trying to download from.  As the error message suggests,
the --dcpath parameter may be useful.

Comment 3 Richard W.M. Jones 2015-11-12 14:58:14 UTC
Actually I think it's *without* the slash, so:

https://vCenterServer/folder

Comment 4 Alessandro 2015-11-12 15:32:39 UTC
I did try that but I am honestly not sure with folder what you mean. The URL I have on the VMware Web UI is not even close to looking like the one virt-v2v is
trying to download from.

This is the URL if I navigate to the datastore folder containing the VM:
https://vCenterServerName:9443/vsphere-client/#extensionId=vsphere.core.datastore.manage.filesView;context=com.vmware.core.model%3A%3AServerObjectRef~21F3B6D6-26FB-430E-BA4A-BAF302EBA2F8%3ADatastore%3Adatastore-23633~core.

Plus, as I mentioned before. The exact same command on the exact same VM works with the older version of Virt-v2v.

Comment 5 Alessandro 2015-11-12 19:40:07 UTC
Should I upgrade my Libvirt version maybe?

If I run a yum update though I do not get anything higher than libvirt-client-1.2.17-1.el7.x86_64.

Do I need a specific repo?

Comment 6 Richard W.M. Jones 2015-11-13 08:57:35 UTC
Put https:// + the name of your vCenter server + /folder into the
URL field of your web browser.  It will show a list of data stores.
Navigate through these to your VM.  Find the final URL of the VM's
disk image, and add that as a comment to this bug.

Comment 8 Richard W.M. Jones 2015-11-13 16:20:18 UTC
That's a VM called "conversion-test-vm (....)" whereas the one
you selected in the original description was called
"conversion-test-vm2 (...)".

Assuming that was not a mistake, you might want to try adding
one of the following parameters:

  --dcpath 'Eagan Developer Cloud'
  --dcpath 'Eagan%20Developer%20Cloud'
  --dcpath 'Eagan%2520Developer%2520Cloud'

(Or just rename all your VMs and stuff so you don't have spaces
and other non-alphanumeric characters in them, which could be
the path of least resistance).

Comment 9 Alessandro 2015-11-13 16:23:35 UTC
https://vCenterServername/folder/conversion-test-vm2%20%28f97c63bc-e711-4281-bb57-627c86e06b18%29?dcPath=Data%2520Center%2520Name&dsName=virtual%255fccf0444%255ft1%255fsnap07%255f01

The above is the path to the vm2.

What I still do not understand is why all these vms can be converted fine by the older version of virt-v2v and not from the latest.

Comment 10 Richard W.M. Jones 2015-11-13 16:36:06 UTC
(In reply to Alessandro from comment #9)
> https://vCenterServername/folder/conversion-test-vm2%20%28f97c63bc-e711-4281-
> bb57-
> 627c86e06b18%29?dcPath=Data%2520Center%2520Name&dsName=virtual%255fccf0444%25
> 5ft1%255fsnap07%255f01
> 
> The above is the path to the vm2.

Try --dcpath 'Data%2520Center%2520Name'

This may actually be a quoting bug, but it's hard to tell because there
are so many layers of quoting going on here.

> What I still do not understand is why all these vms can be converted fine by
> the older version of virt-v2v and not from the latest.

By "older version" what *precise* version worked before and what
precise version doesn't work now?

Comment 11 Alessandro 2015-11-13 16:45:09 UTC
I have specified the older version in the first comment:

Additional info:
The same command works using the following tools versions:
Libvirt version:
libvirt-client-1.2.17-1.el7.x86_64

Virt-v2v version:
virt-v2v-1.28.1-1.48.el7.x86_64

However, I tried --dcpath 'Data%2520Center%2520Name' and doesn't work:

virt-v2v: error: vcenter: URL not found: 
https://c111aqz.mgmt.tlrg.com/folder/conversion-test-vm2%20%28f97c63bc-e711-4281-bb57-627c86e06b18%29/conversion-test-vm2%20%28f97c63bc-e711-4281-bb57-627c86e06b18%29-flat.vmdk?dcPath=Eagan%252520Developer%252520Cloud&dsName=virtual%5fccf0444%5ft1%5fsnap07%5f01

The '--dcpath' parameter may be useful.  See the explanation in the 
virt-v2v(1) man page OPTIONS section.

If reporting bugs, run virt-v2v with debugging enabled and include the 
complete output:

  virt-v2v -v -x [...]


Tried this instead, --dcpath 'Data%20Center%20Name' and it fixed the issue, now the tool connects fine and can start the conversion:

[   0.0] Opening the source -i libvirt -ic vpx://TLR%5cm0035016.tlrg.com/Eagan%20Developer%20Cloud/F%20Module/C915PYA/c475zfc.int.thomsonreuters.com/?no_verify=1 conversion-test-vm2 (f97c63bc-e711-4281-bb57-627c86e06b18)
[   2.6] Creating an overlay to protect the source from being modified
[   8.2] Opening the overlay
ecc..

I still would like to understand if I can upgrade Libvirt to another more recent version to avoid adding this parameter.

Regards

Comment 12 Richard W.M. Jones 2015-11-13 16:55:45 UTC
It's almost certainly more fallout from the fix for bug 1256823.

There's no good solution here, but the possible solutions are:

 - Try different --dcpath parameters until something works.  I
   suggest removing layers of quoting, so:

      --dcpath 'Data%20Center%20Name'
      --dcpath 'Data Center Name'

 - Rename your data center so it doesn't contain spaces or other
   special characters.

 - Upgrade to the libvirt version which supports <vmware:datacenterpath>
   https://www.redhat.com/archives/libvir-list/2015-September/thread.html#00201
   However you'll also have to switch to upstream virt-v2v since
   the one in RHEL 7 doesn't have the commits to read this info
   from libvirt.

*** This bug has been marked as a duplicate of bug 1256823 ***

Comment 13 Alessandro 2015-11-15 19:12:17 UTC
Richard,
What do you mean with switch to the virt-v2v upstream? I do not really have the steps clear to do so.
 
Thanks

Comment 14 Richard W.M. Jones 2015-11-15 19:37:47 UTC
My suggestion is that you use Fedora 23 as your virt-v2v conversion
server (instead of whatever you are using now - RHEL 7.1 I think).

This version of Fedora includes a version of libvirt (>= 1.2.20) which
has the <vmware:datacenterpath> feature.

You will still need to use libguestfs >= 1.31.16.  That is not
available in Fedora 23, but there are two options for that:

 - (Easier) Add the Rawhide repo to your Fedora 23 guest and
   install the version of virt-v2v from Rawhide.  The following
   commands should be sufficient:

   # dnf upgrade dnf
   # dnf install dnf-plugins-core fedora-repos-rawhide
   # dnf --enablerepo=rawhide install --best virt-v2v

 - (Harder) Compile libguestfs from source and use virt-v2v like this:

   ./run virt-v2v [...]

   If compiling libguestfs from source, read the README file carefully
   before starting.

Comment 15 Alessandro 2015-11-16 16:11:01 UTC
Thank you Richard,
so if I understand well my only option is to rebuild my environment on a Fedora23 OS instead of using CentOS or keep using the current one with the --dcpath option. Correct?

My last question is, will I be able to run the last version of virt-v2v that will support windows 2012 on my CentOS once it will be released?
If I can, I'd prefer not to rebuild everything from scratch unless I must use libvirt 1.20 for the last virt-v2v release.

Thanks for you help as always!