Bug 1172425 - [RFE]virt-v2v failed to convert VMware ESX VM with snapshot
Summary: [RFE]virt-v2v failed to convert VMware ESX VM with snapshot
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libguestfs
Version: 7.1
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: rc
: ---
Assignee: Richard W.M. Jones
QA Contact: Virtualization Bugs
Jiri Herrmann
URL:
Whiteboard: V2V
: 1311415 1373259 (view as bug list)
Depends On:
Blocks: 952703 1236075
TreeView+ depends on / blocked
 
Reported: 2014-12-10 03:37 UTC by zhoujunqin
Modified: 2019-04-28 13:27 UTC (History)
14 users (show)

Fixed In Version: libguestfs-1.36.10-1.el7
Doc Type: Release Note
Doc Text:
*virt-v2v* can convert VMware guests with snapshots The *virt-v2v* utility has been enhanced to convert VMware guest virtual machines that have snapshots. Note that after the conversion, the status of such a guest is set to the top-most snapshot and the other snapshots are removed.
Clone Of:
Environment:
Last Closed: 2018-04-10 09:15:08 UTC


Attachments (Terms of Use)
esx5.1-rhel7.0-x86_64-withsnapshot-debug.log (2.65 KB, text/plain)
2014-12-10 03:37 UTC, zhoujunqin
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:0677 normal SHIPPED_LIVE libguestfs new packages, bug fix and enhancement update 2018-04-10 13:13:49 UTC
Red Hat Knowledge Base (Solution) 2485581 None None None 2016-08-04 12:54:16 UTC

Description zhoujunqin 2014-12-10 03:37:09 UTC
Created attachment 966593 [details]
esx5.1-rhel7.0-x86_64-withsnapshot-debug.log

Description of problem:
virt-v2v failed to convert VMware ESXi VM with snapshot

Version-Release number of selected component (if applicable):
libguestfs-1.28.1-1.14.el7.x86_64
virt-v2v-1.28.1-1.14.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Create a Vmware ESXi guest, then make some changes on guest:
eg:
# mkdir /test
# touch /test/test

2. Make a snapshot for the guest.

3. Remove the new file create:
# rm -rf /test/test
# ls /test/test
ls: cannot access /test/test: No such file or directory

4. Use virt-v2v convert this guest to local kvm.
# virt-v2v -ic vpx://root@10.66.111.25/tzheng-demo/10.66.4.153/?no_verify=1 --password-file password esx5.1-rhel7.0-x86_64
[   0.0] Opening the source -i libvirt -ic vpx://root@10.66.111.25/tzheng-demo/10.66.4.153/?no_verify=1 esx5.1-rhel7.0-x86_64
curl -q --insecure --user '<hidden>' --head --silent --url 'https://10.66.111.25/folder/esx5.1-rhel7.0-x86%5f64/esx5.1-rhel7.0-x86%5f64-000001-flat.vmdk?dcPath=tzheng-demo&dsName=datastore1%20%281%29'
HTTP/1.1 404 Not Found
Date: Fri, 5 Dec 2014 02:26:21 GMT
Set-Cookie: vmware_soap_session="520bccef-bda3-247d-d590-19349158a09f"; Path=/; HttpOnly; Secure;
Connection: close
Content-Type: text; charset=plain
Content-Length: 0

virt-v2v: error: vcenter: URL not found:
https://10.66.111.25/folder/esx5.1-rhel7.0-x86%5f64/esx5.1-rhel7.0-x86%5f64-000001-flat.vmdk?dcPath=tzheng-demo&dsName=datastore1%20%281%29

If reporting bugs, run virt-v2v with debugging enabled and include the
complete output:

  virt-v2v -v -x [...]


Actual results:
As step4 output, failed to convert guest to local kvm.

Expected results:
Can convert guest to local kvm successfully and after guest boot up, there should be no file /test/test existing.

Additional info:
1. Login esxi host and check:
/vmfs/volumes/4f59a359-4cc3fa06-cac0-4437e66df86c/esx5.1-rhel7.0-x86_64 # pwd
/vmfs/volumes/datastore1 (1)/esx5.1-rhel7.0-x86_64

/vmfs/volumes/4f59a359-4cc3fa06-cac0-4437e66df86c/esx5.1-rhel7.0-x86_64 # ls
esx5.1-rhel7.0-x86_64-000001-delta.vmdk  esx5.1-rhel7.0-x86_64.nvram              esx5.1-rhel7.0-x86_64.vmxf               vmware-4.log
esx5.1-rhel7.0-x86_64-000001.vmdk        esx5.1-rhel7.0-x86_64.vmdk               vmware-1.log                             vmware-5.log
esx5.1-rhel7.0-x86_64-Snapshot2.vmsn     esx5.1-rhel7.0-x86_64.vmsd               vmware-2.log                             vmware-6.log
esx5.1-rhel7.0-x86_64-flat.vmdk          esx5.1-rhel7.0-x86_64.vmx                vmware-3.log                             vmware.log

2. I will attach detail debug log.

Comment 4 Richard W.M. Jones 2015-09-23 14:51:05 UTC
I don't realistically think we'll ever implement this, but I'm
moving to 7.3 anyway.

Comment 6 Richard W.M. Jones 2016-03-03 12:17:57 UTC
*** Bug 1311415 has been marked as a duplicate of this bug. ***

Comment 7 Michal Skrivanek 2016-04-01 07:58:00 UTC
Maybe skipping over that snapshot instead of failing would be possible?

Comment 8 Richard W.M. Jones 2016-04-01 12:56:54 UTC
The fix for this is probably simpler than I was thinking.  In fact,
there was code for this already in old virt-v2v:

https://git.fedorahosted.org/cgit/virt-v2v.git/tree/lib/Sys/VirtConvert/Transfer/ESX.pm#n347

So if we're happy with just ignoring snapshots and converting
(I think) the top image only then we can do this for 7.3.  I've
added it back to the to-do list.

Comment 9 Tomáš Golembiovský 2016-04-10 19:32:49 UTC
From what I understand from the VMware docs, dropping the snapshot
(*-NNNNNN.vmdk/*-NNNNNN-delta.vmdk files) would not import the latest
version of the disk. It would rather import the state at the time the
first snapshot was created (is that what you mean by "top image"?). In
the example in the report it would be the state with the file /test/test
still there.

Also, by looking at the original v2v code, I appears to me that this is
what old v2v used to do.

Nevertheles I don't think that silently converting older version of the
image is what a user would expect.

Comment 13 Tomáš Golembiovský 2016-06-09 21:31:24 UTC
(In reply to Tomáš Golembiovský from comment #9)
> From what I understand from the VMware docs, dropping the snapshot
> (*-NNNNNN.vmdk/*-NNNNNN-delta.vmdk files) would not import the latest
> version of the disk. It would rather import the state at the time the
> first snapshot was created (is that what you mean by "top image"?). In
> the example in the report it would be the state with the file /test/test
> still there.
> 
> Also, by looking at the original v2v code, I appears to me that this is
> what old v2v used to do.
> 
> Nevertheles I don't think that silently converting older version of the
> image is what a user would expect.

Just for the record, I have created clone of a machine with snapshot and ended up with a VM that reports the virtual disk is "[my_datastore] vm1/vm1-000001.vmdk" and file "vm-000001-flat.vmdk" is a proper name of the associated flat file. In this case trying to be smart and striiping the "-NNNNNN" suffix from files would produce an error.

Comment 14 Richard W.M. Jones 2016-06-22 20:52:05 UTC
(In reply to Tomáš Golembiovský from comment #13)
> (In reply to Tomáš Golembiovský from comment #9)
> > From what I understand from the VMware docs, dropping the snapshot
> > (*-NNNNNN.vmdk/*-NNNNNN-delta.vmdk files) would not import the latest
> > version of the disk. It would rather import the state at the time the
> > first snapshot was created (is that what you mean by "top image"?). In
> > the example in the report it would be the state with the file /test/test
> > still there.
> > 
> > Also, by looking at the original v2v code, I appears to me that this is
> > what old v2v used to do.
> > 
> > Nevertheles I don't think that silently converting older version of the
> > image is what a user would expect.
> 
> Just for the record, I have created clone of a machine with snapshot and
> ended up with a VM that reports the virtual disk is "[my_datastore]
> vm1/vm1-000001.vmdk" and file "vm-000001-flat.vmdk" is a proper name of the
> associated flat file. In this case trying to be smart and striiping the
> "-NNNNNN" suffix from files would produce an error.

Sure, but the old virt-v2v tried vm-000001-flat.vmdk first, and only
if that failed (with a 404) did it chop off the number and try again.

I'm going to have to test the behaviour of ESXi to really understand
what's going on here.  The documentation is very vague.

Comment 15 Tomáš Golembiovský 2016-06-23 06:49:05 UTC
To get the real filename of the backing file, we can first fetch the .vmdk file reported by libvirt (which is plain text file) and parse it. There will be a line in the following format:

# Extent description
RW 8388608 VMFS "foo-flat.vmdk"

I.e.: <mode> <size> <content_type> <file_name>

Similarily for the (snapshot) delta file:

# Extent description
RW 8388608 VMFSSPARSE "foo-000001-delta.vmdk"

I am however not sure that VMFSSPARSE is proof enough that the filesystem is incomplete. It may be necessary to check also the 'parentCID' field in the same .vmdk file. Or alternatively, to be on the safe side, use a libvirt call to check for snapshot existence first. And only after that look into the .vmdk file for the filename of the flat file.

The minor inconvenience here is that both the plain text file with the description and the real backing file have .vmdk extension. It would be wise to put a limit on the HTTP request when fetching the plain text file to prevent accidentaly downloading gigs of data.

Comment 16 Tomas Jelinek 2016-09-13 06:39:28 UTC
*** Bug 1373259 has been marked as a duplicate of this bug. ***

Comment 17 mxie@redhat.com 2016-11-15 10:05:14 UTC
Hi rjones,

 I see below comment in the https://bugzilla.redhat.com/show_bug.cgi?id=1256687#c3

  Basically virt-v2v doesn't support conversion of snapshots.
  Same applies to VMware conversions.

 Whether this bug could be closed as WONTFIX?

Comment 18 Richard W.M. Jones 2016-11-15 10:08:34 UTC
No it's an RFE so we may fix it in future.

Comment 19 Richard W.M. Jones 2017-02-16 15:24:22 UTC
No clear direction upstream, so moving to RHEL 7.5.

Comment 21 Richard W.M. Jones 2017-10-13 16:28:02 UTC
Posted here:
https://www.redhat.com/archives/libguestfs/2017-October/msg00140.html

Comment 25 mxie@redhat.com 2017-10-25 06:40:44 UTC
I can reproduce the bug with builds:
virt-v2v-1.36.7-1.el7.x86_64
libguestfs-1.36.7-1.el7.x86_64

Try to verify the bug with builds:
virt-v2v-1.36.10-1.el7.x86_64
libguestfs-1.36.10-1.el7.x86_64
libvirt-3.8.0-1.el7.x86_64
qemu-kvm-rhev-2.10.0-3.el7.x86_64
virtio-win-1.9.3-1.el7.noarch
libguestfs-winsupport-7.2-2.el7.x86_64

Steps:
Scenario1:
1.Create a snapshot for win2016 guest on vSphere client,then convert the guest from vmware to rhv by virt-v2v, there is no v2v error during conversion
# virt-v2v -ic vpx://root@10.73.75.182/data/10.73.72.61/?no_verify=1 esx6.0-win2016-x86_64 --password-file /tmp/passwd -o rhv -os 10.73.131.93:/home/nfs_export -of qcow2 -b ovirtmgmt 
[   0.0] Opening the source -i libvirt -ic vpx://root@10.73.75.182/data/10.73.72.61/?no_verify=1 esx6.0-win2016-x86_64
[   1.4] Creating an overlay to protect the source from being modified
[   2.0] Initializing the target -o rhv -os 10.73.131.93:/home/nfs_export
[   2.2] Opening the overlay
[  14.2] Inspecting the overlay
[ 178.2] Checking for sufficient free disk space in the guest
[ 178.2] Estimating space required on target for each disk
[ 178.2] Converting Windows Server 2016 Standard to run on KVM
virt-v2v: warning: /usr/share/virt-tools/pnp_wait.exe is missing.  
Firstboot scripts may conflict with PnP.
virt-v2v: warning: there is no QXL driver for this version of Windows (10.0 
x86_64).  virt-v2v looks for this driver in /usr/share/virtio-win

The guest will be configured to use a basic VGA display driver.
virt-v2v: This guest has virtio drivers installed.
[ 213.1] Mapping filesystem data to avoid copying unused and blank areas
[ 214.1] Closing the overlay
[ 214.2] Checking if the guest needs BIOS or UEFI to boot
[ 214.2] Assigning disks to buses
[ 214.2] Copying disk 1/1 to /tmp/v2v.ZC6sUl/bdf9c90b-e6f0-439c-aa9f-6305fd5fad7e/images/d5d66baf-e208-4a30-b6ee-fd64987772d0/8ff85f06-3cfb-46da-ade8-d88ab0309f37 (qcow2)
    (100.00/100%)
[6135.7] Creating output metadata
[6135.8] Finishing off

2.Check guest after finishing v2v conversion, all checkpoints are passed


Scenario2:
1.Create a snapshot for linux guest on vSphere client,then convert the guest from vmware to rhv by virt-v2v, there is no error during conversion
# virt-v2v -ic vpx://vsphere.local%5cAdministrator@10.73.199.71/data/10.73.196.89/?no_verify=1 esx6.5-rhel6.9-x86_64 -o rhv -os 10.73.131.93:/home/nfs_export --password-file /tmp/passwd -n ovirtmgmt -b ovirtmgmt
[   0.0] Opening the source -i libvirt -ic vpx://vsphere.local%5cAdministrator@10.73.199.71/data/10.73.196.89/?no_verify=1 esx6.5-rhel6.9-x86_64
[   1.4] Creating an overlay to protect the source from being modified
[   2.1] Initializing the target -o rhv -os 10.73.131.93:/home/nfs_export
[   2.6] Opening the overlay
[  21.3] Inspecting the overlay
[ 156.3] Checking for sufficient free disk space in the guest
[ 156.3] Estimating space required on target for each disk
[ 156.3] Converting Red Hat Enterprise Linux Server release 6.9 (Santiago) to run on KVM
virt-v2v: This guest has virtio drivers installed.
[ 942.6] Mapping filesystem data to avoid copying unused and blank areas
[ 944.5] Closing the overlay
[ 944.6] Checking if the guest needs BIOS or UEFI to boot
[ 944.6] Assigning disks to buses
[ 944.6] Copying disk 1/1 to /tmp/v2v.XsAYcS/bdf9c90b-e6f0-439c-aa9f-6305fd5fad7e/images/09660df8-406b-4071-a531-f578e85a3d59/360efa0f-ed8e-4ceb-ae82-6226e2f27e55 (raw)
    (100.00/100%)
[2743.2] Creating output metadata
[2743.4] Finishing off

2.Check guest after finishing v2v conversion, all checkpoints are passed


According to above verification result, the bug has been fixed, so move this bug from ON_QA to VERIFIED

Comment 28 errata-xmlrpc 2018-04-10 09:15:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0677


Note You need to log in before you can comment on or make changes to this bug.