Bug 1753121
| Summary: | [v2v][RHV][Scale] v2v Migration to RHV failed on timed out waiting for transfer to finalize [rhel-7.7.z] | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | RAD team bot copy to z-stream <autobot-eus-copy> |
| Component: | libguestfs | Assignee: | Richard W.M. Jones <rjones> |
| Status: | CLOSED ERRATA | QA Contact: | Virtualization Bugs <virt-bugs> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 7.6 | CC: | dagur, derez, dmetzger, fdupont, istein, jomurphy, jsuchane, juzhou, michal.skrivanek, mtessun, mxie, mzhan, nsoffer, ptoscano, pvauter, rjones, tnisan, tzheng, xiaodwan, ytale, zili |
| Target Milestone: | rc | Keywords: | Regression, TestBlocker, ZStream |
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | V2V | ||
| Fixed In Version: | libguestfs-1.40.2-5.el7_7.2 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | 1680361 | Environment: | |
| Last Closed: | 2019-10-15 17:48:09 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 1680361 | ||
| Bug Blocks: | |||
|
Description
RAD team bot copy to z-stream
2019-09-18 07:11:33 UTC
Based on the original bug 1680361, comment #38: In RHV conversion host, run: $ yum downgrade libguestfs libguestfs-tools-c virt-v2v python-libguestfs, brings interim version: 1.40.2-5.el7.1.bz1680361.v3.1.x86_64 (based on Derez fix). Using this repo: http://brew-task-repos.usersys.redhat.com/repos/scratch/rjones/libguestfs/1.40.2/5.el7.1.bz1680361.v3.1/ Error: 'RuntimeError: timed out waiting for transfer to finalize\n' no longer appeared. I managed to run v2v migration, VMware->RHV ISCSI->FC, VDDK, for a single VM, with 20GB disk successfully. This RHEL-7.7 bug can be moved to VERIFIED. Verify the bug with builds: virt-v2v-1.40.2-5.el7_7.2.x86_64 libguestfs-1.40.2-5.el7_7.2.x86_64 libvirt-4.5.0-23.el7_7.1.x86_64 qemu-kvm-rhev-2.12.0-33.el7_7.4.x86_64 nbdkit-1.8.0-1.el7.x86_64 VMware-vix-disklib-6.7.3-14389676.x86_64.tar.gz RHV:4.3.6.5-0.1.el7 vdsm-4.30.30-1.el7ev.x86_64 Steps: Scenario1: 1.1 Convert a rhel7 guest which has disk size more than 100GB from VMware host to rhv's FC domain on standalone v2v conversion server by virt-v2v # virt-v2v -ic esx://root.75.219/?no_verify=1 rhv-4.3-150G -it vddk -io vddk-libdir=/home/vmware-vix-disklib-distrib -io vddk-thumbprint=89:89:AE:6D:DE:B2:89:01:B0:CF:FC:D3:C8:A8:1A:5E:14:E4:B4:54 -o rhv-upload -oo rhv-cafile=/root/ca.pem -oo rhv-direct -oc https://hp-dl360eg8-03.lab.eng.pek2.redhat.com/ovirt-engine/api -op /tmp/rhvpasswd -os fc_data -b ovirtmgmt -oo rhv-cluster=FC --password-file /tmp/esxpw -of raw -oa preallocated Exception AttributeError: "'module' object has no attribute 'dump_plugin'" in <module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored [ 0.5] Opening the source -i libvirt -ic esx://root.75.219/?no_verify=1 rhv-4.3-150G -it vddk -io vddk-libdir=/home/vmware-vix-disklib-distrib -io vddk-thumbprint=89:89:AE:6D:DE:B2:89:01:B0:CF:FC:D3:C8:A8:1A:5E:14:E4:B4:54 [ 1.9] Creating an overlay to protect the source from being modified nbdkit: error: VixDiskLib: VixDiskLibIsLegacyConnParams: the instance of VixDiskLibConnectParams is NOT allocated by VixDiskLib_AllocateConnectParams. The new features in 6.7 or later are not supported. [ 3.0] Opening the overlay nbdkit: error: VixDiskLib: VixDiskLibIsLegacyConnParams: the instance of VixDiskLibConnectParams is NOT allocated by VixDiskLib_AllocateConnectParams. The new features in 6.7 or later are not supported. [ 25.9] Inspecting the overlay [ 60.8] Checking for sufficient free disk space in the guest [ 60.8] Estimating space required on target for each disk [ 60.8] Converting Red Hat Enterprise Linux Server 7.6 (Maipo) to run on KVM virt-v2v: This guest has virtio drivers installed. [ 224.8] Mapping filesystem data to avoid copying unused and blank areas [ 225.2] Closing the overlay [ 225.5] Assigning disks to buses [ 225.5] Checking if the guest needs BIOS or UEFI to boot [ 225.5] Initializing the target -o rhv-upload -oa preallocated -oc https://hp-dl360eg8-03.lab.eng.pek2.redhat.com/ovirt-engine/api -op /tmp/rhvpasswd -os fc_data [ 227.0] Copying disk 1/1 to qemu URI json:{ "file.driver": "nbd", "file.path": "/var/tmp/rhvupload.MYycIT/nbdkit0.sock", "file.export": "/" } (raw) nbdkit: error: VixDiskLib: VixDiskLibIsLegacyConnParams: the instance of VixDiskLibConnectParams is NOT allocated by VixDiskLib_AllocateConnectParams. The new features in 6.7 or later are not supported. (100.00/100%) [4883.7] Creating output metadata [4894.0] Finishing off 1.2 Power on guest and checkpoints of guest are passed 1.3 Convert a rhel7 guest which has disk size more than 100GB from VMware vCenter to rhv's iscsi data domain on standalone v2v conversion server by virt-v2v virt-v2v -ic vpx://root.73.141/data/10.73.75.219/?no_verify=1 -it vddk -io vddk-libdir=/home/vmware-vix-disklib-distrib -io vddk-thumbprint=1F:97:34:5F:B6:C2:BA:66:46:CB:1A:71:76:7D:6B:50:1E:03:00:EA rhv-4.3-150G -o rhv-upload -oo rhv-cafile=/root/ca.pem -oo rhv-direct -oc https://hp-dl360eg8-03.lab.eng.pek2.redhat.com/ovirt-engine/api -op /tmp/rhvpasswd -os iscsi_data -b ovirtmgmt -oo rhv-cluster=ISCSI --password-file /tmp/passwd -of raw -oa preallocated Exception AttributeError: "'module' object has no attribute 'dump_plugin'" in <module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored [ 0.5] Opening the source -i libvirt -ic vpx://root.73.141/data/10.73.75.219/?no_verify=1 rhv-4.3-150G -it vddk -io vddk-libdir=/home/vmware-vix-disklib-distrib -io vddk-thumbprint=1F:97:34:5F:B6:C2:BA:66:46:CB:1A:71:76:7D:6B:50:1E:03:00:EA [ 2.5] Creating an overlay to protect the source from being modified nbdkit: error: VixDiskLib: VixDiskLibIsLegacyConnParams: the instance of VixDiskLibConnectParams is NOT allocated by VixDiskLib_AllocateConnectParams. The new features in 6.7 or later are not supported. [ 3.9] Opening the overlay nbdkit: error: VixDiskLib: VixDiskLibIsLegacyConnParams: the instance of VixDiskLibConnectParams is NOT allocated by VixDiskLib_AllocateConnectParams. The new features in 6.7 or later are not supported. [ 8.8] Inspecting the overlay [ 44.8] Checking for sufficient free disk space in the guest [ 44.8] Estimating space required on target for each disk [ 44.8] Converting Red Hat Enterprise Linux Server 7.6 (Maipo) to run on KVM virt-v2v: This guest has virtio drivers installed. [ 220.2] Mapping filesystem data to avoid copying unused and blank areas [ 220.6] Closing the overlay [ 220.9] Assigning disks to buses [ 220.9] Checking if the guest needs BIOS or UEFI to boot [ 220.9] Initializing the target -o rhv-upload -oa preallocated -oc https://hp-dl360eg8-03.lab.eng.pek2.redhat.com/ovirt-engine/api -op /tmp/rhvpasswd -os iscsi_data [ 222.4] Copying disk 1/1 to qemu URI json:{ "file.driver": "nbd", "file.path": "/var/tmp/rhvupload.OBpy6z/nbdkit0.sock", "file.export": "/" } (raw) nbdkit: error: VixDiskLib: VixDiskLibIsLegacyConnParams: the instance of VixDiskLibConnectParams is NOT allocated by VixDiskLib_AllocateConnectParams. The new features in 6.7 or later are not supported. (100.00/100%) [5383.7] Creating output metadata [5392.0] Finishing off Scenario2: 2.1 Import the guest which has disk size more than 100GB from VMware on RHV GUI Click import option on Virtual Machine interface-> input VMware host info to load the guest-> select the guest to import 2.2 The importing is finished without error,power on guest and checkpoints of guest are passed Sceanrio3: 3.1 Convert a rhel7 guest which has disk size more than 100GB from VMware to rhv on rhv-node by virt-v2v(rhv-node and data domain belong to different datacenter) # virt-v2v -ic esx://root.75.219/?no_verify=1 rhv-4.3-150G -it vddk -io vddk-libdir=/home/vmware-vix-disklib-distrib -io vddk-thumbprint=89:89:AE:6D:DE:B2:89:01:B0:CF:FC:D3:C8:A8:1A:5E:14:E4:B4:54 -o rhv-upload -oo rhv-cafile=/home/ca.pem -oo rhv-direct -oc https://ibm-x3250m5-03.rhts.eng.pek2.redhat.com/ovirt-engine/api -op /tmp/rhvpasswd -os rhvh_data -b ovirtmgmt -oo rhv-cluster=rhvh --password-file /tmp/esxpw -of raw Exception AttributeError: "'module' object has no attribute 'dump_plugin'" in <module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored [ 0.3] Opening the source -i libvirt -ic esx://root.75.219/?no_verify=1 rhv-4.3-150G -it vddk -io vddk-libdir=/home/vmware-vix-disklib-distrib -io vddk-thumbprint=89:89:AE:6D:DE:B2:89:01:B0:CF:FC:D3:C8:A8:1A:5E:14:E4:B4:54 [ 1.8] Creating an overlay to protect the source from being modified nbdkit: error: VixDiskLib: VixDiskLibIsLegacyConnParams: the instance of VixDiskLibConnectParams is NOT allocated by VixDiskLib_AllocateConnectParams. The new features in 6.7 or later are not supported. [ 2.8] Opening the overlay libvirt needs authentication to connect to libvirt URI qemu:///system (see also: http://libvirt.org/auth.html http://libvirt.org/uri.html) Please enter your authentication name: test Please enter your password: nbdkit: error: VixDiskLib: VixDiskLibIsLegacyConnParams: the instance of VixDiskLibConnectParams is NOT allocated by VixDiskLib_AllocateConnectParams. The new features in 6.7 or later are not supported. [ 11.2] Inspecting the overlay [ 40.0] Checking for sufficient free disk space in the guest [ 40.0] Estimating space required on target for each disk [ 40.0] Converting Red Hat Enterprise Linux Server 7.6 (Maipo) to run on KVM virt-v2v: This guest has virtio drivers installed. [ 186.0] Mapping filesystem data to avoid copying unused and blank areas [ 186.3] Closing the overlay [ 187.1] Assigning disks to buses [ 187.1] Checking if the guest needs BIOS or UEFI to boot [ 187.1] Initializing the target -o rhv-upload -oc https://ibm-x3250m5-03.rhts.eng.pek2.redhat.com/ovirt-engine/api -op /tmp/rhvpasswd -os rhvh_data [ 188.4] Copying disk 1/1 to qemu URI json:{ "file.driver": "nbd", "file.path": "/var/tmp/rhvupload.aHfrRq/nbdkit0.sock", "file.export": "/" } (raw) nbdkit: error: VixDiskLib: VixDiskLibIsLegacyConnParams: the instance of VixDiskLibConnectParams is NOT allocated by VixDiskLib_AllocateConnectParams. The new features in 6.7 or later are not supported. (100.00/100%) [11289.8] Creating output metadata [11301.0] Finishing off 3.2 Power on guest and checkpoints of guest are passed In reply to comment #5: I tried already several times, and also today, to run 20 VMs of 20GB on the exact same RHV/hosts, that has virt-v2v-1.40.2-5.el7_7.1.x86_64. Bug did not reproduce. Based on that I cannot verify the bug. In case the bug will occur, though the fix, i'll update this bug. Correction to comment #7: the v2v MVware->RHV run was for 20 VMs with __100GB__ disk, 66% usage. Because the bug happened with a low frequency and is hard to reproduce now (both Ilanit and me can't reproduce),besides,the bug doesn't happen with fixed build according to comment8, move the bug from ON_QA to VERIFIED status. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:3071 |