Description of problem: Can't convert guest from VMware via vmx+ssh by virt-v2v due to openssh protocol problem Version-Release number of selected component (if applicable): openssh-8.0p1-2.el8.x86_64 How reproducible: 100% Steps to Reproduce: 1.Convert a windows guest from VMware to KVM managed by libvirt via vmx+ssh using virt-v2v but the conversion is failed # virt-v2v -i vmx -it ssh ssh://root.75.219/vmfs/volumes/5aefd41e-1d448cf8-0b1f-001018d0c8e0/esx6.7-win2019-x86_64-efi/esx6.7-win2019-x86_64-efi.vmx -of raw -o local -os /home [ 0.0] Opening the source -i vmx ssh://root.75.219/vmfs/volumes/5aefd41e-1d448cf8-0b1f-001018d0c8e0/esx6.7-win2019-x86_64-efi/esx6.7-win2019-x86_64-efi.vmx protocol error: filename does not match request virt-v2v: error: could not copy the VMX file from the remote server, see earlier error messages If reporting bugs, run virt-v2v with debugging enabled and include the complete output: virt-v2v -v -x [...] 2.Downgrade openssh from openssh-8.0p1-2.el8.x86_64 to openssh-7.8p1-4.el8.x86_64 #yum downgrade http://download.eng.bos.redhat.com/brewroot/vol/rhel-8/packages/openssh/7.8p1/4.el8/x86_64/openssh-7.8p1-4.el8.x86_64.rpm http://download.eng.bos.redhat.com/brewroot/vol/rhel-8/packages/openssh/7.8p1/4.el8/x86_64/openssh-server-7.8p1-4.el8.x86_64.rpm http://download.eng.bos.redhat.com/brewroot/vol/rhel-8/packages/openssh/7.8p1/4.el8/x86_64/openssh-clients-7.8p1-4.el8.x86_64.rpm http://download.eng.bos.redhat.com/brewroot/vol/rhel-8/packages/openssh/7.8p1/4.el8/x86_64/openssh-askpass-7.8p1-4.el8.x86_64.rpm -y # rpm -q openssh openssh-7.8p1-4.el8.x86_64 3.Convert a windows guest from VMware to KVM managed by libvirt via vmx+ssh using virt-v2v again, the conversion can be finished without error # virt-v2v -i vmx -it ssh ssh://root.75.219/vmfs/volumes/5aefd41e-1d448cf8-0b1f-001018d0c8e0/esx6.7-win2019-x86_64-efi/esx6.7-win2019-x86_64-efi.vmx -of raw -o local -os /home [ 0.0] Opening the source -i vmx ssh://root.75.219/vmfs/volumes/5aefd41e-1d448cf8-0b1f-001018d0c8e0/esx6.7-win2019-x86_64-efi/esx6.7-win2019-x86_64-efi.vmx [ 0.6] Creating an overlay to protect the source from being modified [ 0.7] Initializing the target -o local -os /home [ 0.7] Opening the overlay [ 5.5] Inspecting the overlay [ 13.0] Checking for sufficient free disk space in the guest [ 13.0] Estimating space required on target for each disk [ 13.0] Converting Windows Server 2019 Standard to run on KVM virt-v2v: warning: /usr/share/virt-tools/pnp_wait.exe is missing. Firstboot scripts may conflict with PnP. virt-v2v: warning: there is no QXL driver for this version of Windows (10.0 x86_64). virt-v2v looks for this driver in /usr/share/virtio-win/virtio-win.iso The guest will be configured to use a basic VGA display driver. virt-v2v: This guest has virtio drivers installed. [ 19.9] Mapping filesystem data to avoid copying unused and blank areas virt-v2v: warning: fstrim on guest filesystem /dev/sda2 failed. Usually you can ignore this message. To find out more read "Trimming" in virt-v2v(1). Original message: fstrim: fstrim: /sysroot/: the discard operation is not supported [ 20.4] Closing the overlay [ 20.5] Checking if the guest needs BIOS or UEFI to boot virt-v2v: This guest requires UEFI on the target to boot. [ 20.5] Assigning disks to buses [ 20.5] Copying disk 1/1 to /home/esx6.7-win2019-x86_64-efi-sda (raw) (100.00/100%) [ 448.6] Creating output metadata [ 448.6] Finishing off Actual results: Expected results: Can convert guest from VMware via vmx+ssh by virt-v2v with latest openssh Additional info:
Please, provide the debug logs. This sounds like consequence of fix for CVE-2019-6111. Try to pass the -T switch to the scp, if that is using scp.
See related bug #1722229 which broke oracle for possible workarounds.
Here are my comments from IRC: 10:50 < rjones> mxie: it's likely to be a quoting problem, so you could try copying a remote file with a space in the name 10:51 < rjones> and see if quoting it with "file' 'name" works or whatever 10:51 < rjones> or backslashes or quotes 10:52 < rjones> mxie: see also: 10:52 < rjones> https://github.com/libguestfs/libguestfs/blob/6d251e3828ff94deac7589b5892df174430e01f9/v2v/input_vmx.ml#L82 10:53 < rjones> and also https://stackoverflow.com/questions/19858176/how-to-escape-spaces-in-path-during-scp-copy-in-linux 10:53 < rjones> notice all the backslashes in the error message
As already mentioned in the previous comments, there are two possible workarounds: * Use sftp for sensible file transfers. * Use -T switch to avoid these checks if you are using trusted servers
sftp (command line tool) isn't going to work for us because it's interactive. There's a batch mode (-b option) which may be worth looking into. Using sftp from libssh is also possible, but a rather huge change. I need to make sure that VMware's ssh actually supports sftp because it could be disabled. We can easily add the -T option. The server is a VMware server so I guess we trust it. It's at least always owned by the same people running virt-v2v.
Richard, do you want me to change this bug to your component or can I close it as duplicate of #1722229?
I don't really agree with breaking scp for everyone, but here we are.
Patch posted: https://www.redhat.com/archives/libguestfs/2019-July/msg00268.html
Thank you. Technically, the upstream fix was trying to accommodate as much as possible use cases, but because of the flexibility of this "API" and many possible various bash expansions or quotes ignorance that could be used in the scp, it is really not possible to catch everything.
Upstream in: https://github.com/libguestfs/libguestfs/commit/7692c31494f7b1d37e380eed9eb99c5952940dbf (Sorry, I kept the "lazy", I quite like it)
Verify the bug with builds: virt-v2v-1.40.2-12.module+el8.1.0+3908+8a8c5ed4.x86_64 libguestfs-1.40.2-12.module+el8.1.0+3908+8a8c5ed4.x86_64 libvirt-5.6.0-1.module+el8.1.0+3890+4d3d259c.x86_64 qemu-kvm-4.0.0-6.module+el8.1.0+3736+a2aefea3.x86_64 nbdkit-1.12.5-1.module+el8.1.0+3868+35f94834.x86_64 libssh-0.9.0-4.el8.x86_64 openssh-8.0p1-3.el8.x86_64 kernel-4.18.0-128.el8.x86_64 Steps: Scenario1: 1.1 Convert guests from VMware to RHV via vmx+ssh using virt-v2v # virt-v2v -i vmx -it ssh ssh://root.75.219//vmfs/volumes/5aefd41e-1d448cf8-0b1f-001018d0c8e0//esx6.7-rhel7.6-uefi-raid//esx6.7-rhel7.6-uefi-raid.vmx -o rhv-upload -oo rhv-cafile=/home/ca.pem -oo rhv-direct -oc https://ibm-x3250m5-03.rhts.eng.pek2.redhat.com/ovirt-engine/api -op /tmp/rhvpasswd --password-file /tmp/passwd -os nfs_data -b ovirtmgmt -oo rhv-cluster=nfs # virt-v2v -i vmx -it ssh ssh://root.75.219//vmfs/volumes/5aefd41e-1d448cf8-0b1f-001018d0c8e0//esx6.7-rhel7.6-uefi-raid//esx6.7-rhel7.6-uefi-raid.vmx -o rhv-upload -oo rhv-cafile=/home/ca.pem -oo rhv-direct -oc https://ibm-x3250m5-03.rhts.eng.pek2.redhat.com/ovirt-engine/api -op /tmp/rhvpasswd --password-file /tmp/passwd -os nfs_data -b ovirtmgmt -oo rhv-cluster=nfs [ 0.5] Opening the source -i vmx ssh://root.75.219//vmfs/volumes/5aefd41e-1d448cf8-0b1f-001018d0c8e0//esx6.7-rhel7.6-uefi-raid//esx6.7-rhel7.6-uefi-raid.vmx [ 1.5] Creating an overlay to protect the source from being modified [ 2.6] Opening the overlay [ 23.3] Inspecting the overlay [ 50.5] Checking for sufficient free disk space in the guest [ 50.5] Estimating space required on target for each disk [ 50.5] Converting Red Hat Enterprise Linux Server 7.6 (Maipo) to run on KVM virt-v2v: warning: guest tools directory ‘linux/el7’ is missing from the virtio-win directory or ISO. Guest tools are only provided in the RHV Guest Tools ISO, so this can happen if you are using the version of virtio-win which contains just the virtio drivers. In this case only virtio drivers can be installed in the guest, and installation of Guest Tools will be skipped. virt-v2v: This guest has virtio drivers installed. [ 240.8] Mapping filesystem data to avoid copying unused and blank areas virt-v2v: warning: fstrim on guest filesystem /dev/md127 failed. Usually you can ignore this message. To find out more read "Trimming" in virt-v2v(1). Original message: fstrim: fstrim: /sysroot/: the discard operation is not supported [ 242.7] Closing the overlay [ 242.8] Assigning disks to buses [ 242.8] Checking if the guest needs BIOS or UEFI to boot virt-v2v: This guest requires UEFI on the target to boot. [ 242.8] Initializing the target -o rhv-upload -oc https://ibm-x3250m5-03.rhts.eng.pek2.redhat.com/ovirt-engine/api -op /tmp/rhvpasswd -os nfs_data [ 246.4] Copying disk 1/3 to qemu URI json:{ "file.driver": "nbd", "file.path": "/var/tmp/rhvupload.LTgO8Q/nbdkit0.sock", "file.export": "/" } (raw) (100.00/100%) [ 744.1] Copying disk 2/3 to qemu URI json:{ "file.driver": "nbd", "file.path": "/var/tmp/rhvupload.LTgO8Q/nbdkit1.sock", "file.export": "/" } (raw) (100.00/100%) [ 868.2] Copying disk 3/3 to qemu URI json:{ "file.driver": "nbd", "file.path": "/var/tmp/rhvupload.LTgO8Q/nbdkit2.sock", "file.export": "/" } (raw) (100.00/100%) [1055.7] Creating output metadata [1074.4] Finishing off 1.2 Power on guest and checkpoint of guest are passed 1.3 Check if "scp -T " is used in v2v vmx+ssh conversion # cat vmx-ssh-fast.log |grep "scp -T" LANG=C scp -T |& grep "unknown option" scp -T 'root'@'10.73.75.219':''\''//vmfs/volumes/5aefd41e-1d448cf8-0b1f-001018d0c8e0//esx6.7-rhel7.6-uefi-raid//esx6.7-rhel7.6-uefi-raid.vmx'\''' '/var/tmp/vmx.GxVglt/source.vmx' Scenario2: 2.1 Convert guests from VMware to RHV via vmx using virt-v2v # virt-v2v -i vmx esx6.5-win2019-x86_64/esx6.5-win2019-x86_64.vmx -o rhv-upload -oo rhv-cafile=/home/ca.pem -oo rhv-direct -oc https://ibm-x3250m5-03.rhts.eng.pek2.redhat.com/ovirt-engine/api -op /tmp/rhvpasswd --password-file /tmp/passwd -os nfs_data -b ovirtmgmt -oo rhv-cluster=nfs -of raw [ 0.5] Opening the source -i vmx esx6.5-win2019-x86_64/esx6.5-win2019-x86_64.vmx [ 0.5] Creating an overlay to protect the source from being modified [ 0.6] Opening the overlay [ 4.8] Inspecting the overlay [ 7.8] Checking for sufficient free disk space in the guest [ 7.8] Estimating space required on target for each disk [ 7.8] Converting Windows Server 2019 Standard to run on KVM virt-v2v: warning: /usr/share/virt-tools/pnp_wait.exe is missing. Firstboot scripts may conflict with PnP. virt-v2v: warning: there is no QXL driver for this version of Windows (10.0 x86_64). virt-v2v looks for this driver in /usr/share/virtio-win/virtio-win.iso The guest will be configured to use a basic VGA display driver. virt-v2v: This guest has virtio drivers installed. [ 15.2] Mapping filesystem data to avoid copying unused and blank areas [ 16.1] Closing the overlay [ 16.2] Assigning disks to buses [ 16.2] Checking if the guest needs BIOS or UEFI to boot [ 16.2] Initializing the target -o rhv-upload -oc https://ibm-x3250m5-03.rhts.eng.pek2.redhat.com/ovirt-engine/api -op /tmp/rhvpasswd -os nfs_data [ 17.5] Copying disk 1/1 to qemu URI json:{ "file.driver": "nbd", "file.path": "/var/tmp/rhvupload.NwNXbH/nbdkit0.sock", "file.export": "/" } (raw) (100.00/100%) [ 796.1] Creating output metadata [ 819.8] Finishing off 2.2 Power on guest and checkpoint of guest are passed 2.3 Check if "scp -T " is used in v2v vmx conversion # cat vmx.log |grep "scp -T" nothing Scenario3: 3.1 Convert a guest from Xen to rhv by virt-v2v # virt-v2v -ic xen+ssh://root.3.21 xen-hvm-sles12sp1-x86_64 -of raw -o rhv-upload -oo rhv-cafile=/home/ca.pem -oo rhv-direct -oc https://ibm-x3250m5-03.rhts.eng.pek2.redhat.com/ovirt-engine/api -op /tmp/rhvpasswd --password-file /tmp/passwd -os nfs_data -b ovirtmgmt -oo rhv-cluster=nfs [ 0.5] Opening the source -i libvirt -ic xen+ssh://root.3.21 xen-hvm-sles12sp1-x86_64 [ 1.2] Creating an overlay to protect the source from being modified [ 1.7] Opening the overlay [ 6.7] Inspecting the overlay [ 24.7] Checking for sufficient free disk space in the guest [ 24.7] Estimating space required on target for each disk [ 24.7] Converting SUSE Linux Enterprise Server 12 SP1 to run on KVM virt-v2v: warning: guest tools directory ‘linux/lp151’ is missing from the virtio-win directory or ISO. Guest tools are only provided in the RHV Guest Tools ISO, so this can happen if you are using the version of virtio-win which contains just the virtio drivers. In this case only virtio drivers can be installed in the guest, and installation of Guest Tools will be skipped. virt-v2v: This guest has virtio drivers installed. [ 69.5] Mapping filesystem data to avoid copying unused and blank areas [ 70.3] Closing the overlay [ 70.3] Assigning disks to buses [ 70.3] Checking if the guest needs BIOS or UEFI to boot [ 70.3] Initializing the target -o rhv-upload -oc https://ibm-x3250m5-03.rhts.eng.pek2.redhat.com/ovirt-engine/api -op /tmp/rhvpasswd -os nfs_data [ 71.7] Copying disk 1/1 to qemu URI json:{ "file.driver": "nbd", "file.path": "/var/tmp/rhvupload.MAEmTd/nbdkit0.sock", "file.export": "/" } (raw) (100.00/100%) [ 509.1] Creating output metadata [ 532.7] Finishing off 3.2 Power on guest and checkpoint of guest are passed 3.3 Check if "scp -T " is used in v2v xen conversion # cat xen-fast.log |grep "scp -T" nothing Result: Virt-v2v only use ‘scp -T’ to convert guest via vmx+ssh, so move the bug from ON_QA to VERIFIED
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:3723