Bug 1837337 - nbdkit: curl[1]: error: server does not support 'range' (byte range) requests with HTTP/2 server in VMware ESXi 7
Summary: nbdkit: curl[1]: error: server does not support 'range' (byte range) requests...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: nbdkit
Version: 8.2
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: 8.3
Assignee: Richard W.M. Jones
QA Contact: liuzi
URL:
Whiteboard: V2V
Depends On:
Blocks: 1837328 1840126 1841038
TreeView+ depends on / blocked
 
Reported: 2020-05-19 10:09 UTC by Richard W.M. Jones
Modified: 2020-07-28 07:14 UTC (History)
12 users (show)

Fixed In Version: nbdkit-1.16.2-4.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1837328
: 1841038 (view as bug list)
Environment:
Last Closed: 2020-07-28 07:13:31 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2020:3172 0 None None None 2020-07-28 07:14:27 UTC

Description Richard W.M. Jones 2020-05-19 10:09:20 UTC
+++ This bug was initially created as a clone of Bug #1837328 +++

Description of problem:
Virt-v2v cannot convert guest from ESXI7.0 server without vddk

Version-Release number of selected component (if applicable):
virt-v2v-1.40.2-22.module+el8.3.0+6423+e4cb6418.x86_64
libguestfs-1.40.2-22.module+el8.3.0+6423+e4cb6418.x86_64
VMware-vix-disklib-6.7.3-14389676.x86_64.tar.gz

How reproducible:
100%

Steps to Reproduce:
See Bug #1837328

nbdkit fails against the HTTP/2 server saying:
nbdkit: curl[1]: error: server does not support 'range' (byte range) requests

Comment 2 Richard W.M. Jones 2020-05-19 17:50:14 UTC
remote: *** Checking commit 56f65713b98b89bc4f0252a050beeba33650794e
remote: *** Resolves:
remote: ***   Unapproved:
remote: ***     rhbz#1837337 (qa_ack?, internal_target_release=8.2.1, devel_ack+, pm_ack?, mirror+, release?, Triaged)
remote: *** No approved issue IDs referenced in log message or changelog for 56f65713b98b89bc4f0252a050beeba33650794e
remote: *** Unapproved issue IDs referenced in log message or changelog for 56f65713b98b89bc4f0252a050beeba33650794e
remote: *** Commit 56f65713b98b89bc4f0252a050beeba33650794e denied
remote: *** Current checkin policy requires:
remote:     (release == + and internal_target_release == 8.2.1)

Comment 5 liuzi 2020-05-28 07:40:09 UTC
Test bug with builds:
nbdkit-1.16.2-4.module+el8.2.1+6710+effcb1df.x86_64
virt-v2v-1.40.2-23.module+el8.2.1+6797+0db00a40.x86_64
libguestfs-1.40.2-23.module+el8.2.1+6797+0db00a40.x86_64

Steps:
1.Convert a guest from ESXI7.0 to rhv without VDDK
# virt-v2v -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1  esx7.0-rhel8.2-x86_64 -o rhv -os 10.66.144.40:/home/nfs_export -ip /home/passwd
[   0.0] Opening the source -i libvirt -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1 esx7.0-rhel8.2-x86_64
[   1.7] Creating an overlay to protect the source from being modified
qemu-img: /var/tmp/v2vovl56bced.qcow2: CURL: Error opening file: Server does not support 'range' (byte ranges).
Could not open backing image to determine size.
virt-v2v: error: qemu-img command failed, see earlier errors

If reporting bugs, run virt-v2v with debugging enabled and include the 
complete output:

  virt-v2v -v -x [...]

Result:virt-v2v cannot convert guest from esxi7.0 without vddk.


Hi,Rjones,
I find even though bug1837337 and Bug1837328 all on_qa,virt-v2v still cannot convert guest from esxi7.0 without vddk.

Comment 6 Richard W.M. Jones 2020-05-28 08:06:52 UTC
> libguestfs-1.40.2-23.module+el8.2.1+6797+0db00a40.x86_64
> qemu-img: /var/tmp/v2vovl56bced.qcow2: CURL: Error opening file: Server does not support 'range' (byte ranges).

I forgot that this version of libguestfs does not use nbdkit for
accessing https servers.  It uses the curl driver inside qemu.
So while this bug is fixed correctly for nbdkit, it will not fix
virt-v2v in RHEL 8.2.1.

If we still want to fix this issue in 8.2.1 we'll need to clone this
bug and reassign it to qemu.  qemu needs the following commits to be
added:

  commit 69032253c33ae1774233c63cedf36d32242a85fc
  block/curl: HTTP header field names are case insensitive

  commit 7788a319399f17476ff1dd43164c869e320820a2
  block/curl: HTTP header fields allow whitespace around values

Let me ask Martin T what he thinks.

Comment 7 Richard W.M. Jones 2020-05-28 09:14:31 UTC
Finally got a simple reproducer for this bug:

nbdkit -fv curl https://centos.mirror.liteserver.nl/8/BaseOS/x86_64/os/images/boot.iso --run 'qemu-img info $nbd'

Comment 8 liuzi 2020-06-09 13:07:29 UTC
Verify bug with builds:
virt-v2v-1.40.2-23.module+el8.2.1+6797+0db00a40.x86_64
libguestfs-1.40.2-23.module+el8.2.1+6797+0db00a40.x86_64
nbdkit-1.16.2-4.module+el8.2.1+6710+effcb1df.x86_64
qemu-kvm-4.2.0-24.module+el8.2.1+6959+9b840e7c.x86_64

Steps:
Scenario 1:Convert a rhel guest from ESXI7.0 to RHV without vddk:
#  virt-v2v -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1  esx7.0-rhel8.2-x86_64 -o rhv -os 10.66.144.40:/home/nfs_export -ip /home/passwd
[   0.0] Opening the source -i libvirt -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1 esx7.0-rhel8.2-x86_64
[   1.6] Creating an overlay to protect the source from being modified
[   2.1] Opening the overlay
[  24.7] Inspecting the overlay
[ 142.7] Checking for sufficient free disk space in the guest
[ 142.7] Estimating space required on target for each disk
[ 142.7] Converting Red Hat Enterprise Linux 8.2 (Ootpa) to run on KVM
virt-v2v: warning: guest tools directory ‘linux/el8’ is missing from 
the virtio-win directory or ISO.

Guest tools are only provided in the RHV Guest Tools ISO, so this can 
happen if you are using the version of virtio-win which contains just the 
virtio drivers.  In this case only virtio drivers can be installed in the 
guest, and installation of Guest Tools will be skipped.
virt-v2v: This guest has virtio drivers installed.
[1395.2] Mapping filesystem data to avoid copying unused and blank areas
[1397.5] Closing the overlay
[1397.8] Assigning disks to buses
[1397.8] Checking if the guest needs BIOS or UEFI to boot
[1397.8] Initializing the target -o rhv -os 10.66.144.40:/home/nfs_export
[1398.4] Copying disk 1/1 to /tmp/v2v.XbPNhl/5792ab14-7746-4340-a27d-845dc12b8db9/images/d2b68c18-08b6-46a8-8cde-d95ff2cdc5d0/baa7869f-7020-40ad-892a-ac3cbaefb1c0 (raw)
    (100.00/100%)
[2379.0] Creating output metadata
[2379.2] Finishing off

Result 1:virt-v2v can convert guest successfully,and guest can pass normal checkpoints.

Scenario 2:convert a windows guest from ESXI7.0 to RHV
#  virt-v2v -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1  esx7.0-win2019-x86_64 -o rhv -os 10.66.144.40:/home/nfs_export -ip /home/passwd
[   0.0] Opening the source -i libvirt -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1 esx7.0-win2019-x86_64
[   1.7] Creating an overlay to protect the source from being modified
[   2.1] Opening the overlay

[  17.0] Inspecting the overlay
[ 135.5] Checking for sufficient free disk space in the guest
[ 135.5] Estimating space required on target for each disk
[ 135.5] Converting Windows Server 2019 Standard to run on KVM
virt-v2v: warning: /usr/share/virt-tools/pnp_wait.exe is missing.  
Firstboot scripts may conflict with PnP.
virt-v2v: warning: QEMU Guest Agent MSI not found on tools ISO/directory. 
You may want to install the guest agent manually after conversion.
virt-v2v: warning: there are no virtio drivers available for this version 
of Windows (10.0 x86_64 Server).  virt-v2v looks for drivers in 
/usr/share/virtio-win

The guest will be configured to use slower emulated devices.
virt-v2v: This guest does not have virtio drivers installed.
[ 149.1] Mapping filesystem data to avoid copying unused and blank areas
[ 150.2] Closing the overlay
[ 150.5] Assigning disks to buses
[ 150.5] Checking if the guest needs BIOS or UEFI to boot
[ 150.5] Initializing the target -o rhv -os 10.66.144.40:/home/nfs_export
[ 150.8] Copying disk 1/1 to /tmp/v2v.QiIxGP/5792ab14-7746-4340-a27d-845dc12b8db9/images/bf1deb79-387c-45ab-91a4-5a2f6f48dc7e/2cce187b-3a19-4398-ae54-474cbe2d3834 (raw)
    (100.00/100%)
[2098.5] Creating output metadata
[2098.7] Finishing off

Result 2:virt-v2v can convert guest successfully,and guest can pass normal checkpoints.

Scenario 3:Use non-admin user to convert a rhel guest from ESXI7.0 to RHV without vddk:
#  virt-v2v -ic  vpx://vsphere.local%5clz.198.169/data/10.73.199.217/?no_verify=1  esx7.0-rhel8.2-x86_64 -o rhv -os 10.66.144.40:/home/nfs_export -ip /home/passwd
[   0.0] Opening the source -i libvirt -ic vpx://vsphere.local%5clz.198.169/data/10.73.199.217/?no_verify=1 esx7.0-rhel8.2-x86_64
[   1.7] Creating an overlay to protect the source from being modified
[   2.2] Opening the overlay
[  25.0] Inspecting the overlay
[ 143.4] Checking for sufficient free disk space in the guest
[ 143.4] Estimating space required on target for each disk
[ 143.4] Converting Red Hat Enterprise Linux 8.2 (Ootpa) to run on KVM
virt-v2v: warning: guest tools directory ‘linux/el8’ is missing from 
the virtio-win directory or ISO.

Guest tools are only provided in the RHV Guest Tools ISO, so this can 
happen if you are using the version of virtio-win which contains just the 
virtio drivers.  In this case only virtio drivers can be installed in the 
guest, and installation of Guest Tools will be skipped.
virt-v2v: This guest has virtio drivers installed.
[1395.9] Mapping filesystem data to avoid copying unused and blank areas
[1398.1] Closing the overlay
[1398.4] Assigning disks to buses
[1398.4] Checking if the guest needs BIOS or UEFI to boot
[1398.4] Initializing the target -o rhv -os 10.66.144.40:/home/nfs_export
[1401.0] Copying disk 1/1 to /tmp/v2v.nW2DaR/5792ab14-7746-4340-a27d-845dc12b8db9/images/e5808d3f-2732-47e9-815c-f2d684a1ce8d/5e4054d3-866b-422a-93f8-b9d7fa4e388a (raw)
    (100.00/100%)
[2491.4] Creating output metadata
[2491.8] Finishing off

Result 3:virt-v2v can convert guest successfully,and guest can pass normal checkpoints.

Result:virt-v2v can convert guest from ESXI7.0 successfully without vddk,so change bug status from ON_QA to VERIFIED.

Comment 10 errata-xmlrpc 2020-07-28 07:13:31 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:3172


Note You need to log in before you can comment on or make changes to this bug.