Description of problem: In one of the test cases in CDI, we found a possible bug in qemu-img curl. We moved from qemu-img curl to nbdkit curl, and the behavior is different. More details in [1]. From Richard's comment [2], it seems that qemu-img is making a wrong range request. You can find the test image invalid-qcow-large-json.img in [3] [1] https://github.com/kubevirt/containerized-data-importer/pull/1610#issuecomment-787832360 [2] https://github.com/kubevirt/containerized-data-importer/pull/1610#issuecomment-787852077 [3] https://github.com/kubevirt/containerized-data-importer/tree/main/tests/images/invalid_qcow_images Version-Release number of selected component (if applicable): How reproducible: Always Steps to Reproduce: 1. Start a http server with the image 2. query the information from the http endpoint: qemu-img info --output=json <endpoint> 3. If you download the image locally, qemu-img info returns the information (as in the case of nbdkit -U - curl --run 'qemu-img info $nbd') Actual results: Returning the error: qemu-img: curl: The requested URL returned error: 416 Requested Range Not Satisfiable qemu-img: Could not open 'http://cdi-file-host.cdi/invalid_qcow_images/invalid-qcow-large-json.img': Failed to read snapshot table: Input/output error Expected results: Same behavior as with using nbdkit curl
I patched qemu's block/curl.c to enable verbose debugging (https://github.com/qemu/qemu/blob/51db2d7cf26d05a961ec0ee0eb773594b32cc4a1/block/curl.c#L38) and it shows that the curl driver is issuing requests beyond the bounds of the remote disk image: > GET /invalid-qcow-large-json.img HTTP/1.1 Host: scratch.home.annexia.org Range: bytes=197136-459279 Accept: */* * Mark bundle as not supporting multiuse * The requested URL returned error: 416 Requested Range Not Satisfiable * Closing connection 0 Notice how the file is 197120 bytes in size, so requesting byte range 197136-459279 (beyond the end of the file) is wrong. I wonder if qemu internally is getting confused about the virtual vs file size?
About nbdkit-curl-plugin, here's the command which runs successfully: $ nbdkit curl http://example.org/invalid-qcow-large-json.img --run 'qemu-img info $uri' If you enable debugging: $ nbdkit -fv -D curl.verbose=1 curl http://example.org/invalid-qcow-large-json.img --run 'qemu-img info $uri' then you can see the NBD_CMD_READ requests that qemu-img makes (grep for " pread "), and they are all valid, ie. within the range of the file. It's peculiar - I don't understand why qemu-img -> block/curl.c should behave differently from qemu-img -> block/nbd.c -> nbdkit -> curl.
Is this an image that is not sector-aligned? qemu's block/nbd.c implementation of NBD_CMD_READ intentionally takes care of fuzzing the tail of an image to 0 when qemu requests a 512-sector at the end of the file beyond the byte-accurate size advertised by NBD, but block/curl.c does not do the same fuzzing. See also this upstream thread, where Kevin proposed making the two the same: https://lists.gnu.org/archive/html/qemu-devel/2021-03/msg06332.html
This message is a reminder that Fedora 33 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora 33 on 2021-11-30. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '33'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 33 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
Fedora 33 changed to end-of-life (EOL) status on 2021-11-30. Fedora 33 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. If you are unable to reopen this bug, please file a new report against the current release. If you experience problems, please add a comment to this bug. Thank you for reporting this bug and we are sorry it could not be fixed.