Created attachment 1702415 [details] trace of failing qemu-img convert command Description of problem: Converting image to NBD URL fails with exit code 1 and no error message, but checking the target image the convert was successful. How to reproduce: $ echo "disk data" > disk1.raw $ truncate -s 1g disk1.raw $ truncate -s 1g test.tar Note: I'm skpping the steps to create a the tar file, since they are not need to reproduce the issue. In another shell start qemu-nbd, exporting this image: $ qemu-nbd --socket=/tmp/nbd.sock --persistent --format=raw --offset 1536 test.tar Note: --offset is not required to reproduce this issue. The goal is to write a qcow2 compressed image directly into tar file. The image is exposed as raw: $ qemu-img info nbd+unix:///?socket=/tmp/nbd.sock image: nbd+unix://?socket=/tmp/nbd.sock file format: raw virtual size: 1 GiB (1073741824 bytes) disk size: unavailable Convert the image: $ qemu-img convert -f raw -O qcow2 -c disk1.raw nbd+unix:///?socket=/tmp/nbd.sock; echo $? 1 qemu-img failed without any error message! But the image was converted: $ qemu-img info nbd+unix:///?socket=/tmp/nbd.sock image: nbd+unix://?socket=/tmp/nbd.sock file format: qcow2 virtual size: 1 GiB (1073741824 bytes) disk size: unavailable cluster_size: 65536 Format specific information: compat: 1.1 lazy refcounts: false refcount bits: 16 corrupt: false $ qemu-img check nbd+unix:///?socket=/tmp/nbd.sock No errors were found on the image. 1/16384 = 0.01% allocated, 100.00% fragmented, 100.00% compressed clusters Image end offset: 393216 $ qemu-img compare disk1.raw nbd+unix:///?socket=/tmp/nbd.sock Images are identical. Lets extract the image from the tar file: $ qemu-img convert -f qcow2 -O qcow2 nbd+unix:///?socket=/tmp/nbd.sock extracted.qcow2 $ qemu-img info extracted.qcow2 image: extracted.qcow2 file format: qcow2 virtual size: 1 GiB (1073741824 bytes) disk size: 324 KiB cluster_size: 65536 Format specific information: compat: 1.1 lazy refcounts: false refcount bits: 16 corrupt: false $ qemu-img compare extracted.qcow2 disk1.raw Images are identical. Version-Release number of selected component (if applicable): $ rpm -q qemu-img qemu-img-4.2.0-29.module+el8.2.1+7297+a825794d.x86_64 Same issue exists in: - qemu-img-5.0.0-0.3.rc2.fc31.x86_64 - qemu-img built from git, commit b0ce3f021e0157e9a5ab836cb162c48caac132e1. How reproducible: Always Addtional info: I reproduced this also with: $ qemu-nbd --socket=/tmp/nbd.sock --persistent json:'{"driver": "raw", "offset": 1523, "file": {"driver": "file", "filename": "test.tar"}}' And with nbdkit - the logs show that the conversion was successful. We see the expected writes, flush, and clean disconnect. (I did not try to use offset, it is not required to reproduce). $ nbdkit --unix /tmp/nbd.sock --verbose --foreground file test.tar nbdkit: debug: TLS disabled: could not load TLS certificates nbdkit: debug: registering plugin /usr/local/lib/nbdkit/plugins/nbdkit-file-plugin.so nbdkit: debug: registered plugin /usr/local/lib/nbdkit/plugins/nbdkit-file-plugin.so (name file) nbdkit: debug: file: load nbdkit: debug: file: config key=file, value=test.tar nbdkit: debug: file: config_complete nbdkit: debug: using thread model: parallel nbdkit: debug: file: get_ready nbdkit: debug: bound to unix socket /tmp/nbd.sock nbdkit: debug: file: after_fork nbdkit: debug: accepted connection nbdkit: file[1]: debug: file: preconnect nbdkit: file[1]: debug: newstyle negotiation: flags: global 0x3 nbdkit: file[1]: debug: newstyle negotiation: client flags: 0x3 nbdkit: file[1]: debug: newstyle negotiation: NBD_OPT_STRUCTURED_REPLY: client requested structured replies nbdkit: file[1]: debug: newstyle negotiation: NBD_OPT_SET_META_CONTEXT: client requested export '' nbdkit: file[1]: debug: newstyle negotiation: NBD_OPT_SET_META_CONTEXT: set count: 1 nbdkit: file[1]: debug: newstyle negotiation: NBD_OPT_SET_META_CONTEXT: set base:allocation nbdkit: file[1]: debug: newstyle negotiation: NBD_OPT_SET_META_CONTEXT: replying with base:allocation id 1 nbdkit: file[1]: debug: newstyle negotiation: NBD_OPT_SET_META_CONTEXT: reply complete nbdkit: file[1]: debug: newstyle negotiation: NBD_OPT_GO: client requested export '' nbdkit: file[1]: debug: file: open readonly=0 nbdkit: file[1]: debug: file: open returned handle 0x7fe3e4001d70 nbdkit: file[1]: debug: file: prepare readonly=0 nbdkit: file[1]: debug: file: get_size nbdkit: file[1]: debug: file: can_write nbdkit: file[1]: debug: file: can_zero nbdkit: file[1]: debug: file: can_fast_zero nbdkit: file[1]: debug: file: can_trim nbdkit: file[1]: debug: file: can_fua nbdkit: file[1]: debug: file: can_flush nbdkit: file[1]: debug: file: is_rotational nbdkit: file[1]: debug: file: can_multi_conn nbdkit: file[1]: debug: file: can_cache nbdkit: file[1]: debug: file: can_extents nbdkit: file[1]: debug: newstyle negotiation: flags: export 0x5ed nbdkit: file[1]: debug: newstyle negotiation: NBD_OPT_GO: ignoring NBD_INFO_* request 3 (NBD_INFO_BLOCK_SIZE) nbdkit: file[1]: debug: handshake complete, processing requests with 16 threads nbdkit: debug: starting worker thread file.0 nbdkit: debug: starting worker thread file.1 nbdkit: debug: starting worker thread file.2 nbdkit: file.0: debug: file: zero count=512 offset=0 may_trim=1 fua=0 fast=0 nbdkit: debug: starting worker thread file.3 nbdkit: debug: starting worker thread file.4 nbdkit: debug: starting worker thread file.5 nbdkit: debug: starting worker thread file.8 nbdkit: debug: starting worker thread file.6 nbdkit: debug: starting worker thread file.7 nbdkit: file.1: debug: file: flush nbdkit: debug: starting worker thread file.9 nbdkit: debug: starting worker thread file.10 nbdkit: debug: starting worker thread file.11 nbdkit: debug: starting worker thread file.13 nbdkit: debug: starting worker thread file.12 nbdkit: debug: starting worker thread file.14 nbdkit: debug: starting worker thread file.15 nbdkit: file.8: debug: client sent NBD_CMD_DISC, closing connection nbdkit: file.8: debug: exiting worker thread file.8 nbdkit: file.0: debug: exiting worker thread file.0 nbdkit: file.3: debug: exiting worker thread file.3 nbdkit: file.5: debug: exiting worker thread file.5 nbdkit: file.6: debug: exiting worker thread file.6 nbdkit: file.9: debug: exiting worker thread file.9 nbdkit: file.13: debug: exiting worker thread file.13 nbdkit: file.15: debug: exiting worker thread file.15 nbdkit: file.2: debug: exiting worker thread file.2 nbdkit: file.10: debug: exiting worker thread file.10 nbdkit: debug: accepted connection nbdkit: file.4: debug: exiting worker thread file.4 nbdkit: file.12: debug: exiting worker thread file.12 nbdkit: file.14: debug: exiting worker thread file.14 nbdkit: file.1: debug: exiting worker thread file.1 nbdkit: file.7: debug: exiting worker thread file.7 nbdkit: file.11: debug: exiting worker thread file.11 nbdkit: file[2]: debug: file: preconnect nbdkit: file[2]: debug: newstyle negotiation: flags: global 0x3 nbdkit: file[2]: debug: newstyle negotiation: client flags: 0x3 nbdkit: file[2]: debug: newstyle negotiation: NBD_OPT_STRUCTURED_REPLY: client requested structured replies nbdkit: file[1]: debug: file: finalize nbdkit: file[2]: debug: newstyle negotiation: NBD_OPT_SET_META_CONTEXT: client requested export '' nbdkit: file[2]: debug: newstyle negotiation: NBD_OPT_SET_META_CONTEXT: set count: 1 nbdkit: file[2]: debug: newstyle negotiation: NBD_OPT_SET_META_CONTEXT: set base:allocation nbdkit: file[2]: debug: newstyle negotiation: NBD_OPT_SET_META_CONTEXT: replying with base:allocation id 1 nbdkit: file[1]: debug: file: close nbdkit: file[2]: debug: newstyle negotiation: NBD_OPT_SET_META_CONTEXT: reply complete nbdkit: file[2]: debug: newstyle negotiation: NBD_OPT_GO: client requested export '' nbdkit: file[2]: debug: file: open readonly=0 nbdkit: file[2]: debug: file: open returned handle 0x7fe3ac001d70 nbdkit: file[2]: debug: file: prepare readonly=0 nbdkit: file[2]: debug: file: get_size nbdkit: file[2]: debug: file: can_write nbdkit: file[2]: debug: file: can_zero nbdkit: file[2]: debug: file: can_fast_zero nbdkit: file[2]: debug: file: can_trim nbdkit: file[2]: debug: file: can_fua nbdkit: file[2]: debug: file: can_flush nbdkit: file[2]: debug: file: is_rotational nbdkit: file[2]: debug: file: can_multi_conn nbdkit: file[2]: debug: file: can_cache nbdkit: file[2]: debug: file: can_extents nbdkit: file[2]: debug: newstyle negotiation: flags: export 0x5ed nbdkit: file[2]: debug: newstyle negotiation: NBD_OPT_GO: ignoring NBD_INFO_* request 3 (NBD_INFO_BLOCK_SIZE) nbdkit: file[2]: debug: handshake complete, processing requests with 16 threads nbdkit: debug: starting worker thread file.0 nbdkit: debug: starting worker thread file.1 nbdkit: debug: starting worker thread file.2 nbdkit: debug: starting worker thread file.3 nbdkit: debug: starting worker thread file.4 nbdkit: debug: starting worker thread file.5 nbdkit: debug: starting worker thread file.6 nbdkit: file.0: debug: file: pwrite count=65536 offset=0 fua=0 nbdkit: debug: starting worker thread file.7 nbdkit: debug: starting worker thread file.8 nbdkit: debug: starting worker thread file.9 nbdkit: debug: starting worker thread file.10 nbdkit: debug: starting worker thread file.11 nbdkit: debug: starting worker thread file.12 nbdkit: debug: starting worker thread file.13 nbdkit: debug: starting worker thread file.14 nbdkit: debug: starting worker thread file.15 nbdkit: file.6: debug: file: pwrite count=131072 offset=65536 fua=0 nbdkit: file.2: debug: file: pread count=104 offset=0 nbdkit: file.3: debug: file: pread count=65536 offset=65536 nbdkit: file.4: debug: file: pread count=8 offset=104 nbdkit: file.5: debug: file: pread count=65536 offset=131072 nbdkit: file.1: debug: file: pwrite count=65536 offset=0 fua=0 nbdkit: file.7: debug: file: pwrite count=65536 offset=131072 fua=0 nbdkit: file.8: debug: file: flush nbdkit: file.9: debug: file: pwrite count=16 offset=196608 fua=0 nbdkit: file.9: debug: file: flush nbdkit: file.0: debug: file: pwrite count=12 offset=36 fua=0 nbdkit: file.0: debug: file: flush nbdkit: file.11: debug: file: pwrite count=8 offset=24 fua=0 nbdkit: file.11: debug: file: flush nbdkit: file.13: debug: file: flush nbdkit: file.14: debug: file: flush nbdkit: file.15: debug: file: flush nbdkit: file.6: debug: file: flush nbdkit: file.2: debug: file: flush nbdkit: file.3: debug: file: pread count=104 offset=0 nbdkit: file.4: debug: file: pread count=16 offset=196608 nbdkit: file.5: debug: file: pread count=65536 offset=65536 nbdkit: file.1: debug: file: pread count=8 offset=104 nbdkit: file.1: debug: file: pread count=8 offset=400 nbdkit: file.7: debug: file: flush nbdkit: file.8: debug: file: flush nbdkit: file.9: debug: file: flush nbdkit: file.10: debug: file: flush nbdkit: file.10: debug: client sent NBD_CMD_DISC, closing connection nbdkit: file.10: debug: exiting worker thread file.10 nbdkit: file.12: debug: exiting worker thread file.12 nbdkit: file.13: debug: exiting worker thread file.13 nbdkit: file.14: debug: exiting worker thread file.14 nbdkit: file.2: debug: exiting worker thread file.2 nbdkit: file.4: debug: exiting worker thread file.4 nbdkit: file.6: debug: exiting worker thread file.6 nbdkit: file.11: debug: exiting worker thread file.11 nbdkit: file.0: debug: exiting worker thread file.0 nbdkit: file.3: debug: exiting worker thread file.3 nbdkit: file.15: debug: exiting worker thread file.15 nbdkit: file.5: debug: exiting worker thread file.5 nbdkit: file.1: debug: exiting worker thread file.1 nbdkit: file.7: debug: exiting worker thread file.7 nbdkit: file.8: debug: exiting worker thread file.8 nbdkit: debug: accepted connection nbdkit: file.9: debug: exiting worker thread file.9 nbdkit: file[3]: debug: file: preconnect nbdkit: file[3]: debug: newstyle negotiation: flags: global 0x3 nbdkit: file[2]: debug: file: finalize nbdkit: file[3]: debug: newstyle negotiation: client flags: 0x3 nbdkit: file[2]: debug: file: close nbdkit: file[3]: debug: newstyle negotiation: NBD_OPT_STRUCTURED_REPLY: client requested structured replies nbdkit: file[3]: debug: newstyle negotiation: NBD_OPT_SET_META_CONTEXT: client requested export '' nbdkit: file[3]: debug: newstyle negotiation: NBD_OPT_SET_META_CONTEXT: set count: 1 nbdkit: file[3]: debug: newstyle negotiation: NBD_OPT_SET_META_CONTEXT: set base:allocation nbdkit: file[3]: debug: newstyle negotiation: NBD_OPT_SET_META_CONTEXT: replying with base:allocation id 1 nbdkit: file[3]: debug: newstyle negotiation: NBD_OPT_SET_META_CONTEXT: reply complete nbdkit: file[3]: debug: newstyle negotiation: NBD_OPT_GO: client requested export '' nbdkit: file[3]: debug: file: open readonly=0 nbdkit: file[3]: debug: file: open returned handle 0x7fe3a0001d70 nbdkit: file[3]: debug: file: prepare readonly=0 nbdkit: file[3]: debug: file: get_size nbdkit: file[3]: debug: file: can_write nbdkit: file[3]: debug: file: can_zero nbdkit: file[3]: debug: file: can_fast_zero nbdkit: file[3]: debug: file: can_trim nbdkit: file[3]: debug: file: can_fua nbdkit: file[3]: debug: file: can_flush nbdkit: file[3]: debug: file: is_rotational nbdkit: file[3]: debug: file: can_multi_conn nbdkit: file[3]: debug: file: can_cache nbdkit: file[3]: debug: file: can_extents nbdkit: file[3]: debug: newstyle negotiation: flags: export 0x5ed nbdkit: file[3]: debug: newstyle negotiation: NBD_OPT_GO: ignoring NBD_INFO_* request 3 (NBD_INFO_BLOCK_SIZE) nbdkit: file[3]: debug: handshake complete, processing requests with 16 threads nbdkit: debug: starting worker thread file.0 nbdkit: file.0: debug: file: pread count=104 offset=0 nbdkit: debug: starting worker thread file.2 nbdkit: debug: starting worker thread file.3 nbdkit: file.0: debug: file: pread count=16 offset=196608 nbdkit: debug: starting worker thread file.4 nbdkit: debug: starting worker thread file.5 nbdkit: debug: starting worker thread file.6 nbdkit: debug: starting worker thread file.7 nbdkit: debug: starting worker thread file.8 nbdkit: file.2: debug: file: pread count=65536 offset=65536 nbdkit: debug: starting worker thread file.9 nbdkit: debug: starting worker thread file.11 nbdkit: debug: starting worker thread file.12 nbdkit: file.3: debug: file: pread count=8 offset=104 nbdkit: debug: starting worker thread file.10 nbdkit: debug: starting worker thread file.13 nbdkit: debug: starting worker thread file.14 nbdkit: file.0: debug: file: pread count=8 offset=400 nbdkit: debug: starting worker thread file.15 nbdkit: debug: starting worker thread file.1 nbdkit: file.4: debug: file: pread count=65536 offset=131072 nbdkit: file.5: debug: file: pwrite count=65536 offset=131072 fua=0 nbdkit: file.6: debug: file: flush nbdkit: file.7: debug: file: pwrite count=65536 offset=262144 fua=0 nbdkit: file.8: debug: file: flush nbdkit: file.9: debug: file: pwrite count=8 offset=196608 fua=0 nbdkit: file.2: debug: file: flush nbdkit: file.11: debug: file: pwrite count=96 offset=327680 fua=0 nbdkit: file.11: debug: file: pwrite count=65536 offset=131072 fua=0 nbdkit: file.3: debug: file: flush nbdkit: file.10: debug: file: pwrite count=65536 offset=262144 fua=0 nbdkit: file.13: debug: file: flush nbdkit: file.14: debug: file: flush nbdkit: file.0: debug: file: flush nbdkit: file.15: debug: file: flush nbdkit: file.1: debug: client sent NBD_CMD_DISC, closing connection nbdkit: file.1: debug: exiting worker thread file.1 nbdkit: file.4: debug: exiting worker thread file.4 nbdkit: file.5: debug: exiting worker thread file.5 nbdkit: file.7: debug: exiting worker thread file.7 nbdkit: file.6: debug: exiting worker thread file.6 nbdkit: file.11: debug: exiting worker thread file.11 nbdkit: file.9: debug: exiting worker thread file.9 nbdkit: file.2: debug: exiting worker thread file.2 nbdkit: file.12: debug: exiting worker thread file.12 nbdkit: file.8: debug: exiting worker thread file.8 nbdkit: file.3: debug: exiting worker thread file.3 nbdkit: file.10: debug: exiting worker thread file.10 nbdkit: file.13: debug: exiting worker thread file.13 nbdkit: file.14: debug: exiting worker thread file.14 nbdkit: file.0: debug: exiting worker thread file.0 nbdkit: file.15: debug: exiting worker thread file.15 nbdkit: file[3]: debug: file: finalize nbdkit: file[3]: debug: file: close We want to used this for creating OVA files with compressed qcow2 images. We have a workaround, expose the offset in the tar file as a loop device but this requires root. I tied to run strace: $ strace -f -tt -T -o convert.trace qemu-img convert -f raw -O qcow2 -c disk1.raw nbd+unix:///?socket=/tmp/nbd.sock; echo $? I don't see any failure in the trace, see attachment 1 [details].
Note: the correct attachment is attachment 1702415 [details].
Disabling compression works: $ qemu-img convert -f raw -O qcow2 disk1.raw nbd+unix:///?socket=/tmp/nbd.sock; echo $? 0 $ qemu-img info nbd+unix:///?socket=/tmp/nbd.sock image: nbd+unix://?socket=/tmp/nbd.sock file format: qcow2 virtual size: 1 GiB (1073741824 bytes) disk size: unavailable cluster_size: 65536 Format specific information: compat: 1.1 lazy refcounts: false refcount bits: 16 corrupt: false $ qemu-img check nbd+unix:///?socket=/tmp/nbd.sock No errors were found on the image. 1/16384 = 0.01% allocated, 0.00% fragmented, 0.00% compressed clusters Image end offset: 393216 So this seem to be an issue in qcow2 (or qemu-img) not in nbd.
Debugging qemu-img from git show that the error come from: # qemu-img.c 2119 if (s->compressed && !s->ret) { 2120 /* signal EOF to align */ 2121 ret = blk_pwrite_compressed(s->target, 0, NULL, 0); 2122 if (ret < 0) { 2123 return ret; 2124 } 2125 } (gdb) bt #0 bdrv_co_truncate (child=0x555555866670, offset=1073740288, exact=false, prealloc=PREALLOC_MODE_OFF, flags=0, errp=0x0) at /home/nsoffer/src/qemu/block/io.c:3397 #1 0x00005555555cb57c in qcow2_co_pwritev_compressed_part (bs=0x5555558b70d0, offset=0, bytes=0, qiov=0x7fffffffcfc0, qiov_offset=0) at /home/nsoffer/src/qemu/block/qcow2.c:4522 #2 0x0000555555615bfe in bdrv_driver_pwritev_compressed (bs=0x5555558b70d0, offset=0, bytes=0, qiov=0x7fffffffcfc0, qiov_offset=0) at /home/nsoffer/src/qemu/block/io.c:1277 #3 0x0000555555617ba4 in bdrv_aligned_pwritev (child=0x5555558c6bc0, req=0x7fffe50d7e10, offset=0, bytes=0, align=1, qiov=0x7fffffffcfc0, qiov_offset=0, flags=32) at /home/nsoffer/src/qemu/block/io.c:2015 #4 0x00005555556183e8 in bdrv_co_pwritev_part (child=0x5555558c6bc0, offset=0, bytes=0, qiov=0x7fffffffcfc0, qiov_offset=0, flags=BDRV_REQ_WRITE_COMPRESSED) at /home/nsoffer/src/qemu/block/io.c:2186 #5 0x00005555555fddcb in blk_do_pwritev_part (blk=0x5555558442a0, offset=0, bytes=0, qiov=0x7fffffffcfc0, qiov_offset=0, flags=BDRV_REQ_WRITE_COMPRESSED) at /home/nsoffer/src/qemu/block/block-backend.c:1260 #6 0x00005555555fdf40 in blk_write_entry (opaque=0x7fffffffcfa0) at /home/nsoffer/src/qemu/block/block-backend.c:1310 #7 0x00005555556d35e6 in coroutine_trampoline (i0=1436206016, i1=21845) at /home/nsoffer/src/qemu/util/coroutine-ucontext.c:173 #8 0x00007ffff6b84d20 in ?? () from /lib64/libc.so.6 #9 0x00007fffffffc760 in ?? () #10 0x0000000000000000 in ?? () (gdb) frame 0 #0 bdrv_co_truncate (child=0x555555866670, offset=1073740288, exact=false, prealloc=PREALLOC_MODE_OFF, flags=0, errp=0x0) at /home/nsoffer/src/qemu/block/io.c:3397 3397 ret = -ENOTSUP; (gdb) list 3392 ret = drv->bdrv_co_truncate(bs, offset, exact, prealloc, flags, errp); 3393 } else if (bs->file && drv->is_filter) { 3394 ret = bdrv_co_truncate(bs->file, offset, exact, prealloc, flags, errp); 3395 } else { 3396 error_setg(errp, "Image format driver does not support resize"); 3397 ret = -ENOTSUP; So we have 2 issues: - Failure to truncate the image, as the last step in the convert when using nbd backend. not sure why we try to truncate. The call to truncate looks wrong: 4513 if (bytes == 0) { 4514 /* 4515 * align end of file to a sector boundary to ease reading with 4516 * sector based I/Os 4517 */ 4518 int64_t len = bdrv_getlength(bs->file->bs); 4519 if (len < 0) { 4520 return len; 4521 } 4522 return bdrv_co_truncate(bs->file, len, false, PREALLOC_MODE_OFF, 0, 4523 NULL); 4524 } The comment suggest that we are aligning the end of the file to sector boundary, but we align it to the current length of the file, there is no alignment. We can see that we called bdrv_co_truncate() with len=1073740288, which is the size reported by qemu-nbd: 1024**3 - 1536 = 1073740288. - The error message "Image format driver does not support resize" is not written to stderr. This is really bad, making debugging very hard. 4522 return bdrv_co_truncate(bs->file, len, false, PREALLOC_MODE_OFF, 0, 4523 NULL); Here we call bdrv_co_truncate with NULL errp, so we hide possible error set in bdrv_co_truncate().
I posted a fix to nbd driver here: https://lists.nongnu.org/archive/html/qemu-block/2020-07/msg01543.html
Reproduced with qemu-kvm-5.0.0-2.module+el8.3.0+7379+0505d6ca.x86_64, disable compression, it didn't hit the issue, enable compression it hit the issue. Version: kernel-4.18.0-227.el8.x86_64 qemu-kvm-5.0.0-2.module+el8.3.0+7379+0505d6ca.x86_64 Test steps: 1. echo "disk data" > disk1.raw 2. truncate -s 1g disk1.raw 3. truncate -s 1g test.tar 4. qemu-nbd --socket=/tmp/nbd.sock --persistent --format=raw --offset 1536 test.tar With compression 5. qemu-img convert -f raw -O qcow2 -c disk1.raw nbd+unix:///?socket=/tmp/nbd.sock; echo $? Without compression 6. qemu-img convert -f raw -O qcow2 disk1.raw nbd+unix:///?socket=/tmp/nbd.sock; echo $? Actual results: After step 5, it returns 1, execute Step 1-4 and 6, it returns 1. Expected result 1. returns 0 Additional info: After Step 5 1. # qemu-img info nbd+unix:///?socket=/tmp/nbd.sock image: nbd+unix://?socket=/tmp/nbd.sock file format: qcow2 virtual size: 1 GiB (1073741824 bytes) disk size: unavailable cluster_size: 65536 Format specific information: compat: 1.1 compression type: zlib lazy refcounts: false refcount bits: 16 corrupt: false 2. # qemu-img check nbd+unix:///?socket=/tmp/nbd.sock No errors were found on the image. Image end offset: 262144 After Step 6, [root@hp-dl388pg8-01 bug_test]# qemu-img info nbd+unix:///?socket=/tmp/nbd.sock image: nbd+unix://?socket=/tmp/nbd.sock file format: qcow2 virtual size: 1 GiB (1073741824 bytes) disk size: unavailable cluster_size: 65536 Format specific information: compat: 1.1 compression type: zlib lazy refcounts: false refcount bits: 16 corrupt: false [root@hp-dl388pg8-01 bug_test]# qemu-img check nbd+unix:///?socket=/tmp/nbd.sock No errors were found on the image. 1/16384 = 0.01% allocated, 0.00% fragmented, 0.00% compressed clusters Image end offset: 393216 NeedInfo, does this bug need to be copied to 8.3.0 av?
I see Eric has posted PULL requests for qemu-5.1: https://lists.nongnu.org/archive/html/qemu-block/2020-07/msg01706.html https://lists.nongnu.org/archive/html/qemu-block/2020-07/msg01709.html Once accepted, we can move this bz straight to POST with the upstream commit id in Devel Whiteboard
Test with qemu-kvm-5.1.0-2.module+el8.3.0+7652+b30e6901, this issue is fixed. Test version: kernel-4.18.0-232.el8.x86_64 qemu-kvm-5.1.0-2.module+el8.3.0+7652+b30e6901.x86_64 Steps: 1. prepare stg0.raw on nbd server. 2. create a tar file. # tar -cvf test.tar image_convert.raw 3. export nbd img; # qemu-nbd --socket=/tmp/nbd.sock --persistent --format=raw --offset 1536 test.tar 4. convert with compression: # qemu-img convert -f raw -O qcow2 -c /home/kvm_autotest_root/images/stg0.raw nbd+unix:///?socket=/tmp/nbd.sock; echo $? 0 5. convert without compression: qemu-img convert -f raw -O qcow2 -c /home/kvm_autotest_root/images/stg0.raw nbd+unix:///?socket=/tmp/nbd.sock; echo $? 0 Actual result: 1. After Step4 & 5, command return 0. 2. Check img info without error: # qemu-img info nbd+unix:///?socket=/tmp/nbd.sock image: nbd+unix://?socket=/tmp/nbd.sock file format: qcow2 virtual size: 20 GiB (21478375424 bytes) disk size: unavailable cluster_size: 65536 Format specific information: compat: 1.1 compression type: zlib lazy refcounts: false refcount bits: 16 corrupt: false 3. check img without error: # qemu-img check nbd+unix:///?socket=/tmp/nbd.sock No errors were found on the image. 86939/327734 = 26.53% allocated, 80.51% fragmented, 79.23% compressed clusters Image end offset: 3180789760 Expected result: Same as the actual result.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (virt:8.3 bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:5137