Created attachment 1851351 [details]
Reproducer script

Description of problem:

When using qemu-nbd --allocation-depth option, and registering "qemu:allocation-depth"
during handshake, and using raw image, NBD_CMD_BLOCK_STATUS returns correct reply in 
the first call, but in the next call for the same byte range is reported as data
(flags=0).

This may break clients (e.g. backup applications) if they issue multiple
NBD_CMD_BLOCK_STATUS commands for the same range. If the client use the second
reply, it may read the byte range and copy it over the newtwork for no reason.
This can cause a severe performance regression for users.

Version-Release number of selected component (if applicable):
qemu-img-6.2.0-2.module+el8.6.0+13738+17338784.x86_64

How reproducible:
Always

Steps to Reproduce:

1. Start qemu-nbd with --allocation-depth option and raw image

$ truncate -s1g empty.raw\

$ qemu-nbd --read-only \
    --persistent \
    --allocation-depth \
    --socket=/tmp/nbd.sock \
    --format=raw \
    empty.raw &

2. Connect to qemu-nbd with nbd client.

I'm using ovirt-imageio nbd client, but this should be reproducible
with libnbd. To install the client use:

# dnf copr enable nsoffer/ovirt-imageio-preview
# dnf install ovirt-imageio-client

$ python3
Python 3.6.8 (default, Oct 15 2021, 10:57:33) 
[GCC 8.5.0 20210514 (Red Hat 8.5.0-3)] on linux
Type "help", "copyright", "credits" or "license" for more information.

Enable debug logs so we can see all nbd commands and replies...

>>> import logging
>>> logging.basicConfig(level=logging.DEBUG)

Connect to qemu-nbd:

>>> from ovirt_imageio._internal import nbd
>>> c = nbd.Client(nbd.UnixAddress("/tmp/nbd.sock"))
DEBUG:nbd:Connecting address='/tmp/nbd.sock' export_name='' dirty=False
DEBUG:nbd:Received server flags: 3
DEBUG:nbd:Sending client flags: 1
DEBUG:nbd:Sending option: 8 data: b''
DEBUG:nbd:Received reply magic=3e889045565a9 option=8 type=1 len=0
DEBUG:nbd:Structured reply enabled
DEBUG:nbd:Sending option: 10 data: bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x0fbase:allocation\x00\x00\x00\x15qemu:allocation-depth')
DEBUG:nbd:Received reply magic=3e889045565a9 option=10 type=4 len=19
DEBUG:nbd:Meta context base:allocation is available id=0
DEBUG:nbd:Received reply magic=3e889045565a9 option=10 type=4 len=25
DEBUG:nbd:Meta context qemu:allocation-depth is available id=1
DEBUG:nbd:Received reply magic=3e889045565a9 option=10 type=1 len=0
DEBUG:nbd:Sending option: 7 data: bytearray(b'\x00\x00\x00\x00\x00\x00')
DEBUG:nbd:Received reply magic=3e889045565a9 option=7 type=3 len=14
DEBUG:nbd:Received block size info minimum=1 preferred=4096 maximum=33554432
DEBUG:nbd:Received reply magic=3e889045565a9 option=7 type=3 len=12
DEBUG:nbd:Received export info size=1073741824 flags=1423
DEBUG:nbd:Received reply magic=3e889045565a9 option=7 type=1 len=0
DEBUG:nbd:Ready for transmission

Get extents (first call):

>>> from pprint import pprint
>>> pprint(c.extents(0, c.export_size))
DEBUG:nbd:Sending NBD_CMD_BLOCK_STATUS handle=0 offset=0 length=1073741824 flags=0
DEBUG:nbd:Extent length=1073741824 flags=3 context=0
DEBUG:nbd:Extent length=1073741824 flags=1 context=1
{'base:allocation': [Extent(length=1073741824, flags=3)],
 'qemu:allocation-depth': [Extent(length=1073741824, flags=0)]}

We got the flags = 3  (NBD_STATE_HOLE | NBD_STATE_ZERO)

Get extents (second call):

>>> pprint(c.extents(0, c.export_size))
DEBUG:nbd:Sending NBD_CMD_BLOCK_STATUS handle=1 offset=0 length=1073741824 flags=0
DEBUG:nbd:Extent length=1073741824 flags=0 context=0
DEBUG:nbd:Extent length=1073741824 flags=1 context=1
{'base:allocation': [Extent(length=1073741824, flags=0)],
 'qemu:allocation-depth': [Extent(length=1073741824, flags=0)]}

We got the flags = 0  (data)


Actual results:
NBD_CMD_BLOCK_STATUS reply is inconsistent, second call does not match
first call.

Expected results:
Getting same reply in all calls.

Additional info:

This does not happen if we disable --allocation-depth:

$ qemu-nbd --read-only \
    --persistent \
    --socket=/tmp/nbd.sock \
    --format=raw \
    empty.raw &

$ python
>>> from ovirt_imageio._internal import nbd
>>> from pprint import pprint
>>> c = nbd.Client(nbd.UnixAddress("/tmp/nbd.sock"))
Failed to activate qemu:allocation-depth
>>> pprint(c.extents(0, c.export_size))
{'base:allocation': [Extent(length=1073741824, flags=3)]}
>>> pprint(c.extents(0, c.export_size))
{'base:allocation': [Extent(length=1073741824, flags=3)]}

Got the expected reply in both calls.


Using git bisect with the attached reproducer script show this commit:

commit 0bc329fbb009f8601cec23bf2bc48ead0c5a5fa2
Author: Hanna Reitz <hreitz>
Date:   Thu Aug 12 10:41:44 2021 +0200

    block: block-status cache for data regions

Upstream discussion:
https://lists.nongnu.org/archive/html/qemu-block/2022-01/msg00292.html