From Kevin's in https://bugzilla.redhat.com/show_bug.cgi?id=1523614#c7

(In reply to Benny Zlotnik from comment #6)
> Kevin, is this a known 0.10 behaviour/issue?

Not previously, but now it's known. :-)

The thing is that qemu-img convert detects that the space after the end of the backing file reads as zeros. With compat=1.1 images, it can just use the efficient zero cluster support, so you don't really see any increased file size. With compat=0.10 it doesn't have this feature and writes explicit zeros. This happens only with -o backing_file=... because this is the case where the overlay needs to "overwrite" any data from the backing file, so generally you can't just leave things unallocated if they read as zeros.

Of course, qemu-img could be clever enough to figure out that after the end of the backing file, even in this case zeros can be represented by simply leaving a cluster unallocated. The same is true if a cluster is unallocated throughout the whole backing chain. That's just not implemented today.

Sent an upstream series: http://lists.nongnu.org/archive/html/qemu-devel/2018-05/msg00109.html

Fix included in qemu-kvm-rhev-2.12.0-6.el7

Reproduced the issue like below

Tested packages:
qemu-kvm-rhev-2.10.0-21.el7_5.1
kernel-3.10.0-862.el7

Steps:
1.Create base file
# qemu-img create -f qcow2 -o compat=0.10 base.qcow2 512M
Formatting 'base.qcow2', fmt=qcow2 size=536870912 compat=0.10 cluster_size=65536 lazy_refcounts=off refcount_bits=16
2. Create overlay
# qemu-img create -f qcow2 -o compat=0.10 -b base.qcow2 source.qcow2 1G
Formatting 'source.qcow2', fmt=qcow2 size=1073741824 compat=0.10 backing_file=base.qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16
3. Check info
# qemu-img info source.qcow2 
image: source.qcow2
file format: qcow2
virtual size: 1.0G (1073741824 bytes)
disk size: 196K
cluster_size: 65536
backing file: base.qcow2
Format specific information:
    compat: 0.10
    refcount bits: 16
# qemu-img info base.qcow2 
image: base.qcow2
file format: qcow2
virtual size: 512M (536870912 bytes)
disk size: 196K
cluster_size: 65536
Format specific information:
    compat: 0.10
    refcount bits: 16
4. Convert overlay
# qemu-img convert -p -f qcow2 -O qcow2 -o compat=0.10,backing_file=base.qcow2 source.qcow2 target.qcow2
    (100.00/100%)
5. Check info of the target file
# qemu-img info target.qcow2 
image: target.qcow2
file format: qcow2
virtual size: 1.0G (1073741824 bytes)
disk size: 512M    ----------> Reproduced.
cluster_size: 65536
backing file: base.qcow2
Format specific information:
    compat: 0.10
    refcount bits: 16



Verified this as below, according the result, set the bug as verified.

Verified packages:
qemu-kvm-rhev-2.12.0-7.el7
kernel-3.10.0-918.el7

Steps:
1. Create base and overlay files
# qemu-img create -f qcow2 -o compat=0.10 base.qcow2 1G
Formatting 'base.qcow2', fmt=qcow2 size=1073741824 compat=0.10 cluster_size=65536 lazy_refcounts=off refcount_bits=1
# qemu-img create -f qcow2 -o compat=0.10 -b base.qcow2 source.qcow2 2G
Formatting 'source.qcow2', fmt=qcow2 size=2147483648 compat=0.10 backing_file=base.qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16
2. Check info
# qemu-img info base.qcow2 
image: base.qcow2
file format: qcow2
virtual size: 1.0G (1073741824 bytes)
disk size: 196K
cluster_size: 65536
Format specific information:
    compat: 0.10
    refcount bits: 16
# qemu-img info source.qcow2 
image: source.qcow2
file format: qcow2
virtual size: 2.0G (2147483648 bytes)
disk size: 196K
cluster_size: 65536
backing file: base.qcow2
Format specific information:
    compat: 0.10
    refcount bits: 16
3. Convert the overlay file
# qemu-img convert -p -f qcow2 -O qcow2 -o compat=0.10,backing_file=base.qcow2 source.qcow2 target.qcow2
    (100.00/100%)
4. Check disk size of the target file
# qemu-img info target.qcow2 
image: target.qcow2
file format: qcow2
virtual size: 2.0G (2147483648 bytes)
disk size: 196K ---------------------> Expected
cluster_size: 65536
backing file: base.qcow2
Format specific information:
    compat: 0.10
    refcount bits: 16



Run case 122 for I/O test:

Steps:
1. Get source code from SRPM and set test environment
# rpm -ivhf qemu-kvm-rhev-2.12.0-7.el7.src.rpm
# rpmbuild -bp /root/rpmbuild/SPECS/qemu-kvm.spec --nodeps
# cd /root/rpmbuild/BUILD/qemu-2.12.0/
# ./configure
# export QEMU_PROG=/usr/libexec/qemu-kvm
# export QEMU_IMG_PROG=/usr/bin/qemu-img
# export QEMU_IO_PROG=/usr/bin/qemu-io
# export QEMU_NBD_PROG=/usr/bin/qemu-nbd
# cd tests/qemu-iotests
2. Run 122 case
# ./check -qcow2 122
QEMU          -- "/usr/libexec/qemu-kvm" -nodefaults -machine accel=qtest
QEMU_IMG      -- "/usr/bin/qemu-img" 
QEMU_IO       -- "/usr/bin/qemu-io"  --cache writeback -f qcow2
QEMU_NBD      -- "/usr/bin/qemu-nbd" 
IMGFMT        -- qcow2 (compat=1.1)
IMGPROTO      -- file
PLATFORM      -- Linux/x86_64 hp-dl388g8-17 3.10.0-918.el7.x86_64
TEST_DIR      -- /root/rpmbuild/BUILD/qemu-2.12.0/tests/qemu-iotests/scratch
SOCKET_SCM_HELPER -- 

122        
Passed all 1 tests

Hi Max,

Based on my test result, when convert a QCOWv2 image(sn.qcow2) to a new QCOWv2 image(convert.qcow2) and set the backing file as new image(base.qcow2, QCOW version of base.qcow2 doesn't affect the test result), qemu-img doesn't write zeros to convert.qcow2 when the size past the end of the backing file(base.qcow2). However qemu-img reads zeros from backing file(base.qcow2) and writes them to the area. This happens when specifying the convert.qcow2 as QCOWv2.
IMO, I think qemu-img should leave cluster unallocated for all the space, not just past the end of the backing file.

Test Secnario:
1. Create two qcow2 images and specify the QCOW version as v2
# qemu-img create -f qcow2 -o compat=0.10 base.qcow2 10G
# qemu-img create -f qcow2 -o compat=0.10 sn.qcow2 20G

2. Convert the sn.qcow2 to a new QCOWv2 image and set the backing file as base.qcow2.
# qemu-img convert -p -f qcow2 -O qcow2 -o compat=0.10,backing_file=base.qcow2 sn.qcow2 convert.qcow2

3. Get the image information 
# qemu-img info --backing-chain convert.qcow2

4. Dump the metadata of the images
# qemu-img map --output=json base.qcow2
# qemu-img map --output=json sn.qcow2
# qemu-img map --output=json convert.qcow2


Test results with qemu-kvm-rhev-2.10.0-21.el7:
Step 3:
# qemu-img info --backing-chain convert.qcow2
image: convert.qcow2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 27G
cluster_size: 65536
backing file: base.qcow2
Format specific information:
    compat: 0.10
    refcount bits: 16

image: base.qcow2
file format: qcow2
virtual size: 10G (10737418240 bytes)
disk size: 196K
cluster_size: 65536
Format specific information:
    compat: 0.10
    refcount bits: 16
# ll -lsh convert.qcow2 
28G -rw-r--r-- 1 root root 21G Jul 12 04:45 convert.qcow2

Step 4:
# qemu-img map --output=json base.qcow2
[{ "start": 0, "length": 10737418240, "depth": 0, "zero": true, "data": false}]
# qemu-img map --output=json sn.qcow2 
[{ "start": 0, "length": 21474836480, "depth": 0, "zero": true, "data": false}]
# qemu-img map --output=json convert.qcow2 
[{ "start": 0, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 327680},
{ "start": 536870912, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 537264128},
{ "start": 1073741824, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 1074200576},
{ "start": 1610612736, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 1611137024},
{ "start": 2147483648, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 2148139008},
{ "start": 2684354560, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 2685075456},
{ "start": 3221225472, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 3222011904},
{ "start": 3758096384, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 3758948352},
{ "start": 4294967296, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 4295950336},
{ "start": 4831838208, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 4832886784},
{ "start": 5368709120, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 5369823232},
{ "start": 5905580032, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 5906759680},
{ "start": 6442450944, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 6443761664},
{ "start": 6979321856, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 6980698112},
{ "start": 7516192768, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 7517634560},
{ "start": 8053063680, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 8054571008},
{ "start": 8589934592, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 8591572992},
{ "start": 9126805504, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 9128509440},
{ "start": 9663676416, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 9665445888},
{ "start": 10200547328, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 10202382336},
{ "start": 10737418240, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 10739384320},
{ "start": 11274289152, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 11276320768},
{ "start": 11811160064, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 11813257216},
{ "start": 12348030976, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 12350193664},
{ "start": 12884901888, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 12887195648},
{ "start": 13421772800, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 13424132096},
{ "start": 13958643712, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 13961068544},
{ "start": 14495514624, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 14498004992},
{ "start": 15032385536, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 15035006976},
{ "start": 15569256448, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 15571943424},
{ "start": 16106127360, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 16108879872},
{ "start": 16642998272, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 16645816320},
{ "start": 17179869184, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 17182818304},
{ "start": 17716740096, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 17719754752},
{ "start": 18253611008, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 18256691200},
{ "start": 18790481920, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 18793627648},
{ "start": 19327352832, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 19330629632},
{ "start": 19864223744, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 19867566080},
{ "start": 20401094656, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 20404502528},
{ "start": 20937965568, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 20941438976}]

Test results with qemu-kvm-rhev-2.12.0-7.el7:
Step 3:
# qemu-img info --backing-chain convert.qcow2
image: convert.qcow2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 10G
cluster_size: 65536
backing file: base.qcow2
Format specific information:
    compat: 0.10
    refcount bits: 16

image: base.qcow2
file format: qcow2
virtual size: 10G (10737418240 bytes)
disk size: 196K
cluster_size: 65536
Format specific information:
    compat: 0.10
    refcount bits: 16

Step 4:
# qemu-img map --output=json base.qcow2 
[{ "start": 0, "length": 10737418240, "depth": 0, "zero": true, "data": false}]
# qemu-img map --output=json sn.qcow2 
[{ "start": 0, "length": 21474836480, "depth": 0, "zero": true, "data": false}]
# qemu-img map --output=json convert.qcow2 
[{ "start": 0, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 327680},
{ "start": 536870912, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 537264128},
{ "start": 1073741824, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 1074200576},
{ "start": 1610612736, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 1611137024},
{ "start": 2147483648, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 2148139008},
{ "start": 2684354560, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 2685075456},
{ "start": 3221225472, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 3222011904},
{ "start": 3758096384, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 3758948352},
{ "start": 4294967296, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 4295950336},
{ "start": 4831838208, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 4832886784},
{ "start": 5368709120, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 5369823232},
{ "start": 5905580032, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 5906759680},
{ "start": 6442450944, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 6443761664},
{ "start": 6979321856, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 6980698112},
{ "start": 7516192768, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 7517634560},
{ "start": 8053063680, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 8054571008},
{ "start": 8589934592, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 8591572992},
{ "start": 9126805504, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 9128509440},
{ "start": 9663676416, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 9665445888},
{ "start": 10200547328, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 10202382336},
{ "start": 10737418240, "length": 10737418240, "depth": 0, "zero": true, "data": false}]

While that might be possible, it would mean some more difficult changes than what was necessary for this BZ here.  I don’t think that is worth it considering v2 is basically a legacy format anyway.

Also, I don’t think it makes much sense to add a backing file during qemu-img convert.  qemu-img’s man page says this:

> You can use the backing_file option to force the output image to be created as
> a copy on write image of the specified base image; the backing_file should have
> the same content as the input's base image, however the path, image format, etc
> may differ.

It says “should”, so you can do something else if you know what you’re doing, but then you may get funny results.  In this case, if the input does not have a base image (backing file), you just get a fully allocated output.

Compare this with when your input already had a backing file:

$ qemu-img create -f qcow2 -o compat=0.10 base-s.qcow2 64M
$ qemu-img create -f qcow2 -o compat=0.10 base-t.qcow2 64M
$ qemu-img create -f qcow2 -o compat=0.10 -b base-s.qcow2 source.qcow2 64M
$ qemu-img convert -O qcow2 -o compat=0.10 -B base-t.qcow2 source.qcow2 target.qcow2
$ qemu-img info target.qcow2                                                     
image: target.qcow2
file format: qcow2
virtual size: 64M (67108864 bytes)
disk size: 196K
[...]

There really is no good reason why you’d want to add a backing file during convert when your original image didn’t have one.  If you want to do that, you should use qemu-img rebase.

Here, this would look like this:

$ qemu-img create -f qcow2 -o compat=0.10 base-t.qcow2 64M
$ qemu-img create -f qcow2 -o compat=0.10 source.qcow2 64M
$ qemu-img convert -O qcow2 -o compat=0.10 source.qcow2 target.qcow2
$ qemu-img rebase -b base-t.qcow2 target.qcow2
qemu-img: Could not open old backing file '': The 'file' block driver requires a file name

Oh.  Oops.  Apparently, that never worked.  OK, to me that shows that adding a backing file after the fact is something people just don’t do.  But maybe we should get this working anyway, I don’t think it’d be too difficult.

Anyway, as I don’t quite see the point of adding a backing file during qemu-img convert, and because qcow2 v2 is basically a legacy format anyway, I don’t think we should bother with making that operation more efficient.

Max

(In reply to Max Reitz from comment #11)
> While that might be possible, it would mean some more difficult changes than
> what was necessary for this BZ here.  I don’t think that is worth it
> considering v2 is basically a legacy format anyway.
> 
> Also, I don’t think it makes much sense to add a backing file during
> qemu-img convert.  qemu-img’s man page says this:
> 
> > You can use the backing_file option to force the output image to be created as
> > a copy on write image of the specified base image; the backing_file should have
> > the same content as the input's base image, however the path, image format, etc
> > may differ.
> 
> It says “should”, so you can do something else if you know what you’re
> doing, but then you may get funny results.  In this case, if the input does
> not have a base image (backing file), you just get a fully allocated output.
> 
> Compare this with when your input already had a backing file:
> 
> $ qemu-img create -f qcow2 -o compat=0.10 base-s.qcow2 64M
> $ qemu-img create -f qcow2 -o compat=0.10 base-t.qcow2 64M
> $ qemu-img create -f qcow2 -o compat=0.10 -b base-s.qcow2 source.qcow2 64M
> $ qemu-img convert -O qcow2 -o compat=0.10 -B base-t.qcow2 source.qcow2
> target.qcow2
> $ qemu-img info target.qcow2                                                
> 
> image: target.qcow2
> file format: qcow2
> virtual size: 64M (67108864 bytes)
> disk size: 196K
> [...]
> 
> There really is no good reason why you’d want to add a backing file during
> convert when your original image didn’t have one.  If you want to do that,
> you should use qemu-img rebase.
> 
> Here, this would look like this:
> 
> $ qemu-img create -f qcow2 -o compat=0.10 base-t.qcow2 64M
> $ qemu-img create -f qcow2 -o compat=0.10 source.qcow2 64M
> $ qemu-img convert -O qcow2 -o compat=0.10 source.qcow2 target.qcow2
> $ qemu-img rebase -b base-t.qcow2 target.qcow2
> qemu-img: Could not open old backing file '': The 'file' block driver
> requires a file name
> 
> Oh.  Oops.  Apparently, that never worked.  OK, to me that shows that adding
> a backing file after the fact is something people just don’t do.  But maybe
> we should get this working anyway, I don’t think it’d be too difficult.
> 
> Anyway, as I don’t quite see the point of adding a backing file during
> qemu-img convert, and because qcow2 v2 is basically a legacy format anyway,
> I don’t think we should bother with making that operation more efficient.

It is up to you to make the decision.
Set the bug as verified. Thanks.

> 
> Max

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3443