Bug 1277705

*** Bug 1277706 has been marked as a duplicate of this bug. ***

Just to be clear, this is only for --in-place mode, not copying mode.

(In reply to Richard W.M. Jones from comment #3)
> Just to be clear, this is only for --in-place mode, not copying mode.

So what will happen in copy mode? It'll copy the whole disk? The snapshot tree?

(In reply to Yaniv Kaul from comment #4)
> (In reply to Richard W.M. Jones from comment #3)
> > Just to be clear, this is only for --in-place mode, not copying mode.
> 
> So what will happen in copy mode? It'll copy the whole disk? The snapshot
> tree?

It'll copy the whole disk.

Patch posted:
https://www.redhat.com/archives/libguestfs/2015-November/msg00022.html

Discussion of this continued on the mailing list.  I'm dropping
the above patch for now.

https://www.redhat.com/archives/libguestfs/2015-November/thread.html#00025

Kevin identified a particular commit which causes the
'qemu-img commit' not to be propagated down to backing
files.

The problematic commit is:

https://lists.gnu.org/archive/html/qemu-stable/2015-06/msg00014.html

His comments are:

https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg00817.html

Assuming this can be fixed in qemu, then it would be possible
to run:

virt-sparsify --in-place overlay.qcow2
qemu-img commit overlay.qcow2

which would propagate sparseness through the whole backing chain.

Or:

virt-sparsify --in-place overlay.qcow2
qemu-img commit -b base overlay.qcow2

to propagate sparseness through all layers from the overlay
down to 'base' (but not to lower layers).

Of course we will need to check that qemu gets fixed and
this really works, so let's leave this bug open for now.

Richard, the above is great news. I wonder if I should rephrase the title of the bug to reflect that, or we can close the bug altogether (or move it to qemu-kvm).

However, I assume that unless it is carefully coordinated, this is uber dangerous - the backing file might be used by multiple snapshots, no? Some of which may be running.
oVirt can orchestrate it safely, of course, but generally, you need to ensure / warn / provide a '--force' option.

Agreed on both points.

Upstream patches waiting to be merged:

https://lists.gnu.org/archive/html/qemu-devel/2016-01/msg02139.html

This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune with any questions

To verify this bug, please check that zero areas smaller than 10M are committed as 'zeroed' cluster in base:

qemu-img create /var/tmp/base.qcow2 -f qcow2 1G
qemu-io /var/tmp/base.qcow2 -c 'write -P 1 0 1G'
qemu-img create /var/tmp/top.qcow2 -f qcow2 -b /var/tmp/base.qcow2
qemu-io /var/tmp/top.qcow2 -c 'write -z 0 64k'
qemu-img commit /var/tmp/top.qcow2
qemu-img map --output=json /var/tmp/base.qcow2


Before fix:

Formatting '/var/tmp/base.qcow2', fmt=qcow2 size=1073741824 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16
wrote 1073741824/1073741824 bytes at offset 0
1 GiB, 1 ops; 0:00:02.97 (343.968 MiB/sec and 0.3359 ops/sec)
Formatting '/var/tmp/top.qcow2', fmt=qcow2 size=1073741824 backing_file='/var/tmp/base.qcow2' encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16
wrote 65536/65536 bytes at offset 0
64 KiB, 1 ops; 0.0233 sec (2.672 MiB/sec and 42.7515 ops/sec)
Image committed.
[{ "start": 0, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 327680},
{ "start": 536870912, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 537264128}]


After:

Formatting '/var/tmp/base.qcow2', fmt=qcow2 size=1073741824 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16
wrote 1073741824/1073741824 bytes at offset 0
1 GiB, 1 ops; 0:00:03.54 (288.653 MiB/sec and 0.2819 ops/sec)
Formatting '/var/tmp/top.qcow2', fmt=qcow2 size=1073741824 backing_file=/var/tmp/base.qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16
wrote 65536/65536 bytes at offset 0
64 KiB, 1 ops; 0.0278 sec (2.247 MiB/sec and 35.9492 ops/sec)
Image committed.
[{ "start": 0, "length": 65536, "depth": 0, "zero": true, "data": false},
{ "start": 65536, "length": 536805376, "depth": 0, "zero": false, "data": true, "offset": 393216},
{ "start": 536870912, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 537264128}]

Verified with:
qemu-kvm-rhev-2.6.0-22.el7.x86_64
qemu-img-rhev-2.6.0-22.el7.x86_64
kernel-3.10.0-495.el7.x86_64

Steps:
qemu-img create /var/tmp/base.qcow2 -f qcow2 1G
qemu-io /var/tmp/base.qcow2 -c 'write -P 1 0 1G'
qemu-img create /var/tmp/top.qcow2 -f qcow2 -b /var/tmp/base.qcow2
qemu-io /var/tmp/top.qcow2 -c 'write -z 0 64k'
qemu-img commit /var/tmp/top.qcow2
qemu-img map --output=json /var/tmp/base.qcow2

Results:
[{ "start": 0, "length": 65536, "depth": 0, "zero": true, "data": false},
{ "start": 65536, "length": 536805376, "depth": 0, "zero": false, "data": true, "offset": 393216},
{ "start": 536870912, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 537264128}]

Compared with the result on:
qemu-img-rhev-2.3.0-6.el7.x86_64
qemu-kvm-rhev-2.3.0-6.el7.x86_64

[{ "start": 0, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 327680},
{ "start": 536870912, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 537264128}]

Moving to Verified.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2673.html