RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1277705 - Fix handling of thin write correctly in active-commit and drive-mirror (including qemu-img commit)
Summary: Fix handling of thin write correctly in active-commit and drive-mirror (inclu...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm-rhev
Version: 7.3
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: rc
: ---
Assignee: Fam Zheng
QA Contact: Qianqian Zhu
URL:
Whiteboard:
: 1277706 (view as bug list)
Depends On:
Blocks: 734120 1288337 1305606 1459730
TreeView+ depends on / blocked
 
Reported: 2015-11-03 21:11 UTC by Yaniv Kaul
Modified: 2017-06-08 00:54 UTC (History)
15 users (show)

Fixed In Version: QEMU 2.6
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1459730 (view as bug list)
Environment:
Last Closed: 2016-11-07 20:51:26 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:2673 0 normal SHIPPED_LIVE qemu-kvm-rhev bug fix and enhancement update 2016-11-08 01:06:13 UTC

Description Yaniv Kaul 2015-11-03 21:11:46 UTC 
Description of problem:
Per my email discussion with Richard:

> 1.When sparsifying, virt-sparsify is only working on the current snap? Is
> that the intention?
virt-sparsify isn't really tested on snapshots at all.  It's expecting
a single base image that it can sparsify.

> 2. Perhaps it should at least warn that there's work to be done on the base?

It could check that, and should probably just stop if it encounters
that.  I can't think of a case where you'd want to sparsify a
snapshot.
Comment 2 Pino Toscano 2015-11-04 09:02:24 UTC 
*** Bug 1277706 has been marked as a duplicate of this bug. ***
Comment 3 Richard W.M. Jones 2015-11-04 09:59:53 UTC 
Just to be clear, this is only for --in-place mode, not copying mode.
Comment 4 Yaniv Kaul 2015-11-04 10:06:38 UTC 
(In reply to Richard W.M. Jones from comment #3)
> Just to be clear, this is only for --in-place mode, not copying mode.

So what will happen in copy mode? It'll copy the whole disk? The snapshot tree?
Comment 5 Richard W.M. Jones 2015-11-04 10:12:18 UTC 
(In reply to Yaniv Kaul from comment #4)
> (In reply to Richard W.M. Jones from comment #3)
> > Just to be clear, this is only for --in-place mode, not copying mode.
> 
> So what will happen in copy mode? It'll copy the whole disk? The snapshot
> tree?

It'll copy the whole disk.
Comment 6 Richard W.M. Jones 2015-11-04 10:12:33 UTC 
Patch posted:
https://www.redhat.com/archives/libguestfs/2015-November/msg00022.html
Comment 7 Richard W.M. Jones 2015-11-04 12:28:22 UTC 
Discussion of this continued on the mailing list.  I'm dropping
the above patch for now.
Comment 8 Richard W.M. Jones 2015-11-04 12:29:16 UTC 
https://www.redhat.com/archives/libguestfs/2015-November/thread.html#00025
Comment 9 Richard W.M. Jones 2015-11-04 21:03:41 UTC 
Kevin identified a particular commit which causes the
'qemu-img commit' not to be propagated down to backing
files.

The problematic commit is:

https://lists.gnu.org/archive/html/qemu-stable/2015-06/msg00014.html

His comments are:

https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg00817.html

Assuming this can be fixed in qemu, then it would be possible
to run:

virt-sparsify --in-place overlay.qcow2
qemu-img commit overlay.qcow2

which would propagate sparseness through the whole backing chain.

Or:

virt-sparsify --in-place overlay.qcow2
qemu-img commit -b base overlay.qcow2

to propagate sparseness through all layers from the overlay
down to 'base' (but not to lower layers).

Of course we will need to check that qemu gets fixed and
this really works, so let's leave this bug open for now.
Comment 10 Yaniv Kaul 2015-11-05 11:07:39 UTC 
Richard, the above is great news. I wonder if I should rephrase the title of the bug to reflect that, or we can close the bug altogether (or move it to qemu-kvm).

However, I assume that unless it is carefully coordinated, this is uber dangerous - the backing file might be used by multiple snapshots, no? Some of which may be running.
oVirt can orchestrate it safely, of course, but generally, you need to ensure / warn / provide a '--force' option.
Comment 11 Richard W.M. Jones 2015-11-05 12:24:10 UTC 
Agreed on both points.
Comment 14 Fam Zheng 2016-01-15 07:57:29 UTC 
Upstream patches waiting to be merged:

https://lists.gnu.org/archive/html/qemu-devel/2016-01/msg02139.html
Comment 16 Mike McCune 2016-03-28 22:35:49 UTC 
This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune with any questions
Comment 18 Fam Zheng 2016-09-09 03:13:42 UTC 
To verify this bug, please check that zero areas smaller than 10M are committed as 'zeroed' cluster in base:

qemu-img create /var/tmp/base.qcow2 -f qcow2 1G
qemu-io /var/tmp/base.qcow2 -c 'write -P 1 0 1G'
qemu-img create /var/tmp/top.qcow2 -f qcow2 -b /var/tmp/base.qcow2
qemu-io /var/tmp/top.qcow2 -c 'write -z 0 64k'
qemu-img commit /var/tmp/top.qcow2
qemu-img map --output=json /var/tmp/base.qcow2


Before fix:

Formatting '/var/tmp/base.qcow2', fmt=qcow2 size=1073741824 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16
wrote 1073741824/1073741824 bytes at offset 0
1 GiB, 1 ops; 0:00:02.97 (343.968 MiB/sec and 0.3359 ops/sec)
Formatting '/var/tmp/top.qcow2', fmt=qcow2 size=1073741824 backing_file='/var/tmp/base.qcow2' encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16
wrote 65536/65536 bytes at offset 0
64 KiB, 1 ops; 0.0233 sec (2.672 MiB/sec and 42.7515 ops/sec)
Image committed.
[{ "start": 0, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 327680},
{ "start": 536870912, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 537264128}]


After:

Formatting '/var/tmp/base.qcow2', fmt=qcow2 size=1073741824 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16
wrote 1073741824/1073741824 bytes at offset 0
1 GiB, 1 ops; 0:00:03.54 (288.653 MiB/sec and 0.2819 ops/sec)
Formatting '/var/tmp/top.qcow2', fmt=qcow2 size=1073741824 backing_file=/var/tmp/base.qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16
wrote 65536/65536 bytes at offset 0
64 KiB, 1 ops; 0.0278 sec (2.247 MiB/sec and 35.9492 ops/sec)
Image committed.
[{ "start": 0, "length": 65536, "depth": 0, "zero": true, "data": false},
{ "start": 65536, "length": 536805376, "depth": 0, "zero": false, "data": true, "offset": 393216},
{ "start": 536870912, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 537264128}]
Comment 19 Qianqian Zhu 2016-09-09 06:38:19 UTC 
Verified with:
qemu-kvm-rhev-2.6.0-22.el7.x86_64
qemu-img-rhev-2.6.0-22.el7.x86_64
kernel-3.10.0-495.el7.x86_64

Steps:
qemu-img create /var/tmp/base.qcow2 -f qcow2 1G
qemu-io /var/tmp/base.qcow2 -c 'write -P 1 0 1G'
qemu-img create /var/tmp/top.qcow2 -f qcow2 -b /var/tmp/base.qcow2
qemu-io /var/tmp/top.qcow2 -c 'write -z 0 64k'
qemu-img commit /var/tmp/top.qcow2
qemu-img map --output=json /var/tmp/base.qcow2

Results:
[{ "start": 0, "length": 65536, "depth": 0, "zero": true, "data": false},
{ "start": 65536, "length": 536805376, "depth": 0, "zero": false, "data": true, "offset": 393216},
{ "start": 536870912, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 537264128}]

Compared with the result on:
qemu-img-rhev-2.3.0-6.el7.x86_64
qemu-kvm-rhev-2.3.0-6.el7.x86_64

[{ "start": 0, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 327680},
{ "start": 536870912, "length": 536870912, "depth": 0, "zero": false, "data": true, "offset": 537264128}]

Moving to Verified.
Comment 21 errata-xmlrpc 2016-11-07 20:51:26 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2673.html
Note You need to log in before you can comment on or make changes to this bug.