Bug 1475947

Summary: Make use of qemu-img measure (estimates the size of QCOW2 image converted from a raw image)
Product: Red Hat Enterprise Linux 7 Reporter: Ademar Reis <areis>
Component: libvirtAssignee: Peter Krempa <pkrempa>
Status: CLOSED CANTFIX QA Contact: Han Han <hhan>
Severity: high Docs Contact:
Priority: high    
Version: 7.4CC: amureini, chayang, coli, dyuan, eblake, jcall, juzhang, kchamart, knoel, lmen, michen, mlipchuk, mtessun, ngu, nsoffer, pingl, rbalakri, stefanha, tnisan, virt-maint, xuzhang
Target Milestone: rcKeywords: FutureFeature
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: 1433670 Environment:
Last Closed: 2018-02-19 08:46:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1433670    
Bug Blocks:    

Description Ademar Reis 2017-07-27 15:27:09 UTC
qemu-img measure was introduced in QEMU-2.9 and it might be useful for some features in libvirt, such as live storage migration.

This BZ is to track the investigation and implementation of such improvements.

+++ This bug was initially created as a clone of Bug #1433670 +++

Description of problem:
I was wondering if that will be possible to provide a new API that estimates the size of qcow2 image converted from a raw image.
We could use this new API to allocate the size more precisely before the convert operation.

The operation that we are trying to do is to convert a raw sparse image from NFS or from block device to qcow2 image on thin provisioned block device

In ovirt thin provisioned block device is a regular lv, and we like
to allocate only the required size for the the qcow file.

Our current (stupid) solution is to allocate the entire LV using the
size of the raw image.


Here is an example flow:

    $ truncate -s 10G test.raw

We don't know what will be the size of the qcow on the block storage,
so we allocate the entire LV:

    $ lvcreate --size 10G vg/lv

Then we convert the file to the new LV:

    $ qemu-img convert -f raw -O qcow2 test.raw /dev/vg/lv

After the copy we can check the actual size:

    $ qemu-img check /dev/vg/lv

And reduce the LV:

    $ lvreduce -L128m vg/lv

But we like to avoid the allocation, and allocate only the needed size
before we convert the image.

We think that the best way to solve this issue is to return this info
from qemu-img, maybe as a flag to qemu-img convert that will
calculate the size of the converted image without doing any writes.


--- Additional comment from Stefan Hajnoczi on 2017-07-14 06:01:00 BRT ---

The following commits are now in qemu.git/master and will be included in the next qemu-kvm-rhev rebase:

32a1681 iotests: add test 178 for qemu-img measure
217a068 qemu-iotests: support per-format golden output files
fd03c2b qemu-img: add measure subcommand
c501c35 qcow2: add bdrv_measure() support
0eb4a8c qcow2: extract image creation option parsing
7c5bcc4 qcow2: make refcount size calculation conservative
95c67e3 qcow2: extract preallocation calculation function
a843a22 raw-format: add bdrv_measure() support
90880ff block: add bdrv_measure() API

Comment 1 Eric Blake 2017-07-27 15:33:49 UTC
(In reply to Ademar Reis from comment #0)
> qemu-img measure was introduced in QEMU-2.9 and it might be useful for some
> features in libvirt, such as live storage migration.

First available in qemu 2.10, actually.

Comment 2 Peter Krempa 2018-02-19 08:46:41 UTC
This feature is not accessible via the monitor so it's not very useful for libvirt.

Comment 3 Nir Soffer 2018-02-19 11:42:12 UTC
I think this should be a vdsm bug. Vdsm is using qemu code ported to python to 
measure images. We should drop that code and use the new measure command instead.

Comment 4 Ademar Reis 2018-02-22 03:31:51 UTC
(In reply to Nir Soffer from comment #3)
> I think this should be a vdsm bug. Vdsm is using qemu code ported to python
> to 
> measure images. We should drop that code and use the new measure command
> instead.

Yes, the qemu BZ was originally opened by the RHV team and blocked several RHV BZs. Do you think we still need an additional clone for RHV to make use of this feature? The QEMU BZ is: Bug 1433670.

Comment 5 Nir Soffer 2018-02-22 11:34:14 UTC
(In reply to Ademar Reis from comment #4)
Thanks for the info, but I don't think we need a clone for this. I'll open an
upstream ovirt bug to consume qemu-img measure.