Bug 1547960 - Use qemu-img measure to estimate image size
Summary: Use qemu-img measure to estimate image size
Alias: None
Product: vdsm
Classification: oVirt
Component: Core
Version: 4.20.15
Hardware: Unspecified
OS: Unspecified
medium vote
Target Milestone: ovirt-4.3.0
: ---
Assignee: Eyal Shenitzky
QA Contact: Yosi Ben Shimon
Depends On: 1433670
TreeView+ depends on / blocked
Reported: 2018-02-22 11:51 UTC by Nir Soffer
Modified: 2019-02-13 07:43 UTC (History)
3 users (show)

Fixed In Version: v4.30.5
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2019-02-13 07:43:12 UTC
oVirt Team: Storage
rule-engine: ovirt-4.3+

Attachments (Terms of Use)

System ID Priority Status Summary Last Updated
oVirt gerrit 93570 master MERGED image: use qemuimg.measure() to estimate image size 2018-09-06 14:40:27 UTC
oVirt gerrit 93813 master MERGED qemuimg: add qemu-img measure to estimate image size 2018-08-30 13:19:40 UTC
oVirt gerrit 93814 master MERGED image_test: add tests for estimate_qcow2_size() 2018-09-06 14:12:05 UTC
Red Hat Bugzilla 1433670 None CLOSED Provide an API that estimates the size of QCOW2 image converted from a raw image 2019-07-09 01:38:17 UTC

Internal Links: 1433670

Description Nir Soffer 2018-02-22 11:51:19 UTC
Description of problem:
qemu-img 2.10 provides now new "measure" command for estimating the size of an
image, when converting images between formats.
See bug 1433670 for details.

Vdsm already includes some of the features provided now by qemu-img measure,
added in:

commit e776295b32bc22427d300082be18e323ab31983b
Author: Maor Lipchuk <mlipchuk@redhat.com>
Date:   Mon Feb 20 13:30:57 2017 +0200

    qcow2: Calculate estimated size for converted qcow volumes.
    Adding a utility module for estimating qcow2 file size once
    converted from raw to qcow2.
    The estimation takes in consider also the qcow header size
    based on the implementation being done in qemu block/qcow2.c
    The following is a test being done on a 10GB QCOW volume
    with compatibility of 1.1.
    The actual size of the disk after executing qemu-convert was 1.00036621
    GB, while the estimated size by the new function was 1.00177 GB
    Change-Id: Ia87df2b0d5378f93c5cb2cc68a37458fe3b4467b
    Bug-Url: https://bugzilla.redhat.com/1358717
    Signed-off-by: Maor Lipchuk <mlipchuk@redhat.com>

Same code is used also for estimating conversion from qcow2 to qcow2 format, 
added in:

commit 52f2c6078b8b393bfde5da9cd9c78a262f733719
Author: Nir Soffer <nsoffer@redhat.com>
Date:   Thu Sep 21 15:30:32 2017 +0300

    qcow2: Support compressed qcow2 uploads
    If a compressed qcow2 image is uploaded, copying the image to block
    storage is likely to fail, because we used the source image size for
    allocating the destination image.
    This patch enables qcow2 source image in qcow2.estimate_size() and use
    it to estimate the destination file size when copying qcow2 image to
    another storage domain.
    Change-Id: I4ece05285a293da2495bcbebf19cad97fb2f32d8
    Bug-Url: https://bugzilla.redhat.com/1470435
    Signed-off-by: Nir Soffer <nsoffer@redhat.com>

We want to drop the vdsm code and use instead qemu-img messure.

This has several advantages:
- No need to maintain and port the code to python 3
- correctness and robustness, qemu-img measure will not break when qcow2 format
  is changed
- qemu-img measure is more efficient, current vdsm code is calling qemu-img map
  generating possibly huge json for very sparse images.
- being able to use additional features provided by qemu-img measure

Needed changes:

- Add qemuimg.measure() function calling qemu-img measure
- Use the new function when we use now storage.qcow2.estimate_size()
- remove storage.qcow2 module and its tests

Comment 1 Allon Mureinik 2018-02-22 12:02:36 UTC
@Dan - didn't mean to assign this to you, that was a stupid BZ race.
Sorry for the noise.

Comment 2 Yosi Ben Shimon 2018-12-26 09:38:37 UTC
Can you please provide steps to reproduce or how/what exactly to test here.

Comment 3 Eyal Shenitzky 2018-12-26 09:50:41 UTC
There is none.

Just verify that there are no regressions, mainly around copying images/chain (create template/clone/copy disk).

Comment 4 Eyal Shenitzky 2018-12-26 09:52:09 UTC
(In reply to Eyal Shenitzky from comment #3)
> There is none.
> Just verify that there are no regressions, mainly around copying
> images/chain (create template/clone/copy disk).

with qcow2 disks.

Comment 5 Yosi Ben Shimon 2019-02-07 12:43:53 UTC
No regressions related to this were found.
Currently, moving to VERIFIED.

Comment 6 Yosi Ben Shimon 2019-02-07 14:03:23 UTC
Last execution envrinoment:

Comment 7 Sandro Bonazzola 2019-02-13 07:43:12 UTC
This bugzilla is included in oVirt 4.3.0 release, published on February 4th 2019.

Since the problem described in this bug report should be
resolved in oVirt 4.3.0 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.

Note You need to log in before you can comment on or make changes to this bug.