Bug 1547960

Summary: Use qemu-img measure to estimate image size
Product: [oVirt] vdsm Reporter: Nir Soffer <nsoffer>
Component: CoreAssignee: Eyal Shenitzky <eshenitz>
Status: CLOSED CURRENTRELEASE QA Contact: Yosi Ben Shimon <ybenshim>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 4.20.15CC: bugs, ebenahar, eshenitz
Target Milestone: ovirt-4.3.0Flags: rule-engine: ovirt-4.3+
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: v4.30.5 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-02-13 07:43:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Storage RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1433670    
Bug Blocks:    

Description Nir Soffer 2018-02-22 11:51:19 UTC
Description of problem:
qemu-img 2.10 provides now new "measure" command for estimating the size of an
image, when converting images between formats.
See bug 1433670 for details.

Vdsm already includes some of the features provided now by qemu-img measure,
added in:

commit e776295b32bc22427d300082be18e323ab31983b
Author: Maor Lipchuk <mlipchuk>
Date:   Mon Feb 20 13:30:57 2017 +0200

    qcow2: Calculate estimated size for converted qcow volumes.
    
    Adding a utility module for estimating qcow2 file size once
    converted from raw to qcow2.
    The estimation takes in consider also the qcow header size
    based on the implementation being done in qemu block/qcow2.c
    
    The following is a test being done on a 10GB QCOW volume
    with compatibility of 1.1.
    The actual size of the disk after executing qemu-convert was 1.00036621
    GB, while the estimated size by the new function was 1.00177 GB
    
    Change-Id: Ia87df2b0d5378f93c5cb2cc68a37458fe3b4467b
    Bug-Url: https://bugzilla.redhat.com/1358717
    Signed-off-by: Maor Lipchuk <mlipchuk>

Same code is used also for estimating conversion from qcow2 to qcow2 format, 
added in:

commit 52f2c6078b8b393bfde5da9cd9c78a262f733719
Author: Nir Soffer <nsoffer>
Date:   Thu Sep 21 15:30:32 2017 +0300

    qcow2: Support compressed qcow2 uploads
    
    If a compressed qcow2 image is uploaded, copying the image to block
    storage is likely to fail, because we used the source image size for
    allocating the destination image.
    
    This patch enables qcow2 source image in qcow2.estimate_size() and use
    it to estimate the destination file size when copying qcow2 image to
    another storage domain.
    
    Change-Id: I4ece05285a293da2495bcbebf19cad97fb2f32d8
    Bug-Url: https://bugzilla.redhat.com/1470435
    Signed-off-by: Nir Soffer <nsoffer>

We want to drop the vdsm code and use instead qemu-img messure.

This has several advantages:
- No need to maintain and port the code to python 3
- correctness and robustness, qemu-img measure will not break when qcow2 format
  is changed
- qemu-img measure is more efficient, current vdsm code is calling qemu-img map
  generating possibly huge json for very sparse images.
- being able to use additional features provided by qemu-img measure

Needed changes:

- Add qemuimg.measure() function calling qemu-img measure
- Use the new function when we use now storage.qcow2.estimate_size()
- remove storage.qcow2 module and its tests

Comment 1 Allon Mureinik 2018-02-22 12:02:36 UTC
@Dan - didn't mean to assign this to you, that was a stupid BZ race.
Sorry for the noise.

Comment 2 Yosi Ben Shimon 2018-12-26 09:38:37 UTC
Hi,
Can you please provide steps to reproduce or how/what exactly to test here.

Comment 3 Eyal Shenitzky 2018-12-26 09:50:41 UTC
There is none.

Just verify that there are no regressions, mainly around copying images/chain (create template/clone/copy disk).

Comment 4 Eyal Shenitzky 2018-12-26 09:52:09 UTC
(In reply to Eyal Shenitzky from comment #3)
> There is none.
> 
> Just verify that there are no regressions, mainly around copying
> images/chain (create template/clone/copy disk).

with qcow2 disks.

Comment 5 Yosi Ben Shimon 2019-02-07 12:43:53 UTC
No regressions related to this were found.
Currently, moving to VERIFIED.

Comment 6 Yosi Ben Shimon 2019-02-07 14:03:23 UTC
Last execution envrinoment:
ovirt-engine-4.3.0.4-0.1.el7.noarch
vdsm-4.30.8-2.el7ev.x86_64
qemu-img-rhev-2.12.0-21.el7.x86_64

Comment 7 Sandro Bonazzola 2019-02-13 07:43:12 UTC
This bugzilla is included in oVirt 4.3.0 release, published on February 4th 2019.

Since the problem described in this bug report should be
resolved in oVirt 4.3.0 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.