Bug 1977232

+++ This bug was initially created as a clone of Bug #1973345 +++

Description of problem:

Creating a template from vm when source disk is on block storage, and the
volume does not have a parent fails with:

engine.log:

VDSM host3 command HSMGetAllTasksStatusesVDS failed: value=low level Image copy failed: ('Destination volume 0195a9f2-51f6-49c8-8520-9c630cbdfd59 error: Command [\'/usr/bin/qemu-img\', \'measure\', \'--output\', \'json\', \'-O\', \'qcow2\', \'json:{"file": {"driver": "file", "filename": "/rhev/data-center/mnt/blockSD/feab3738-c158-4d48-8a41-b5a95c057a50/images/3e8214f8-7a70-4e6b-a2fe-84c7103cd8ae/2eda0099-4dbc-4602-a418-fbd1ccee3357"}, "driver": "qcow2"}\'] failed with rc=1 out=b\'\' err=b\'qemu-img: Could not open \\\'json:{"file": {"driver": "file", "filename": "/rhev/data-center/mnt/blockSD/feab3738-c158-4d48-8a41-b5a95c057a50/images/3e8214f8-7a70-4e6b-a2fe-84c7103cd8ae/2eda0099-4dbc-4602-a418-fbd1ccee3357"}, "driver": "qcow2"}\\\': \\\'file\\\' driver requires \\\'/rhev/data-center/mnt/blockSD/feab3738-c158-4d48-8a41-b5a95c057a50/images/3e8214f8-7a70-4e6b-a2fe-84c7103cd8ae/2eda0099-4dbc-4602-a418-fbd1ccee3357\\\' to be a regular file\\n\'',) abortedcode=261

vdsm.log:

2021-06-17 03:14:17,571+0300 INFO  (tasks/1) [storage.ThreadPool.WorkerThread] START task 62624777-82b1-4d25-8db7-1bffbf34f
f90 (cmd=<bound method Task.commit of <vdsm.storage.task.Task object at 0x7fa62aee1f60>>, args=None) (threadPool:146)
2021-06-17 03:14:17,925+0300 INFO  (tasks/1) [storage.Image] sdUUID=feab3738-c158-4d48-8a41-b5a95c057a50 vmUUID= srcImgUUID
=3e8214f8-7a70-4e6b-a2fe-84c7103cd8ae srcVolUUID=2eda0099-4dbc-4602-a418-fbd1ccee3357 dstImgUUID=77eec8ed-7dce-4250-8575-42
5d661e64e6 dstVolUUID=0195a9f2-51f6-49c8-8520-9c630cbdfd59 dstSdUUID=feab3738-c158-4d48-8a41-b5a95c057a50 volType=6 volForm
at=COW preallocate=SPARSE force=False postZero=False discard=True (image:635)
2021-06-17 03:14:17,926+0300 INFO  (tasks/1) [storage.VolumeManifest] Volume: preparing volume feab3738-c158-4d48-8a41-b5a9
5c057a50/2eda0099-4dbc-4602-a418-fbd1ccee3357 (volume:599)
2021-06-17 03:14:17,934+0300 INFO  (tasks/1) [storage.LVM] Activating lvs: vg=feab3738-c158-4d48-8a41-b5a95c057a50 lvs=['2e
da0099-4dbc-4602-a418-fbd1ccee3357'] (lvm:1755)
2021-06-17 03:14:18,298+0300 ERROR (tasks/1) [storage.Image] Unexpected error (image:729)
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/vdsm/storage/image.py", line 673, in copyCollapsed
    sdUUID, volParams, dstSdUUID, dstVolFormat)
  File "/usr/lib/python3.6/site-packages/vdsm/storage/image.py", line 850, in calculate_vol_alloc
    return self.estimate_qcow2_size(src_vol_params, dst_sd_id)
  File "/usr/lib/python3.6/site-packages/vdsm/storage/image.py", line 117, in estimate_qcow2_size
    output_format=qemuimg.FORMAT.QCOW2)
  File "/usr/lib/python3.6/site-packages/vdsm/storage/qemuimg.py", line 156, in measure
    out = _run_cmd(cmd)
  File "/usr/lib/python3.6/site-packages/vdsm/storage/qemuimg.py", line 578, in _run_cmd
    raise cmdutils.Error(cmd, rc, out, err)
vdsm.common.cmdutils.Error: Command ['/usr/bin/qemu-img', 'measure', '--output', 'json', '-O', 'qcow2', 'json:{"file": {"driver": "file", "filename": "/rhev/data-center/mnt/blockSD/feab3738-c158-4d48-8a41-b5a95c057a50/images/3e8214f8-7a70-4e6b-a2fe-84c7103cd8ae/2eda0099-4dbc-4602-a418-fbd1ccee3357"}, "driver": "qcow2"}'] failed with rc=1 out=b'' err=b'qemu-img: Could not open \'json:{"file": {"driver": "file", "filename": "/rhev/data-center/mnt/blockSD/feab3738-c158-4d48-8a41-b5a95c057a50/images/3e8214f8-7a70-4e6b-a2fe-84c7103cd8ae/2eda0099-4dbc-4602-a418-fbd1ccee3357"}, "driver": "qcow2"}\': \'file\' driver requires \'/rhev/data-center/mnt/blockSD/feab3738-c158-4d48-8a41-b5a95c057a50/images/3e8214f8-7a70-4e6b-a2fe-84c7103cd8ae/2eda0099-4dbc-4602-a418-fbd1ccee3357\' to be a regular file\n'

The issues is using:

    'json:{"file": {"driver": "file", "filename": "/path"}, "driver": "qcow2"}'

When /path is block storage. It should be:

    'json:{"file": {"driver": "host_device", "filename": "/path"}, "driver": "qcow2"}'

The root cause is not calling qemuimg.measure() without is_block=True.

This is a regression introduce in:
$ git describe 1718b8784e841405574c44abe2357997e3235723
v4.40.32-4-g1718b8784

Before this change, qemu-img measure was called with a path instead of json: uri,
and qemu-img detected the right driver.

Version-Release number of selected component (if applicable):
vdsm-4.40.70.4

How reproducible:
100$

Steps to Reproduce:
1. Create think vm on block storage (iSCSI/FC)
   The vm disk must not have a parent.
2. Create template from the VM using qcow2 format

The operation fails.

--- Additional comment from RHEL Program Management on 2021-06-18 11:27:44 UTC ---

This bug report has Keywords: Regression or TestBlocker.
Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP.

--- Additional comment from Nir Soffer on 2021-06-19 15:43:38 UTC ---

Should be available in vdsm 4.40.70.5.

--- Additional comment from Lukas Svaty on 2021-06-21 10:43:12 UTC ---

Based on Infro from mavital adding blocker+

--- Additional comment from  on 2021-06-28 09:06:17 UTC ---

Could't reproduce it on older version (VDSM 4.40.7.4) according to these steps mentioned above.

1) created a VM from Blank (no template)
2) Attached new disk that I created with thin provision on iscsi storage domain
3) From virtual machines -> choose the new VM -> Make Template using qcow2 format.

The process passes successfully and the template and can be seen in the template section under compute.


Nir, Can you please add additional steps if something is missing here?

--- Additional comment from  on 2021-06-28 09:07:57 UTC ---

correcting : VDSM 4.40.70.4

--- Additional comment from Nir Soffer on 2021-06-28 11:53:37 UTC ---

(In reply to sshmulev from comment #4)
This isssue can be reproduced only with qemu-6.0.0 available in RHEL 8.5 nightly repos
or on Centos Stream.

On RHEL 8.4 we have qemu 5.2.0:

[root@storage-ge12-vdsm3 ~]# rpm -q qemu-img
qemu-img-5.2.0-16.module+el8.4.0+10806+b7d97207.x86_64

With this version trying to measure block device with incorrect json uri does
not fail, it only logs a warning:

# qemu-img info /dev/0bc333f0-815d-4613-87a4-341dbde9413e/eb83cee8-424d-4edc-9695-ca1570dea42c
image: /dev/0bc333f0-815d-4613-87a4-341dbde9413e/eb83cee8-424d-4edc-9695-ca1570dea42c
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: 0 B
cluster_size: 65536
Format specific information:
    compat: 1.1
    compression type: zlib
    lazy refcounts: false
    refcount bits: 16
    corrupt: false
    extended l2: false

# qemu-img measure -O qcow2 'json:{"driver": "qcow2", "file":{"driver": "file", "filename": "/dev/0bc333f0-815d-4613-87a4-341dbde9413e/eb83cee8-424d-4edc-9695-ca1570dea42c"}}'; echo $?
qemu-img: warning: Opening a block device as a file using the 'file' driver is deprecated
required size: 1900544
fully allocated size: 10739318784
bitmaps size: 0
0

So this bug affects oVirt 4.4.7 *now* on Centos Stream, but it will affect RHV only when
RHEL 8.5 will be released.

To test this properly, you need to upgrade qemu to version 6.0.0. This requires also 
upgrading libvirt. It can be done by using RHEL 8.5.0 nightly repos instead of the 8.4.0
nightly repos.