Bug 1836645 - Package ovirt-imageio-client is not included in latest 4.4 build
Summary: Package ovirt-imageio-client is not included in latest 4.4 build
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-distribution
Classification: oVirt
Component: ovirt-host
Version: 4.4.0
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: ovirt-4.4.0
: 4.4.0
Assignee: Nir Soffer
QA Contact: Pavol Brilla
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-05-17 15:38 UTC by Ilan Zuckerman
Modified: 2020-05-25 06:26 UTC (History)
8 users (show)

Fixed In Version: ovirt-host-4.4.1-1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-05-25 06:26:35 UTC
oVirt Team: Storage
Embargoed:
sbonazzo: ovirt-4.4?


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 109088 0 master MERGED spec: Add ovirt-imageio-client 2020-12-08 17:39:13 UTC

Description Ilan Zuckerman 2020-05-17 15:38:11 UTC
Description of problem:

The package named 'ovirt-imageio-client' is absent from build rhv-release-4.4.0-36-001.noarch
Making the testing of new imageio not feasible (using SDK), as download and upload disks flows require this package to exist.

According to release notes the package named 'python3-ovirt-engine-sdk4' hasn't been changed from the build before.
The version is python3-ovirt-engine-sdk4-4.4.2-1.el8ev.x86_64

Version-Release number of selected component (if applicable):

rhv-release-4.4.0-36-001.noarch
All of the packages for this release are specified in 'version table' for this release:
http://bob.eng.lab.tlv.redhat.com/builds/4.4/rhv-4.4.0-36/el8/rhv-4.4.0-36_version_info.html

Comment 2 Nir Soffer 2020-05-17 16:04:31 UTC
Moving to ovirt-host-deploy since this is the component that should install
ovirt-imageio-client.

Comment 3 Nir Soffer 2020-05-17 16:06:36 UTC
Didi, is this the right component for 4.4? There is no 4.4 milestone
for this component.

Comment 5 Avihai 2020-05-18 05:30:57 UTC
Not having ovirt-imageio-client blocks QE from testing upload/download disk via SDK and also incremental backup(see SDK example backup_vm.py ).

Raising severity according to this as this effects multiple features functionality as delaying testing efforts for 4.4GA.

Comment 6 Michal Skrivanek 2020-05-18 13:24:05 UTC
ovirt-imageio-client is not a supported part of the product, it's status is similar to vdsm-client. It's fine to use it, but issues with it don't warrant urgent handling.

Comment 7 Nir Soffer 2020-05-18 14:50:38 UTC
(In reply to Michal Skrivanek from comment #6)
> ovirt-imageio-client is not a supported part of the product, it's status is
> similar to vdsm-client. It's fine to use it, but issues with it don't
> warrant urgent handling.

vdsm-client is internal tool that should not be used by users of the product but
it is used by sosreport. imageio client is public facing tool that should be used
by users to perform image transfer and backup, and is the recommended way to use
the product.

Its status is like python3-ovirt-engine-sdk4 - you are expected to use it but
you can write your own client using the REST API.

Comment 8 Ilan Zuckerman 2020-05-24 11:20:16 UTC
Verified on rhv-4.4.1-1
http://bob.eng.lab.tlv.redhat.com/builds/4.4/rhv-4.4.1-1/el8/rhv-4.4.1-1_version_info.html

[root@storage-ge5-vdsm1 ~]# rpm -qa | grep ovirt-imageio-client
ovirt-imageio-client-2.0.6-0.el8ev.x86_64



Upload from host:

root@storage-ge5-vdsm1 ~]# python3 /usr/share/doc/python3-ovirt-engine-sdk4/examples/upload_disk.py virtio-win-0.1.141.iso --engine-url https://storage-ge-05.scl.lab.tlv.redhat.com --username admin@internal --disk-format qcow2 --disk-sparse --sd-name iscsi_0 -c pki.cer --insecure
Checking image...
Image format: raw
Disk format: cow
Disk content type: iso
Disk provisioned size: 316628992
Disk initial size: 316997632
Disk name: virtio-win-0.1.141.qcow2
Connecting...
Password: 
Creating disk...
Disk id: 47dcbff3-fb4b-454d-a03d-6b25f113f374
Creating image transfer...
Transfer ID: 22a79db1-48a9-45c2-9f67-a432770f4daf
Transfer host: host_mixed_2
Uploading image...
[ 100.00% ] 301.96 MiB, 3.29 seconds, 91.72 MiB/s                              
Finalizing image transfer...
Upload completed successfully


Download to host:

[root@storage-ge5-vdsm1 ~]# python3 /usr/share/doc/python3-ovirt-engine-sdk4/examples/download_disk.py 943f8c56-7082-4384-b861-c009efb1b54d downloaded.iso --engine-url https://storage-ge-05.scl.lab.tlv.redhat.com --username admin@internal -c pki.cer --insecure --format qcow2
Connecting...
Password: 
Creating image transfer...
Transfer ID: 70e3e5e7-b56d-4465-8f82-3e1f3098b5b5
Transfer host: host_mixed_3
Downloading image...
Formatting 'downloaded.iso', fmt=qcow2 size=10737418240 cluster_size=65536 lazy_refcounts=off refcount_bits=16
[ 100.00% ] 10.00 GiB, 23.12 seconds, 442.87 MiB/s                             
Finalizing image transfer...

Comment 9 Nir Soffer 2020-05-24 11:43:45 UTC
Ilan, did you notice the Disk id in the upload output?

    Disk id: 47dcbff3-fb4b-454d-a03d-6b25f113f374

You can download the same disk after the upload without going
to the UI or API to find a disk id.

To verify that both upload and download are correct, you can do this:

1. upload file to disk
2. download disk to file
3. compare file contents

   qemu-img compare upload.img download.img

Also note that you downloaded ISO file to qcow2 image:

    Formatting 'downloaded.iso', fmt=qcow2 size=10737418240 cluster_size=65536 lazy_refcounts=off refcount_bits=16

This works, but when the disk is uploaded again it will not be considered ISO
image, since ISO images are always raw. So you want to download ISO as raw:

    # python3 /usr/share/doc/python3-ovirt-engine-sdk4/examples/download_disk.py 943f8c56-7082-4384-b861-c009efb1b54d downloaded.iso --engine-url https://storage-ge-05.scl.lab.tlv.redhat.com --username admin@internal -c pki.cer --insecure --format raw

With this the downloaded file will be considered as ISO when you upload it again.

Also use use --insecure, but the is no need to use this flag when you specify
a cafile.

We may need to update the example download script to check the disk content
type before the download, and automatically use the correct format, rejecting
request to download ISO as qcow2.

I think the verification fine for the purpose of this bug, but we need to do
much more detailed testing of the client functionallity.

Comment 10 Ilan Zuckerman 2020-05-24 13:39:37 UTC
Nir, your feedback is much appreciated.
I re-ran the test this time trying to implement your suggestions.

1. I created a 1GB file with random data on the host:
head -c 1G </dev/urandom >myfile

2. I uploaded it as qcow2
python3 /usr/share/doc/python3-ovirt-engine-sdk4/examples/upload_disk.py myfile --engine-url https://storage-ge-05.scl.lab.tlv.redhat.com --username admin@internal --disk-format qcow2 --disk-sparse --sd-name iscsi_0 -c pki.ce

Checking image...
Image format: raw
Disk format: cow
Disk content type: data
Disk provisioned size: 1073741824
Disk initial size: 1074135040
Disk name: myfile.qcow2
Connecting...
Password: 
Creating disk...
Disk id: d717fa3d-ab30-4c0e-a7cc-025334839c56
Creating image transfer...
Transfer ID: 29f59424-8518-46c4-bba9-50c4ab219d8b
Transfer host: host_mixed_3
Uploading image...
[ 100.00% ] 1.00 GiB, 11.15 seconds, 91.87 MiB/s                               
Finalizing image transfer...


3. I downloaded it 

python3 /usr/share/doc/python3-ovirt-engine-sdk4/examples/download_disk.py d717fa3d-ab30-4c0e-a7cc-025334839c56 downloaded --engine-url https://storage-ge-05.scl.lab.tlv.redhat.com --username admin@internal -c pki.cer --format qcow2

Connecting...
Password: 
Creating image transfer...
Transfer ID: 6576c28a-df38-4455-bd6f-cf520ed92940
Transfer host: host_mixed_1
Downloading image...
Formatting 'downloaded', fmt=qcow2 size=1073741824 cluster_size=65536 lazy_refcounts=off refcount_bits=16
[ 100.00% ] 1.00 GiB, 7.61 seconds, 134.55 MiB/s                               
Finalizing image transfer...


4. i compared the contents of both files:

[root@storage-ge5-vdsm1 ~]# file downloaded 
downloaded: QEMU QCOW Image (v3), 1073741824 bytes

[root@storage-ge5-vdsm1 ~]# qemu-img compare myfile downloaded 
Images are identical.

Comment 11 Sandro Bonazzola 2020-05-25 06:26:35 UTC
This bugzilla is included in oVirt 4.4.0 release, published on May 20th 2020.

Since the problem described in this bug report should be
resolved in oVirt 4.4.0 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.