Description of problem: VDSM verifying for uploaded QCOW image should also check if the QCOW's compat level is valid. Steps to Reproduce: 1. Upload a QCOW image with a compat level which is not valid Actual results: Should fail.
Please also already open a bug on removing this limitation, and make it depends on the qcow2v3 support that we wish to include in next release.
This bug resolution will add functionality that confirms that the uploaded QCOW compat level is supported in the VDSM's machine. so once VDSM provides that QCOW2v3 is supported, an uploaded QCOW2v3 will succeed. Anyhow, We need to make sure that this indeed happens, so I opened Bug 1345792 to add tests for that verification, that will depend on both Bug 821546 and this one.
(In reply to Amit Aviram from comment #2) > This bug resolution will add functionality that confirms that the uploaded > QCOW compat level is supported in the VDSM's machine. so once VDSM provides > that QCOW2v3 is supported, an uploaded QCOW2v3 will succeed. > > Anyhow, We need to make sure that this indeed happens, so I opened Bug > 1345792 to add tests for that verification, that will depend on both Bug > 821546 and this one. Excellent, thanks.
https://gerrit.ovirt.org/#/q/status:open+project:vdsm+branch:master+topic:verify-qcow2-compat
We should support any format supported by qemu image. Limiting the format to the old 0.1 version means that nobody will be able to upload images, and we will waste support time on this. Kevin, do you see any issue with uploading qcow2 comapt 1.1, and then creating snapshots on this image using qcow compat 0.11? This seems to work fine in my tests.
(In reply to Amit Aviram from comment #0) > Steps to Reproduce: > 1. Upload a QCOW image with a compat level which is not valid Can you reproduce this issue? Please try to create qcow2 image with invalid compat level and upload it.
This bug is about verifying that the compat level is supported in the host- reproducing this bug is basically just uploading an image..
(In reply to Amit Aviram from comment #7) > This bug is about verifying that the compat level is supported in the host- > reproducing this bug is basically just uploading an image.. In comment 0 you mention qcow file with invalid compat level. There is nothing invalid about compat level 1.1. I think it is a good idea to verify that uploaded images uses known compat values, and so accepting only compat 0.11 or 1.1 is fine. Limiting the feature to accept only the old 0.11 format means it will make this feature useless for most users, to make it possible to attach a storage domain to a legacy setup running legacy rhel version that does not support qcow2 compat 1.1. Instead of making the product worse for most users, we can solve this by introducing a solution for the user that are interested in this compatiblity, without harming most of the users that do not care about it. You can amend a qcow2 image using compat=1.1 to compat=0.11: qemu-img amend -f qcow2 -o compat=1.1 test.qcow2 The best way to handle this is to add a "make image compatible with legacy systems" checkbox to the upload ui. When the option is checked, you will ask vdsm to make the image compatible during verification. Vdsm will run qemu-img amend on the image. When the option is not checked, we will accept any qcow file with known comapt value.
Adding back need info for Kevin, see comment 5.
(In reply to Nir Soffer from comment #8) > (In reply to Amit Aviram from comment #7) > > This bug is about verifying that the compat level is supported in the host- > > reproducing this bug is basically just uploading an image.. > > In comment 0 you mention qcow file with invalid compat level. There is > nothing > invalid about compat level 1.1. > > I think it is a good idea to verify that uploaded images uses known compat > values, and so accepting only compat 0.11 or 1.1 is fine. > > Limiting the feature to accept only the old 0.11 format means it will > make this feature useless for most users, to make it possible to attach > a storage domain to a legacy setup running legacy rhel version that does > not support qcow2 compat 1.1. > > Instead of making the product worse for most users, we can solve this by > introducing a solution for the user that are interested in this compatiblity, > without harming most of the users that do not care about it. > > You can amend a qcow2 image using compat=1.1 to compat=0.11: > > qemu-img amend -f qcow2 -o compat=1.1 test.qcow2 btw the user also can amend the version to 0.11 to upload the image > > The best way to handle this is to add a "make image compatible with legacy > systems" > checkbox to the upload ui. > > When the option is checked, you will ask vdsm to make the image compatible > during verification. Vdsm will run qemu-img amend on the image. > > When the option is not checked, we will accept any qcow file with known > comapt > value. That makes sense, but anyway we need to make sure that this is what our customers really need. Yaniv&Allon, you know the customers better than us- what do you think? In case the bug is not detailed enough- I'll add a brief of what's going on here: - Currently oVirt images' version is an old version of QCOW (0.11), as this is the supported version of QCOW in RHEL6. every usage of a newer version (snapshots, etc) won't work in a host with RHEL6. (which we support today in 3.6 afaik) - However, version 1.1 is more popular today (based on Nir's comment, I don't know that as a fact) and generally better than 0.11. so an image that a user will want to upload to the system is likely to be 1.1. - This bug is about verifying that the uploaded image is 0.1, so the image will be usable to any host in the system. So my question to you- do we need: 1. enforce 0.1 ? 2. add a checkbox that will make VDSM convert an 1.1 image to 0.1 as Nir suggested? 3. add a checkbox that will disable the enforcement of the QCOW version? 3. not enforcing 0.1 at all, making the storage domains possibly not compatible with all the possible DCs that it can be attached to? (specifically, DC3.6 with RHEL6 hosts) Let us know, thanks.
(In reply to Amit Aviram from comment #10) > So my question to you- do we need: > 1. enforce 0.1 ? > 2. add a checkbox that will make VDSM convert an 1.1 image to 0.1 as Nir > suggested? > 3. add a checkbox that will disable the enforcement of the QCOW version? > 3. not enforcing 0.1 at all, making the storage domains possibly not > compatible with all the possible DCs that it can be attached to? > (specifically, DC3.6 with RHEL6 hosts) The point of this BZ is to align image upload to the other flows. Currently, ALL our qcows are compat=0.10, unless you manually tweak a conf value in VDSM. Image upload should do the same - enforce that the uploaded qcow has the same compat version as the conf value. For 4.1, we'll introduce a better solution and properly support compat=1.1, which image upload will have to be a part of.
(In reply to Nir Soffer from comment #5) > Kevin, do you see any issue with uploading qcow2 comapt 1.1, and then > creating snapshots on this image using qcow compat 0.11? This seems to work > fine in my tests. No, this is completely fine. Of course, it still means that you need a QEMU version >= 1.1 in order to use both images together. (In reply to Allon Mureinik from comment #11) > For 4.1, we'll introduce a better solution and properly support compat=1.1, > which image upload will have to be a part of. Finally! It has only been four years. :-)
The automation script jumped the gun here. All these patches are just merged on master, they still need to be backported. Returning to POST.
All needed patches have been backported and merged to the 4.0 branch.
Adam/Amit - this behavior needs documenting. Not sure if it's already documented somewhere else, but if not, it should be documented here.
(In reply to Amit Aviram from comment #0) > Description of problem: > VDSM verifying for uploaded QCOW image should also check if the QCOW's > compat level is valid. > > Steps to Reproduce: > 1. Upload a QCOW image with a compat level which is not valid > > Actual results: > Should fail. At what point should I expect it to fail?
End of upload(In reply to Natalie Gavrielov from comment #16) > (In reply to Amit Aviram from comment #0) > > Description of problem: > > VDSM verifying for uploaded QCOW image should also check if the QCOW's > > compat level is valid. > > > > Steps to Reproduce: > > 1. Upload a QCOW image with a compat level which is not valid > > > > Actual results: > > Should fail. > > At what point should I expect it to fail? The upload should fail in the verification phase, which is currently only when the file is fully uploaded. We have Bug 1344285 that will also add this check right after starting the upload, and 1357548 that will verify compat level at the frontend, but it is not merged yet.
Amit, What do I need to configure in order to support the upload of qcow2 v3 disk images?
AFAIK, In /usr/lib/python2.7/site-packages/vdsm/config.py , you need to set the "qcow2_compat" field in the "irs" section to "1.1", and then restart VDSM. Adam, is this correct?
Amit is right about the config setting but you change it in /etc/vdsm/vdsm.conf and then restart vdsm.
Verified using the following builds: vdsm-4.18.10-1.el7ev.x86_64 ovirt-imageio-daemon-0.3.0-0.el7ev.noarch ovirt-imageio-common-0.3.0-0.el7ev.noarch rhevm-4.0.2.4-0.1.el7ev.noarch ovirt-imageio-proxy-0.3.0-0.el7ev.noarch Test flow: 1. Upload qcow2 (compat = 0.10) to both block and file storage domains 2. Attach disk image to a VM - making sure all files on the disk image are available after mount operation (this works fine). Performed step 1 for qcow2, compat = 1.1 (both block and file storage) - As expected after the upload finishes the disk is displayed with status "Illegal" (vdsm's default configuration supports compat 0.10 and not 1.1). Then, changed the configuration of these files: 1. /usr/lib/python2.7/site-packages/vdsm/config.py under section # Section: [irs] 'qcow2_compat', '1.1', <== was 0.10 2. /etc/vdsm/vdsm.conf Just added the following at the end of the file: [irs] qcow2_compat = 1.1 Restart vdsm. Uploaded 4 disk images: 1. qcow2 compat 1.1 to file storage, result: successful. 2. qcow2 compat 1.1 to block storage, result: successful. 3. qcow2 compat 0.10 to file, result: after the upload switches state to Illegal. 4. qcow2 compat 0.10 to block, result: after the upload switches state to Illegal. Bottom line: works as expected.