Description of problem: Raw + Sparse isn't working with iSCSI storage domain. There are IMHO three large problems here, on the oVirt side: (1) We do not know through what combinations of {raw,qcow2} & {sparse,preallocated} are supported by the storage domain. Also we shouldn't have to know this. How does the user know if they should use raw or qcow2? It's a ridiculous detail to have to choose. The storage domain should just do the right thing. (2) imageio works at the wrong level. When the format is qcow2 we shouldn't have to send actual qcow2 bits over the wire. We should (like VDDK) send data blocks and the imageio backend should do the conversion (and possibly sparseness) automatically. You can see why imageio is wrong by some simple thought experiments: What happens if the client generates an old or incompatible variant of qcow2 - eg. qcow2 v0.10; or qcow2 v1.1 when the backend only supports 0.10? What happens if the client sends broken qcow2? What happens if the client sends malicious qcow2? (3) If sparse was requested but only preallocated is available, imageio should fall back to preallocated. Doing this alone would solve the iSCSI problem we are seeing. Version-Release number of selected component (if applicable): RHV 4.2.3.5 How reproducible: Every time Steps to Reproduce: 1. Use virt-v2v CLI or Infrastructure Migration tooling to migrate a VM from VMware to RHV backed by an iSCSI storage domain 2. 3. Actual results: Image upload fails. Expected results: Image upload works. Additional info: Upload works for NFS based storage domain.
Indeed - imageio is the wrong tool to use here. We need something representing some higher order functions, like v2v, which is already in the works.
(This is for general information of virt-v2v users only, it does not address the problem in this bug) https://people.redhat.com/~rjones/virt-v2v-RHEL-7.5-rhv-preview/ virt-v2v >= -6.10.rhvpreview from the repository above removes the restriction on sparse. You can now use ‘-oa preallocated’ on the command line which should work for iSCSI & FC transfers. It doesn't (because it cannot) solve the problem that different storage domains have unknowable requirements which is the focus of this bug.
related to this bug https://bugzilla.redhat.com/show_bug.cgi?id=1574438 Bug 1574438 - [upstream][v2v] migration from VMware to RHV fails
(In reply to Brett Thurber from comment #0) > Description of problem: > Raw + Sparse isn't working with iSCSI storage domain. This combination will never work, as block devices do not support sparseness. > There are IMHO three large problems here, on the oVirt side: > > (1) We do not know through what combinations of {raw,qcow2} & > {sparse,preallocated} are supported by the storage domain. Also we > shouldn't have to know this. How does the user know if they should > use raw or qcow2? It's a ridiculous detail to have to choose. The > storage domain should just do the right thing. oVirt does not have the magic to select the format for the user. We expect the user (oVirt administrator) to select the format based on the needs. We don't plan to select the format for the user via the SDK. We do this in the UI, which should probably be changed to let the user choose the format. > (2) imageio works at the wrong level. When the format is qcow2 we > shouldn't have to send actual qcow2 bits over the wire. We should > (like VDDK) send data blocks and the imageio backend should do the > conversion (and possibly sparseness) automatically. imageio works at the right level to allow uploading qcow2 files directly from the UI. In 4.3 it will be available to upload raw data into a qcow2 image, via nbd integration, planned for incremental backup. > You can see why imageio is wrong by some simple thought experiments: > What happens if the client generates an old or incompatible variant of > qcow2 - eg. qcow2 v0.10; or qcow2 v1.1 when the backend only supports > 0.10? When uploading files via the UI, we detect qcow2 version and block upload of qcow2 version 1.1 to storage domain version < v4. This was needed in the past for sharing storage domain between RHEL 6 and RHEL 7 systems. This is a non-issue now when RHV is supported only on RHEL 7 now. On storage domain v4 and later we support both qcow2 version 0.10 and 1.1. > What happens if the client sends broken qcow2? What happens if > the client sends malicious qcow2? We verify the qcow2 file with qemu-img at the end the upload, and reject the upload if the image is broken. If the image is malicious, it is qemu-img responsibility to handle it. Any image on storage may be modified by malicious user that gain access to the underlying storage, so qemu-img and qemu must be ready to handle such images. > (3) If sparse was requested but only preallocated is available, > imageio should fall back to preallocated. Doing this alone would > solve the iSCSI problem we are seeing. There are no such fallbacks in oVirt and I don't think we should have. The user creates a disk using certain format. Once the user created the disk, imageio cannot change anything. > Actual results: > Image upload fails. No, creating a disk fail - this is not an imageio problem. Creating disks is the responsibility of ovirt-engine. imageio is only about I/O. Please open separate RFE for supporting on-the-fly conversion from raw to qcow2.
> (1) We do not know through what combinations of {raw,qcow2} & > {sparse,preallocated} are supported by the storage domain. The combinations are documented in RHV documentation: https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.1/html-single/administration_guide/index#Understanding_virtual_disks What is missing to use this info is the type of the storage domain, which is not reported in the SDK, but I think an API returning the different combinations is better since the capabilities may change and it does not make sense to keep the table in all clients. Please open new RFE to provide this info if this is needed to improve the user experience of virt-v2v.