Bug 1574734 - raw + sparse isn't working with iSCSI storage domain for upload
Summary: raw + sparse isn't working with iSCSI storage domain for upload
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-imageio-daemon
Version: 4.2.3
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ovirt-4.4.0
: ---
Assignee: Daniel Erez
QA Contact: Elad
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-05-03 22:07 UTC by Brett Thurber
Modified: 2018-11-17 19:10 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-11-17 18:51:18 UTC
oVirt Team: Storage
Target Upstream Version:


Attachments (Terms of Use)

Description Brett Thurber 2018-05-03 22:07:16 UTC
Description of problem:
Raw + Sparse isn't working with iSCSI storage domain.

There are IMHO three large problems here, on the oVirt side:

(1) We do not know through what combinations of {raw,qcow2} &
{sparse,preallocated} are supported by the storage domain. Also we
shouldn't have to know this. How does the user know if they should
use raw or qcow2? It's a ridiculous detail to have to choose. The
storage domain should just do the right thing.

(2) imageio works at the wrong level. When the format is qcow2 we
shouldn't have to send actual qcow2 bits over the wire. We should
(like VDDK) send data blocks and the imageio backend should do the
conversion (and possibly sparseness) automatically.

You can see why imageio is wrong by some simple thought experiments:
What happens if the client generates an old or incompatible variant of
qcow2 - eg. qcow2 v0.10; or qcow2 v1.1 when the backend only supports
0.10? What happens if the client sends broken qcow2? What happens if
the client sends malicious qcow2?

(3) If sparse was requested but only preallocated is available,
imageio should fall back to preallocated. Doing this alone would
solve the iSCSI problem we are seeing.


Version-Release number of selected component (if applicable):
RHV 4.2.3.5

How reproducible:
Every time

Steps to Reproduce:
1.  Use virt-v2v CLI or Infrastructure Migration tooling to migrate a VM from VMware to RHV backed by an iSCSI storage domain
2.
3.

Actual results:
Image upload fails.

Expected results:
Image upload works.


Additional info:
Upload works for NFS based storage domain.

Comment 1 Allon Mureinik 2018-05-07 11:47:22 UTC
Indeed - imageio is the wrong tool to use here. We need something representing some higher order functions, like v2v, which is already in the works.

Comment 4 Richard W.M. Jones 2018-05-15 07:58:28 UTC
(This is for general information of virt-v2v users only, it does
not address the problem in this bug)

https://people.redhat.com/~rjones/virt-v2v-RHEL-7.5-rhv-preview/

virt-v2v >= -6.10.rhvpreview from the repository above removes the
restriction on sparse.  You can now use ‘-oa preallocated’ on the
command line which should work for iSCSI & FC transfers.

It doesn't (because it cannot) solve the problem that different
storage domains have unknowable requirements which is the
focus of this bug.

Comment 5 Daniel Gur 2018-05-23 17:17:10 UTC
related to this bug https://bugzilla.redhat.com/show_bug.cgi?id=1574438 
Bug 1574438 - [upstream][v2v] migration from VMware to RHV fails

Comment 6 Nir Soffer 2018-11-17 18:51:18 UTC
(In reply to Brett Thurber from comment #0)
> Description of problem:
> Raw + Sparse isn't working with iSCSI storage domain.

This combination will never work, as block devices do not support sparseness.
 
> There are IMHO three large problems here, on the oVirt side:
> 
> (1) We do not know through what combinations of {raw,qcow2} &
> {sparse,preallocated} are supported by the storage domain. Also we
> shouldn't have to know this. How does the user know if they should
> use raw or qcow2? It's a ridiculous detail to have to choose. The
> storage domain should just do the right thing.

oVirt does not have the magic to select the format for the user.
We expect the user (oVirt administrator) to select the format based on
the needs.

We don't plan to select the format for the user via the SDK. We do this
in the UI, which should probably be changed to let the user choose the format.

> (2) imageio works at the wrong level. When the format is qcow2 we
> shouldn't have to send actual qcow2 bits over the wire. We should
> (like VDDK) send data blocks and the imageio backend should do the
> conversion (and possibly sparseness) automatically.

imageio works at the right level to allow uploading qcow2 files directly from
the UI.

In 4.3 it will be available to upload raw data into a qcow2 image, via nbd
integration, planned for incremental backup.

> You can see why imageio is wrong by some simple thought experiments:
> What happens if the client generates an old or incompatible variant of
> qcow2 - eg. qcow2 v0.10; or qcow2 v1.1 when the backend only supports
> 0.10? 

When uploading files via the UI, we detect qcow2 version and block upload
of qcow2 version 1.1 to storage domain version < v4. This was needed in the
past for sharing storage domain between RHEL 6 and RHEL 7 systems.

This is a non-issue now when RHV is supported only on RHEL 7 now. On storage
domain v4 and later we support both qcow2 version 0.10 and 1.1.

> What happens if the client sends broken qcow2? What happens if
> the client sends malicious qcow2?

We verify the qcow2 file with qemu-img at the end the upload, and reject the 
upload if the image is broken.

If the image is malicious, it is qemu-img responsibility to handle it.
Any image on storage may be modified by malicious user that gain access to the
underlying storage, so qemu-img and qemu must be ready to handle such images.

> (3) If sparse was requested but only preallocated is available,
> imageio should fall back to preallocated. Doing this alone would
> solve the iSCSI problem we are seeing.

There are no such fallbacks in oVirt and I don't think we should have. The user
creates a disk using certain format. Once the user created the disk, imageio
cannot change anything.

> Actual results:
> Image upload fails.

No, creating a disk fail - this is not an imageio problem. Creating disks is 
the responsibility of ovirt-engine. imageio is only about I/O.

Please open separate RFE for supporting on-the-fly conversion from raw to qcow2.

Comment 7 Nir Soffer 2018-11-17 19:10:14 UTC
> (1) We do not know through what combinations of {raw,qcow2} &
> {sparse,preallocated} are supported by the storage domain.

The combinations are documented in RHV documentation:
https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.1/html-single/administration_guide/index#Understanding_virtual_disks

What is missing to use this info is the type of the storage domain, which is not
reported in the SDK, but I think an API returning the different combinations 
is better since the capabilities may change and it does not make sense to keep
the table in all clients.

Please open new RFE to provide this info if this is needed to improve the user
experience of virt-v2v.


Note You need to log in before you can comment on or make changes to this bug.