Bug 1273284 - Engine- Support initial allocation size for volume on block storage in order to allow v2v to complete
Engine- Support initial allocation size for volume on block storage in order ...
Status: CLOSED CURRENTRELEASE
Product: ovirt-engine
Classification: oVirt
Component: BLL.Storage (Show other bugs)
3.5.0
Unspecified Unspecified
high Severity high (vote)
: ovirt-3.6.1
: 3.6.1
Assigned To: Fred Rolland
Nisim Simsolo
:
Depends On:
Blocks: 952703 1236075
  Show dependency treegraph
 
Reported: 2015-10-20 03:09 EDT by Fred Rolland
Modified: 2016-02-18 05:55 EST (History)
24 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1221603
Environment:
Last Closed: 2016-02-18 05:55:10 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Storage
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
rule-engine: ovirt‑3.6.z+
ylavi: planning_ack+
rule-engine: devel_ack+
rule-engine: testing_ack+


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 47178 master MERGED engine : Initial size for thin provisioning disk Never
oVirt gerrit 47647 ovirt-engine-3.6 MERGED engine : Initial size for thin provisioning disk Never

  None (edit)
Description Fred Rolland 2015-10-20 03:09:23 EDT
Engine side

+++ This bug was initially created as a clone of Bug #1221603 +++

Description of problem:
Currently when allocating a sparse volume on block based storage, it always allocates 1g on the storage and set the virtual size to the requested volume size.
Now consider the following case: we convert a VM from external system with a 10g virtual size and 5g actual size and the user asks that the target disk will reside in block storage and be sparsed. So we have two options:
1. Keeping the initial allocating to be 1g and then to extend the volume as we write to it. Might be possible, much more complex.
2. To make the initial allocation size configurable so instead of allocating 1g, we will initially allocate the actual size of the disk so virt-v2v will be able to write the converted data to the volume.

#2 seems to be much simpler, just adding a parameter for the initial allocation size to the Volume.create verb.

--- Additional comment from Allon Mureinik on 2015-05-17 10:39:38 EDT ---

Just to get the terminology right - you want to create a QCow with an initial size of 10GB, right?

--- Additional comment from Arik on 2015-05-17 10:48:07 EDT ---

(In reply to Allon Mureinik from comment #1)
> Just to get the terminology right - you want to create a QCow with an
> initial size of 10GB, right?

No, in the example above I want to create a QCow with an intial size of 5GB and that the disk will be marked somehow as one that can grow up to 10GB when the VM starts (with the watermark mechanism).

--- Additional comment from Allon Mureinik on 2015-05-17 11:10:25 EDT ---

Talked to Arik f2f to clarify this. The requirement is to be able to define to properties for a volume - the virtual size and the initial allocation.
(Note this will also be needed for the upload image feature)

So, I can see two options here:
1. Add a parameter to createVolume (so you have virutalSize and initialSize instead of just size). Offhand, this doesn't seem to fit the SDM design.
2. Call createVolume and then allocate

--- Additional comment from Nir Soffer on 2015-08-20 17:44:09 EDT ---

Adding initial size to createVolumeContainer is nice, and is also useful for
cloning disk structure when moving disks. I don't see any conflict with sdm
design.

This can also simplify internal flows in vdsm - instead of creating a volume
and extending it immediately, just create it in the right size. Why perform 
2 lvm operations when one does what you need?

--- Additional comment from Arik on 2015-09-13 04:08:43 EDT ---

This is a must-have capability for 3.6 for the integration with virt-v2v.

--- Additional comment from Allon Mureinik on 2015-09-13 05:36:47 EDT ---

Yaniv, this requires changing the API, which I'm not sure we'd want to do at this point of time.

Let's discuss?

--- Additional comment from Arik on 2015-09-13 05:56:16 EDT ---

(In reply to Allon Mureinik from comment #6)
> Yaniv, this requires changing the API, which I'm not sure we'd want to do at
> this point of time.

The implication of not solving this issue is that customers that use block storage will not be able to set converted disks of VMs that are being imported from VMware to thin-provision.

Allon, I think that if we go with the second solution (the one that Nir commented on in comment 4) we can add the initial capacity as default-parameter so the API will not break. But anyway, even doing it as we talked several months ago (by doing 2 operations in AddDiskCommand) will be ok. I think that since block storage and thin-provision is probably the most popular settings, not supporting this will prevent many from using this new feature.

--- Additional comment from Moran Goldboim on 2015-09-17 05:05:09 EDT ---

(In reply to Arik from comment #7)
> (In reply to Allon Mureinik from comment #6)
> > Yaniv, this requires changing the API, which I'm not sure we'd want to do at
> > this point of time.
> 
> The implication of not solving this issue is that customers that use block
> storage will not be able to set converted disks of VMs that are being
> imported from VMware to thin-provision.
> 
> Allon, I think that if we go with the second solution (the one that Nir
> commented on in comment 4) we can add the initial capacity as
> default-parameter so the API will not break. But anyway, even doing it as we
> talked several months ago (by doing 2 operations in AddDiskCommand) will be
> ok. I think that since block storage and thin-provision is probably the most
> popular settings, not supporting this will prevent many from using this new
> feature.

sounds like this functionality is something we would definitely need for 3.6, Allon what would be the safest way you see to introduce the functionality?

--- Additional comment from Allon Mureinik on 2015-09-17 06:49:36 EDT ---

(In reply to Moran Goldboim from comment #8)
> (In reply to Arik from comment #7)
> > (In reply to Allon Mureinik from comment #6)
> > > Yaniv, this requires changing the API, which I'm not sure we'd want to do at
> > > this point of time.
> > 
> > The implication of not solving this issue is that customers that use block
> > storage will not be able to set converted disks of VMs that are being
> > imported from VMware to thin-provision.
> > 
> > Allon, I think that if we go with the second solution (the one that Nir
> > commented on in comment 4) we can add the initial capacity as
> > default-parameter so the API will not break. But anyway, even doing it as we
> > talked several months ago (by doing 2 operations in AddDiskCommand) will be
> > ok. I think that since block storage and thin-provision is probably the most
> > popular settings, not supporting this will prevent many from using this new
> > feature.
> 
> sounds like this functionality is something we would definitely need for
> 3.6, Allon what would be the safest way you see to introduce the
> functionality?

The SAFEST? Not to introduce it.
:-)

Fred - let's see if we can introduce this (vdsm+engine) in a way that doesn't destabilize the rest of the system.

--- Additional comment from RHEL Product and Program Management on 2015-10-19 07:24:09 EDT ---

This request has been proposed as a blocker, but a release flag has
not been requested. Please set a release flag to ? to ensure we may
track this bug against the appropriate upcoming release, and reset
the blocker flag to ?.

--- Additional comment from Nir Soffer on 2015-10-19 12:57:49 EDT ---

This is vdsm bug - the vdsm side is merged in ovirt-3.6 branch - this should
be MODIFIED now.

The engine patch should be tracked elsewhere.
Comment 1 Allon Mureinik 2015-10-28 10:11:43 EDT
Fred - is this supported externally? Can a user issue a RESTAPI call to create a thinly provisioned disk on a block domain with an initial size allocated?
Comment 2 Fred Rolland 2015-10-29 01:07:16 EDT
It is not supported through RESTAPI. Only Vdsm client.
Comment 3 Yaniv Lavi 2015-10-29 08:46:30 EDT
In oVirt testing is done on single release by default. Therefore I'm removing the 4.0 flag. If you think this bug must be tested in 4.0 as well, please re-add the flag. Please note we might not have testing resources to handle the 4.0 clone.
Comment 4 Nisim Simsolo 2016-01-20 03:04:42 EST
Verified: 
rhevm-3.6.2.5-0.1.el6
sanlock-3.2.4-1.el7.x86_64
vdsm-4.17.17-0.el7ev.noarch
libvirt-client-1.2.17-13.el7_2.2.x86_64
qemu-kvm-rhev-2.3.0-31.el7_2.4.x86_64

VMware Version 6.0.0 Build 2594327

Verification scenario: 
1. Create VMware VM with 16GB iSCSI disk (thin provision type).
2. Install RHEL7 OS on that VM.
3. Create 10G file (copy 10G from /dev/zero to a file).
3. Power off VMware VM and import it using RHEVM, select iSCSI storage domain as a destination disk).
4. After import completed, run both VMs and compare the next things between source VM and imported VM:
Total disk size, virtual size, actual size, number of partitions, partitions size, available space on each partition and 10G files created.
5. Repeat test steps 1-4, this time use NFS storage domain as a destination disk.
6. Repeat test steps 1-4, this time use VMware iSCSI disk, type: Thick provision lazy zeroed.
7. Repeat test steps 1-4, this time use VMware iSCSI disk, type: Thick provision eager zeroed.

Note You need to log in before you can comment on or make changes to this bug.