To enable fast zero on upload image, 'sparse' flag should be added to the image ticket. See bug 1615122 for more information on imageio side.
Engine should mark tickets as sparse when the underlying disk is considered as thin and storage is file based. Engine must not mark a ticket as sparse for block storage. We may limit this restriction in the future by checking if the user selected wipe-after-delete for the underlying disk. Idan, do we have an easy way to detect a disk on block storage that can be safely discarded during upload?
Yes, a disk can be discarded if it's not marked with WAD and its storage domain supports discard.
(In reply to Nir Soffer from comment #1) > Engine should mark tickets as sparse when the underlying disk is considered > as thin > and storage is file based. > > Engine must not mark a ticket as sparse for block storage. We may limit this > restriction in the future by checking if the user selected wipe-after-delete > for the underlying disk. > > Idan, do we have an easy way to detect a disk on block storage that can be > safely > discarded during upload? @Nir - we can check if wipe-after-delete is selected on the disk (we have it also in upload flow). Should we do it now or just allow sparse for thin disk on file domain?
(In reply to Daniel Erez from comment #3) > @Nir - we can check if wipe-after-delete is selected on the disk (we have it > also in upload flow). Should we do it now or just allow sparse for thin disk > on file domain? I would defer this to another RFE, since on imageio side we don't support yet trim for block storage, and I don't want to risk this feature for 4.2.6.
(In reply to Nir Soffer from comment #4) > (In reply to Daniel Erez from comment #3) > > @Nir - we can check if wipe-after-delete is selected on the disk (we have it > > also in upload flow). Should we do it now or just allow sparse for thin disk > > on file domain? > > I would defer this to another RFE, since on imageio side we don't support yet > trim for block storage, and I don't want to risk this feature for 4.2.6. ok, opened an RFE for that: https://bugzilla.redhat.com/show_bug.cgi?id=1619006
Daniel, can you please provide the scenario for verification? Thanks
(In reply to Elad from comment #6) > Daniel, can you please provide the scenario for verification? > Thanks We currently only use it in v2v flow, i.e. sparse is not supported using engine api/ui. So the best way to verify it is by checking AddImageTicket in the log (e.g. [*]). It should be true for thin disks on file domain. [*] INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.AddImageTicketVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [b32b5c96-6d02-4eac-acad-b90520c1b43b] START, AddImageTicketVDSCommand(HostName = host1, AddImageTicketVDSCommandParameters:{hostId='0e7696d2-c45a-4436-ac2c-392581f56e99', ticketId='0fa87884-f87a-4c58-951b-237bace1b6a6', timeout='300', operations='[write]', size='46137344', url='file:///rhev/data-center/mnt/derezengine.usersys:_home_data_nfs_data1/8f3a6be6-c65e-4640-91aa-31580fe91262/images/5a2203bc-395b-4ba1-8bbb-f3118ed48b5c/4b5e174d-118c-45d0-ade3-67355b862977', filename='null', sparse='true'}), log id: 40a62d65
Even better, check that ovirt-imageio-daemon got a ticket with "sparse": True. 1. Start upload from the UI to NFS storage domain 2. Look in daemon log for the ADD log message for this transfer: 2018-08-19 10:15:47,714 INFO (Thread-1) [tickets] [local] ADD ticket={u'url': u'file:///rhev/data-center/mnt/blockSD/bee83d01-aab7-4581-921f-d0145235c757/images/b2f50c31-c878-4a1b-96ed-010060e6ca49/f96d3922-03cc-4169-ae9f-6e37d520a30a', u'ops': [u'write'], u'uuid': u'4fe94bbe-db25-465c-a2a5-e83e254882b0', u'timeout': 300, u'size': 107374182400} There should be a u'sparse': True key in the ticket dict. Checking that imageio does the right thing for sparse image should be done at bug 1615122. To check the entire flow, we need the imageio client, supporting sparse uploads.
Does download sprase require a different RFE?
(In reply to Yaniv Lavi from comment #9) > Does download sprase require a different RFE? Yes, should open a new RFE for it.
Thanks Yaniv and Daniel.Veritas will open a new RFE for this. Best, Pavan.
Verified using: ovirt-engine-4.2.6.4-0.0.master.20180823141941.gitb3a5b18.el7.noarch vdsm-4.20.38-1.gitb49b34d.el7.x86_64 Uploaded the next image using WebAdmin: # qemu-img info temp_img.qcow2 image: temp_img.qcow2 file format: qcow2 virtual size: 10G (10737418240 bytes) disk size: 50M cluster_size: 65536 Format specific information: compat: 0.10 refcount bits: 16 From engine log: 2018-08-28 11:50:34,337+03 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.AddImageTicketVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-87) [e6365c22-a317-4f08-8d8d-728b248b867f] START, AddImageTicketVDSCommand(HostName = host_mixed_2, AddImageTicketVDSCommandParameters:{hostId='e03db7f3-8d22-4644-b6b3-e1df54d294f7', ticketId='b6339314-e880-4765-bf93-fc3abefd2ed1', timeout='300', operations='[write]', size='10737418240', url='file:///rhev/data-center/mnt/yellow-vdsb.qa.lab.tlv.redhat.com:_Storage__NFS_storage__local__ge16__nfs__1/45c4a7f6-394e-4583-92c8-1bd73ce43dac/images/5d6afe53-0f21-4d7c-81e4-2acc7f2305b7/efe38bf9-47fb-4858-b457-1e7f6d89c30d', filename='null', sparse='true'}), log id: 739c7704 From daemon log: 2018-08-28 11:50:34,373 INFO (Thread-1) [tickets] [local] ADD ticket={u'uuid': u'b6339314-e880-4765-bf93-fc3abefd2ed1', u'ops': [u'write'], u'url': u'file:///rhev/data-center/mnt/yellow-vdsb.qa.lab.tlv.redhat.com:_Storage__NFS_storage__local__ge16__nfs__1/45c4a7f6-394e-4583-92c8-1bd73ce43dac/images/5d6afe53-0f21-4d7c-81e4-2acc7f2305b7/efe38bf9-47fb-4858-b457-1e7f6d89c30d', u'sparse': True, u'timeout': 300, u'size': 10737418240}
QE verification bot: the bug was verified upstream