Bug 1391859 - [RFE] When creating thick allocated disks on file-based storage, use '-o preallocation=falloc' instead of 'dd' with zeros.
Summary: [RFE] When creating thick allocated disks on file-based storage, use '-o prea...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.Storage
Version: 4.0.4
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: ovirt-4.2.0
: 4.2.0
Assignee: Denis Chaplygin
QA Contact: Kevin Alon Goldblatt
URL:
Whiteboard:
Depends On:
Blocks: 1429286
TreeView+ depends on / blocked
 
Reported: 2016-11-04 09:23 UTC by Sahina Bose
Modified: 2021-09-09 12:01 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
New feature greatly reduces the time required to create thick allocated disks on file-based storage.
Clone Of:
Environment:
Last Closed: 2017-12-20 11:17:40 UTC
oVirt Team: Storage
Embargoed:
rule-engine: ovirt-4.2+
ratamir: testing_plan_complete-
ylavi: planning_ack+
rule-engine: devel_ack+
ratamir: testing_ack+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHV-43438 0 None None None 2021-09-09 12:01:06 UTC
oVirt gerrit 69848 0 master MERGED qemuimg: Add preallocation support to qemuimg.create() 2020-06-19 23:59:15 UTC
oVirt gerrit 70882 0 master MERGED volume: Use fallocate for prellocation of file-based volumes 2020-06-19 23:59:15 UTC
oVirt gerrit 70883 0 master MERGED storage: Add helper for posix_fallocate() call 2020-06-19 23:59:15 UTC
oVirt gerrit 70884 0 master MERGED volume: Implemented preallocated volume resizing with fallocate 2020-06-19 23:59:15 UTC

Description Sahina Bose 2016-11-04 09:23:10 UTC
Description of problem:

Preallocation of images is slow as zero'es are written to the image that is created. "qemu-img create" supports flags 'falloc', 'metadata' and 'full'
Users should be provided option to pass these flags while preallocating the images

Version-Release number of selected component (if applicable):
4.0

How reproducible:
NA

Steps to Reproduce:
NA

Comment 1 Yaniv Kaul 2016-11-06 06:27:53 UTC
From mailing list discussions seems the real value is in switching our 'dd' implementation to using 'falloc' in the qemu-img create command line. It is not exactly a 1:1 mapping (the fact the file system marked entries as allocated has nothing to do with what the underlying storage has performed) - but so is the case with writing zeros...

The metadata allocation for qcow2 is discussed elsewhere (but also, unless file-based, may be of little value unfortunately).

Comment 2 Yaniv Lavi 2016-11-07 12:17:01 UTC
This RFE has come in very late. Can you give some background on this request?
Do you expect this to happen for 4.1?

Comment 3 Sahina Bose 2016-11-07 15:31:26 UTC
(In reply to Yaniv Dary from comment #2)
> This RFE has come in very late. Can you give some background on this request?
> Do you expect this to happen for 4.1?

Customer was experiencing poor performance in pre-allocating disk images and on digging further, we realise the preallocation flags are not used.

If not feasible for 4.1, please move it to another release

Comment 7 Raz Tamir 2017-05-25 18:35:51 UTC
Hi Sahina,

How can we test that this new option to pass flags to qemu-img?

Comment 8 Sahina Bose 2017-07-07 09:03:28 UTC
(In reply to Raz Tamir from comment #7)
> Hi Sahina,
> 
> How can we test that this new option to pass flags to qemu-img?

I don't this fix uses qemu-img create, but calls posix_fallocate to preallocate.
Denis can provide more info

Comment 9 Nir Soffer 2017-07-12 11:06:39 UTC
Raz, we should test this feature by timing creation of a new preallocated disk on
file storage.

The result differ, based on the underlying file system:
- NFS 3,4,4.1 - use emulation, should be about 1.5X faster compared with older 
  version using dd (we cannot guarantee exact number).
- NFS 4.2,glusterfs,localfs - should take no time, using fallocate()

This is basically a user experience thing, so we should test this from the point of
view of a user.

Comment 10 Denis Chaplygin 2017-07-12 14:32:27 UTC
(In reply to Sahina Bose from comment #8)
> (In reply to Raz Tamir from comment #7)
> > Hi Sahina,
> > 
> > How can we test that this new option to pass flags to qemu-img?
> 
> I don't this fix uses qemu-img create, but calls posix_fallocate to
> preallocate.


Yes, we use posix_fallocate here. Nir provided a good approach to test that issue.

Comment 15 Kevin Alon Goldblatt 2017-08-17 14:37:43 UTC
(In reply to Denis Chaplygin from comment #14)
> Could you please try nfsv4, glusterfs or any local fs? Nfsv3 doesn't
> supports that syscall.
> 
> It definitely works with gluster, so i recommend verifying on it.

Created the same 10G with Glusterfs and the latest code:


>>>>>> It took 7 seconds


Verified with the following code:
---------------------------------------------------
ovirt-engine-4.2.0-0.0.master.20170813134654.gitaee967b.el7.centos.noarch
vdsm-4.20.2-77.gite43f776.el7.centos.x86_64


Moving to VERIFIED!

Comment 16 Sandro Bonazzola 2017-12-20 11:17:40 UTC
This bugzilla is included in oVirt 4.2.0 release, published on Dec 20th 2017.

Since the problem described in this bug report should be
resolved in oVirt 4.2.0 release, published on Dec 20th 2017, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.