Bug 734120 - [RFE] use virt-sparsify to reduce image size
[RFE] use virt-sparsify to reduce image size
Status: NEW
Product: ovirt-engine
Classification: oVirt
Component: RFEs (Show other bugs)
---
Unspecified Unspecified
high Severity high with 1 vote (vote)
: ovirt-4.1.0-alpha
: ---
Assigned To: Shmuel Melamud
meital avital
: FutureFeature
Depends On: 1277705
Blocks:
  Show dependency treegraph
 
Reported: 2011-08-29 09:50 EDT by Federico Simoncelli
Modified: 2016-07-04 05:29 EDT (History)
13 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Virt
RHEL 7.3 requirements from Atomic Host:
michal.skrivanek: ovirt‑4.1.0?
rule-engine: planning_ack?
michal.skrivanek: devel_ack+
rule-engine: testing_ack?


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 28328 None None None Never
oVirt gerrit 57347 master NEW storage: Add inplace virt-sparsify support 2016-07-18 08:49 EDT

  None (edit)
Description Federico Simoncelli 2011-08-29 09:50:55 EDT
Description of problem:
VDSM should use zerofree to reduce the image size before cloning disks and (probably) collapsing disks.
Comment 2 Richard W.M. Jones 2012-01-05 06:57:08 EST
(In reply to comment #0)
> Description of problem:
> VDSM should use zerofree to reduce the image size before cloning disks and
> (probably) collapsing disks.

I would recommend using virt-sparsify instead or in addition ...

http://libguestfs.org/virt-sparsify.1.html

Zerofree really needs to be properly supported by our filesystem
team before we can start using it with customers.  That's the
reason why it's not in RHEL at the moment.  There is large scope
for it to do very bad things to customer data.

Also virt-sparsify, although it's much less efficient than
zerofree, has the advantage that it works with any type of
filesystem.  In particular with NTFS, so it can sparsify Windows
guests, which obviously zerofree (being ext2/3-based) cannot.

I'm planning to put virt-sparsify in RHEL 6.3.
Comment 3 Itamar Heim 2013-03-12 05:47:11 EDT
Rich - any updated thinking/status on this?
Comment 4 Richard W.M. Jones 2013-03-12 10:18:30 EDT
The current status is that virt-sparsify (the "copying" version
of it) is in RHEL 6.4, RHEL 7 and Fedora everything.

In future we want to have a version of virt-sparsify which
works in-place.  This is something that might happen in RHEL 7.1.
It requires changes to qemu, kernel and more, see:

http://libguestfs.org/guestfs-faq.1.html#why-doesnt-virt-sparsify-work-on-the-disk-image-in-place-

HTH
Comment 5 Richard W.M. Jones 2013-03-12 10:20:46 EDT
(In reply to comment #4)
> http://libguestfs.org/guestfs-faq.1.html#why-doesnt-virt-sparsify-work-on-
> the-disk-image-in-place-

Grrr, this link doesn't work unless you manually add a '-' to the
end of it ...
Comment 6 Itamar Heim 2013-12-01 05:02:10 EST
thoughts on impact vs. effort?
Comment 7 Ayal Baron 2013-12-01 07:21:22 EST
Impact -
After a while of usage probably >10% possibly >50% (i.e. cut usage down by half!).

Effort - since this is a cold process and it starts by copying all the data from source to target then in our case the flow would be:
1. create a target disk
2. virt-sparsify (in case of error, delete target)
3. if all went well, the rest depends on the use case.
there are several use cases:
 A. Attach to VM and detach original disk (copy all vm-disk properties) - to allow user to test the new disk.  User would have to manually delete the old disk.  Need to make sure they cannot be attached concurrently to avoid conflicts.
 B. Make the disk into a template (no point in copying again) - i.e. virt-sparsify would be a checkbox when creating template
 C. Keep as floating disk to be used with another VM (i.e. virt-sparsify would be a checkbox in clone disk operation or something).
Probably a lot more.

Effort depends on which use cases we would like to implement.  The one for same VM is probably the most complex due to the logic around the existing copy vs. new one.
Comment 8 Richard W.M. Jones 2013-12-01 07:27:47 EST
(In reply to Ayal Baron from comment #7)
> Effort - since this is a cold process and it starts by copying all the data
> from source to target

It depends on the backing store being used, but starting with RHEL 7
it should be possible to make virt-sparsify work entirely in-place.
(Note that no one's actually done that: virt-sparsify in RHEL 7.0
still works by copying ... just saying it's possible).

Adding Pino Toscano to CC as he is working on libguestfs things.
Comment 9 Ayal Baron 2013-12-01 09:43:21 EST
(In reply to Richard W.M. Jones from comment #8)
> (In reply to Ayal Baron from comment #7)
> > Effort - since this is a cold process and it starts by copying all the data
> > from source to target
> 
> It depends on the backing store being used, but starting with RHEL 7
> it should be possible to make virt-sparsify work entirely in-place.
> (Note that no one's actually done that: virt-sparsify in RHEL 7.0
> still works by copying ... just saying it's possible).

Not sure we'd want to do this considering the scope of changes and the risk of loosing data...
However, it would be nice to split this out of virt-sparsify to allow doing more clever things (e.g. instead of full copy, use storage based snapshot which would be instantaneous and would not take up much space).


> 
> Adding Pino Toscano to CC as he is working on libguestfs things.
Comment 10 Allon Mureinik 2015-03-05 13:05:04 EST
The current patches, both those merges in VDSM and those not yet merged in the engine assume an SPM.
It would be a shame to invest work in doing something that will have to be re-written in a couple of weeks/months.
Let's wait till we have a proper SPM-less solution and then move on.
Comment 13 Michal Skrivanek 2016-03-23 07:29:42 EDT
We want to continue development, but due to RHEL 7.3 dependency to make it useful I'm postponing this to 4.1.
Shahar, also note the need to properly lock the whole chain while the operation is in progress
Comment 14 Michal Skrivanek 2016-03-23 07:31:34 EDT
we're going with virt-sparsify only, majority of remaining work is on engine side

Note You need to log in before you can comment on or make changes to this bug.