Bug 734120 - [RFE] use virt-sparsify to reduce image size
[RFE] use virt-sparsify to reduce image size
Status: CLOSED CURRENTRELEASE
Product: ovirt-engine
Classification: oVirt
Component: RFEs (Show other bugs)
---
Unspecified Unspecified
high Severity high with 1 vote (vote)
: ovirt-4.1.0-beta
: 4.1.0.2
Assigned To: Shmuel Melamud
sefi litmanovich
http://www.ovirt.org/documentation/ad...
: FutureFeature
Depends On: 1277705
Blocks: 1420197
  Show dependency treegraph
 
Reported: 2011-08-29 09:50 EDT by Federico Simoncelli
Modified: 2017-03-16 10:49 EDT (History)
19 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
With this update, the ability to sparsify a thin-provisioned disk has been added to Red Hat Virtualization. When a virtual machine is shut down the user can sparsify the disk to convert the free space within the disk image back to free space on the host.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-03-16 10:49:32 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Virt
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
eheftman: needinfo-
rule-engine: ovirt‑4.1+
bmcclain: priority_rfe_tracking+
gklein: testing_plan_complete+
mgoldboi: planning_ack+
michal.skrivanek: devel_ack+
mavital: testing_ack+


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 28328 None None None Never
oVirt gerrit 57347 master NEW storage: Add inplace virt-sparsify support 2016-09-13 10:34 EDT
oVirt gerrit 60960 master NEW core: Added SparsifyImageCommand 2016-09-18 13:00 EDT
oVirt gerrit 62858 master NEW ui: Add Sparsify button to VM Disks subtab 2016-09-13 09:40 EDT
oVirt gerrit 67023 master MERGED restapi: Implement Disk.sparsify REST API action 2016-11-21 03:24 EST
oVirt gerrit 72049 master MERGED ui: Added 4.2 to the list of NFS versions 2017-02-14 09:57 EST

  None (edit)
Description Federico Simoncelli 2011-08-29 09:50:55 EDT
Description of problem:
VDSM should use zerofree to reduce the image size before cloning disks and (probably) collapsing disks.
Comment 2 Richard W.M. Jones 2012-01-05 06:57:08 EST
(In reply to comment #0)
> Description of problem:
> VDSM should use zerofree to reduce the image size before cloning disks and
> (probably) collapsing disks.

I would recommend using virt-sparsify instead or in addition ...

http://libguestfs.org/virt-sparsify.1.html

Zerofree really needs to be properly supported by our filesystem
team before we can start using it with customers.  That's the
reason why it's not in RHEL at the moment.  There is large scope
for it to do very bad things to customer data.

Also virt-sparsify, although it's much less efficient than
zerofree, has the advantage that it works with any type of
filesystem.  In particular with NTFS, so it can sparsify Windows
guests, which obviously zerofree (being ext2/3-based) cannot.

I'm planning to put virt-sparsify in RHEL 6.3.
Comment 3 Itamar Heim 2013-03-12 05:47:11 EDT
Rich - any updated thinking/status on this?
Comment 4 Richard W.M. Jones 2013-03-12 10:18:30 EDT
The current status is that virt-sparsify (the "copying" version
of it) is in RHEL 6.4, RHEL 7 and Fedora everything.

In future we want to have a version of virt-sparsify which
works in-place.  This is something that might happen in RHEL 7.1.
It requires changes to qemu, kernel and more, see:

http://libguestfs.org/guestfs-faq.1.html#why-doesnt-virt-sparsify-work-on-the-disk-image-in-place-

HTH
Comment 5 Richard W.M. Jones 2013-03-12 10:20:46 EDT
(In reply to comment #4)
> http://libguestfs.org/guestfs-faq.1.html#why-doesnt-virt-sparsify-work-on-
> the-disk-image-in-place-

Grrr, this link doesn't work unless you manually add a '-' to the
end of it ...
Comment 6 Itamar Heim 2013-12-01 05:02:10 EST
thoughts on impact vs. effort?
Comment 7 Ayal Baron 2013-12-01 07:21:22 EST
Impact -
After a while of usage probably >10% possibly >50% (i.e. cut usage down by half!).

Effort - since this is a cold process and it starts by copying all the data from source to target then in our case the flow would be:
1. create a target disk
2. virt-sparsify (in case of error, delete target)
3. if all went well, the rest depends on the use case.
there are several use cases:
 A. Attach to VM and detach original disk (copy all vm-disk properties) - to allow user to test the new disk.  User would have to manually delete the old disk.  Need to make sure they cannot be attached concurrently to avoid conflicts.
 B. Make the disk into a template (no point in copying again) - i.e. virt-sparsify would be a checkbox when creating template
 C. Keep as floating disk to be used with another VM (i.e. virt-sparsify would be a checkbox in clone disk operation or something).
Probably a lot more.

Effort depends on which use cases we would like to implement.  The one for same VM is probably the most complex due to the logic around the existing copy vs. new one.
Comment 8 Richard W.M. Jones 2013-12-01 07:27:47 EST
(In reply to Ayal Baron from comment #7)
> Effort - since this is a cold process and it starts by copying all the data
> from source to target

It depends on the backing store being used, but starting with RHEL 7
it should be possible to make virt-sparsify work entirely in-place.
(Note that no one's actually done that: virt-sparsify in RHEL 7.0
still works by copying ... just saying it's possible).

Adding Pino Toscano to CC as he is working on libguestfs things.
Comment 9 Ayal Baron 2013-12-01 09:43:21 EST
(In reply to Richard W.M. Jones from comment #8)
> (In reply to Ayal Baron from comment #7)
> > Effort - since this is a cold process and it starts by copying all the data
> > from source to target
> 
> It depends on the backing store being used, but starting with RHEL 7
> it should be possible to make virt-sparsify work entirely in-place.
> (Note that no one's actually done that: virt-sparsify in RHEL 7.0
> still works by copying ... just saying it's possible).

Not sure we'd want to do this considering the scope of changes and the risk of loosing data...
However, it would be nice to split this out of virt-sparsify to allow doing more clever things (e.g. instead of full copy, use storage based snapshot which would be instantaneous and would not take up much space).


> 
> Adding Pino Toscano to CC as he is working on libguestfs things.
Comment 10 Allon Mureinik 2015-03-05 13:05:04 EST
The current patches, both those merges in VDSM and those not yet merged in the engine assume an SPM.
It would be a shame to invest work in doing something that will have to be re-written in a couple of weeks/months.
Let's wait till we have a proper SPM-less solution and then move on.
Comment 13 Michal Skrivanek 2016-03-23 07:29:42 EDT
We want to continue development, but due to RHEL 7.3 dependency to make it useful I'm postponing this to 4.1.
Shahar, also note the need to properly lock the whole chain while the operation is in progress
Comment 14 Michal Skrivanek 2016-03-23 07:31:34 EDT
we're going with virt-sparsify only, majority of remaining work is on engine side
Comment 15 Moran Goldboim 2016-11-30 07:51:16 EST
Michal, do we have a recommendation on when to perform this sparsification? for example actual size is bigger than the virtual size.
Comment 16 Shmuel Melamud 2016-11-30 08:03:24 EST
I would say, when virtual disk on storage takes significantly more space than used space shown by df command.
Comment 17 Richard W.M. Jones 2016-11-30 08:05:11 EST
And of course you can determine that using virt-df:
http://libguestfs.org/virt-df.1.html
Comment 18 sefi litmanovich 2016-12-11 09:05:05 EST
Tried to test this feature on current upstream and couldn't get any results.
According to smelamud in order for the feature to work libguestfs should have rw permissions to access libvirtd - If this is the case then we should either configure this from our side or make sure in documentation of the feature to explain what and how should be configured in order for this feature to work.
Comment 19 Pino Toscano 2016-12-12 05:08:22 EST
(In reply to sefi litmanovich from comment #18)
> Tried to test this feature on current upstream and couldn't get any results.

What did you try? How did you check that the sparsification yield any result?
You can use `du -s` to check the actual size taken by a file -- usually raw files are sparse.

> According to smelamud in order for the feature to work libguestfs should
> have rw permissions to access libvirtd

libguestfs needs a rw connection to be able to launch its own VMs using libvirt; you can always pass your own connection URI to libguestfs, using the -c parameter: -c qemu:///system, for example. If that is not feasible, you can always change the backend to the "direct" one (exporting LIBGUESTFS_BACKEND=direct).
libguestfs in RHEL 7+ and Fedora 19+ is built to use libvirt by default to launch its own VMs.
Comment 20 Michal Skrivanek 2016-12-12 07:10:16 EST
see section about "Sparsifying a Virtual Disk" in the admin guide
Comment 21 Sven Kieske 2016-12-12 07:15:24 EST
From the man page of virt-sparsify:

IMPORTANT LIMITATIONS

    The virtual machine must be shut down before using this tool.


Am I misreading the feature, or shouldn't this warning be added to:

http://www.ovirt.org/documentation/admin-guide/administration-guide/

?

Thanks

Sven
Comment 22 Richard W.M. Jones 2016-12-12 08:18:46 EST
(In reply to Sven Kieske from comment #21)
> From the man page of virt-sparsify:
> 
> IMPORTANT LIMITATIONS
> 
>     The virtual machine must be shut down before using this tool.

This is absolutely true.  However doesn't ovirt itself enforce
this condition?  (I certainly hope so ...)
Comment 23 Shmuel Melamud 2016-12-12 08:28:45 EST
(In reply to Richard W.M. Jones from comment #22)
> >     The virtual machine must be shut down before using this tool.
> 
> This is absolutely true.  However doesn't ovirt itself enforce
> this condition?  (I certainly hope so ...)

Enforces, for sure.
Comment 24 sefi litmanovich 2016-12-12 09:48:05 EST
(In reply to Pino Toscano from comment #19)
> (In reply to sefi litmanovich from comment #18)
> > Tried to test this feature on current upstream and couldn't get any results.
> 
> What did you try? How did you check that the sparsification yield any result?
> You can use `du -s` to check the actual size taken by a file -- usually raw
> files are sparse.
> 
I used du -sh on the storage domain in the image's folder.

> > According to smelamud in order for the feature to work libguestfs should
> > have rw permissions to access libvirtd
> 
> libguestfs needs a rw connection to be able to launch its own VMs using
> libvirt; you can always pass your own connection URI to libguestfs, using
> the -c parameter: -c qemu:///system, for example. If that is not feasible,
> you can always change the backend to the "direct" one (exporting
> LIBGUESTFS_BACKEND=direct).
> libguestfs in RHEL 7+ and Fedora 19+ is built to use libvirt by default to
> launch its own VMs.

If that's the prerequisite for this feature to work then we should make sure that this is configured as expected on the host, or add this as a comment in feature's documentation
Comment 25 Richard W.M. Jones 2016-12-12 10:02:24 EST
To be clear, DO NOT use LIBGUESTFS_BACKEND=direct unless you fully
understand the implications of that.  libguestfs is already used by
oVirt for various purposes and it seems to have no problem getting
a r/w libvirt connection for those, so likely the problem you are
seeing is something else.

You said:

> Tried to test this feature on current upstream and couldn't get any results.

but I don't even understand what that means.  What did you try?  What
error message did you see?
Comment 26 sefi litmanovich 2016-12-12 10:30:42 EST
I tried to use the new 'sparsify' button added in this patch to ovirt-engine.
What I get is that action is successful but took 0.01 secs so I guess nothing was done or there was some uncought error or something:

from vdsm log:

2016-12-12 17:05:48,521 INFO  (jsonrpc/0) [dispatcher] Run and protect: sdm_sparsify_volume(job_id=u'a245582e-eed4-4359-9d98-1d22b15faf19', vol_info={u'img_id': u'928afe69-b268-4d12-8979-c38b7c1b4da4', u'sd_id': u'5fd07c1e-d75a-4fbb-9cf5-
a6aefaa712f1', u'vol_id': u'3cffb204-d1f5-49de-a0c9-656a2b0d22a2'}) (logUtils:49)
2016-12-12 17:05:48,523 INFO  (jsonrpc/0) [dispatcher] Run and protect: sdm_sparsify_volume, Return response: None (logUtils:52)
2016-12-12 17:05:48,525 INFO  (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call SDM.sparsify_volume succeeded in 0.01 seconds (__init__:515)

After this I was talking with smelamud and tried to run 'libguestfs-test-tool' on my host and I get:

libguestfs: opening libvirt handle: URI = qemu:///system, auth = default+wrapper, flags = 0
libvirt needs authentication to connect to libvirt URI qemu:///system
(see also: http://libvirt.org/auth.html http://libvirt.org/uri.html)
Please enter your authentication name: 

I do not know the authentication name and password.

I now succeeded running 'libguestfs-test-tool' after setting:
auth_unix_rw="none"
in libvirtd.conf and export LIBGUESTFS_BACKEND=direct.
then restarting libvirtd.

But when I tried the sparsify option in the engine again I cannot see any change at file size using du -sh, vdsm reports successful run after 0.01 secs as it did the first time.
Comment 27 Richard W.M. Jones 2016-12-12 10:37:21 EST
(In reply to sefi litmanovich from comment #26)
> I tried to use the new 'sparsify' button added in this patch to ovirt-engine.
> What I get is that action is successful but took 0.01 secs so I guess
> nothing was done or there was some uncought error or something:
> 
> from vdsm log:
> 
> 2016-12-12 17:05:48,521 INFO  (jsonrpc/0) [dispatcher] Run and protect:
> sdm_sparsify_volume(job_id=u'a245582e-eed4-4359-9d98-1d22b15faf19',
> vol_info={u'img_id': u'928afe69-b268-4d12-8979-c38b7c1b4da4', u'sd_id':
> u'5fd07c1e-d75a-4fbb-9cf5-
> a6aefaa712f1', u'vol_id': u'3cffb204-d1f5-49de-a0c9-656a2b0d22a2'})
> (logUtils:49)
> 2016-12-12 17:05:48,523 INFO  (jsonrpc/0) [dispatcher] Run and protect:
> sdm_sparsify_volume, Return response: None (logUtils:52)
> 2016-12-12 17:05:48,525 INFO  (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call
> SDM.sparsify_volume succeeded in 0.01 seconds (__init__:515)

I'm not totally familiar with how this is implemented in VDSM.  Maybe
this is just sending the start message over to the SDM where virt-sparsify
actually is run.

> After this I was talking with smelamud and tried to run
> 'libguestfs-test-tool' on my host and I get:
> 
> libguestfs: opening libvirt handle: URI = qemu:///system, auth =
> default+wrapper, flags = 0
> libvirt needs authentication to connect to libvirt URI qemu:///system
> (see also: http://libvirt.org/auth.html http://libvirt.org/uri.html)
> Please enter your authentication name: 
> 
> I do not know the authentication name and password.

This bit is not relevant.  What you need to do is to run 'libguestfs-test-tool'
on the SDM node, and run it as exactly the same user as VDSM with the same
environment etc.  Which is generally quite difficult for end users to do.

> I now succeeded running 'libguestfs-test-tool' after setting:
> auth_unix_rw="none"
> in libvirtd.conf and export LIBGUESTFS_BACKEND=direct.
> then restarting libvirtd.

These settings are contradictory anyway.  LIBGUESTFS_BACKEND=direct
says "don't use libvirt", so changing libvirt's configuration makes
no difference.  In any case VDSM does use libvirt, and we want to keep
it that way, see earlier comment.

> But when I tried the sparsify option in the engine again I cannot see any
> change at file size using du -sh, vdsm reports successful run after 0.01
> secs as it did the first time.

So that's obviously a problem.

I think you need to track down the log file from the SDM or SPM node to find out
what really happened.
Comment 30 Emma Heftman 2017-02-20 11:36:31 EST
Shmuel, in order to document this feature, please list all limitations that should appear in the documentation.
From what I have read, this should include 

1. a list of NFS types supported and not supported.
2. Whether libguestfs should have rw permissions to access libvirtd or whether this has been preconfigured.

I would also like to understand whether this was implemented as a button in the UI, as the feature page suggested.
Comment 31 Emma Heftman 2017-02-21 10:21:27 EST
Shmuel, in addition to my questions in Comment 30, please let me know whether Sparsify is supported in Storage Domains defined as Posix.
Comment 32 Shmuel Melamud 2017-02-22 08:51:03 EST
(In reply to emma heftman from comment #30)
Unfortunately, the feature page for virt-sparsify is very outdated. I've wrote a new text, but it is still not approved. You can read it in my pull request here: https://github.com/oVirt/ovirt-site/pull/772/commits/7e4d76881572210f70f1ee21ca1b1e6d406ec92f

Also I've demonstrated the feature and how it is implemented in Deep Dive: https://www.youtube.com/watch?v=ayseKlGLwHI&feature=share

NFS supported is 4.2 or newer.

libguestfs must have rw permissions to access libvirtd, without it virt-sparsify will not work. And yes, this should be configured automatically on host deploy, but I didn't verify this.

The feature is implemented as "Sparsify" button on "Disks" subtab of "Virtual Machines" tab.

About POSIX Storage Domain I'll answer later, need to recheck this.
Comment 33 Emma Heftman 2017-02-22 10:31:32 EST
(In reply to Shmuel Melamud from comment #32)
> (In reply to emma heftman from comment #30)
> Unfortunately, the feature page for virt-sparsify is very outdated. I've
> wrote a new text, but it is still not approved. You can read it in my pull
> request here:
> https://github.com/oVirt/ovirt-site/pull/772/commits/
> 7e4d76881572210f70f1ee21ca1b1e6d406ec92f
> 
> Also I've demonstrated the feature and how it is implemented in Deep Dive:
> https://www.youtube.com/watch?v=ayseKlGLwHI&feature=share
> 
> NFS supported is 4.2 or newer.
> 
> libguestfs must have rw permissions to access libvirtd, without it
> virt-sparsify will not work. And yes, this should be configured
> automatically on host deploy, but I didn't verify this.
> 
> The feature is implemented as "Sparsify" button on "Disks" subtab of
> "Virtual Machines" tab.
> 
> About POSIX Storage Domain I'll answer later, need to recheck this.

Shmuel, I removed the needinfo from this bug and added it to the documentation bug: 

https://bugzilla.redhat.com/show_bug.cgi?id=1420197

there you can see Yaniv's answer and the remaining open issues.
Comment 34 sefi litmanovich 2017-03-16 05:49:06 EDT
It was decided that this RFE shouldn't be blocked by bz#1425161 which is a corner case.
The basic tier-1 flows of this RFE are working fine and users can consume the feature easily.

Test run:
https://polarion.engineering.redhat.com/polarion/#/project/RHEVM3/testrun?id=16032017

Note You need to log in before you can comment on or make changes to this bug.