Bug 1531071 - [RFE] OVA export: Decrease OVA export file size
Summary: [RFE] OVA export: Decrease OVA export file size
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.Virt
Version: 4.2.0.2
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: ---
Assignee: Shmuel Melamud
QA Contact: Nisim Simsolo
URL:
Whiteboard:
Depends On:
Blocks: 1981095
TreeView+ depends on / blocked
 
Reported: 2018-01-04 14:08 UTC by Nisim Simsolo
Modified: 2022-04-10 08:26 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1981095 (view as bug list)
Environment:
Last Closed: 2022-04-10 08:26:47 UTC
oVirt Team: Virt
Embargoed:
pm-rhel: ovirt-4.5?
rule-engine: planning_ack?
rule-engine: devel_ack?
rule-engine: testing_ack?


Attachments (Terms of Use)

Description Nisim Simsolo 2018-01-04 14:08:00 UTC
Description of problem:
Currently, when exporting VM as OVA, the tar file in not compressed, for example:
Importing OVA from external provider with file size of 691M and afterward exporting it as OVA, creates OVA file of 1.8G.
It would be nice if OVA export file will be compressed in order to save disk space.

Version-Release number of selected component (if applicable):
ovirt-engine-4.2.1-0.0.master.20180102220038.git8f941b3.el7.centos.noarch
vdsm-4.20.9-95.git2b98daf.el7.centos.x86_64
qemu-kvm-common-ev-2.9.0-16.el7_4.11.1.x86_64
virt-v2v-1.36.3-6.el7_4.3.x86_64

Comment 1 RHEL Program Management 2020-06-04 13:44:28 UTC
This request has been proposed for two releases. This is invalid flag usage. The ovirt-future release flag has been cleared. If you wish to change the release flag, you must clear one release flag and then set the other release flag to ?.

Comment 2 Tomáš Golembiovský 2020-06-17 16:24:57 UTC
The OVF specification is explicit in that the: 

485  A compressed OVF package shall be created by using the TAR format that complies with the USTAR
486  (Uniform Standard Tape Archive) format as defined by the ISO/IEC/IEEE 9945:20

"Compressed" in this context means that it is a single file, not that the size is reduced by any compression algorithm.

The fact that other vendors and individuals use gzip on the TAR file or using ZIP compression (and ignore the TAR requirement altogether) does not mean we should deviate from the specs too. If we would like to make the generated OVA smaller we may consider writing sparse QCOW instead of RAW files.

Comment 3 Arik 2020-07-06 11:50:31 UTC
(In reply to Tomáš Golembiovský from comment #2)
> The fact that other vendors and individuals use gzip on the TAR file or
> using ZIP compression (and ignore the TAR requirement altogether) does not
> mean we should deviate from the specs too.

+1

> If we would like to make the generated OVA smaller we may consider writing
> sparse QCOW instead of RAW files.

We actually convert each disk to a single QCOW volume when generating the OVA.
We thought about using QCOW compression but there's a trade-off between the output size and the time it takes to export/import (assuming that on the storage domain we don't want these volumes to be compressed). With lower compression values, the benefit appeared to be negligible. With higher compression values, it took much longer to export to an OVA. So if think about using QCOW compression, we probably want to make it configurable.

Comment 4 Arik 2021-06-14 12:06:37 UTC
(In reply to Arik from comment #3)
> > If we would like to make the generated OVA smaller we may consider writing
> > sparse QCOW instead of RAW files.
> 
> We actually convert each disk to a single QCOW volume when generating the
> OVA.
> We thought about using QCOW compression but there's a trade-off between the
> output size and the time it takes to export/import (assuming that on the
> storage domain we don't want these volumes to be compressed). With lower
> compression values, the benefit appeared to be negligible. With higher
> compression values, it took much longer to export to an OVA. So if think
> about using QCOW compression, we probably want to make it configurable.

More things to think about:
1. What should be the default compression rate
2. What to do on import-from-OVA - whether to keep the volumes compressed to decompress them

Comment 5 Michal Skrivanek 2021-08-20 07:53:57 UTC
is still doesn't look really clear what to do here. Is it really committed to?

Comment 6 Arik 2021-08-20 08:16:19 UTC
(In reply to Michal Skrivanek from comment #5)
> is still doesn't look really clear what to do here. Is it really committed
> to?

Yes, we have a plan for this

Comment 7 Arik 2021-08-20 09:36:31 UTC
and in some more details - 
to make the compression level configurable, the default will be determined after inspecting the overhead once the writes are aligned to 4K blocks (which also includes the overhead on the 'measure' command)
when extracting the OVA, the disks would be decompressed to the created volumes in the storage domain

Comment 8 Sandro Bonazzola 2022-03-29 16:16:40 UTC
We are past 4.5.0 feature freeze, please re-target.

Comment 9 Michal Skrivanek 2022-04-08 16:55:55 UTC
didn't make it into 4.5, untargetting

Comment 10 Arik 2022-04-10 08:26:47 UTC
This appeared to be more difficult than we thought -
We current measure the images and based on that create a skeleton that is aligned to 4k blocks, then qemu-img writes the converted image(s) to the right place(s)
It is possible to instruct qemu-img convert to compress the converted images but then their size would be different than the measured size and the structure of the OVA file would have to change in order to produce a smaller OVA file. This requires a big change.
If one wants to save disk space, it would be simpler to compress the entire OVA file and decompress it before importing the VM/template. Adding this to RHV/oVirt is debatable not only because it would make the export-to-OVA operation slower but also because the created OVA won't conform the OVF specification. It can be automated on the client side instead.


Note You need to log in before you can comment on or make changes to this bug.