Bug 1274065 - [RFE] Hosted-Engine: make OVA extraction faster
[RFE] Hosted-Engine: make OVA extraction faster
Status: NEW
Product: ovirt-hosted-engine-setup
Classification: oVirt
Component: RFEs (Show other bugs)
---
Unspecified Unspecified
high Severity medium (vote)
: ---
: ---
Assigned To: Simone Tiraboschi
meital avital
: FutureFeature
Depends On: 1198507
Blocks:
  Show dependency treegraph
 
Reported: 2015-10-21 15:51 EDT by Fabian Deutsch
Modified: 2017-11-16 13:38 EST (History)
11 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Integration
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
fdeutsch: ovirt‑future?
rule-engine: planning_ack?
rule-engine: devel_ack?
rule-engine: testing_ack?


Attachments (Terms of Use)

  None (edit)
Description Fabian Deutsch 2015-10-21 15:51:56 EDT
Description of problem:
When using the appliance flow in HE setup, the extraction is taking a large portion of the setup time.

Version-Release number of selected component (if applicable):
3.6

How reproducible:
always

Steps to Reproduce:
1. Use appliance flow in
2.
3.

Actual results:
Extraction process takes long

Expected results:
Extraction process is quick

Additional info:
Comment 1 Fabian Deutsch 2015-10-22 07:35:22 EDT
Raising the priority, because the flow could be nice, but is really slowed down due to this bug.

Ryan, could you quantify the slow down compared to the whole installation duration?
Comment 2 Simone Tiraboschi 2015-10-22 07:42:03 EDT
(In reply to Fabian Deutsch from comment #1)
> Raising the priority, because the flow could be nice, but is really slowed
> down due to this bug.
> 
> Ryan, could you quantify the slow down compared to the whole installation
> duration?

It's not really a bug, it's an RFE: it could be faster but it's working.
The issue is about python not efficiently handling sparse files so the real gain depends just from the image sparseness.
Comment 3 Fabian Deutsch 2015-10-22 07:47:25 EDT
Agreed, it's an RFE.
Comment 4 Yaniv Kaul 2015-10-22 15:10:01 EDT
Are we running virt-sparsify and virt-sysprep on the image before packing it?
Comment 5 Fabian Deutsch 2017-01-27 04:51:28 EST
Yes, IIRC.

The files should be sparse, but as Simone says: Python does not handle those efficiently when extracting tars.
Comment 6 Yaniv Kaul 2017-11-16 08:42:09 EST
Simone, the new installation flow in 4.2 makes a difference here?
Comment 7 Simone Tiraboschi 2017-11-16 09:14:21 EST
(In reply to Yaniv Kaul from comment #6)
> Simone, the new installation flow in 4.2 makes a difference here?

We are using system tar via ansible with --sparse option:
https://github.com/oVirt/ovirt-hosted-engine-setup/blob/master/src/ansible/bootstrap_local_vm.yml#L36

On my test system with a 7200 rpm disk is taking about 50 seconds to extract a 2.4 qcow2 sparse image from a 800M ova file:

 2017-11-16 12:14:51,291+0100 INFO otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils.v2_playbook_on_task_start:164 TASK [Extract appliance to local vm dir]
 2017-11-16 12:15:41,756+0100 INFO otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils.v2_runner_on_ok:120 changed: [localhost]
 
 [root@c74he20171031h1 ~]# ls -lh /usr/share/ovirt-engine-appliance/ovirt-engine-appliance-4.2-20171114.1.el7.centos.ova
 -rw-r--r--. 1 root root 808M 14 nov 14.59 /usr/share/ovirt-engine-appliance/ovirt-engine-appliance-4.2-20171114.1.el7.centos.ova
 [root@c74he20171031h1 ~]# du -h /var/tmp/localvm/images/d1746233-6cf3-42b8-9efb-b481954a8d3f/d7cdd432-9bc6-47f6-a6d8-73340e59647b
 2,4G	/var/tmp/localvm/images/d1746233-6cf3-42b8-9efb-b481954a8d3f/d7cdd432-9bc6-47f6-a6d8-73340e59647b
 [root@c74he20171031h1 ~]# file /var/tmp/localvm/images/d1746233-6cf3-42b8-9efb-b481954a8d3f/d7cdd432-9bc6-47f6-a6d8-73340e59647b
 /var/tmp/localvm/images/d1746233-6cf3-42b8-9efb-b481954a8d3f/d7cdd432-9bc6-47f6-a6d8-73340e59647b: QEMU QCOW Image (v3), 53687091200 bytes

If we want to improve, I think we should evaluate avoiding the RPM(OVA(QCOW2)) packaging and just ship the qcow2 disk in an rpm file using it in place with a snapshot to revert on issues.
Comment 8 Yaniv Kaul 2017-11-16 13:38:54 EST
Alternatively we should use the RHEL cloud image, and virt-customize it on the spot with latest Engine, etc. 
Should take a lot more time, but would reduce the initial size and ensure we use the latest-greatest.

Note You need to log in before you can comment on or make changes to this bug.