Bug 1336655 - Parse the tar header on the configuration volume and download a minimal amount of data
Summary: Parse the tar header on the configuration volume and download a minimal amoun...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.HostedEngine
Version: 4.1.1
Hardware: Unspecified
OS: Unspecified
medium
low
Target Milestone: ovirt-4.1.3
: 4.1.3
Assignee: Andrej Krejcir
QA Contact: Artyom
URL:
Whiteboard:
: 1339321 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-05-17 07:09 UTC by Roy Golan
Modified: 2020-05-28 10:46 UTC (History)
12 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
undefined
Clone Of:
Environment:
Last Closed: 2017-07-06 14:01:27 UTC
oVirt Team: SLA
Embargoed:
rule-engine: ovirt-4.1+
rule-engine: exception+
ylavi: planning_ack+
sbonazzo: devel_ack+
pstehlik: testing_ack+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1480039 0 medium CLOSED Additional HE host deploy fails due to 'received downloaded data size is wrong' 2021-02-22 00:41:40 UTC
oVirt gerrit 61793 0 'None' MERGED jsonrpc: expose Volume.setDescription as setVolumeDescription 2020-09-29 09:37:23 UTC
oVirt gerrit 75490 0 'None' ABANDONED conf: use the system tar to force a custom record size 2020-09-29 09:37:17 UTC
oVirt gerrit 75498 0 'None' ABANDONED conf: use a json description for the configuration volume 2020-09-29 09:37:17 UTC
oVirt gerrit 75499 0 'None' ABANDONED conf: use a json description for the configuration volume 2020-09-29 09:37:23 UTC

Internal Links: 1480039

Description Roy Golan 2016-05-17 07:09:28 UTC
Description of problem:

The disk images created by the setup, specifically the configuration disk and will be in use to as answer file when deploying hosted engine from the engine.

When the engine will download that volume, we need not to download the whole 1GB it takes on the disk but rather the size of the tar image on it. Since we know the size of the tar when we create it, we can put the size in the description, same as what engine does when it serializes OVF into OVF_STORE volumes.

example of OVF volume description:
 {"Updated":true,"Size":10240,"Last Updated":"Wed Apr 20 07:51:46 UTC 2016","Storage Domains":[{"uuid":"fe8f242c-e945-4822-a9f1-ea36d4f56132"}],"Disk Description":"OVF_STORE"}. 

Once the real size is there, we can use RerieveImageCommand with a Range=0-10240 to download the tar with the hosted-engine.conf inside which is 10KiB instead of 1Gib


Actual results:

The description is one word - HostedEngineConfigurationImage with no size and other standard parsable structure by engine

Expected results:
Have the same json structure the engine expects like OVF_STORE in the volume description, specially 'size' . 

Additional info:

Comment 2 Sandro Bonazzola 2016-08-18 06:10:34 UTC
Rafael, can you please update Doc-Text for this bug?

Comment 3 Rafael Martins 2016-08-29 13:22:37 UTC
(In reply to Sandro Bonazzola from comment #2)
> Rafael, can you please update Doc-Text for this bug?

I'm moving this back to ASSIGNED and will see if storage guys can help on this

Comment 4 Yaniv Lavi 2016-09-14 15:06:56 UTC
*** Bug 1339321 has been marked as a duplicate of this bug. ***

Comment 5 Yaniv Kaul 2016-12-01 13:16:04 UTC
(In reply to Rafael Martins from comment #3)
> (In reply to Sandro Bonazzola from comment #2)
> > Rafael, can you please update Doc-Text for this bug?
> 
> I'm moving this back to ASSIGNED and will see if storage guys can help on
> this

Any news?

Comment 6 Simone Tiraboschi 2017-04-14 11:47:21 UTC
Probably the engine has to be fixed as well to correctly detect the disk with json description and one with a simple string for compatibility issues:

https://gerrit.ovirt.org/gitweb?p=ovirt-engine.git;a=blob;f=backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/hostedengine/HostedEngineConfigFetcher.java;h=d4accd062c5fe7ad8263b9b3bd28d51d30cb3ad4;hb=HEAD#l124

Comment 7 Simone Tiraboschi 2017-04-20 09:04:26 UTC
The effort to modify the engine code to extract the tar file size from the image description is comparable with the effort to extract the tar file size from the tar header so duplicating the info doesn't seam worth.
Moving the bug to the engine to have a single fix there.

Comment 8 Martin Sivák 2017-05-03 14:40:05 UTC
Lets do a quick fix for 4.1 here, just define a fixed size to download (a 100KiB or so should be enough). We will have to adapt to different API for 4.2.

Comment 9 Andrej Krejcir 2017-05-15 12:41:18 UTC
The download size is already limited by the configuration value HostedEngineConfigDiskSizeInBytes. By default it is set to 20 KiB.

Comment 10 Artyom 2017-06-04 07:02:15 UTC
Hi, can you please provide verification steps?

Comment 11 Artyom 2017-06-08 11:47:41 UTC
Andrej helped me with verification.
Verified on ovirt-engine-4.1.3.1-0.1.el7.noarch

Comment 12 nijin ashok 2017-07-03 11:49:49 UTC
With the default HostedEngineConfigDiskSizeInBytes of 20480, it can cause issues while validating conf volume archive in RetrieveImageDataVDSCommand . For one of the customer , it's failing with error "received downloaded data size is wrong (requested 20480, received 10240)" while deploying HE using RHV-M. 

===
2017-06-30 03:24:18,866 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.RetrieveImageDataVDSCommand] (ForkJoinPool.commonPool-worker-3) [2707a5e6] Command 'RetrieveImageDataVDSCommand(HostName = =, ImageHttpAccessVDSCommandParameters:{runAsync='true', hostId='949da47b-23c6-4cd1-8509-760f5f9fa316'})' execution failed: VDSGenericException: VDSErrorException: received downloaded data size is wrong (requested 20480, received 10240)
====

We have to manually adjust the  HostedEngineConfigDiskSizeInBytes to 10240 to make this work. 

Is reading the size from tar header still in consideration ? 

I believe conf volume size can vary depending upon the environment and the version where it was created as I can see  new conf parameters added in answer.conf and hosted-engine.conf in newer version compared to the conf volume customer is having.


Note You need to log in before you can comment on or make changes to this bug.