Bug 1254104

Summary:	[hosted-engine-setup] Deployment over iSCSI using RHEVM-appliance fails with endless 'WARNING otopi.plugins.ovirt_hosted_engine_setup.vm.image image._disk_customization:124 Not enough free space' messages
Product:	Red Hat Enterprise Virtualization Manager	Reporter:	Elad <ebenahar>
Component:	ovirt-hosted-engine-setup	Assignee:	Simone Tiraboschi <stirabos>
Status:	CLOSED ERRATA	QA Contact:	Elad <ebenahar>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	3.5.4	CC:	acanan, fdeutsch, gklein, lsurette, sbonazzo, ykaul, ylavi
Target Milestone:	ovirt-3.6.0-rc	Keywords:	ZStream
Target Release:	3.6.0
Hardware:	x86_64
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:	Previously, hosted engine deployment over iSCSI using the RHEVM-appliance failed with an endless loop if the destination storage had insufficient space. This has now been fixed.	Story Points:	---
Clone Of:
Clones:	1258786 (view as bug list)		Environment:
Last Closed:	2016-03-09 19:14:24 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	Integration	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1258786

Description Elad 2015-08-17 07:43:45 UTC

Description of problem:
Tried to deploy hosted-engine over iSCSI using RHEVM-appliance. During the deployment, I was informed that the OVF does not contain a valid image description and that it will use the default one.
Then the setup failed with the following message that appeared endlessly:

2015-08-17 10:21:05 WARNING otopi.plugins.ovirt_hosted_engine_setup.vm.image image._disk_customization:124 Not enough free space, about 46 GiB are available

Version-Release number of selected component (if applicable):
ovirt-hosted-engine-setup-1.2.5.2-1.el7ev.noarch
vdsm-4.16.24-2.el7ev.x86_64

How reproducible:
While using RHEVM-appliance that doesn't have a valid image description 

Steps to Reproduce:
1. Deploy hosted-engine using RHEVM-appliance over a LUN exposed by iSCSI (51G LUN in my case)


Actual results:

2015-08-17 10:21:33 WARNING otopi.plugins.ovirt_hosted_engine_setup.vm.image image._disk_customization:124 Not enough free space, about 46 GiB are available
2015-08-17 10:21:33 WARNING otopi.plugins.ovirt_hosted_engine_setup.vm.image image._disk_customization:124 Not enough free space, about 46 GiB are available
2015-08-17 10:21:33 WARNING otopi.plugins.ovirt_hosted_engine_setup.vm.image image._disk_customization:124 Not enough free space, about 46 GiB are available
2015-08-17 10:21:33 WARNING otopi.plugins.ovirt_hosted_engine_setup.vm.image image._disk_customization:124 Not enough free space, about 46 GiB are available
2015-08-17 10:21:33 WARNING otopi.plugins.ovirt_hosted_engine_setup.vm.image image._disk_customization:124 Not enough free space, about 46 GiB are available
2015-08-17 10:21:33 WARNING otopi.plugins.ovirt_hosted_engine_setup.vm.image image._disk_customization:124 Not enough free space, about 46 GiB are available
2015-08-17 10:21:33 WARNING otopi.plugins.ovirt_hosted_engine_setup.vm.image image._disk_customization:124 Not enough free space, about 46 GiB are available
2015-08-17 10:21:33 WARNING otopi.plugins.ovirt_hosted_engine_setup.vm.image image._disk_customization:124 Not enough free space, about 46 GiB are available
2015-08-17 10:21:33 WARNING otopi.plugins.ovirt_hosted_engine_setup.vm.image image._disk_customization:124 Not enough free space, about 46 GiB are available


2 issues here:
1) Deployment failed with the mentioned message. The message says that there is not enough free space, about 46G are available. 46G should be enough for hosted-engine image.
2) The message was thrown endlessly.

Expected results:
Deployment using RHEVM-appliance using iSCSI should succeed while having 46G available in the used LUN. Also, the message should not appear endlessly.

Additional info:
sosreport: http://file.tlv.redhat.com/ebenahar/sosreport-green-vdsb.qa.lab.tlv.redhat.com-20150817103641.tar.xz

Comment 1 Simone Tiraboschi 2015-08-27 09:58:11 UTC

(In reply to Elad from comment #0)
> 2 issues here:
> 1) Deployment failed with the mentioned message. The message says that there
> is not enough free space, about 46G are available. 46G should be enough for
> hosted-engine image.

This one was indeed correct: from the log I see that the appliance was sized for 50 GB
2015-08-17 10:20:08 DEBUG otopi.context context.dumpEnvironment:500 ENV OVEHOSTED_STORAGE/imgSizeGB=int:'50'

And the storage domain structures on a block device requires something less than 5 GB so the free space on a 51G LUN after the creation of an empty storage domain is just about 46 GB which is not enough to deploy a 50 GB large appliance.
Please see: https://bugzilla.redhat.com/1163088

> 2) The message was thrown endlessly.

Were you running it not interactively via answerfile?
On interactive setup it should loop letting you choose a different storage device while answerfile is always provinding the same answers and so the infinite loop.
We should detect it and raise breaking the loop.

Comment 2 Elad 2015-08-27 12:10:51 UTC

(In reply to Simone Tiraboschi from comment #1)
> (In reply to Elad from comment #0)
> > 2 issues here:
> > 1) Deployment failed with the mentioned message. The message says that there
> > is not enough free space, about 46G are available. 46G should be enough for
> > hosted-engine image.
> 
> This one was indeed correct: from the log I see that the appliance was sized
> for 50 GB
> 2015-08-17 10:20:08 DEBUG otopi.context context.dumpEnvironment:500 ENV
> OVEHOSTED_STORAGE/imgSizeGB=int:'50'
> 
> And the storage domain structures on a block device requires something less
> than 5 GB so the free space on a 51G LUN after the creation of an empty
> storage domain is just about 46 GB which is not enough to deploy a 50 GB
> large appliance.
> Please see: https://bugzilla.redhat.com/1163088
> 
> > 2) The message was thrown endlessly.
> 
> Were you running it not interactively via answerfile?
> On interactive setup it should loop letting you choose a different storage
> device while answerfile is always provinding the same answers and so the
> infinite loop.
> We should detect it and raise breaking the loop.

I deployed interactively. I got this message infinitely, without the ability to answer it. It was thrown several times per second as you can see in the timestamps in the logs.

Comment 3 Simone Tiraboschi 2015-08-27 13:13:51 UTC

(In reply to Elad from comment #2)
> I deployed interactively. I got this message infinitely, without the ability
> to answer it. It was thrown several times per second as you can see in the
> timestamps in the logs.

Ok, sorry: if you used an appliance it takes the image size from the OVF so, regarding this specific question, it's always not interactively.
The posted patch will solve as well.

Comment 6 Elad 2015-11-09 09:09:59 UTC

In case a LUN which its size is not sufficient for hosted-engine deployment using appliance is picked, the setup is aborted nicely and not enters to a loop.

[WARNING] Not enough free space, about 46 GiB are available
[ ERROR ] Failed to execute stage 'Environment customization': Not enough free space, about 46 GiB are available
[ INFO  ] Stage: Clean up
[ INFO  ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20151109070741.conf'
[ INFO  ] Stage: Pre-termination
[ INFO  ] Stage: Termination


Verified using ovirt-hosted-engine-setup-1.3.0-1.el7ev.noarch

Comment 8 errata-xmlrpc 2016-03-09 19:14:24 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-0375.html