Bug 1258786

Summary: [hosted-engine-setup] Deployment over iSCSI using RHEVM-appliance fails with endless 'WARNING otopi.plugins.ovirt_hosted_engine_setup.vm.image image._disk_customization:124 Not enough free space' messages
Product: Red Hat Enterprise Virtualization Manager Reporter: rhev-integ
Component: ovirt-hosted-engine-setupAssignee: Simone Tiraboschi <stirabos>
Status: CLOSED ERRATA QA Contact: Elad <ebenahar>
Severity: high Docs Contact:
Priority: unspecified    
Version: 3.5.4CC: aburden, acanan, ecohen, fdeutsch, gklein, lsurette, sbonazzo, ylavi
Target Milestone: ovirt-3.5.5Keywords: ZStream
Target Release: 3.5.5Flags: ylavi: Triaged+
Hardware: x86_64   
OS: Unspecified   
Whiteboard: integration
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Previously, hosted-engine deployment using the RHEV-M Virtual Appliance over iSCSI storage would fail with an endless error message if the destination storage had insufficient space. Now, this scenario raises an appropriate error and fails gracefully.
Story Points: ---
Clone Of: 1254104 Environment:
Last Closed: 2015-10-26 21:38:54 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1254104    
Bug Blocks: 1263137    

Description rhev-integ 2015-09-01 09:44:47 UTC
+++ This bug is a RHEV-M zstream clone. The original bug is: +++
+++   https://bugzilla.redhat.com/show_bug.cgi?id=1254104. +++
+++ Requested by "sbonazzo" +++
======================================================================



----------------------------------------------------------------------
Following comment by ebenahar on August 17 at 07:43:45, 2015

Description of problem:
Tried to deploy hosted-engine over iSCSI using RHEVM-appliance. During the deployment, I was informed that the OVF does not contain a valid image description and that it will use the default one.
Then the setup failed with the following message that appeared endlessly:

2015-08-17 10:21:05 WARNING otopi.plugins.ovirt_hosted_engine_setup.vm.image image._disk_customization:124 Not enough free space, about 46 GiB are available

Version-Release number of selected component (if applicable):
ovirt-hosted-engine-setup-1.2.5.2-1.el7ev.noarch
vdsm-4.16.24-2.el7ev.x86_64

How reproducible:
While using RHEVM-appliance that doesn't have a valid image description 

Steps to Reproduce:
1. Deploy hosted-engine using RHEVM-appliance over a LUN exposed by iSCSI (51G LUN in my case)


Actual results:

2015-08-17 10:21:33 WARNING otopi.plugins.ovirt_hosted_engine_setup.vm.image image._disk_customization:124 Not enough free space, about 46 GiB are available
2015-08-17 10:21:33 WARNING otopi.plugins.ovirt_hosted_engine_setup.vm.image image._disk_customization:124 Not enough free space, about 46 GiB are available
2015-08-17 10:21:33 WARNING otopi.plugins.ovirt_hosted_engine_setup.vm.image image._disk_customization:124 Not enough free space, about 46 GiB are available
2015-08-17 10:21:33 WARNING otopi.plugins.ovirt_hosted_engine_setup.vm.image image._disk_customization:124 Not enough free space, about 46 GiB are available
2015-08-17 10:21:33 WARNING otopi.plugins.ovirt_hosted_engine_setup.vm.image image._disk_customization:124 Not enough free space, about 46 GiB are available
2015-08-17 10:21:33 WARNING otopi.plugins.ovirt_hosted_engine_setup.vm.image image._disk_customization:124 Not enough free space, about 46 GiB are available
2015-08-17 10:21:33 WARNING otopi.plugins.ovirt_hosted_engine_setup.vm.image image._disk_customization:124 Not enough free space, about 46 GiB are available
2015-08-17 10:21:33 WARNING otopi.plugins.ovirt_hosted_engine_setup.vm.image image._disk_customization:124 Not enough free space, about 46 GiB are available
2015-08-17 10:21:33 WARNING otopi.plugins.ovirt_hosted_engine_setup.vm.image image._disk_customization:124 Not enough free space, about 46 GiB are available


2 issues here:
1) Deployment failed with the mentioned message. The message says that there is not enough free space, about 46G are available. 46G should be enough for hosted-engine image.
2) The message was thrown endlessly.

Expected results:
Deployment using RHEVM-appliance using iSCSI should succeed while having 46G available in the used LUN. Also, the message should not appear endlessly.

Additional info:
sosreport: http://file.tlv.redhat.com/ebenahar/sosreport-green-vdsb.qa.lab.tlv.redhat.com-20150817103641.tar.xz

----------------------------------------------------------------------
Following comment by stirabos on August 27 at 09:58:11, 2015

(In reply to Elad from comment #0)
> 2 issues here:
> 1) Deployment failed with the mentioned message. The message says that there
> is not enough free space, about 46G are available. 46G should be enough for
> hosted-engine image.

This one was indeed correct: from the log I see that the appliance was sized for 50 GB
2015-08-17 10:20:08 DEBUG otopi.context context.dumpEnvironment:500 ENV OVEHOSTED_STORAGE/imgSizeGB=int:'50'

And the storage domain structures on a block device requires something less than 5 GB so the free space on a 51G LUN after the creation of an empty storage domain is just about 46 GB which is not enough to deploy a 50 GB large appliance.
Please see: https://bugzilla.redhat.com/1163088

> 2) The message was thrown endlessly.

Were you running it not interactively via answerfile?
On interactive setup it should loop letting you choose a different storage device while answerfile is always provinding the same answers and so the infinite loop.
We should detect it and raise breaking the loop.

----------------------------------------------------------------------
Following comment by ebenahar on August 27 at 12:10:51, 2015

(In reply to Simone Tiraboschi from comment #1)
> (In reply to Elad from comment #0)
> > 2 issues here:
> > 1) Deployment failed with the mentioned message. The message says that there
> > is not enough free space, about 46G are available. 46G should be enough for
> > hosted-engine image.
> 
> This one was indeed correct: from the log I see that the appliance was sized
> for 50 GB
> 2015-08-17 10:20:08 DEBUG otopi.context context.dumpEnvironment:500 ENV
> OVEHOSTED_STORAGE/imgSizeGB=int:'50'
> 
> And the storage domain structures on a block device requires something less
> than 5 GB so the free space on a 51G LUN after the creation of an empty
> storage domain is just about 46 GB which is not enough to deploy a 50 GB
> large appliance.
> Please see: https://bugzilla.redhat.com/1163088
> 
> > 2) The message was thrown endlessly.
> 
> Were you running it not interactively via answerfile?
> On interactive setup it should loop letting you choose a different storage
> device while answerfile is always provinding the same answers and so the
> infinite loop.
> We should detect it and raise breaking the loop.

I deployed interactively. I got this message infinitely, without the ability to answer it. It was thrown several times per second as you can see in the timestamps in the logs.

----------------------------------------------------------------------
Following comment by stirabos on August 27 at 13:13:51, 2015

(In reply to Elad from comment #2)
> I deployed interactively. I got this message infinitely, without the ability
> to answer it. It was thrown several times per second as you can see in the
> timestamps in the logs.

Ok, sorry: if you used an appliance it takes the image size from the OVF so, regarding this specific question, it's always not interactively.
The posted patch will solve as well.

Comment 2 Elad 2015-10-07 14:47:24 UTC
A wrong disk size error message is raised once and the setup fails nicely.

2015-10-07 17:44:56 ERROR otopi.context context._executeMethod:161 Failed to execute stage 'Environment customization': Not enough free space, about 40 GiB are available


Verified using vt17.1 ovirt-hosted-engine-setup-1.2.6-1.el7ev.noarch

Comment 4 errata-xmlrpc 2015-10-26 21:38:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-1941.html