Bug 1253167

Summary: [RFE] [hosted-engine-setup] Deployment using appliance should not be aborted automatically in case engine is not reachable after engine-setup is completed
Product: [oVirt] ovirt-engine Reporter: Elad <ebenahar>
Component: RFEsAssignee: Balakrishnan, Radhesh <rbalakri>
Status: CLOSED DEFERRED QA Contact: Gil Klein <gklein>
Severity: high Docs Contact:
Priority: unspecified    
Version: 3.6.0CC: bugs, lpeer, lsurette, rbalakri, srevivo, ykaul
Target Milestone: ---Keywords: FutureFeature
Target Release: ---Flags: ylavi: ovirt-future?
rule-engine: planning_ack?
rule-engine: devel_ack?
rule-engine: testing_ack?
Hardware: x86_64   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-11-16 13:43:26 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Integration RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Elad 2015-08-13 07:28:25 UTC
Description of problem:
Tried to deploy hosted-engine over iSCSI using ovirt-appliance. After engine-setup finished, the engine wasn't reachable for a time period few minutes. Therefore, the host wasn't added to the engine and the hosted-engine setup got aborted. 

This behaviour is wrong, the setup should not be aborted in this case. The user should be asked if he wants to continue waiting.


Version-Release number of selected component (if applicable):
ovirt-hosted-engine-setup-1.3.0-0.0.master.20150729070044.git26149d7.el7.noarch
ovirt-engine-appliance-20150802.0-1.el7.centos.noarch

How reproducible:
In case the engine is not reachable for more than around 3 minutes after engine-setup is completed

Steps to Reproduce:
1. Deploy hosted-engine over iSCSI using ovirt-appliance 


Actual results:
Deployment fails with the following message in setup.log:


2015-08-12 19:40:54 DEBUG otopi.plugins.otopi.dialog.human dialog.__logString:218 DIALOG:SEND                 |- HE_APPLIANCE_ENGINE_SETUP_SUCCESS
2015-08-12 19:40:54 DEBUG otopi.plugins.ovirt_hosted_engine_setup.engine.health health._closeup:145 Engine-setup successfully completed 
2015-08-12 19:40:54 INFO otopi.plugins.ovirt_hosted_engine_setup.engine.health health._closeup:146 Engine-setup successfully completed 
2015-08-12 19:40:54 DEBUG otopi.plugins.ovirt_hosted_engine_setup.engine.health appliance_esetup._appliance_disconnect:127 Closing connection to appliance
2015-08-12 19:40:54 DEBUG otopi.ovirt_hosted_engine_setup.check_liveliness check_liveliness.isEngineUp:158 Checking for Engine health status
2015-08-12 19:40:54 INFO otopi.ovirt_hosted_engine_setup.check_liveliness check_liveliness.isEngineUp:180 Engine is still unreachable
2015-08-12 19:40:54 INFO otopi.plugins.ovirt_hosted_engine_setup.engine.health health._closeup:159 Engine is still not reachable, waiting...
2015-08-12 19:41:04 DEBUG otopi.ovirt_hosted_engine_setup.check_liveliness check_liveliness.isEngineUp:158 Checking for Engine health status
2015-08-12 19:41:04 INFO otopi.ovirt_hosted_engine_setup.check_liveliness check_liveliness.isEngineUp:180 Engine is still unreachable
2015-08-12 19:41:04 INFO otopi.plugins.ovirt_hosted_engine_setup.engine.health health._closeup:159 Engine is still not reachable, waiting...
2015-08-12 19:41:14 DEBUG otopi.ovirt_hosted_engine_setup.check_liveliness check_liveliness.isEngineUp:158 Checking for Engine health status
2015-08-12 19:41:14 INFO otopi.ovirt_hosted_engine_setup.check_liveliness check_liveliness.isEngineUp:180 Engine is still unreachable
2015-08-12 19:41:14 INFO otopi.plugins.ovirt_hosted_engine_setup.engine.health health._closeup:159 Engine is still not reachable, waiting...
2015-08-12 19:41:24 DEBUG otopi.ovirt_hosted_engine_setup.check_liveliness check_liveliness.isEngineUp:158 Checking for Engine health status
2015-08-12 19:41:24 INFO otopi.ovirt_hosted_engine_setup.check_liveliness check_liveliness.isEngineUp:180 Engine is still unreachable
2015-08-12 19:41:24 INFO otopi.plugins.ovirt_hosted_engine_setup.engine.health health._closeup:159 Engine is still not reachable, waiting...
2015-08-12 19:41:34 DEBUG otopi.ovirt_hosted_engine_setup.check_liveliness check_liveliness.isEngineUp:158 Checking for Engine health status
2015-08-12 19:41:34 INFO otopi.ovirt_hosted_engine_setup.check_liveliness check_liveliness.isEngineUp:180 Engine is still unreachable
2015-08-12 19:41:34 INFO otopi.plugins.ovirt_hosted_engine_setup.engine.health health._closeup:159 Engine is still not reachable, waiting...
2015-08-12 19:41:44 DEBUG otopi.ovirt_hosted_engine_setup.check_liveliness check_liveliness.isEngineUp:158 Checking for Engine health status
2015-08-12 19:41:44 INFO otopi.ovirt_hosted_engine_setup.check_liveliness check_liveliness.isEngineUp:180 Engine is still unreachable
2015-08-12 19:41:44 INFO otopi.plugins.ovirt_hosted_engine_setup.engine.health health._closeup:159 Engine is still not reachable, waiting...
2015-08-12 19:41:54 DEBUG otopi.ovirt_hosted_engine_setup.check_liveliness check_liveliness.isEngineUp:158 Checking for Engine health status
2015-08-12 19:41:54 INFO otopi.ovirt_hosted_engine_setup.check_liveliness check_liveliness.isEngineUp:180 Engine is still unreachable
2015-08-12 19:41:54 ERROR otopi.plugins.ovirt_hosted_engine_setup.engine.health health._closeup:156 Engine is still not reachable
2015-08-12 19:41:54 DEBUG otopi.context context._executeMethod:155 method exception
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/otopi/context.py", line 145, in _executeMethod
    method['method']()
  File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/ovirt-hosted-engine-setup/engine/health.py", line 157, in _closeup
    raise RuntimeError(_('Engine is still not reachable'))
RuntimeError: Engine is still not reachable
2015-08-12 19:41:54 ERROR otopi.context context._executeMethod:164 Failed to execute stage 'Closing up': Engine is still not reachable
2015-08-12 19:41:54 DEBUG otopi.context context.dumpEnvironment:499 ENVIRONMENT DUMP - BEGIN



Expected results:
The setup should not be aborted in this case. The user should be asked if he wants to continue waiting.

Additional info:

sosreport from host: http://file.tlv.redhat.com/ebenahar/sosreport-green-vdsb.qa.lab.tlv.redhat.com-20150813102357.tar.xz

Logs from engine: http://file.tlv.redhat.com/ebenahar/bug1.tar.gz

Comment 1 Scott Herold 2015-08-21 01:57:06 UTC
RFE flagged for 3.6 after dev freeze.  Moving to rhevm-future.

Comment 2 Yaniv Kaul 2017-11-16 13:43:26 UTC
I don't see the value in waiting. It should be done in 3 minutes. Otherwise, it's a bug or a very very slow storage!