Bug 1261971 - hosted-engine vm-status does note clearly report the engine state on the host where the engine was previously running
hosted-engine vm-status does note clearly report the engine state on the host...
Status: CLOSED NOTABUG
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-hosted-engine-ha (Show other bugs)
3.5.0
x86_64 Linux
medium Severity medium
: ---
: ---
Assigned To: Martin Sivák
Ilanit Stein
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-09-10 10:18 EDT by Anand Nande
Modified: 2017-03-07 06:48 EST (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-09-11 10:14:55 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: SLA
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 1609893 None None None Never

  None (edit)
Description Anand Nande 2015-09-10 10:18:34 EDT
Description of problem: 

"hosted-engine --vm-status" command output does not clearly reply with the state of the engine

Executed on Host1:
---------------------------------------------------------------
[root@ftcstanrhev01 ~]# hosted-engine --vm-status


--== Host 1 status ==--

Status up-to-date                  : True
Hostname                           : ftcstanrhev01.ftcisp.com
Host ID                            : 1
Engine status                      : {"health": "good", "vm": "up", "detail": "up"}
Score                              : 2400
Local maintenance                  : False
Host timestamp                     : 58429
Extra metadata (valid at timestamp):
	metadata_parse_version=1
	metadata_feature_version=1
	timestamp=58429 (Wed Sep  9 08:34:03 2015)
	host-id=1
	score=2400
	maintenance=False
	state=EngineUp


--== Host 2 status ==--

Status up-to-date                  : True
Hostname                           : ftcstanrhev02.ftcisp.com
Host ID                            : 2
Engine status                      : {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}
Score                              : 2400
Local maintenance                  : False
Host timestamp                     : 1510
Extra metadata (valid at timestamp):
	metadata_parse_version=1
	metadata_feature_version=1
	timestamp=1510 (Wed Sep  9 08:34:13 2015)
	host-id=2
	score=2400
	maintenance=False
	state=EngineDown

---------------------------------------------------------------

Executed on Host2:
----------------------------------------------------------------------
[root@ftcstanrhev02 ~]# hosted-engine --vm-status


--== Host 1 status ==--

Status up-to-date                  : True
Hostname                           : ftcstanrhev01.ftcisp.com
Host ID                            : 1
Engine status                      : {"health": "good", "vm": "up", "detail": "up"}
Score                              : 2400
Local maintenance                  : False
Host timestamp                     : 58439
Extra metadata (valid at timestamp):
	metadata_parse_version=1
	metadata_feature_version=1
	timestamp=58439 (Wed Sep  9 08:34:14 2015)
	host-id=1
	score=2400
	maintenance=False
	state=EngineUp


--== Host 2 status ==--

Status up-to-date                  : True
Hostname                           : ftcstanrhev02.ftcisp.com
Host ID                            : 2
Engine status                      : {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}
Score                              : 2400
Local maintenance                  : False
Host timestamp                     : 1510
Extra metadata (valid at timestamp):
	metadata_parse_version=1
	metadata_feature_version=1
	timestamp=1510 (Wed Sep  9 08:34:13 2015)
	host-id=2
	score=2400
	maintenance=False
	state=EngineDown

----------------------------------------------------------------------

the state EngineDown and EngineUp are confusing on the hosts.

Reading through the output, I see that engineVM is now running on Host1.
This was previously running on Host2.

Looking at the "hosted-engine --vm-status" output, I think that the older HA metadata was not cleared.
Rather this is a disability of the hosted engine HA broker to update the metadata on older hosts about the 
engine state.



Version-Release number of selected component (if applicable): rhev-3.5


How reproducible: always


Steps to Reproduce:
1. set local maintenance on the host1 where the engine-VM is running
2. the engine moves to host2. 
3. set local-maintenance on host-2. enginevm moves to host1. Then check the output of "hosted-engine --vm-status" on both the hosts.

Actual results: "state:EngineUp" is confusing 

Expected results: Should clearly report the engineUP state only on the host where the engineVM is running "currently"


Additional info:
Comment 1 Martin Sivák 2015-09-10 11:40:38 EDT
I am not sure I see what you describe. When I remove all unnecessary lines I get the following:


[root@ftcstanrhev01 ~]# hosted-engine --vm-status

--== Host 1 status ==--
Hostname                           : ftcstanrhev01.ftcisp.com
Engine status                      : {"health": "good", "vm": "up", "detail": "up"}
state=EngineUp

--== Host 2 status ==--
Hostname                           : ftcstanrhev02.ftcisp.com
Engine status                      : {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}
state=EngineDown



[root@ftcstanrhev02 ~]# hosted-engine --vm-status

--== Host 1 status ==--
Hostname                           : ftcstanrhev01.ftcisp.com
Engine status                      : {"health": "good", "vm": "up", "detail": "up"}
state=EngineUp

--== Host 2 status ==--
Hostname                           : ftcstanrhev02.ftcisp.com
Engine status                      : {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}
state=EngineDown




Both reports are correct and show that the engine is running on ftcstanrhev01.ftcisp.com (hosted engine id 1) - the VM is UP there and the ovirt-engine service is healthy. The VM is DOWN on host 2 as expected, there can be only one engine VM running.

Also all reports show that the status is up-to-date and both reports agree with each other.

The state= line in the extra section reports some internal data (the state machine state) about the host and those lines also agree with the data in the main block (Engine status).
Comment 2 Anand Nande 2015-09-11 08:08:26 EDT
Thanks - please close this bug.

Note You need to log in before you can comment on or make changes to this bug.