Bug 1026351

Summary:	hosted-engine --vm-status reports about hosts that not participating in HA anymore.
Product:	Red Hat Enterprise Virtualization Manager	Reporter:	Leonid Natapov <lnatapov>
Component:	ovirt-hosted-engine-ha	Assignee:	Martin Sivák <msivak>
Status:	CLOSED ERRATA	QA Contact:	Artyom <alukiano>
Severity:	unspecified	Docs Contact:
Priority:	high
Version:	unspecified	CC:	acathrow, dfediuck, iheim, mavital, msivak, pstehlik, sbonazzo
Target Milestone:	---	Keywords:	Triaged
Target Release:	3.3.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:	sla
Fixed In Version:	ovirt-hosted-engine-ha-0.1.0-0.5.1.beta1.el6ev	Doc Type:	Bug Fix
Doc Text:	Cause: Metedata updates from all hosts that ever participated in the hosted engine environment are persisted indefinitely. The timestamp is used to distinguish between old and up-to-date data. Consequence: The administrator had to notice that the unix timestamp of the report is too old when using hosted-engine --vm-status. Fix: The hosted-engine tool was enhanced to explicitly identify the old metadata record as being too old by changing the state to "unknown stale-data" and showing Status up-to-date: False line. Result: The administrator can now easily identify old metadata records.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2014-01-21 16:51:11 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	SLA	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Leonid Natapov 2013-11-04 13:40:28 UTC

I have a host that not participating in HA anymore (host was completely reinstalled) but still reported by hosted-engine. 
The problem is in userspace reporting. We should report stale data differently.

Here is the output of hosted-engine --vm-status. Host green-vdsb is not there anymore but still reported with it's last known status. So you can see two hosts with "vm-up good-health-status" engine status.
-------------------------------------------------
[root@green-vdsa ha_agent]# hosted-engine --vm-status


--== Host 1 status ==--

Hostname                           : green-vdsb.qa.lab.tlv.redhat.com
Host ID                            : 1
Engine status                      : vm-up good-health-status
Score                              : 2400
Host timestamp                     : 1383462493
Extra metadata                     :
	metadata_parse_version=1
	metadata_feature_version=1
	timestamp=1383462493 (Sun Nov  3 09:08:13 2013)
	host-id=1
	score=2400
	bridge=True
	cpu-load=0.24
	engine-health=vm-up good-health-status
	gateway=True
	mem-free=11279
	mem-load=0.00188075982697


--== Host 2 status ==--

Hostname                           : green-vdsa.qa.lab.tlv.redhat.com
Host ID                            : 2
Engine status                      : vm-down
Score                              : 0
Host timestamp                     : 1383572285
Extra metadata                     :
	metadata_parse_version=1
	metadata_feature_version=1
	timestamp=1383572285 (Mon Nov  4 15:38:05 2013)
	host-id=2
	score=0
	bridge=True
	cpu-load=0.265
	engine-health=vm-down
	gateway=True
	mem-free=15468
	mem-load=0.000250767976929


--== Host 3 status ==--

Hostname                           : green-vdsc.qa.lab.tlv.redhat.com
Host ID                            : 3
Engine status                      : vm-up good-health-status
Score                              : 2400
Host timestamp                     : 1383571757
Extra metadata                     :
	metadata_parse_version=1
	metadata_feature_version=1
	timestamp=1383571757 (Mon Nov  4 15:29:17 2013)
	host-id=3
	score=2400
	bridge=True
	cpu-load=0.03125
	engine-health=vm-up good-health-status
	gateway=True
	mem-free=13065
	mem-load=0.00119211946292
------------------------------------------------------------

Comment 1 Martin Sivák 2013-11-04 14:14:23 UTC

This is just how hosted-engine reports stale data. The metadata show the last known state and the timeout the time when it was valid.

We should somehow make it easier to spot stale data to avoid user confusion though.

Comment 2 Doron Fediuck 2013-11-04 18:20:42 UTC

(In reply to Martin Sivák from comment #1)
> This is just how hosted-engine reports stale data. The metadata show the
> last known state and the timeout the time when it was valid.
> 
> We should somehow make it easier to spot stale data to avoid user confusion
> though.

Maybe have a shorter report if age > configurable_value?
ie- Skipping host XXX, last updated on YYY?

Comment 3 Sandro Bonazzola 2013-11-05 10:08:32 UTC

patches merged on upstream master and 1.0 branches.

Comment 5 Artyom 2013-11-08 07:50:52 UTC

Verified on ovirt-hosted-engine-ha-0.1.0-0.5.1.beta1.el6ev.noarch
After remove host from rhevm, set Engine status: unknown stale-data
After erase package ovirt-hosted-engine-ha-0.1.0-0.5.1.beta1.el6ev.noarch from host, set Status up-to-date: False

Comment 6 Charlie 2013-11-28 01:41:40 UTC

This bug is currently attached to errata RHEA-2013:15591. If this change is not to be documented in the text for this errata please either remove it from the errata, set the requires_doc_text flag to 
minus (-), or leave a "Doc Text" value of "--no tech note required" if you do not have permission to alter the flag.

Otherwise to aid in the development of relevant and accurate release documentation, please fill out the "Doc Text" field above with these four (4) pieces of information:

* Cause: What actions or circumstances cause this bug to present.
* Consequence: What happens when the bug presents.
* Fix: What was done to fix the bug.
* Result: What now happens when the actions or circumstances above occur. (NB: this is not the same as 'the bug doesn't present anymore')

Once filled out, please set the "Doc Type" field to the appropriate value for the type of change made and submit your edits to the bug.

For further details on the Cause, Consequence, Fix, Result format please refer to:

https://bugzilla.redhat.com/page.cgi?id=fields.html#cf_release_notes 

Thanks in advance.

Comment 7 errata-xmlrpc 2014-01-21 16:51:11 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2014-0080.html