Bug 1394570

Summary:	[HE] Engine VM automatically migrates to host on local maintenance mode
Product:	[oVirt] ovirt-hosted-engine-ha	Reporter:	Gonza <grafuls>
Component:	Agent	Assignee:	Denis Chaplygin <dchaplyg>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Nikolai Sednev <nsednev>
Severity:	high	Docs Contact:
Priority:	high
Version:	2.0.4	CC:	bugs, dchaplyg, dfediuck, gklein, mavital, michal.skrivanek, ylavi
Target Milestone:	ovirt-4.1.1	Keywords:	Triaged
Target Release:	2.1.0.4	Flags:	rule-engine: ovirt-4.0.z+ rule-engine: ovirt-4.1+ ylavi: planning_ack+ dfediuck: devel_ack+ mavital: testing_ack+
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:	Cause: Lack of check for other HE hosts statuses. Consequence: HE agent tried to move VM from current host, ignoring status of other HE hosts, thus migrating VM to host under maintenance. Fix: Before entering maintenance mode HE agent (and engine itself, in case maintenance mode is activated using engine UI) checks, that there are hosts available, capable to run engine VM. Result: Maintenance mode request is cancelled if there are not other hosts, capable to run the engine VM.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2017-04-21 09:50:06 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	SLA	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Gonza 2016-11-13 17:48:02 UTC

Description of problem:
On an HE with 2 hosted engine agent hosts, when host that is holding the engine VM is set to local maintenance, the engine vm is migrated to the next hosted engine agent even if this agent is set to local maintenance too.

Version-Release number of selected component (if applicable):
ovirt-hosted-engine-ha-2.0.4-1.el7ev.noarch

How reproducible:
100%

Steps to Reproduce:
1. Have a HE with 2 hosted engine agent hosts
2. have an agent in local maintenance mode
3. engine vm should be on the next agent, set this to local maintenance and see vm migrates to other agent

Actual results:
engine vm is migrated to host in local maintenance

Expected results:
engine vm should not migrate

Comment 1 Martin Sivák 2016-11-23 13:30:33 UTC

I believe the expected behavior here should be that the host is not moved to local maintenance if it is the last host available (last active host with score higher than 0). This will also prevent the migration.

I expect this to happen in two source code places - 1) the agent's state machine and the 2) data collecting code that computes the best alternative host.

Comment 2 Sandro Bonazzola 2016-12-11 06:11:59 UTC

Moving back to post, a referenced patch is still under review.
Denis, are you working on this BZ? please set yourself as assignee.

Comment 3 Sandro Bonazzola 2017-01-25 07:56:14 UTC

4.0.6 has been the last oVirt 4.0 release, please re-target this bug.

Comment 4 Sandro Bonazzola 2017-02-01 16:02:52 UTC

oVirt 4.1.0 GA has been released, re-targeting to 4.1.1.
Please check if this issue is correctly targeted or already included in 4.1.0.

Comment 5 Denis Chaplygin 2017-03-02 06:02:34 UTC

As far as i understand, patches are already merged to required branches.

Comment 6 Nikolai Sednev 2017-03-05 20:19:34 UTC

puma18 is not running HE-VM.
puma18 ~]# hosted-engine --set-maintenance --mode=local
Score                              : 0
Local maintenance                  : True

puma19 is running HE-VM.
puma19 ~]# hosted-engine --set-maintenance --mode=local
Unable to enter local maintenance mode: there are no available hosts capable of running the engine VM.

Components on hosts:
libvirt-client-2.0.0-10.el7_3.4.x86_64
qemu-kvm-rhev-2.6.0-28.el7_3.6.x86_64
rhevm-appliance-20160721.0-2.el7ev.noarch
mom-0.5.9-1.el7ev.noarch
ovirt-hosted-engine-setup-2.1.0.3-1.el7ev.noarch
ovirt-engine-sdk-python-3.6.9.1-1.el7ev.noarch
sanlock-3.4.0-1.el7.x86_64
ovirt-vmconsole-host-1.0.4-1.el7ev.noarch
vdsm-4.19.6-1.el7ev.x86_64
ovirt-host-deploy-1.6.0-1.el7ev.noarch
ovirt-vmconsole-1.0.4-1.el7ev.noarch
ovirt-imageio-common-1.0.0-0.el7ev.noarch
ovirt-imageio-daemon-1.0.0-0.el7ev.noarch
ovirt-setup-lib-1.1.0-1.el7ev.noarch
ovirt-hosted-engine-ha-2.1.0.3-1.el7ev.noarch
Linux version 3.10.0-514.6.1.el7.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC) ) #1 SMP Sat Dec 10 11:15:38 EST 2016
Linux 3.10.0-514.6.1.el7.x86_64 #1 SMP Sat Dec 10 11:15:38 EST 2016 x86_64 x86_64 x86_64 GNU/Linux
Red Hat Enterprise Linux Server release 7.3 (Maipo)

On engine:
rhev-guest-tools-iso-4.1-3.el7ev.noarch
rhevm-dependencies-4.1.0-1.el7ev.noarch
rhevm-doc-4.1.0-2.el7ev.noarch
rhevm-branding-rhev-4.1.0-1.el7ev.noarch
rhevm-setup-plugins-4.1.0-1.el7ev.noarch
rhevm-4.1.1.2-0.1.el7.noarch
Linux version 3.10.0-514.6.2.el7.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC) ) #1 SMP Fri Feb 17 19:21:31 EST 2017
Linux 3.10.0-514.6.2.el7.x86_64 #1 SMP Fri Feb 17 19:21:31 EST 2017 x86_64 x86_64 x86_64 GNU/Linux
Red Hat Enterprise Linux Server release 7.3 (Maipo)

Moving to verified.