Bug 1157385

Summary:

[engine-backend] hosted-engine: Migrating the HostedEngine VM is blocked because the DST host is reported as non-HE host although it is a HE host

Product:

Red Hat Enterprise Virtualization Manager

Reporter:

Elad <ebenahar>

Component:

ovirt-engine

Assignee:

Doron Fediuck <dfediuck>

Status:

CLOSED CURRENTRELEASE

QA Contact:

Elad <ebenahar>

Severity:

high

Docs Contact:

Priority:

unspecified

Version:

3.5.0

CC:

acanan, dfediuck, ebenahar, ecohen, gklein, iheim, lpeer, lsurette, mavital, rbalakri, Rhev-m-bugs, sherold, yeylon

Target Milestone:

---

Keywords:

Reopened

Target Release:

3.5.0

Hardware:

x86_64

OS:

Unspecified

Whiteboard:

sla

Fixed In Version:

Doc Type:

Bug Fix

Doc Text:

Story Points:

---

Clone Of:

Environment:

Last Closed:

2015-02-17 17:11:42 UTC

Type:

Bug

Regression:

---

Mount Type:

---

Documentation:

---

CRM:

Verified Versions:

Category:

---

oVirt Team:

SLA

RHEL 7.3 requirements from Atomic Host:

Cloudforms Team:

---

Target Upstream Version:

Embargoed:

Bug Depends On:

1166002, 1169290

Bug Blocks:

Attachments:

Description	Flags
engine.log	none
logs from the DST host	none
ha logs	none

Description Elad 2014-10-27 07:07:07 UTC

Created attachment 950895 [details]
engine.log

Description of problem:
On a hosted-engine envi., I deployed a second hosted-engine host using hosted-engine --depoy on it (followed the instructions from: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.3/html/Installation_Guide/Installing_Additional_Hosts_to_a_Self-Hosted_Environment.html)
The host was added to the exiting cluster as hosted_engine_2.
I tried to migrate the HostedEngine VM from the first host to the second and blocked because the new host is reported as non-hosted-engine host. 


Version-Release number of selected component (if applicable):
rhev 3.5 vt7
rhel6.6 host
ovirt-hosted-engine-setup-1.2.1-1.el6ev.noarch
rhevm-3.5.0-0.17.beta.el6ev.noarch
vdsm-4.16.7.1-1.el6ev.x86_64


How reproducible:
Always

Steps to Reproduce:
1. Deploy hosted-engine (I used iSCSI storage)
2. When the engine is up and running, add a second host to the hosted-engine cluster by 'hosted-engine --deploy'.
3. Try to migrate the HostedEngine VM to the new host.

Actual results:
Migrating the HostedEngine VM to the new hosted-engine host is blocked on CDA:


2014-10-27 08:51:09,401 WARN  [org.ovirt.engine.core.bll.MigrateVmCommand] (ajp-/127.0.0.1:8702-5) [476f0344] CanDoAction of action MigrateVm failed. Reasons:VAR__ACTION__MIGRATE,VAR__TYPE__VM,SCHEDULING_ALL_HOSTS_FILTERED_OUT,VAR__FILTERTYPE__INTERNAL,$hostName hosted_engine_2,$filterName HA,VAR__DETAIL__NOT_HE_HOST,SCHEDULING_HOST_FILTERED_REASON_WITH_DETAIL




Expected results:
It can be one of 2 things:
1) The HostedEngine VM migration should not be blocked  
2) The documention is wrong

Additional info: engine.log

Comment 1 Doron Fediuck 2014-10-28 15:36:32 UTC

Can you provide the relevant hosted engine logs from both hosts?

Comment 2 Elad 2014-10-29 08:10:48 UTC

Created attachment 951660 [details]
logs from the DST host

logs from the DST host attached

Comment 3 Jiri Moskovcak 2014-10-29 08:21:53 UTC

(In reply to Elad from comment #2)
> Created attachment 951660 [details]
> logs from the DST host
> 
> logs from the DST host attached

None of the setup logs shows completed installation actually every one fails with some error, the last one because of wrong iscsi password, can you please double check that the installation was completed successfully? Also if the installation was finished ovirt-ha-agent and ovirt-ha-broker services should be running and actually the presence of these services is what makes HE host from normal host. If those services are running please attach the agent.log and broker.log

Comment 4 Elad 2014-10-29 08:31:35 UTC

Created attachment 951662 [details]
ha logs

Comment 5 Jiri Moskovcak 2014-10-29 08:42:26 UTC

Is this a clean install or was the host used before? The problem is that the host is in LocalMaintenance state right after it starts, so I guess there already was a host with the same id (2) whioch was put to the LocalMaintenance and the freshly installed agent just picks up the previous state. Plea try to run this command on the host:

hosted-engine --set-maintenance --mode=none

Comment 6 Jiri Moskovcak 2014-10-29 08:42:53 UTC

it should disable the maintenance and allow the migrations

Comment 7 Elad 2014-10-29 08:55:56 UTC

(In reply to Jiri Moskovcak from comment #5)
> Is this a clean install or was the host used before? The problem is that the
> host is in LocalMaintenance state right after it starts, so I guess there
> already was a host with the same id (2) whioch was put to the
> LocalMaintenance and the freshly installed agent just picks up the previous
> state. Plea try to run this command on the host:
> 
> hosted-engine --set-maintenance --mode=none

The host wan't part of another HE setups before. The other HE setup failures in the logs are from after I removed it from this setup. The host was up and running in the setup as an additional HE host.

Comment 8 Doron Fediuck 2014-11-17 17:13:53 UTC

(In reply to Elad from comment #7)
> (In reply to Jiri Moskovcak from comment #5)
> > Is this a clean install or was the host used before? The problem is that the
> > host is in LocalMaintenance state right after it starts, so I guess there
> > already was a host with the same id (2) whioch was put to the
> > LocalMaintenance and the freshly installed agent just picks up the previous
> > state. Plea try to run this command on the host:
> > 
> > hosted-engine --set-maintenance --mode=none
> 
> The host wan't part of another HE setups before. The other HE setup failures
> in the logs are from after I removed it from this setup. The host was up and
> running in the setup as an additional HE host.

Elad, this scenario (installing HE on more than one host) is being done almost every day, so it could only be an environmental issue, like a dirty setup (storage included) or specifying a different storage path during the installation, which will get you 2 hosted engine setups.

For now closing this issue.
If you can reproduce this on a clean setup (fresh install of the OS and storage)
please re-open and provide the actual setup steps for each node with the relevant setup and HA log files.

Comment 9 Aharon Canan 2014-11-19 09:48:39 UTC

Elad, 
Lets try to reproduce like Doron wrote in comment #8

Doron,
Did you had chance to check the logs? Do we have something there? I saw Jiri's comments but Elad comment about Jiri's comments.

anyway, I think we should give it a try before closing it.

Comment 10 Elad 2014-11-20 09:10:05 UTC

Currently, cannot check due to https://bugzilla.redhat.com/show_bug.cgi?id=1166002

Comment 13 Elad 2014-12-01 09:36:00 UTC

Cannot be tested due to https://bugzilla.redhat.com/show_bug.cgi?id=1169290

Comment 14 Elad 2014-12-09 14:08:41 UTC

Migration of the hosted engine VM is allowed while using iSCSI 

Used rhev 3.5 vt13.1

Comment 16 Eyal Edri 2015-02-17 17:11:42 UTC

rhev 3.5.0 was released. closing.