Bug 1157385 - [engine-backend] hosted-engine: Migrating the HostedEngine VM is blocked because the DST host is reported as non-HE host although it is a HE host
Summary: [engine-backend] hosted-engine: Migrating the HostedEngine VM is blocked beca...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine
Version: 3.5.0
Hardware: x86_64
OS: Unspecified
unspecified
high
Target Milestone: ---
: 3.5.0
Assignee: Doron Fediuck
QA Contact: Elad
URL:
Whiteboard: sla
Depends On: 1166002 1169290
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-10-27 07:07 UTC by Elad
Modified: 2016-02-10 20:16 UTC (History)
13 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-02-17 17:11:42 UTC
oVirt Team: SLA
Target Upstream Version:


Attachments (Terms of Use)
engine.log (52.17 KB, application/x-gzip)
2014-10-27 07:07 UTC, Elad
no flags Details
logs from the DST host (958.83 KB, application/x-gzip)
2014-10-29 08:10 UTC, Elad
no flags Details
ha logs (370.20 KB, application/x-gzip)
2014-10-29 08:31 UTC, Elad
no flags Details

Description Elad 2014-10-27 07:07:07 UTC
Created attachment 950895 [details]
engine.log

Description of problem:
On a hosted-engine envi., I deployed a second hosted-engine host using hosted-engine --depoy on it (followed the instructions from: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.3/html/Installation_Guide/Installing_Additional_Hosts_to_a_Self-Hosted_Environment.html)
The host was added to the exiting cluster as hosted_engine_2.
I tried to migrate the HostedEngine VM from the first host to the second and blocked because the new host is reported as non-hosted-engine host. 


Version-Release number of selected component (if applicable):
rhev 3.5 vt7
rhel6.6 host
ovirt-hosted-engine-setup-1.2.1-1.el6ev.noarch
rhevm-3.5.0-0.17.beta.el6ev.noarch
vdsm-4.16.7.1-1.el6ev.x86_64


How reproducible:
Always

Steps to Reproduce:
1. Deploy hosted-engine (I used iSCSI storage)
2. When the engine is up and running, add a second host to the hosted-engine cluster by 'hosted-engine --deploy'.
3. Try to migrate the HostedEngine VM to the new host.

Actual results:
Migrating the HostedEngine VM to the new hosted-engine host is blocked on CDA:


2014-10-27 08:51:09,401 WARN  [org.ovirt.engine.core.bll.MigrateVmCommand] (ajp-/127.0.0.1:8702-5) [476f0344] CanDoAction of action MigrateVm failed. Reasons:VAR__ACTION__MIGRATE,VAR__TYPE__VM,SCHEDULING_ALL_HOSTS_FILTERED_OUT,VAR__FILTERTYPE__INTERNAL,$hostName hosted_engine_2,$filterName HA,VAR__DETAIL__NOT_HE_HOST,SCHEDULING_HOST_FILTERED_REASON_WITH_DETAIL




Expected results:
It can be one of 2 things:
1) The HostedEngine VM migration should not be blocked  
2) The documention is wrong

Additional info: engine.log

Comment 1 Doron Fediuck 2014-10-28 15:36:32 UTC
Can you provide the relevant hosted engine logs from both hosts?

Comment 2 Elad 2014-10-29 08:10:48 UTC
Created attachment 951660 [details]
logs from the DST host

logs from the DST host attached

Comment 3 Jiri Moskovcak 2014-10-29 08:21:53 UTC
(In reply to Elad from comment #2)
> Created attachment 951660 [details]
> logs from the DST host
> 
> logs from the DST host attached

None of the setup logs shows completed installation actually every one fails with some error, the last one because of wrong iscsi password, can you please double check that the installation was completed successfully? Also if the installation was finished ovirt-ha-agent and ovirt-ha-broker services should be running and actually the presence of these services is what makes HE host from normal host. If those services are running please attach the agent.log and broker.log

Comment 4 Elad 2014-10-29 08:31:35 UTC
Created attachment 951662 [details]
ha logs

Comment 5 Jiri Moskovcak 2014-10-29 08:42:26 UTC
Is this a clean install or was the host used before? The problem is that the host is in LocalMaintenance state right after it starts, so I guess there already was a host with the same id (2) whioch was put to the LocalMaintenance and the freshly installed agent just picks up the previous state. Plea try to run this command on the host:

hosted-engine --set-maintenance --mode=none

Comment 6 Jiri Moskovcak 2014-10-29 08:42:53 UTC
it should disable the maintenance and allow the migrations

Comment 7 Elad 2014-10-29 08:55:56 UTC
(In reply to Jiri Moskovcak from comment #5)
> Is this a clean install or was the host used before? The problem is that the
> host is in LocalMaintenance state right after it starts, so I guess there
> already was a host with the same id (2) whioch was put to the
> LocalMaintenance and the freshly installed agent just picks up the previous
> state. Plea try to run this command on the host:
> 
> hosted-engine --set-maintenance --mode=none

The host wan't part of another HE setups before. The other HE setup failures in the logs are from after I removed it from this setup. The host was up and running in the setup as an additional HE host.

Comment 8 Doron Fediuck 2014-11-17 17:13:53 UTC
(In reply to Elad from comment #7)
> (In reply to Jiri Moskovcak from comment #5)
> > Is this a clean install or was the host used before? The problem is that the
> > host is in LocalMaintenance state right after it starts, so I guess there
> > already was a host with the same id (2) whioch was put to the
> > LocalMaintenance and the freshly installed agent just picks up the previous
> > state. Plea try to run this command on the host:
> > 
> > hosted-engine --set-maintenance --mode=none
> 
> The host wan't part of another HE setups before. The other HE setup failures
> in the logs are from after I removed it from this setup. The host was up and
> running in the setup as an additional HE host.

Elad, this scenario (installing HE on more than one host) is being done almost every day, so it could only be an environmental issue, like a dirty setup (storage included) or specifying a different storage path during the installation, which will get you 2 hosted engine setups.

For now closing this issue.
If you can reproduce this on a clean setup (fresh install of the OS and storage)
please re-open and provide the actual setup steps for each node with the relevant setup and HA log files.

Comment 9 Aharon Canan 2014-11-19 09:48:39 UTC
Elad, 
Lets try to reproduce like Doron wrote in comment #8

Doron,
Did you had chance to check the logs? Do we have something there? I saw Jiri's comments but Elad comment about Jiri's comments.

anyway, I think we should give it a try before closing it.

Comment 10 Elad 2014-11-20 09:10:05 UTC
Currently, cannot check due to https://bugzilla.redhat.com/show_bug.cgi?id=1166002

Comment 13 Elad 2014-12-01 09:36:00 UTC
Cannot be tested due to https://bugzilla.redhat.com/show_bug.cgi?id=1169290

Comment 14 Elad 2014-12-09 14:08:41 UTC
Migration of the hosted engine VM is allowed while using iSCSI 

Used rhev 3.5 vt13.1

Comment 16 Eyal Edri 2015-02-17 17:11:42 UTC
rhev 3.5.0 was released. closing.


Note You need to log in before you can comment on or make changes to this bug.