Bug 1425892 – [downstream clone - 4.0.7] Enable HE deploy option in 3.6 cluster compatibility and add note in hosted engine tab that it will only work in host that are 4.0 and above.

Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.

Bug 1425892 - [downstream clone - 4.0.7] Enable HE deploy option in 3.6 cluster compatibility and add note in hosted engine tab that it will only work in host that are 4.0 and above.

Summary:	[downstream clone - 4.0.7] Enable HE deploy option in 3.6 cluster compatibili...

Status:	CLOSED ERRATA

Aliases:	None

Product:	Red Hat Enterprise Virtualization Manager
Classification:	Red Hat
Component:	ovirt-hosted-engine-ha (Show other bugs)
Sub Component:
Version:	4.0.3
Hardware:	All Linux

Priority	high Severity medium
Target Milestone:	ovirt-4.0.7
Target Release:	---
Assigned To:	Andrej Krejcir
QA Contact:	Artyom
Docs Contact:

URL:
Whiteboard:	infra
Keywords:	Triaged, ZStream

Depends On:	1383156
Blocks:
	Show dependency tree / graph

Reported:	2017-02-22 12:10 EST by rhev-integ
Modified:	2017-03-16 11:29 EDT (History)
CC List:	11 users (show)

See Also:
Fixed In Version:
Doc Type:	Bug Fix
Doc Text:	Cause: Deploying HE on hosts through webadmin was not allowed if the cluster compatibility level was less than 4.0 because the HE agent for 3.6 did not support it. Consequence: It was not possible to update a HE cluster from 3.6 to 4.0. During the update, the cluster compatibility level has to be 3.6, because it contains some 3.6 hosts. Deploying the HE on a 4.0 host is only possible using the webadmin, but at the same time the deployment was disabled because of the compatibility. Fix: Allow deployment of HE in the UI for compatibility version less than 4.0, but show a warning to the user that it should only be done for hosts with repositories for version 4.0 or larger. Result: It is now possible to update a HE cluster to 4.0.
Story Points:	---
Clone Of:	1383156
Environment:
Last Closed:	2017-03-16 11:29:15 EDT
Type:	---
Regression:	---
Mount Type:	---
Documentation:	---
CRM:
Verified Versions:
Category:	---
oVirt Team:	SLA
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---

Attachments	(Terms of Use)
Add an attachment (proposed patch, testcase, etc.)

External Trackers
Tracker	ID	Priority	Status	Summary	Last Updated
oVirt gerrit	72721	None	None	None	2017-02-22 12:12 EST
oVirt gerrit	72752	None	None	None	2017-02-22 12:12 EST
oVirt gerrit	72894	None	None	None	2017-02-22 12:12 EST
oVirt gerrit	72917	None	None	None	2017-02-22 12:12 EST
oVirt gerrit	72922	None	None	None	2017-02-23 05:04 EST
oVirt gerrit	72931	None	None	None	2017-02-23 05:04 EST
Red Hat Product Errata	RHBA-2017:0541	normal	SHIPPED_LIVE	ovirt-hosted-engine-ha bug fix update for 4.0.7	2017-03-16 15:24:59 EDT

Groups: None (edit)

Description rhev-integ 2017-02-22 12:10:53 EST

+++ This bug is a downstream clone. The original bug is: +++
+++   bug 1383156 +++
======================================================================

## Description of problem:

I've recently upgrade from 3.6 to 4.0 (hosted engine).  I'm at the part where i have one new rhvh4.0 hypervisor, and all the rest are from 3.6.  I removed the old 3.6 one via the gui, re-install with rhvh4.0, then try and add it via the gui (with hosted engine) and it says:

"Error while executing action: Cannot add Host.  The action deploy hosted engine host is supported from cluster level 3.6"

## Version-Release number of selected component (if applicable):

RHV4.0
ovirt-hosted-engine-ha
Cluster and DC at 3.6 (still completing upgrade)

## How reproducible:


## Steps to Reproduce:
1.
2.
3.

## Actual results:

To complete the upgrade, it seems I need to:

- remove *all* hosts (except the 4.0 one), then upgrade Cluster/DC to 4.0, then upgrade hosts and add back - but what if I can't run my all my VM's on just one hypervisor?

or 

- rolling remove/upgrade/add of hosts to 4.0, then upgrade Cluster/DC to 4.0, then remove hosts again and re-add again as 'hosted engine' hosts - it seems unnecessary to remove hosts and add them twice, just to enable hosted engine deploy.


## Expected results:

Be able to add a new hosts with hosted engine enabled to a 3.6 cluster.  Or, be able to enable hosted engine after a host has been added (not have to remove and add it).

(Originally by Marcus West)

Comment 3 rhev-integ 2017-02-22 12:11:05 EST

Marcus, I do not understand why is this RFE?
If you cannot add a 4.0 Host to an upgraded environment, then it is a bug. OR I misunderstand something in the flow.

Can you please provide simple steps to reproduce?

(Originally by Marina Kalinin)

Comment 6 rhev-integ 2017-02-22 12:11:18 EST

I have tried again - but this time instead of using the RHEV gui, I tried via the cockpit interface (on the hypervisor), and it worked!

Do we still support adding hosts via the RHEV gui?  If not, may need to clarify this in the documentation:

https://access.redhat.com/documentation/en/red-hat-virtualization/4.0/paged/self-hosted-engine-guide/chapter-7-installing-additional-hosts-to-a-self-hosted-environment

(Originally by Marcus West)

Comment 9 rhev-integ 2017-02-22 12:11:31 EST

Marcus, is your cluster compatibility level set to 3.6? That might be the issue here. We do support deploying of 4.0 hosted engine hosts using the UI, but not when 4.0 features are disabled by the compatibility mode.

We will investigate if that limitation is really necessary and either remove it or update the UI to reflect it.

(Originally by Martin Sivak)

Comment 10 rhev-integ 2017-02-22 12:11:36 EST

Hey Martin, it is set to 3.6 as I am still in the process of upgrading (from 3.6 to 4.0)  I can't change it to 4.0 until I upgrade all the hosts.  Only after I have upgraded all hosts (so I can upgrade cluster and DC to 4.0), then I can deploy another hosted engine host.

If I only have two or thee hosts, it's not a lot of time without a second HE host.  But for a large environment (40 hosts, etc), thats a lot of time with only one host capable of running HostedEngine.

It just seems a bit strange that HE is already running on an upgraded hypervisor, but I can't add another upgraded HE capable hosts.  Also we can do it via the cockpit gui (c#5), just not via the rhevm gui it seems.

If we can't remove this restriction, perhaps we should recommend that all hypervisors be upgraded first? (before upgrading rhevm from 3.6 to 4.0)

(Originally by Marcus West)

Comment 11 rhev-integ 2017-02-22 12:11:40 EST

(In reply to Marcus West from comment #9)
> Hey Martin, it is set to 3.6 as I am still in the process of upgrading (from
> 3.6 to 4.0)  I can't change it to 4.0 until I upgrade all the hosts.  Only
> after I have upgraded all hosts (so I can upgrade cluster and DC to 4.0),
> then I can deploy another hosted engine host.
> 
> If I only have two or thee hosts, it's not a lot of time without a second HE
> host.  But for a large environment (40 hosts, etc), thats a lot of time with
> only one host capable of running HostedEngine.
> 
> It just seems a bit strange that HE is already running on an upgraded
> hypervisor, but I can't add another upgraded HE capable hosts.  Also we can
> do it via the cockpit gui (c#5), just not via the rhevm gui it seems.
> 
> If we can't remove this restriction, perhaps we should recommend that all
> hypervisors be upgraded first? (before upgrading rhevm from 3.6 to 4.0)

Have you followed the "5.4. Upgrading a RHEV-H-Based Self-Hosted Engine Environment." from https://access.redhat.com/documentation/en/red-hat-virtualization/4.0/single/self-hosted-engine-guide/#Upgrading_a_RHEV-H-Based_Self-Hosted_Engine_Environment?

(Originally by Nikolai Sednev)

Comment 12 rhev-integ 2017-02-22 12:11:44 EST

Hello Nikolai,

Yes I have.  Specifically, in the outline:

"
 - Place the high-availability agents that manage the Manager virtual machine into the global maintenance mode.
 - Add a new RHVH 4.0 host to your environment.
 - Migrate the Manager virtual machine to the new host and set as SPM.
 - Run the upgrade utility to upgrade the Manager.
 - Update the hosts.
 - After the Manager virtual machine and all hosts in the cluster have been updated, change the cluster compatibility version to 4.0.
"

For the 2nd last step "Update the hosts", this could take quite a while for a large environment (40+hosts, etc).  That's  long time to run without HostedEngine redundancy.



Cheers,

Marcus

(Originally by Marcus West)

Comment 13 rhev-integ 2017-02-22 12:11:49 EST

(In reply to Marcus West from comment #11)
> Hello Nikolai,
> 
> Yes I have.  Specifically, in the outline:
> 
> "
>  - Place the high-availability agents that manage the Manager virtual
> machine into the global maintenance mode.
>  - Add a new RHVH 4.0 host to your environment.
>  - Migrate the Manager virtual machine to the new host and set as SPM.
>  - Run the upgrade utility to upgrade the Manager.
>  - Update the hosts.
>  - After the Manager virtual machine and all hosts in the cluster have been
> updated, change the cluster compatibility version to 4.0.
> "
> 
> For the 2nd last step "Update the hosts", this could take quite a while for
> a large environment (40+hosts, etc).  That's  long time to run without
> HostedEngine redundancy.
> 
> 
> 
> Cheers,
> 
> Marcus
So I understand that your only issue is redundancy with at least two HE-hosts.
 
I've also followed the same steps, upgraded OK, I agree with your point that redundancy is important for HE, hence for step "Add a new RHVH 4.0 host to your environment.", I've added two hosts, so I would have the redundancy available on my environment regardless all other hosts, which being upgraded.

(Originally by Nikolai Sednev)

Comment 15 rhev-integ 2017-02-22 12:11:58 EST

This bug looks like the same as we're testing in https://bugzilla.redhat.com/show_bug.cgi?id=1420283.

(Originally by Nikolai Sednev)

Comment 17 Artyom 2017-03-02 05:02:57 EST

Verified on the engine:
rhevm-4.0.7.3-0.1.el7ev.noarch

1) Deploy 3.6 HE environment with one host
2) Upgrade host and environment to 4.0
3) Add additional the 3.6 host to the HE cluster
4) Reinstall it with HE deployment via webadmin - PASS

Can see warning message under the deployment tab
Hosted Engine
The cluster compatibility version is less than 4.0. The hosted engine deployment will still work on a host with support for 4.0 and above, but might fail when only older (3.x) packages are available.

5) Undeploy 3.6 host via webadmin - PASS
6) Update 3.6 host packages to 4.0
7) Reinstall it with HE deployment via webadmin - PASS
8) Change cluster compatibility version to 4.0
9) Warning message disappears from the deployment tab

Comment 19 errata-xmlrpc 2017-03-16 11:29:15 EDT

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2017-0541.html

Note You need to log in before you can comment on or make changes to this bug.