Bug 1854401 - Unable to update cluster compatibility, post upgrading the self-hosted-engine setup to RHV 4.4
Summary: Unable to update cluster compatibility, post upgrading the self-hosted-engine...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: rhhi
Version: rhgs-3.5
Hardware: x86_64
OS: Linux
unspecified
urgent
Target Milestone: ---
: RHHI-V 1.8
Assignee: Gobinda Das
QA Contact: SATHEESARAN
URL:
Whiteboard:
Depends On: 1854393
Blocks: RHHI-V-1.8-Engineering-Inflight-BZs
TreeView+ depends on / blocked
 
Reported: 2020-07-07 12:16 UTC by SATHEESARAN
Modified: 2020-08-04 14:52 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1854393
Environment:
Last Closed: 2020-08-04 14:52:36 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2020:3314 0 None None None 2020-08-04 14:52:50 UTC

Description SATHEESARAN 2020-07-07 12:16:32 UTC
Description of problem:
------------------------
I had a 3 node self-hosted engine setup running RHV 4.3.10-async. This is a hyperconverged environment, gluster storage domains are used with brick created on the local disks

Upgrade was smooth, and post upgrade, edited the cluster to 4.4 compatibility, with all the VMs up and running. There was a general warning thrown out, but I have seen null pointer exception.

When turning off all the VMs except HostedEngine VM (4.4 VM), I have seen new set of error, 'CLUSTER_CANNOT_DISABLE_GLUSTER_WHEN_CLUSTER_CONTAINS_VOLUMES'

Version-Release number of selected component (if applicable):
-------------------------------------------------------------
RHV 4.4.1
As there are no new appliance for QE, used rhvm-appliance-4.4-20200604.0.el8ev

How reproducible:
-----------------
Always

Steps to Reproduce:
--------------------
1. Create RHHI-V deployment with 3 node self-hosted-engine setup
2. Upgrade the environment to RHV 4.4
3. Update the cluster compatibility to 4.4 after all the hosts are upgraded

Actual results:
---------------
Unable to update the cluster to 4.4 as the error mentions that there are VM/Templates and asks to change that specifically. Then after turning off all the VMs, I see error - 'CLUSTER_CANNOT_DISABLE_GLUSTER_WHEN_CLUSTER_CONTAINS_VOLUMES'

Expected results:
-----------------
1. When VMs are up, updating cluster compatibility to 4.4 should succeed, with VMs needing to reboot later
2. When all the VMs are shutdown, able to update cluster compatibility

Additional info:
-----------------
I have started upgrade from RHV 4.3.8 to RHV 4.3.10-async, then continued to upgrade to RHV 4.4

--- Additional comment from RHEL Program Management on 2020-07-07 11:57:26 UTC ---

Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

--- Additional comment from SATHEESARAN on 2020-07-07 12:01:48 UTC ---



--- Additional comment from SATHEESARAN on 2020-07-07 12:11:52 UTC ---

For your information, with RHHI-V environment hosts are deployed in to 'Default' cluster.

I see 2 problems here:

1. I am seeing Null Pointer Exception, when updating the compatibility version of the 'Default' cluster, when all the VMs are up and running.

2. When all the VMs are powered down, I see a different problem - 'CLUSTER_CANNOT_DISABLE_GLUSTER_WHEN_CLUSTER_CONTAINS_VOLUMES'
<snip>
2020-07-07 09:46:24,661Z WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-23) [bf38294d-faef-422a-b68a-22870e57e632] EVENT_ID: CANNOT_HIBERNATE_RUNNING_VMS_AFTER_CLUSTER_CPU_UPGRADE(9,006), Hibernation of VMs after CPU upgrade of Cluster Default is not supported. Please stop and restart those VMs in case you wish to hibernate them
2020-07-07 09:46:24,671Z WARN  [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-23) [bf38294d-faef-422a-b68a-22870e57e632] Validation of action 'UpdateCluster' failed for user admin@internal-authz. Reasons: VAR__TYPE__CLUSTER,VAR__ACTION__UPDATE,CLUSTER_CANNOT_DISABLE_GLUSTER_WHEN_CLUSTER_CONTAINS_VOLUMES
</snip>

--- Additional comment from SATHEESARAN on 2020-07-07 12:14:41 UTC ---

Here is the screenshot of the error dialog of when editing RHV default cluster, to update cluster compatibility to 4.4

Comment 2 SATHEESARAN 2020-07-13 08:00:36 UTC
Verified with RHV manager 4.4.1.8-0.7.el8ev

Post upgrading all the hosts with RHVH 4.4.1, with the latest appliance (rhvm-appliance-4.4-20200707.0.el8ev.x86_64),
the cluster could be updated with 4.4 without any errors

Comment 4 errata-xmlrpc 2020-08-04 14:52:36 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (RHHI for Virtualization 1.8 bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2020:3314


Note You need to log in before you can comment on or make changes to this bug.