Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 2079901

Summary:	allow Enroll Certificate action when host is Non Responsive
Product:	[oVirt] ovirt-engine	Reporter:	Michal Skrivanek <michal.skrivanek>
Component:	General	Assignee:	Milan Zamazal <mzamazal>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Petr Kubica <pkubica>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	---	CC:	bugs, fgarciad, lsvaty, mperina, pkubica
Target Milestone:	ovirt-4.5.0-1	Flags:	pm-rhel: ovirt-4.5? lsvaty: exception+
Target Release:	4.5.0.7
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	ovirt-engine-4.5.0.7	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-05-30 06:42:37 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	Infra	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Michal Skrivanek 2022-04-28 13:05:27 UTC

When certificates expire there is no way how to re-enroll without shutting down all the running VMs. The Enroll is possible only in Maintenance state and the only way how to get to Maintenance state from Non Responsive with running VMs is through "confirm host has been rebooted" - i.e. either reboot it or knowingly override the actual state triggering VM restart of HA VMs elsewhere and potentially causing split brain.

We cannot fully re-enroll all certificates on a running system, but we can install new certificates and restart subset of services that allow VMs to be migrated away and move the host to proper Maintenance.

Let's
- allow the Enroll Certificate operation when host is in Non Responsive state
- run the same enrollment code as in Maintenance
- restart/reload libvirt (which in turn restarts vdsm), imageio, and OVN

It introduces a small risk for losing track of ongoing actions (e.g. live merge completion) but we should mostly deal with those gracefully. Not all operations would work (e.g. VNC certificate cannot be reloaded so console connections will not be possible), but being able to control the host and lifecycle of VMs has more priority.

Still, it shall be documented that this is a "desperate measure" for cases where certificates suddenly expire and there's no other way out. Such action must be followed by another re-enroll during Maintenance, and such running VMs *must* be restarted before they are fully functional again.

Comment 1 Petr Kubica 2022-05-24 06:40:43 UTC

verified in ovirt-engine-4.5.0.7-0.9.el8ev.noarch

Verification steps
Host was put into a NonResponsive state after killing vdsmd service. (with and without running VMs on top of the hosts)
Triggered enrolling certificate on the host which successfully completed