Bug 1649367

Summary:	libvirtd failing with error virNetTLSContextLoadCertFromFile:513 : Unable to import server certificate /etc/pki/vdsm/certs/vdsmcert.pem after upgrading host to RHEL 7.6
Product:	Red Hat Enterprise Virtualization Manager	Reporter:	Nirav Dave <ndave>
Component:	libvirt	Assignee:	Martin Perina <mperina>
Status:	CLOSED DUPLICATE	QA Contact:	Lukas Svaty <lsvaty>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	4.1.1	CC:	michal.skrivanek, ndave
Target Milestone:	---	Flags:	lsvaty: testing_plan_complete-
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2018-11-14 11:50:41 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	Infra	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Nirav Dave 2018-11-13 13:43:36 UTC

Description of problem:

After upgrading the host from RHEL 7.5 to RHEL 7.6 VDSM fails:

-------------------------------------
Nov  4 13:01:15  vdsmd_init_common.sh: libvirt is already configured for vdsm
Nov  4 13:01:15  vdsmd_init_common.sh: Current revision of multipath.conf detected, preserving
Nov  4 13:01:15  vdsmd_init_common.sh: vdsm: Running validate_configuration
Nov  4 13:01:15  vdsmd_init_common.sh: Error:  Config is not valid. Check conf files
Nov  4 13:01:15  vdsmd_init_common.sh: FAILED: conflicting vdsm and libvirt-qemu tls configuration.
Nov  4 13:01:15  vdsmd_init_common.sh: vdsm.conf with ssl=False requires the following changes:
Nov  4 13:01:15  vdsmd_init_common.sh: libvirtd.conf: listen_tcp=1, auth_tcp="none", listen_tls=0
Nov  4 13:01:15  vdsmd_init_common.sh: qemu.conf: spice_tls=0.
Nov  4 13:01:15  vdsmd_init_common.sh: Modules libvirt contains invalid configuration
Nov  4 13:01:15  vdsmd_init_common.sh: vdsm: stopped during execute validate_configuration task (task returned with error code 1).
Nov  4 13:01:15  systemd: vdsmd.service: control process exited, code=exited status=1
Nov  4 13:01:15  systemd: Failed to start Virtual Desktop Server Manager.
Nov  4 13:01:15  systemd: Dependency failed for MOM instance configured for VDSM purposes.
Nov  4 13:01:15 systemd: Job mom-vdsm.service/start failed with result 'dependency'.
Nov  4 13:01:15  systemd: Unit vdsmd.service entered failed state.
Nov  4 13:01:15  systemd: vdsmd.service failed.
---------------------------------------------

Restarting the Libvirtd service fails with following error:

-----------------------------
Nov 08 06:26:27  libvirtd[184127]: 184127: info : libvirt version: 4.5.0, package: 10.el7_6.2 (Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>, 2018-10-10-12:29:34, x86-030.build.eng.bos.redhat.com)
Nov 08 06:26:27  libvirtd[184127]: 184127: info : hostname: 
Nov 08 06:26:27  libvirtd[184127]: 184127: error : virNetTLSContextLoadCertFromFile:513 : Unable to import server certificate /etc/pki/vdsm/certs/vdsmcert.pem
-------------------------------


Version-Release number of selected component (if applicable):
- ovirt-engine-4.1.11.2-0.1.el7.noarch

- vdsm-4.20.39.1-1.el7ev.x86_64

Host OS:
REDHAT_BUGZILLA_PRODUCT_VERSION=7.6

How reproducible:
Everytime

Steps to Reproduce:
1.Put the host to be upgraded into maintenance mode 
2.update the host via yum update -y (excluding EPEL)

Actual results:
Host doesn't come up in active state with VDSM failure. 

Expected results:
After upgrading the host should go in active state

Additional info:

I will be attaching logs shortly.

~Nirav Dave

Comment 2 Michal Skrivanek 2018-11-14 06:07:26 UTC

I suppose a duplicate of bug 1648190

Comment 3 Martin Perina 2018-11-14 08:28:12 UTC

(In reply to Michal Skrivanek from comment #2)
> I suppose a duplicate of bug 1648190

Very likely this is the same issue as BZ1648190. We don't include certificates into sos-logcollector reports, but looking at current logs I don't see any "Reinstall" or "Enroll certificate" actions being performed on host s01pqfrhe2 (10.201.91.33) since Apr 22nd 2017 (that's the oldest log we have). 

Nirav, could you please check that CA certificate on RHV Manager has been regenerated or customer skipped that regeneration? If the CA certificate on RHV Manager is correct, could you please that it's really enrolled on all hosts in the cluster?

Comment 6 Martin Perina 2018-11-14 11:50:41 UTC


*** This bug has been marked as a duplicate of bug 1648190 ***