Red Hat Bugzilla – Bug 1291161
Engine grants invalid certificate if host clock is restarted prior installation
Last modified: 2016-08-03 01:14:47 EDT
Created attachment 1105510 [details]
sos report from rhev-h
Description of problem:
libvirt failed to start, because certificate problem
Dec 14 07:26:08 localhost libvirtd: 20180: info : libvirt version: 1.2.17, package: 13.el7_2.2 (Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>, 2015-11-23-07:46:04, x86-019.build.eng.bos.redhat.com)
Dec 14 07:26:08 localhost libvirtd: 20180: error : virNetTLSContextCheckCertTimes:165 : The server certificate /etc/pki/vdsm/certs/vdsmcert.pem is not yet active
# openssl verify -CAfile /etc/pki/vdsm/certs/cacert.pem /etc/pki/vdsm/certs/vdsmcert.pem
/etc/pki/vdsm/certs/vdsmcert.pem: CN = VDSM Certificate Authority
error 9 at 1 depth lookup:certificate is not yet valid
Looks like we generate vdsm certificates, before we have configured network and receive correct date from NTP server, so we have
openssl x509 -in /etc/pki/vdsm/certs/vdsmcert.pem -noout -text
Version: 3 (0x2)
Signature Algorithm: sha256WithRSAEncryption
Issuer: CN=VDSM Certificate Authority
Not Before: Dec 14 08:36:24 2015 GMT
Not After : Dec 13 08:36:24 2016 GMT
Mon Dec 14 07:38:05 UTC 2015
Version-Release number of selected component (if applicable):
Red Hat Enterprise Virtualization Hypervisor (Beta) release 7.2 (20151210.1.el7ev)
I also encounter this one in rhev-hypervisor7-7.1-20151015.0
Steps to Reproduce:
1. Deploy RHEV-H from usb-key
After deployment libvirtd failed to start because certificate problem
After deployment libvirtd succeed to start without any errors
This appears on Node, but doesn't seem to be a node specific problem, as this can also happen on RHEL hosts.
We cannot wait for NTP to sync the host time.
Prior to ovirt-host-deploy we attempted copying time from from Engine to the host.
Also, we made our certificate is valid few hours into the past, so reasonably out-of-sync hosts can be added.
Would reintroducing one of these hacks make sense?
(In reply to Dan Kenigsberg from comment #2)
> We cannot wait for NTP to sync the host time.
> Prior to ovirt-host-deploy we attempted copying time from from Engine to the
> Also, we made our certificate is valid few hours into the past, so
> reasonably out-of-sync hosts can be added.
> Would reintroducing one of these hacks make sense?
Engine grant a valid certificate based on engine clock minus one day.
Host-deploy still sync clock if it is out of sync.
This one time sync is useless, as after reboot you will probably be out of sync once again... but still we do this.
Instead of attaching irrelevant sos reports, please attach host-deploy log, so we can probably see the above.
I opened bug on RHEV-H and problem that I have libvirtd service failed to start before I add host to any engine, so not host-deploy.log
there is no sense in starting libvirt pre-deploy, nor am I sure how /etc/pki/vdsm/certs/vdsmcert.pem was created.
back to ovirt-node, I suggest to close it as NOTABUG.
libvirtd is started by default on Node, which can cause this problem.
because it is an issue, but appearing not to often I'm closing this bug. Please reopen if you encounter this problem often.
For me it happen pretty often, for example from five RHEV-H installation it can happen twice. It also can create additional problem if you want to run automation on RHEV-H. Can we just add some additional hook on network setup, that will regenerate libvirt certificate?
Can we consider setting the certificate validity, 1 day before the current date?
This can easily workaround this kind of problem.
According to comment 3 this is already the case.
Artyom, have you got an idea why the host is initially so out of sync?
Does the DHCP server also provide NTP servers?
Do not really know, maybe RHEV-H use some default time zone?
Yes DHCP server also provide NTP.