Bug 1257876

Summary: [engine] CA cert about to expire is detected as already expired
Product: Red Hat Enterprise Virtualization Manager Reporter: Jiri Belka <jbelka>
Component: ovirt-engineAssignee: Moti Asayag <masayag>
Status: CLOSED CURRENTRELEASE QA Contact: Jiri Belka <jbelka>
Severity: high Docs Contact:
Priority: high    
Version: 3.6.0CC: gklein, lsurette, masayag, oourfali, pstehlik, rbalakri, Rhev-m-bugs, srevivo, ykaul
Target Milestone: ovirt-3.6.1   
Target Release: 3.6.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 3.6.0-12 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-04-20 01:11:49 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Infra RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
engine.log none

Description Jiri Belka 2015-08-28 10:02:40 UTC
Created attachment 1067949 [details]
engine.log

Description of problem:

Seems there's logic issue in code when CA cert is about to expire, it is detected as already expire.

~~~
2025-08-04 10:57:52,102 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-5) [] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: oVirt-engine's CA certification has expired at 2025-08-24.
^C
[root@jb-bz1 ~]# date
Mon Aug  4 11:12:07 CEST 2025
[root@jb-bz1 ~]# openssl x509 -in /etc/pki/ovirt-engine/ca.pem -enddate -noout
notAfter=Aug 24 14:53:21 2025 GMT
~~~

~~~
+ENGINE_CA_CERTIFICATION_HAS_EXPIRED=oVirt-engine's CA certification has expired at ${ExpirationDate}.
+ENGINE_CA_CERTIFICATION_IS_ABOUT_TO_EXPIRE=oVirt-engine's CA certification is about to expire at ${ExpirationDate}.
~~~

I was expected the latter event msg.

https://gerrit.ovirt.org/gitweb?p=ovirt-engine.git;a=commitdiff;h=50e990e8b40643d3ffb066402a8dd6d333e5f1de

Version-Release number of selected component (if applicable):
rhevm-backend-3.6.0-0.12.master.el6.noarch

How reproducible:
100%

Steps to Reproduce:
1. engine-setup
2. move os time to be max 30 days before CA cert expiration period
3.

Actual results:
CA certs is about to expire but engine sends event mesg that is already expired

Expected results:
valid detection/ valid event msg

Additional info:

Comment 1 Jiri Belka 2015-09-11 17:02:17 UTC
ok, rhevm-backend-3.6.0-0.15.master.el6.noarch

[root@jb-bz1 ~]# openssl x509 -in /etc/pki/ovirt-engine/ca.pem -enddate -noout
notAfter=Oct  1 16:47:36 2015 GMT

[root@jb-bz1 ~]# grep 'Message:.*expire' /var/log/ovirt-engine/engine.log 
2015-09-11 18:58:40,250 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) [] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: oVirt-engine's CA certification is about to expire at 2015-10-01.

Comment 2 Jiri Belka 2015-09-15 14:14:02 UTC
There's something odd:

[root@jb-bz1 ~]# date
Mon Sep 21 15:51:16 CEST 2015
[root@jb-bz1 ~]# openssl x509 -in /etc/pki/ovirt-engine/ca.pem -noout -enddate
notAfter=Sep 25 13:14:08 2015 GMT
[root@jb-bz1 ~]# tail -f /var/log/ovirt-engine/engine.log | grep 'Message:.*'
2015-09-21 15:23:13,449 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in.
2015-09-21 15:31:28,407 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-16) [] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: oVirt-engine's CA certification has expired at 2015-09-25.
^C
[root@jb-bz1 ~]# rpm -q rhevm-backend
rhevm-backend-3.6.0-0.15.master.el6.noarch

Comment 3 Moti Asayag 2015-09-16 07:38:13 UTC
There are 2 types of errors:

If the certificate less than 7 days from expiration: "The oVirt-engine's CA/Host certification has expired at ${date_less_than_7_days_from_expiration}"

If the certificate less than 30 days from expiration: "The oVirt-engine's CA/Host certification is about to expire at ${date_less_than_30_days_from_expiration}"

Comment 4 Jiri Belka 2015-09-16 14:34:44 UTC
I can't agree with this wording because:

  > current date: 2015-10-06
  > expire date:  2015-10-05

(OK) 2015-10-06 11:05:57,913 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) [] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Messag\
e: oVirt-engine's CA certification has expired at 2015-10-05.

I'm not native speaker but '...has expired' cannot be same for "your" "less then 7 days till expiration" and for already effective expired cert.

Something should be changes, this is chaotic and thus also makes effect on https://bugzilla.redhat.com/show_bug.cgi?id=1258021 (already verified). Also take into account that there's notifier and thus chaotic mail message.

Comment 5 Jiri Belka 2015-09-16 14:37:11 UTC
same logic for engine certs:

- engine blizi se expiraci

  > current date: 2015-09-16
  > expire date:  2015-09-20

(FAIL) 2015-09-16 12:06:59,727 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-84) [] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Mess\
age: oVirt-engine's certification has expired at 2015-09-20. Please renew the engine's certification.
                                  ^^^^^^^^^^^
(FAIL) 2015-09-16 12:07:00,348 FINE    [org.ovirt.engine.core.notifier.dao.EventsManager getAuditLogEvents] event 0 => AuditLogEvent:{id='6', logTypeName='ENGINE_CERTIFICATION_HAS_EXPIRED', type='alertMessage', \
userId='00000000-0000-0000-0000-000000000000', userName='null', vmId='00000000-0000-0000-0000-000000000000', vmName='null', vmTemplateId='null', vmTemplateName='null', vdsId='null', vdsName='null', storagePoolId\
='00000000-0000-0000-0000-000000000000', storagePoolName='', storageDomainId='00000000-0000-0000-0000-000000000000', storageDomainName='', logTime='2015-09-16 12:06:59.717', severity='ALERT', message='oVirt-engi\
ne's certification has expired at 2015-09-20. Please renew the engine's certification.'}
                   ^^^^^^^^^^^

Comment 6 Moti Asayag 2015-09-17 12:25:58 UTC
There will be 3 event logs for each cert expiration check:
1. Warning when cert is about to expire (less than 30 days)
2. Alert when cert is about to expire (less than 7 days)
3. Alert when cert has expired

Comment 7 Jiri Belka 2015-11-13 12:59:58 UTC
1. about to expire > 7 days < 30 days
   - ca:
     # date ;openssl x509 -in /etc/pki/ovirt-engine/ca.pem -enddate -noout
     Fri Nov 13 11:54:44 CET 2015
     notAfter=Dec  8 10:20:52 2015 GMT

     Engine's CA certification is about to expire at 2015-12-08.

   - engine:
     # date ;openssl x509 -in /etc/pki/ovirt-engine/certs/engine.cer -enddate \
       -noout
     Fri Nov 13 11:55:56 CET 2015
     notAfter=Dec  3 10:20:53 2015 GMT

     Engine's certification is about to expire at 2015-12-03. Please renew the
     engine's certification.

   - host:
     # date ; openssl x509 -in /etc/pki/vdsm/certs/vdsmcert.pem -enddate -noout                                                                                                                        
     Fri Nov 13 12:27:50 UTC 2015
     notAfter=Nov 27 17:26:20 2015 GMT

     Host dell-r210ii-13 certification is about to expire at 2015-11-27. Please
     renew the host's certification.

2 about to expire < 7 days

   - ca:
     # date ; openssl x509 -in /etc/pki/ovirt-engine/ca.pem -enddate -noout
     Thu Nov 12 18:06:27 CET 2015
     notAfter=Nov 18 17:02:35 2015 GMT

   - engine:
     # date ; openssl x509 -in /etc/pki/ovirt-engine/certs/engine.cer -enddate \
       -noout
     Thu Nov 12 18:23:22 CET 2015
     notAfter=Nov 18 17:02:39 2015 GMT

     Engine's certification is about to expire at 2015-11-18. Please renew the 
     engine's certification.

   - host:
     # date ; openssl x509 -in /etc/pki/vdsm/certs/vdsmcert.pem -enddate -noout
     Fri Nov 13 11:50:24 UTC 2015
     notAfter=Nov 18 17:13:03 2015 GMT

     Host dell-r210ii-13 certification is about to expire at 2015-11-18. Please
     renew the host's certification.

3. expired

   - ca:
     # date ; openssl x509 -in /etc/pki/ovirt-engine/ca.pem -enddate -noout
     Tue Dec  1 00:01:16 CET 2015
     notAfter=Nov 18 17:02:35 2015 GMT

     Engine's CA certification has expired at 2015-11-18.
     
   - engine:
     # date ; openssl x509 -in /etc/pki/ovirt-engine/certs/engine.cer -enddate \
       -noout
     Tue Dec  1 00:12:38 CET 2015
     notAfter=Nov 18 17:02:39 2015 GMT
     
     Engine's certification has expired at 2015-11-18. Please renew the engine's 
     certification.

   - host:
     # date ; openssl x509 -in /etc/pki/vdsm/certs/vdsmcert.pem -enddate -noout
     Tue Dec  1 00:22:31 UTC 2015
     notAfter=Nov 27 18:00:10 2015 GMT

     VDSM dell-r210ii-13 command failed: General SSLEngine problem

Comment 8 Jiri Belka 2015-11-13 13:30:06 UTC
FYI https://bugzilla.redhat.com/show_bug.cgi?id=1281815 about not visible event msg about expired host cert.