Bug 2097560 - Warning when ovsdb-server certificates are about to expire(OVN certificate)
Summary: Warning when ovsdb-server certificates are about to expire(OVN certificate)
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine
Version: 4.5.0
Hardware: All
OS: Linux
medium
medium
Target Milestone: ovirt-4.5.2
: ---
Assignee: Milan Zamazal
QA Contact: msheena
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-06-16 01:36 UTC by Marcus West
Modified: 2022-09-08 17:44 UTC (History)
5 users (show)

Fixed In Version: ovirt-engine-4.5.2.4
Doc Type: Enhancement
Doc Text:
Expiration of ovirt-provider-ovn certificate is now checked regularly along with other RHV certificates (engine CA, engine, or hypervisors) and if ovirt-provider-ovn is going to expire or has expired, the warning or alert is raised to the audit log. To renew the ovirt-provider-ovn certificate, run engine-setup. If your ovirt-provider-ovn certificate expires on a previous RHV version, you must upgrade to RHV 4.4 SP1 batch 2 or newer, and the ovirt-provider-ovn certificate will be renewed automatically as part of engine-setup.
Clone Of:
Environment:
Last Closed: 2022-09-08 11:28:53 UTC
oVirt Team: Network
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github oVirt ovirt-engine pull 562 0 None open packaging: Make sure OVN certificates are renewed when needed 2022-07-28 09:17:22 UTC
Github oVirt ovirt-engine pull 576 0 None open Check expiration of ovirt-provider-ovn certificate 2022-08-08 06:55:33 UTC
Github oVirt ovirt-engine pull 603 0 None open Fix checking ovirt-provider-ovn certificate validity 2022-08-19 09:45:18 UTC
Github oVirt ovirt-engine pull 604 0 None open Fix checking ovirt-provider-ovn certificate validity 2022-08-19 12:15:33 UTC
Red Hat Issue Tracker RHV-46439 0 None None None 2022-06-16 01:38:54 UTC
Red Hat Knowledge Base (Solution) 6877501 0 None None None 2022-06-16 01:40:37 UTC
Red Hat Product Errata RHSA-2022:6393 0 None None None 2022-09-08 11:29:26 UTC

Description Marcus West 2022-06-16 01:36:03 UTC
Description of problem:

There is no warning when ovsdb-server is about to expire.  We would like a warning so that preventative steps can be taken

Version-Release number of selected component (if applicable):

ovirt-engine-4.5.0.7-0.9.el8ev.noarch

How reproducible:

Always

Steps to Reproduce:

1. Allow ovsdb-server certs to expire

Actual results:

OVS service stops functioning as expected, and logs get filled with errors

Expected results:

An alert or warning so that the certificates can be renewed before they expire

Additional info:

The renewal process is not handled by `engine-setup`, and needs to be done manually -> https://access.redhat.com/solutions/6877501  Am hoping that this can be fixed in BZ#2097558

Comment 2 Michael Burman 2022-07-28 09:53:08 UTC
Note that the warning will for engine certificate is going to expire. 

We are not going to have a specific check for OVN certificate, because it will be renewed at the same time as engine certificate.
With this fix we will get a warning that engine certificate is going to expire and after renewal also OVN certificate is going to be renewed.
So no special warning specifically for OVN is needed.

Comment 3 Michael Burman 2022-07-28 09:54:42 UTC
(In reply to Michael Burman from comment #2)
> Note that the warning will be for engine certificate is going to expire. 
> 
> We are not going to have a specific check for OVN certificate, because it
> will be renewed at the same time as engine certificate.
> With this fix we will get a warning that engine certificate is going to
> expire and after renewal also OVN certificate is going to be renewed.
> So no special warning specifically for OVN is needed.

Comment 5 Milan Zamazal 2022-07-28 11:42:23 UTC
(In reply to Michael Burman from comment #2)

> We are not going to have a specific check for OVN certificate, because it
> will be renewed at the same time as engine certificate.
> With this fix we will get a warning that engine certificate is going to
> expire and after renewal also OVN certificate is going to be renewed.
> So no special warning specifically for OVN is needed.

Please note there can be a situation, perhaps inherited from older Engine versions, when the OVN certificates expire earlier than the other certificates, see Comment 1. Then there should be a warning about the OVN certificates specifically, or the Engine certificate should be scheduled for renewal earlier, according to the OVN certificates expiration.

Comment 6 Michael Burman 2022-07-28 13:00:40 UTC
(In reply to Milan Zamazal from comment #5)
> (In reply to Michael Burman from comment #2)
> 
> > We are not going to have a specific check for OVN certificate, because it
> > will be renewed at the same time as engine certificate.
> > With this fix we will get a warning that engine certificate is going to
> > expire and after renewal also OVN certificate is going to be renewed.
> > So no special warning specifically for OVN is needed.
> 
> Please note there can be a situation, perhaps inherited from older Engine
> versions, when the OVN certificates expire earlier than the other
> certificates, see Comment 1. Then there should be a warning about the OVN
> certificates specifically, or the Engine certificate should be scheduled for
> renewal earlier, according to the OVN certificates expiration.

Thanks Milan for letting us know that. Good to know there is a dedicated warning for OVN expired certificate for such situation.

Comment 7 Milan Zamazal 2022-08-01 07:50:28 UTC
Looking into the Engine code, we provide expiration alerts only for CA, Engine and host certificates and not the other ones. As Michael pointed out, under normal circumstances, the OVN certificates should be renewed together with the Engine certificate.

Martin, what do you think, is it enough to fix the OVN certificates renewal and then rely on the Engine certificate expiration warnings or should we introduce an additional check for the OVN certificates?

Comment 8 Martin Perina 2022-08-01 08:00:50 UTC
I think relying on engine certificate expiration time is enough, if needed CEE can create a KCS article how to handle OVN certificate expiration before customers would upgrade to RHV 4.4 SP1 batch 2

Comment 9 Milan Zamazal 2022-08-02 15:44:19 UTC
If the expiration dates of Engine and OVN certificates get significantly misaligned (which may happen due to Bug 2097558) and engine-setup is not run at least once a year regularly, it may still happen the OVN certificates expire without a warning. I'm not sure how significant this case is; I'd presume not that much considering this is a special case, engine-setup is usually run at least once a year to renew the web certificates (unless a custom CA is used?), the lifetime of the OVN certificates is 5 years, and running engine-setup is an easy remedy in case the OVN certificates actually expire. Do you agree or does anybody object to closing this bug once the fix for Bug 2097558 is available?

Comment 10 Milan Zamazal 2022-08-03 07:45:53 UTC
> so once engine certificate is renewed on RHV 4.4 SP1 batch 2 or newer version, there is no misalignment between engine certificate expiration and ovsdb-server expiration,

This is not entirely true, if Engine and OVN certificates are not renewed at the same time (e.g. one of them is in the renewal period while the other one not yet when running engine-setup) then their expiration dates are not aligned.

Comment 11 Milan Zamazal 2022-08-05 17:39:05 UTC
As discussed offline, in the end result, we'll add a separate warning for OVN certificates expiration.

Comment 14 Martin Perina 2022-08-08 06:55:33 UTC
During the investigation we have found out, that ovirt-provider-ovn may be created independently of engine certificate, so we needed check ovirt-provider-ovn certificate separately and display specific audit log events for ovirt-provider-ovn:

OVIRT_PROVIDER_OVN_CERTIFICATE_IS_ABOUT_TO_EXPIRE=ovirt-provider-ovn's certificate is about to expire at ${ExpirationDate}. Please renew the certificate.
OVIRT_PROVIDER_OVN_CERTIFICATE_IS_ABOUT_TO_EXPIRE_ALERT=ovirt-provider-ovn's certificate is about to expire at ${ExpirationDate}. Please renew the certificate.
OVIRT_PROVIDER_OVN_CERTIFICATE_HAS_EXPIRED=ovirt-provider-ovn's certificate has expired at ${ExpirationDate}. Please renew the certificate.

Comment 26 msheena 2022-08-23 18:31:42 UTC
Verified on
===========
ovirt-engine-4.5.2.4-0.1.el8ev.noarch

Verification steps
==================
1. Check validity periods for ovirt-provider-ovn certificate and Apache https certificate
[root@engine ~]# openssl x509 -text -noout -in /etc/pki/ovirt-engine/certs/apache.cer | grep -A 2 Validity
        Validity
            Not Before: Aug 21 17:10:19 2022 GMT
            Not After : Sep 24 17:10:19 2023 GMT
[root@engine ~]# openssl x509 -text -noout -in /etc/pki/ovirt-engine/certs/ovirt-provider-ovn.cer | grep -A 2 Validity
        Validity
            Not Before: Aug 21 17:10:21 2022 GMT
            Not After : Aug 23 17:10:21 2027 GMT

2. Forward engine vm time
[root@engine ~]# date -s "Aug 22 17:10:21 2027 GMT"

3. Renew Apache certificate using https://access.redhat.com/solutions/3329431

4. Reduce certificate check period
[root@engine ~]# engine-config -s CertificationValidityCheckTimeInHours=0.02

5. Observe warning about ovirt-provider-ovn certiticate expiration in /var/log/ovirt-engine/engine.log
2027-08-22 20:10:57,188+03 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-1) [5af9c159] EVENT_ID: OVIRT_PROVIDER_OVN_CERTIFICATE_IS_ABOUT_TO_EXPIRE_ALERT(904), ovirt-provider-ovn's certificate is about to expire at 2027-08-23. Please renew the certificate.

Comment 29 Casper (RHV QE bot) 2022-09-04 20:05:04 UTC
This bug has low overall severity and is not going to be further verified by QE. If you believe special care is required, feel free to properly align relevant severity, flags and keywords to raise PM_Score or use one of the Bumps ('PrioBumpField', 'PrioBumpGSS', 'PrioBumpPM', 'PrioBumpQA') in Keywords to raise it's PM_Score above verification threashold (1000).

Comment 32 errata-xmlrpc 2022-09-08 11:28:53 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: RHV Manager (ovirt-engine) [ovirt-4.5.2] bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:6393


Note You need to log in before you can comment on or make changes to this bug.