Bug 1780782
| Summary: | ipa-cert-fix tool fails when the Dogtag CA SSL CSR is missing from CS.cfg | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 8 | Reporter: | Marco Rhodes <mrhodes> |
| Component: | ipa | Assignee: | Florence Blanc-Renaud <frenaud> |
| Status: | CLOSED ERRATA | QA Contact: | ipa-qe <ipa-qe> |
| Severity: | medium | Docs Contact: | |
| Priority: | high | ||
| Version: | --- | CC: | aakkiang, abokovoy, abroy, amore, dpal, ftweedal, jiazhang, ksiddiqu, matt, mharmsen, myusuf, ndehadra, pasik, pcech, rcritten, rhcs-maint, ssidhaye, tmihinto, tscherf, vvanhaft, wrydberg |
| Target Milestone: | rc | Keywords: | TestCaseProvided, Triaged |
| Target Release: | --- | Flags: | jiazhang:
needinfo-
pm-rhel: mirror+ |
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | ipa-4.9.1-1 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-05-18 15:47:50 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 1894575 | ||
|
Description
Marco Rhodes
2019-12-06 22:06:21 UTC
I am moving this issue to pki-server component as the error happens inside the tool 'pki-server cert-fix' which is used internally by ipa-cert-fix. I found upstream PKI tickets to retrieve the certreq and cert from LDAP directly instead of storing a redundant copy in CS.cfg: https://pagure.io/dogtagpki/issue/1552 https://pagure.io/dogtagpki/issue/1598 IMO, there are 2 ways to fix this BZ: Option 1 [Low risk]: ==================== Retrieve the cert request value from /var/lib/certmonger/requests and stick it into CS.cfg This needs to be implemented by ipa-cert-fix Option 2 [High risk]: ===================== Remove the *.certreq and *.cert from CS.cfg and retrieve it from LDAP. This means, we need to handle different installation scenarios. I have to dig deeper to see the impact. This must be implemented in PKI and tested *thoroughly* Question to Marco: is there an empty value in *.sslserver.certreq or the entire key/value pair is missing from CS.cfg? Dinesh: this is probably a symptom of an older bug that caused the .certreq directives to disappear: https://pagure.io/freeipa/issue/7288. As for how to fix, I am OK with Option 1. Because we are root we could construct a CSR ourselves, too. Option 2 is too much effort to make reliable. I'm being hit by this. In anwser to Dinesh's question in #14, here is what I have for certreq in my CS.cfg: [root@localhost ca]# grep certreq CS.cfg ca.audit_signing.certreq=None ca.ocsp_signing.certreq=None ca.subsystem.certreq=None (In reply to Dinesh Prasanth from comment #14) > I found upstream PKI tickets to retrieve the certreq and cert from LDAP > directly > instead of storing a redundant copy in CS.cfg: > https://pagure.io/dogtagpki/issue/1552 > https://pagure.io/dogtagpki/issue/1598 > > IMO, there are 2 ways to fix this BZ: > > Option 1 [Low risk]: > ==================== > Retrieve the cert request value from /var/lib/certmonger/requests and stick > it into CS.cfg > > This needs to be implemented by ipa-cert-fix > > Option 2 [High risk]: > ===================== > Remove the *.certreq and *.cert from CS.cfg and retrieve it from LDAP. This > means, we need to > handle different installation scenarios. > > I have to dig deeper to see the impact. This must be implemented in PKI and > tested *thoroughly* > > > Question to Marco: is there an empty value in *.sslserver.certreq or the > entire key/value pair is > missing from CS.cfg? In each of the 3 occurrences I've seen, the entire key/value pair is missing from CS.cfg. (In reply to Matt Mossholder from comment #16) > I'm being hit by this. In anwser to Dinesh's question in #14, here is what I > have for certreq in my CS.cfg: > > [root@localhost ca]# grep certreq CS.cfg > ca.audit_signing.certreq=None > ca.ocsp_signing.certreq=None > ca.subsystem.certreq=None Note that you are missing the ca.sslserver.certreq .... This is the same as Marco observed. (In reply to Marco Rhodes from comment #17) > In each of the 3 occurrences I've seen, the entire key/value pair is missing > from CS.cfg. As Fraser pointed out in #c15, this looks like the symptom of an older bug that was already fixed. As per #comment 15, I am reassigning this bug to IPA. Another possibility to remediate this issue is to fix up CS.cfg during ipa-server-upgrade. IMO this is more desirable than adding extra complexity to ipa-cert-fix. Thank you taking your time and submitting this request for Red Hat Enterprise Linux 7. Unfortunately, this bug cannot be kept even as a stretch goal and was postponed to RHEL8. Marco, Do you happen to have reproducer steps? We were wondering if setting ca.sslserver.certreq=None is sufficient to workaround this bug. If so, we'd move the component to pki-core and fix this in Dogtag. Otherwise, we'd probably keep this in IPA, as pki-server cert-fix is meant to be run offline (and wouldn't necessarily have LDAP access). Thanks! Reproducer steps: 1. install ipa server with --no-ntp --no-dnssec-validation (for test purpose, as the test will manipulate the date), then stop the services: ipactl stop 2. move the date post expiration: date -s +2years 3. edit /etc/pki/pki-tomcat/ca/CS.cfg and delete the line ca.sslserver.certreq=MII... 4. restart the services: ipactl start -> the command fails as pki fails to start 5. force-restart the services (389ds must be running in order to launch ipa-cert-fix): ipactl start --ignore-service-failures 6. launch ipa-cert-fix -> the command fails because it doesn't find the CSR Summary of what we know so far: =============================== 1. How do we get into this situation: In older versions of IPA, a known issue (https://pagure.io/freeipa/issue/7288) resulted in removal of the directive ca.sslserver.certreq from /etc/pki/pki-tomcat/ca/CS.cfg after the renewal of "Server-Cert cert-pki-ca". 2. What is the consequence: When ipa-cert-fix is launched and the directive is missing, the command fails. The command is internally calling "pki-server cert-fix", that should produce a file containing the renewed cert in /etc/pki/pki-tomcat/certs/sslserver.crt, but: - pki-server cert-fix fails if the directive is missing. - ipa-cert-fix does properly not handle the error returned by pki-server cert-fix and goes on, trying to read the file sslserver.crt that wasn't created. Alex suggested to check the behavior of ipa-cert-fix if we manually write "ca.sslserver.certreq=None" to the CS.cfg file, but this doesn't fix the problem: "pki-server cert-fix" also exits with 1 but sslserver.crt is created as an empty file and ipa-cert-fix fails when trying to initialize the cert from the file content. IMO the fix should be done on IPA side. We can either add an upgrade plugin that re-adds ca.sslserver.certreq into CS.cfg or modify ipa-cert-fix for it to read the CSR from LDAP (ipa-cert-fix already requires the Directory Server to be running when it is launched). We also need to properly handle when "pki-server cert-fix" exits on error. Upstream ticket: https://pagure.io/freeipa/issue/8618 Fixed upstream master: https://pagure.io/freeipa/c/b8ece644e849f8341544bcadba8b19fd888dd90d https://pagure.io/freeipa/c/1a988ba96d7563d66e47db0d6bcc13cad7a85149 https://pagure.io/freeipa/c/98711e8edf052211e5ccc3c1ac3c2367c6f23aff TestCase provided upstream in a new file test_integration/test_ipa_cert_fix.py Fixed upstream ipa-4-9: https://pagure.io/freeipa/c/f36e518b5704b02b81a4b80a1b84c429594cf5ce https://pagure.io/freeipa/c/eb711f781322657b0b3d77332f2462ecfb27db95 https://pagure.io/freeipa/c/7f2be8a45a1d4baff0074cf4d8c446e3d08db795 Fixed upstream ipa-4-8: https://pagure.io/freeipa/c/c9e0aa31cbd614ef7da5e94aeca3c292219d364a https://pagure.io/freeipa/c/8f53cc36723e5ef84fc2c3cc143d8e0f2c7f283b https://pagure.io/freeipa/c/c1207c34a768ed5eb11a161c989484183281892b Pre-verified using compose: rhel-8.4.0-mbs/9665-1253-idm/
Using version :
2021-01-29T07:48:00 name: ipa-server
2021-01-29T07:48:00 release: 1.module+el8.4.0+9665+c9815399
2021-01-29T07:48:00 source: rpm
2021-01-29T07:48:00 version: 4.9.1
============================= test session starts ==============================
platform linux -- Python 3.6.8, pytest-3.10.1, py-1.10.0, pluggy-0.13.1 -- /usr/libexec/platform-python
cachedir: /home/cloud-user/.pytest_cache
metadata: {'Python': '3.6.8', 'Platform': 'Linux-4.18.0-279.el8.x86_64-x86_64-with-redhat-8.4-Ootpa', 'Packages': {'pytest': '3.10.1', 'py': '1.10.0', 'pluggy': '0.13.1'}, 'Plugins': {'metadata': '1.11.0', 'html': '1.22.1', 'multihost': '3.0', 'sourceorder': '0.5'}}
rootdir: /usr/lib/python3.6/site-packages/ipatests, inifile:
plugins: metadata-1.11.0, html-1.22.1, multihost-3.0, sourceorder-0.5
collecting ... collected 1 item
test_integration/test_ipa_cert_fix.py::TestIpaCertFix::test_missing_csr PASSED [100%]
---------------- generated xml file: /home/cloud-user/junit.xml ----------------
----------- generated html file: file:///home/cloud-user/report.html -----------
========================= 1 passed in 1019.04 seconds ==========================
Verified using nightly compose and using version:
2021-02-02T13:14:19+0000 name: ipa-server
2021-02-02T13:14:19+0000 release: 1.module+el8.4.0+9665+c9815399
2021-02-02T13:14:19+0000 source: rpm
2021-02-02T13:14:19+0000 version: 4.9.1
Test log:
============================= test session starts ==============================
platform linux -- Python 3.6.8, pytest-3.10.1, py-1.10.0, pluggy-0.13.1 -- /usr/libexec/platform-python
cachedir: /home/cloud-user/.pytest_cache
metadata: {'Python': '3.6.8', 'Platform': 'Linux-4.18.0-280.el8.x86_64-x86_64-with-redhat-8.4-Ootpa', 'Packages': {'pytest': '3.10.1', 'py': '1.10.0', 'pluggy': '0.13.1'}, 'Plugins': {'metadata': '1.11.0', 'html': '1.22.1', 'multihost': '3.0', 'sourceorder': '0.5'}}
rootdir: /usr/lib/python3.6/site-packages/ipatests, inifile:
plugins: metadata-1.11.0, html-1.22.1, multihost-3.0, sourceorder-0.5
collecting ... collected 1 item
test_integration/test_ipa_cert_fix.py::TestIpaCertFix::test_missing_csr PASSED [100%]
---------------- generated xml file: /home/cloud-user/junit.xml ----------------
----------- generated html file: file:///home/cloud-user/report.html -----------
========================= 1 passed in 1127.29 seconds ==========================
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: idm:DL1 and idm:client security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:1846 |