1040009 – Automatic CA subsystem certificate renewal is broken on CA clones

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1040009 - Automatic CA subsystem certificate renewal is broken on CA clones

Summary: Automatic CA subsystem certificate renewal is broken on CA clones

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 6
Classification:	Red Hat
Component:	ipa
Sub Component:
Version:	6.4
Hardware:	Unspecified
OS:	Unspecified
Priority:	medium
Severity:	unspecified
Target Milestone:	rc
Target Release:	---
Assignee:	Martin Kosek
QA Contact:	Namita Soman
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:	1040018 1061410
TreeView+	depends on / blocked

Reported:	2013-12-10 13:48 UTC by Dmitri Pal
Modified:	2018-12-05 16:44 UTC (History)
CC List:	5 users (show)
Fixed In Version:	ipa-3.0.0-38.el6
Doc Type:	Bug Fix
Doc Text:	Cause: A bug in the python readline module causes a stray escape sequence to be prepended to the output of the script which certmonger uses to acquire renewed certificates on CA clones. Consequence: Certmonger fails to parse the output of the script and certificate renewal fails. Fix: Work around the bug in the python readline module. Result: Certmonger can successfully parse the output of the script and complete the renewal.
Clone Of:
Clones:	1040018 (view as bug list)
Environment:
Last Closed:	2014-10-14 07:32:29 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHBA-2014:1383	0	normal	SHIPPED_LIVE	ipa bug fix and enhancement update	2014-10-14 01:21:36 UTC

Description Dmitri Pal 2013-12-10 13:48:20 UTC

This bug is created as a clone of upstream ticket:
https://fedorahosted.org/freeipa/ticket/4064

On CA clones, certmonger uses the `dogtag-ipa-retrieve-agent-submit` CA helper script to retrieve renewed CA subsystem certificates from master CA. Certmonger expects the script to write the certificate in PEM format to its standard output. The script does that, but prepends an extra "\033[?1034h" to the output, causing certmonger to fail to parse the certificate.

The erroneous output is caused by a bug in readline: http://lists.gnu.org/archive/html/bug-readline/2013-06/msg00000.html, https://bugzilla.redhat.com/show_bug.cgi?id=880393. The Python readline module is not imported in `dogtag-ipa-retrieve-agent-submit` itself, but in some module it imports.

The workaround is to set the `TERM` environment variable to some terminal type which does not support the meta-key capability (such as vt100) before importing modules in `dogtag-ipa-retrieve-agent-submit`.

Comment 1 Martin Kosek 2014-01-02 14:48:58 UTC

Fixed upstream:
master: https://fedorahosted.org/freeipa/changeset/1357eade4c5086e6c837a49f3008616317f88e5f
ipa-3-3: https://fedorahosted.org/freeipa/changeset/854dbb8ff9e898b289c9464567f141421880a050

Comment 2 Namita Soman 2014-01-06 16:39:46 UTC

Please provide steps to verify.

Comment 3 Martin Kosek 2014-01-07 11:52:05 UTC

I think there are 2 approaches:

1) End to end testing (more difficult approach though more complete):
- Install IPA server
- Install IPA replica with CA
- move system time with both close to the end of cert validity, check if certs renew (based on Yi's work)

2) Certmonger resubmit testing (easier, I was testing it today):
- Install IPA server
- Install IPA replica with CA
- On IPA server, resubmit some of the CA subsystem certificates, for example audit cert:
# getcert resubmit -d '/etc/pki/pki-tomcat/alias' -n 'auditSigningCert cert-pki-ca'
- On IPA server, wait until the certificate is generated and status is back to MONITORING with:
# getcert list -d '/etc/pki/pki-tomcat/alias' -n 'auditSigningCert cert-pki-ca'
- On IPA replica we will try to force renewal of this certificate. Run:
# getcert resubmit -d '/etc/pki/pki-tomcat/alias' -n 'auditSigningCert cert-pki-ca'
- On IPA replica, wait until the certificate is renewed and status is back to MONITORING with:
# getcert list -d '/etc/pki/pki-tomcat/alias' -n 'auditSigningCert cert-pki-ca'
- The new certificate should be now updated, see
# certutil -L -d /var/lib/pki/pki-tomcat/alias/ -n 'auditSigningCert cert-pki-ca'
- PKI service should also be running

With old IPA version, the resubmission of certificate on IPA replica will not be successful, certmonger may either crash or report unsuccessful retrieval of the certificate

Jan, please comment if I missed anything

Comment 5 Kaleem 2014-07-25 12:27:40 UTC

Verified using second method mentioned in https://bugzilla.redhat.com/show_bug.cgi?id=1040009#c3

IPA version:
===========
[root@hp-bl280cg6-01 ipa-ca-install]# rpm -q ipa-server
ipa-server-3.0.0-42.el6.x86_64
[root@hp-bl280cg6-01 ipa-ca-install]# 

Snip from automation log:
=========================

On Master
=========

Resubmitting "20140725114312" to "dogtag-ipa-renew-agent".
:: [   PASS   ] :: Resubmiting the auditSigningCert for renewal (Expected 0, got 0)
:: [   PASS   ] :: Checking the status of certificate (Expected 0, got 0)
Number of certificates and requests being tracked: 8.
Request ID '20140725114312':
	status: SUBMITTING
	stuck: no
	key pair storage: type=NSSDB,location='/var/lib/pki-ca/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin='824831973245'
	certificate: type=NSSDB,location='/var/lib/pki-ca/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
	CA: dogtag-ipa-renew-agent
	issuer: CN=Certificate Authority,O=TESTRELM.TEST
	subject: CN=CA Audit,O=TESTRELM.TEST
	expires: 2016-07-14 11:42:32 UTC
	key usage: digitalSignature,nonRepudiation
	pre-save command: /usr/lib64/ipa/certmonger/stop_pkicad
	post-save command: /usr/lib64/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca"
	track: yes
	auto-renew: yes
:: [   PASS   ] :: Running 'cat /tmp/bz_1040009.txt' (Expected 0, got 0)
:: [   PASS   ] :: File '/tmp/bz_1040009.txt' should contain 'SUBMITTING' 
:: [   PASS   ] :: Running 'sleep 120' (Expected 0, got 0)
:: [   PASS   ] :: Checking the status of certificate (Expected 0, got 0)
Number of certificates and requests being tracked: 8.
Request ID '20140725114312':
	status: MONITORING
	stuck: no
	key pair storage: type=NSSDB,location='/var/lib/pki-ca/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin='824831973245'
	certificate: type=NSSDB,location='/var/lib/pki-ca/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
	CA: dogtag-ipa-renew-agent
	issuer: CN=Certificate Authority,O=TESTRELM.TEST
	subject: CN=CA Audit,O=TESTRELM.TEST
	expires: 2016-07-14 12:06:23 UTC
	key usage: digitalSignature,nonRepudiation
	pre-save command: /usr/lib64/ipa/certmonger/stop_pkicad
	post-save command: /usr/lib64/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca"
	track: yes
	auto-renew: yes
:: [   PASS   ] :: Running 'cat /tmp/bz_1040009.txt' (Expected 0, got 0)
:: [   PASS   ] :: File '/tmp/bz_1040009.txt' should contain 'MONITORING'

On Replica
==========

Resubmitting "20140725115536" to "dogtag-ipa-retrieve-agent-submit".
:: [   PASS   ] :: Resubmiting the auditSigningCert for renewal (Expected 0, got 0)
:: [   PASS   ] :: Checking the status of certificate (Expected 0, got 0)
Number of certificates and requests being tracked: 8.
Request ID '20140725115536':
	status: SUBMITTING
	stuck: no
	key pair storage: type=NSSDB,location='/var/lib/pki-ca/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin='655437621144'
	certificate: type=NSSDB,location='/var/lib/pki-ca/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
	CA: dogtag-ipa-retrieve-agent-submit
	issuer: CN=Certificate Authority,O=TESTRELM.TEST
	subject: CN=CA Audit,O=TESTRELM.TEST
	expires: 2016-07-14 11:42:32 UTC
	key usage: digitalSignature,nonRepudiation
	pre-save command: /usr/lib64/ipa/certmonger/stop_pkicad
	post-save command: /usr/lib64/ipa/certmonger/restart_pkicad "auditSigningCert cert-pki-ca"
	track: yes
	auto-renew: yes
:: [   PASS   ] :: Running 'cat /tmp/bz_1040009.txt' (Expected 0, got 0)
:: [   PASS   ] :: File '/tmp/bz_1040009.txt' should contain 'SUBMITTING' 
Fri Jul 25 08:13:44 EDT 2014
:: [   PASS   ] :: Running 'sleep 300' (Expected 0, got 0)
:: [   PASS   ] :: Checking the status of certificate (Expected 0, got 0)
Number of certificates and requests being tracked: 8.
Request ID '20140725115536':
	status: MONITORING
	stuck: no
	key pair storage: type=NSSDB,location='/var/lib/pki-ca/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin='655437621144'
	certificate: type=NSSDB,location='/var/lib/pki-ca/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
	CA: dogtag-ipa-retrieve-agent-submit
	issuer: CN=Certificate Authority,O=TESTRELM.TEST
	subject: CN=CA Audit,O=TESTRELM.TEST
	expires: 2016-07-14 12:06:23 UTC
	key usage: digitalSignature,nonRepudiation
	pre-save command: /usr/lib64/ipa/certmonger/stop_pkicad
	post-save command: /usr/lib64/ipa/certmonger/restart_pkicad "auditSigningCert cert-pki-ca"
	track: yes
	auto-renew: yes
:: [   PASS   ] :: Running 'cat /tmp/bz_1040009.txt' (Expected 0, got 0)
:: [   PASS   ] :: File '/tmp/bz_1040009.txt' should contain 'MONITORING'

Comment 6 errata-xmlrpc 2014-10-14 07:32:29 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-1383.html

Note You need to log in before you can comment on or make changes to this bug.