RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1040018 - Automatic CA subsystem certificate renewal is broken on CA clones
Summary: Automatic CA subsystem certificate renewal is broken on CA clones
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: ipa
Version: 7.0
Hardware: Unspecified
OS: Unspecified
medium
unspecified
Target Milestone: rc
: ---
Assignee: Martin Kosek
QA Contact: Namita Soman
URL:
Whiteboard:
Depends On: 1040009 1049532
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-12-10 14:08 UTC by Dmitri Pal
Modified: 2014-06-18 00:13 UTC (History)
3 users (show)

Fixed In Version: ipa-3.3.3-10.el7
Doc Type: Bug Fix
Doc Text:
Clone Of: 1040009
Environment:
Last Closed: 2014-06-13 10:48:12 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1033273 0 medium CLOSED ipa cert automatic renew: cert automatically renewed, but renewed client cert not be recognized 2021-02-22 00:41:40 UTC

Internal Links: 1033273

Description Dmitri Pal 2013-12-10 14:08:56 UTC
+++ This bug was initially created as a clone of Bug #1040009 +++

This bug is created as a clone of upstream ticket:
https://fedorahosted.org/freeipa/ticket/4064

On CA clones, certmonger uses the `dogtag-ipa-retrieve-agent-submit` CA helper script to retrieve renewed CA subsystem certificates from master CA. Certmonger expects the script to write the certificate in PEM format to its standard output. The script does that, but prepends an extra "\033[?1034h" to the output, causing certmonger to fail to parse the certificate.

The erroneous output is caused by a bug in readline: http://lists.gnu.org/archive/html/bug-readline/2013-06/msg00000.html, https://bugzilla.redhat.com/show_bug.cgi?id=880393. The Python readline module is not imported in `dogtag-ipa-retrieve-agent-submit` itself, but in some module it imports.

The workaround is to set the `TERM` environment variable to some terminal type which does not support the meta-key capability (such as vt100) before importing modules in `dogtag-ipa-retrieve-agent-submit`.

Comment 3 Martin Kosek 2014-01-07 16:19:32 UTC
It was found out that PKI service did not start properly after the renewal, this is being solved upstream:

https://fedorahosted.org/freeipa/ticket/4092

Comment 4 Martin Kosek 2014-01-08 09:02:23 UTC
Ticket 4092 was fixed upstream:

master: 911f5e9eb76099f8e5cfcff1232c1b10ad05b45a
ipa-3-3: edccf59d8018349bc3596e017a660dcb83034932

Comment 5 Martin Kosek 2014-01-08 09:04:01 UTC
Linking to related SELinux issue preventing renewal with enforced SELinux - Bug 1049532.

Comment 6 Scott Poore 2014-02-10 22:38:43 UTC
Martin,

Does this have to be verified on an IPA Replica with CA installed or can it be verified on an IPA "Master" (single server environment)?

Is it enough to do a getcert resubmit  and confirm the expiration or do we need to go all the way to change the date and watch the auto-renewal system resubmit?

Thanks,
Scott

Comment 7 Rob Crittenden 2014-02-11 13:59:44 UTC
Scott, needs another master to fully test this, along with switching dates, etc.

Only one master does the actual renewal. It stuffs the result into LDAP which the other masters get the updated certificates from. It is this process that was going sideways. ASCII garbage was being included when we fetched the certificate from LDAP and provided it to certmonger.

Comment 8 Scott Poore 2014-02-11 14:28:04 UTC
When you say another master, you mean another IPA server created with --setup-ca for the same domain?

Comment 9 Rob Crittenden 2014-02-11 14:34:01 UTC
Correct. The renewal is done differently on the master initiating the renewal and the other masters and this bug presented on the other masters.

Comment 10 Scott Poore 2014-02-11 17:37:11 UTC
Verified.

Version ::

ipa-server-3.3.3-17.el7.x86_64

Results ::

resubmit test:

[root@master ~]# getcert list -d '/etc/pki/pki-tomcat/alias' -n 'auditSigningCert cert-pki-ca'
Number of certificates and requests being tracked: 7.
Request ID '20140211000124':
        status: MONITORING
        stuck: no
        key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin='906281234271'
        certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
        CA: dogtag-ipa-renew-agent
        issuer: CN=Certificate Authority,O=IPA1.EXAMPLE.TEST
        subject: CN=CA Audit,O=IPA1.EXAMPLE.TEST
        expires: 2016-02-01 00:00:45 UTC
        key usage: digitalSignature,nonRepudiation
        pre-save command: /usr/lib64/ipa/certmonger/stop_pkicad
        post-save command: /usr/lib64/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca"
        track: yes


[root@replica1 ~]# getcert list -d '/etc/pki/pki-tomcat/alias' -n 'auditSigningCert cert-pki-ca'
Number of certificates and requests being tracked: 7.
Request ID '20140211143326':
	status: MONITORING
	stuck: no
	key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin='480267000059'
	certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
	CA: dogtag-ipa-retrieve-agent-submit
	issuer: CN=Certificate Authority,O=IPA1.EXAMPLE.TEST
	subject: CN=CA Audit,O=IPA1.EXAMPLE.TEST
	expires: 2016-02-01 00:00:45 UTC
	key usage: digitalSignature,nonRepudiation
	pre-save command: /usr/lib64/ipa/certmonger/stop_pkicad
	post-save command: /usr/lib64/ipa/certmonger/restart_pkicad "auditSigningCert cert-pki-ca"
	track: yes
	auto-renew: yes

[root@master ~]# getcert resubmit -d '/etc/pki/pki-tomcat/alias' -n 'auditSigningCert cert-pki-ca'
Resubmitting "20140211000124" to "dogtag-ipa-renew-agent".

...waited for status to go back to MONITORING...

[root@master ~]# getcert list -d '/etc/pki/pki-tomcat/alias' -n 'auditSigningCert cert-pki-ca'
Number of certificates and requests being tracked: 7.
Request ID '20140211000124':
        status: MONITORING
        stuck: no
        key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin='906281234271'
        certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
        CA: dogtag-ipa-renew-agent
        issuer: CN=Certificate Authority,O=IPA1.EXAMPLE.TEST
        subject: CN=CA Audit,O=IPA1.EXAMPLE.TEST
        expires: 2016-02-01 15:32:03 UTC
        key usage: digitalSignature,nonRepudiation
        pre-save command: /usr/lib64/ipa/certmonger/stop_pkicad
        post-save command: /usr/lib64/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca"
        track: yes
        auto-renew: yes


...Then on replica, run resubmit and check...

[root@replica1 ~]# getcert resubmit -d '/etc/pki/pki-tomcat/alias' -n 'auditSigningCert cert-pki-ca'
Resubmitting "20140211143326" to "dogtag-ipa-retrieve-agent-submit".

...wait for status to change to MONITORING...

[root@replica1 ~]# getcert list -d '/etc/pki/pki-tomcat/alias' -n 'auditSigningCert cert-pki-ca'
Number of certificates and requests being tracked: 7.
Request ID '20140211143326':
	status: MONITORING
	stuck: no
	key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin='480267000059'
	certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
	CA: dogtag-ipa-retrieve-agent-submit
	issuer: CN=Certificate Authority,O=IPA1.EXAMPLE.TEST
	subject: CN=CA Audit,O=IPA1.EXAMPLE.TEST
	expires: 2016-02-01 15:32:03 UTC
	key usage: digitalSignature,nonRepudiation
	pre-save command: /usr/lib64/ipa/certmonger/stop_pkicad
	post-save command: /usr/lib64/ipa/certmonger/restart_pkicad "auditSigningCert cert-pki-ca"
	track: yes
	auto-renew: yes

...finally, confirm certs match:

[root@replica1 ~]# ssh $MASTER "certutil -L -d /var/lib/pki/pki-tomcat/alias/ -n 'auditSigningCert cert-pki-ca'" > cert.master
root.example.test's password: 

[root@replica1 ~]# certutil -L -d /var/lib/pki/pki-tomcat/alias/ -n 'auditSigningCert cert-pki-ca' > cert.replica

[root@replica1 ~]# diff cert.master cert.replica
[root@replica1 ~]# 

Now, automatic renewal with time change...

This was run after a VM rebuild.

ON MASTER:

# ipactl stop
Stopping Directory Service
Stopping ipa-otpd Service
Stopping pki-tomcatd Service
Stopping httpd Service
Stopping ipa_memcached Service
Stopping named Service
Stopping kadmin Service
Stopping krb5kdc Service
ipa: INFO: The ipactl command was successful

# hostname
master.ipa1.example.test

[root@master ~]# getcert list -d '/etc/pki/pki-tomcat/alias' -n 'auditSigningCert cert-pki-ca'
Number of certificates and requests being tracked: 7.
Request ID '20140211161725':
	status: MONITORING
	stuck: no
	key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin='419558786185'
	certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
	CA: dogtag-ipa-renew-agent
	issuer: CN=Certificate Authority,O=IPA1.EXAMPLE.TEST
	subject: CN=CA Audit,O=IPA1.EXAMPLE.TEST
	expires: 2016-02-01 16:16:45 UTC
	key usage: digitalSignature,nonRepudiation
	pre-save command: /usr/lib64/ipa/certmonger/stop_pkicad
	post-save command: /usr/lib64/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca"
	track: yes
	auto-renew: yes

[root@master ~]# date 010415002016
Mon Jan  4 15:00:00 CST 2016

[root@master ~]# ipactl start
Starting Directory Service
Starting krb5kdc Service
Starting kadmin Service
Starting named Service
Starting ipa_memcached Service
Starting httpd Service
Starting pki-tomcatd Service
Starting ipa-otpd Service
ipa: INFO: The ipactl command was successful

ON REPLICA:

# ipactl stop
Stopping Directory Service
Stopping ipa-otpd Service
Stopping pki-tomcatd Service
Stopping httpd Service
Stopping ipa_memcached Service
Stopping named Service
Stopping kadmin Service
Stopping krb5kdc Service

[root@replica1 ~]# getcert list -d '/etc/pki/pki-tomcat/alias' -n 'auditSigningCert cert-pki-ca'
Number of certificates and requests being tracked: 7.
Request ID '20140211164557':
	status: MONITORING
	stuck: no
	key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin='288546002948'
	certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
	CA: dogtag-ipa-retrieve-agent-submit
	issuer: CN=Certificate Authority,O=IPA1.EXAMPLE.TEST
	subject: CN=CA Audit,O=IPA1.EXAMPLE.TEST
	expires: 2016-02-01 16:16:45 UTC
	key usage: digitalSignature,nonRepudiation
	pre-save command: /usr/lib64/ipa/certmonger/stop_pkicad
	post-save command: /usr/lib64/ipa/certmonger/restart_pkicad "auditSigningCert cert-pki-ca"
	track: yes
	auto-renew: yes

[root@replica1 ~]# date 010415002016
Mon Jan  4 15:00:00 CST 2016

[root@replica1 ~]# ipactl start
Starting Directory Service
Starting krb5kdc Service
Starting kadmin Service
Starting named Service
Starting ipa_memcached Service
Starting httpd Service
Starting pki-tomcatd Service
Starting ipa-otpd Service
ipa: INFO: The ipactl command was successful

ON MASTER:

[root@master ~]# getcert list -d '/etc/pki/pki-tomcat/alias' -n 'auditSigningCert cert-pki-ca'
Number of certificates and requests being tracked: 7.
Request ID '20140211161725':
	status: NOTIFYING_VALIDITY
	stuck: no
	key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin='419558786185'
	certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
	CA: dogtag-ipa-renew-agent
	issuer: CN=Certificate Authority,O=IPA1.EXAMPLE.TEST
	subject: CN=CA Audit,O=IPA1.EXAMPLE.TEST
	expires: 2016-02-01 16:16:45 UTC
	key usage: digitalSignature,nonRepudiation
	pre-save command: /usr/lib64/ipa/certmonger/stop_pkicad
	post-save command: /usr/lib64/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca"
	track: yes
	auto-renew: yes

[root@master ~]# getcert list -d '/etc/pki/pki-tomcat/alias' -n 'auditSigningCert cert-pki-ca'
Number of certificates and requests being tracked: 7.
Request ID '20140211161725':
	status: MONITORING
	stuck: no
	key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin='419558786185'
	certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
	CA: dogtag-ipa-renew-agent
	issuer: CN=Certificate Authority,O=IPA1.EXAMPLE.TEST
	subject: CN=CA Audit,O=IPA1.EXAMPLE.TEST
	expires: 2017-12-24 21:14:20 UTC
	key usage: digitalSignature,nonRepudiation
	pre-save command: /usr/lib64/ipa/certmonger/stop_pkicad
	post-save command: /usr/lib64/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca"
	track: yes
	auto-renew: yes

ON REPLICA:

[root@replica1 ~]# getcert list -d '/etc/pki/pki-tomcat/alias' -n 'auditSigningCert cert-pki-ca'
Number of certificates and requests being tracked: 7.
Request ID '20140211164557':
	status: NOTIFYING_VALIDITY
	stuck: no
	key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin='288546002948'
	certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
	CA: dogtag-ipa-retrieve-agent-submit
	issuer: CN=Certificate Authority,O=IPA1.EXAMPLE.TEST
	subject: CN=CA Audit,O=IPA1.EXAMPLE.TEST
	expires: 2016-02-01 16:16:45 UTC
	key usage: digitalSignature,nonRepudiation
	pre-save command: /usr/lib64/ipa/certmonger/stop_pkicad
	post-save command: /usr/lib64/ipa/certmonger/restart_pkicad "auditSigningCert cert-pki-ca"
	track: yes
	auto-renew: yes

[root@replica1 ~]# getcert list -d '/etc/pki/pki-tomcat/alias' -n 'auditSigningCert cert-pki-ca'
Number of certificates and requests being tracked: 7.
Request ID '20140211164557':
	status: MONITORING
	stuck: no
	key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin='288546002948'
	certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
	CA: dogtag-ipa-retrieve-agent-submit
	issuer: CN=Certificate Authority,O=IPA1.EXAMPLE.TEST
	subject: CN=CA Audit,O=IPA1.EXAMPLE.TEST
	expires: 2017-12-24 21:14:20 UTC
	key usage: digitalSignature,nonRepudiation
	pre-save command: /usr/lib64/ipa/certmonger/stop_pkicad
	post-save command: /usr/lib64/ipa/certmonger/restart_pkicad "auditSigningCert cert-pki-ca"
	track: yes
	auto-renew: yes

And to confirm that certs are the same:

[root@replica1 ~]# ssh $MASTER "certutil -L -d /var/lib/pki/pki-tomcat/alias/ -n 'auditSigningCert cert-pki-ca'" > cert.master
root.example.test's password: 

[root@replica1 ~]# certutil -L -d /var/lib/pki/pki-tomcat/alias/ -n 'auditSigningCert cert-pki-ca' > cert.replica

[root@replica1 ~]# diff cert.master cert.replica

Comment 12 Ludek Smid 2014-06-13 10:48:12 UTC
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.


Note You need to log in before you can comment on or make changes to this bug.