Bug 1343796 - Cannot renew expired certificates in IPA 4.2
Summary: Cannot renew expired certificates in IPA 4.2
Keywords:
Status: CLOSED INSUFFICIENT_DATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: ipa
Version: 7.2
Hardware: x86_64
OS: Linux
unspecified
urgent
Target Milestone: rc
: ---
Assignee: IPA Maintainers
QA Contact: Kaleem
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-06-08 01:01 UTC by lmgnid
Modified: 2017-11-22 17:50 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-10-07 12:10:54 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1343798 0 unspecified CLOSED ipa-replica-install fails on adding CA certs to NSS db if multiple versions of the same CA cert is present in LDAP 2021-02-22 00:41:40 UTC
Red Hat Bugzilla 1344093 0 unspecified CLOSED IPA dirsrv start timeout "Detected Disorderly Shutdown last time Directory Server was running, recovering database." 2021-02-22 00:41:40 UTC

Internal Links: 1343798 1344093

Description lmgnid 2016-06-08 01:01:47 UTC
Description of problem:
As in https://bugzilla.redhat.com/show_bug.cgi?id=1330800
I tried what I did with IPA 4.1, but still cannot renew the certificate in IPA 4.2


Version-Release number of selected component (if applicable):
[root@usdev-ops-ipa-01 ~]# rpm -qa | grep ipa
sssd-ipa-1.13.0-40.el7_2.1.x86_64
ipa-admintools-4.2.0-15.el7_2.3.x86_64
python-iniparse-0.4-9.el7.noarch
libipa_hbac-1.13.0-40.el7_2.1.x86_64
python-libipa_hbac-1.13.0-40.el7_2.1.x86_64
ipa-python-4.2.0-15.el7_2.3.x86_64
ipa-client-4.2.0-15.el7_2.3.x86_64
ipa-server-4.2.0-15.el7_2.3.x86_64
ipa-server-dns-4.2.0-15.el7_2.3.x86_64
redhat-access-plugin-ipa-0.9.1-2.el7.noarch

How reproducible:
IPA cannot start as some ceritifdate expired already
[root@usdev-ops-ipa-01 ~]# getcert list | grep expire
        expires: 2018-01-09 00:40:45 UTC
        expires: 2016-05-03 21:14:05 UTC
        expires: 2016-05-03 21:14:01 UTC
        expires: 2016-05-03 21:14:03 UTC
        expires: 2034-05-14 21:13:57 UTC
        expires: 2017-12-29 00:52:21 UTC
        expires: 2018-01-09 00:40:48 UTC
        expires: 2016-05-03 21:15:03 UTC


Steps to Reproduce:
1.ipactl start will fail
2.ipactl restart will fail

Actual results:
1.ipactl status
Directory Service: RUNNING
krb5kdc Service: RUNNING
kadmin Service: RUNNING
named Service: RUNNING
ipa_memcached Service: RUNNING
httpd Service: RUNNING
pki-tomcatd Service: RUNNING
ipa-otpd Service: STOPPED
ipa-dnskeysyncd Service: STOPPED
ipa: INFO: The ipactl command was successful


Expected results:
All IPA service should be running

Additional info:
I already tried what is in https://bugzilla.redhat.com/show_bug.cgi?id=1330800

Comment 1 lmgnid 2016-06-08 01:02:48 UTC
As all method that I can google or in the ticket bellow failed, any suggestion for how to renew the ceritificates?

https://bugzilla.redhat.com/show_bug.cgi?id=1330800

Comment 3 lmgnid 2016-06-10 00:45:44 UTC
could you help to check this as it quite urgent for us. Thanks!

Comment 4 Petr Vobornik 2016-06-13 14:39:17 UTC
Could you paste here exactly the sequence of commands you tried to renew the certificates?

Comment 5 Petr Vobornik 2016-06-13 14:42:03 UTC
Please note that ipa-cacert-manage should be used only for renewing the Certificate Authority(CA) cert. Which is by default in new IPAs valid for 20 years so it doesn't have to be usually renewed.

Comment 6 lmgnid 2016-06-13 18:23:17 UTC
Hi Petr,

Thanks for your reminder and here is what I did, following your suggestion under https://bugzilla.redhat.com/show_bug.cgi?id=1330800

1: Move data back to April:

2: Restart IPA and it was OK:

3: Check the certs status again:
[root@usdev-ops-ipa-01 ~]# getcert list | grep -e expires -e 2016
Request ID '20160109004901':
        expires: 2018-01-09 00:40:45 UTC
Request ID '20160109005301':
        expires: 2016-05-03 21:14:05 UTC
Request ID '20160109005303':
        expires: 2016-05-03 21:14:01 UTC
Request ID '20160109005305':
        expires: 2016-05-03 21:14:03 UTC
Request ID '20160109005307':
        expires: 2034-05-14 21:13:57 UTC
Request ID '20160109005308':
        expires: 2017-12-29 00:52:21 UTC
Request ID '20160109005609':
        expires: 2018-01-09 00:40:48 UTC
Request ID '20160109005623':
        expires: 2016-05-03 21:15:03 UTC

4: Resubmit the above expired certificates again, for example:
[root@usdev-ops-ipa-01 ~]# getcert resubmit -i 20160109005623
Resubmitting "20160109005623" to "dogtag-ipa-ca-renew-agent".

5: But expired certificates are still not renewed:
[root@usdev-ops-ipa-01 ~]# getcert list | grep -e expires -e 2016
Request ID '20160109004901':
        expires: 2018-01-09 00:40:45 UTC
Request ID '20160109005301':
        expires: 2016-05-03 21:14:05 UTC
Request ID '20160109005303':
        expires: 2016-05-03 21:14:01 UTC
Request ID '20160109005305':
        expires: 2016-05-03 21:14:03 UTC
Request ID '20160109005307':
        expires: 2034-05-14 21:13:57 UTC
Request ID '20160109005308':
        expires: 2017-12-29 00:52:21 UTC
Request ID '20160109005609':
        expires: 2018-01-09 00:40:48 UTC
Request ID '20160109005623':
        expires: 2016-05-03 21:15:03 UTC

Comment 7 Matthew Harmsen 2016-06-27 15:37:48 UTC
Per email from Petr, setting this to IPA component for further investigation.

Comment 8 Petr Vobornik 2016-06-28 12:33:28 UTC
More info is needed:
 - http error_log from IPA server after the resubmit
 - related CA log after the resubmit: 
     /var/log/pki/pki-tomcat/ca/transactions
     /var/log/pki/pki-tomcat/ca/debug
 - full `getcert list` output

Comment 9 lmgnid 2016-06-29 00:07:46 UTC
Hi Petr, thanks fore the comment and I already send the results and logs in email to you.

Comment 10 lmgnid 2016-07-06 18:09:46 UTC
Hello Petr, any findings? Thanks!

Comment 11 Petr Vobornik 2016-07-22 11:05:26 UTC
Copying my email response from Jul 15.

"""
based on

it seems that some certs were renewed but then authentication for RA
user fails.  RA cert is used by IPA for communication with CA.

You may need to check
https://www.freeipa.org/page/Troubleshooting#Authentication_Errors

When RA cert is renewed, correct version is in /etc/httpd/alias cert db
and correct mapping is in LDAP, then other renewals should work correctly.


[29/Mar/2016:23:51:43][http-bio-8443-exec-8]: returnConn: mNumConns now 3
[29/Mar/2016:23:51:43][http-bio-8443-exec-8]: CertUserDBAuthentication:
cannot map certificate to any user
[29/Mar/2016:23:51:43][http-bio-8443-exec-8]: SignedAuditEventFactory:
create()
message=[AuditEvent=AUTH_FAIL][SubjectID=$Unidentified$][Outcome=Failure][AuthMgr=certUserDBAuthMgr][AttemptedCred=CN=IPA
RA,O=INTERNAL.COM] authentication failure"

"""

Given the lack of mail response, I'll close this bz as well. 

If the issue is still present, please let us continue on freeipa-users list as I already wrote in bug 1343798, comment 9. More people will be able to help you there. Or open a support case with Red Hat.

Form for joining mailing list can be found at:
https://www.redhat.com/mailman/listinfo/freeipa-users

Comment 12 lmgnid 2016-07-22 18:02:25 UTC
I got this error when renew the RA cert:
# certutil -A -n ipaCert -d /etc/httpd/alias -t u,u,u -a -i /tmp/ranew.crt  -vvv
Notice: Trust flag u is set automatically if the private key is present.
certutil: could not add certificate to token or database: SEC_ERROR_ADDING_CERT: Error adding certificate to database.

Anyway, I will try the mailing list.

Comment 13 Petr Vobornik 2016-08-12 14:11:44 UTC
Per, https://www.redhat.com/archives/freeipa-users/2016-July/msg00460.html

I'm not really sure what to suggest because you did not say what you have tried, what output from commands you got etc.

I would suggest you to do following:
1. Joining mailing list freeipa-users:
https://www.redhat.com/mailman/listinfo/freeipa-users

2. Write an e-mail with all the information as explained in
http://www.chiark.greenend.org.uk/~sgtatham/bugs.html

3. We can re-open this bug at the moment when the root cause is known. Bugzilla is not a support tool, we need to capture only the root cause here. Before the root cause is know, please be so kind and use mailing list.

Thank you for understanding!

Comment 14 lmgnid 2016-08-15 17:16:54 UTC
@Petr, I already provide all information that you requested in this and the related tickets, not sure why you said "I did not say what I have tried"???, pls let me know what else do you need. The mailing list doesn't help either, I don't know what else to provide and pls let me know if anything else.

Comment 15 Petr Vobornik 2016-08-24 16:20:05 UTC
As was mentioned in https://www.redhat.com/archives/freeipa-users/2016-July/msg00460.html (which was mentioned in comment 13)

Comment 16 Petr Vobornik 2016-08-24 16:21:11 UTC
For reference, this was advised in mail:

Not sure if this is still valid, but anyway:

based on
"""
[29/Mar/2016:23:51:43][http-bio-8443-exec-8]: returnConn: mNumConns now 3
[29/Mar/2016:23:51:43][http-bio-8443-exec-8]: CertUserDBAuthentication:
cannot map certificate to any user
[29/Mar/2016:23:51:43][http-bio-8443-exec-8]: SignedAuditEventFactory:
create()
message=[AuditEvent=AUTH_FAIL][SubjectID=$Unidentified$][Outcome=Failure][AuthMgr=certUserDBAuthMgr][AttemptedCred=CN=IPA
RA,O=INTERNAL.COM] authentication failure
"""

it seems that some certs were renewed but then authentication for RA
user fails.  RA cert is used by IPA for communication with CA.

You may need to check
https://www.freeipa.org/page/Troubleshooting#Authentication_Errors

When RA cert is renewed, correct version is in /etc/httpd/alias cert db
and correct mapping is in LDAP, then other renewals should work correctly.

Comment 17 lmgnid 2016-08-25 00:48:43 UTC
@Petr, thanks for your suggestion, it seems the server that cannot renew ceritificates has different ra.crt with other servers, so I tried to copy the correct ranew.crt from other server to here but still got this error:

[root@usdev-ops-ipa-01 ~]# certutil -A -n ipaCert -d /etc/httpd/alias -t u,u,u -a -i /tmp/ranew.crt
Notice: Trust flag u is set automatically if the private key is present.
certutil: could not add certificate to token or database: SEC_ERROR_ADDING_CERT: Error adding certificate to database.

Any further suggestions? Thanks!

Comment 19 Petr Vobornik 2016-09-09 14:37:30 UTC
Your environment is in unknown state for outside viewer. A complete list of stuff you tried (or at least what you remember) and configuration(number of replicas, cert status, with what options server/replica was install, several log files) you use is required in order to investigate.

Other possibility is to open a support case with Red Hat, generate a SOS report by RH tool(collection of logs and configuration) and  and then support engineer can help you. 

I personally don't have a capacity to do such deep investigation and that's why you were asked to try freeipa users list(comment 13). But even there you need to provide as much as info otherwise nobody will be able to help you.

Comment 20 Jan Cholasta 2016-09-12 11:54:45 UTC
Hi,

IPA RA authentication failure usually means the IPA RA user LDAP entry was not updated with the renewed certificate. See http://www.freeipa.org/page/IPA_2x_Certificate_Renewal for information on how to update the entry (look for "For ipaCert, stored in /etc/httpd/alias you have another job to do.").

Comment 21 Petr Vobornik 2016-09-13 15:45:52 UTC
lmgnid, could you check what Jan proposed in comment 20 and me in comment 11(same thing)

Comment 22 lmgnid 2016-09-13 22:50:34 UTC
Hi Petr and Jan,

I followed all the steps in Comment 20, but the certificates still cannot be renewed. Here are some information after all these changes:

1: Serial is "7", the new value for ipaCert
[root@usdev-ops-ipa-01 ~]# certutil -L -d /etc/httpd/alias -n ipaCert | grep -i serial
        Serial Number: 7 (0x7)

2: Here is the cert bin, i took out the middle part
[root@usdev-ops-ipa-01 ~]# certutil -L -d /etc/httpd/alias -n ipaCert -a
-----BEGIN CERTIFICATE-----
MIIDnzCCAoegAwIBAgIBBzANBgkqhkiG9w0BAQsFADBDMSEwHwYDVQQKExhJTlRF
...
PsnY504OGaodT9IWdsaffmpffA==
-----END CERTIFICATE-----

3: Here is the what inside after the change, not sure if I didn't right change (All PEM middle parts were took out)
[root@usdev-ops-ipa-01 ~]# ldapsearch -x -h localhost -p 389 -D 'cn=directory manager' -W -b uid=ipara,ou=People,o=ipaca
Enter LDAP Password:
# extended LDIF
#
# LDAPv3
# base <uid=ipara,ou=People,o=ipaca> with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#

# ipara, people, ipaca
dn: uid=ipara,ou=people,o=ipaca
description: 2;7;CN=Certificate Authority,O=INTERNAL.COM;CN=IPA RA
 ,O=INTERNAL.COM
userCertificate;binary:: TUlJRG56Q0NBb2VnQXdJQkFnSUJCekFOQmdrcWhraUc5dzBCQVFzR
...
 klRYXN1WFNRTWlnTjJYajR4cjc2OFBzblk1MDRPR2FvZFQ5SVdkc2FmZm1wZmZBPT0=
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
objectClass: cmsuser
uid: ipara
sn: ipara
cn: ipara
usertype: agentType
userstate: 1
userCertificate:: MIIDnzCCAoegAwIBAgIBBzANBgkqhkiG9w0BAQsFADBDMSEwHwYDVQQKExhJ
...
 8PsnY504OGaodT9IWdsaffmpffA==
userCertificate:: MIIDojCCAoqgAwIBAgIFA4/9AAowDQYJKoZIhvcNAQELBQAwQzEhMB8GA1UE
...
 BY5uQHsTpO5O4haY68jXGSlX+a/TlKg==

# search result
search: 2
result: 0 Success
# numResponses: 2
# numEntries: 1

4: After ALL steps in Comment 20, certificate are still not renewed:
[root@usdev-ops-ipa-01 ~]# getcert list | grep "expires: 2016"
        expires: 2016-05-03 21:14:05 UTC
        expires: 2016-05-03 21:14:01 UTC
        expires: 2016-05-03 21:14:03 UTC
        expires: 2016-05-03 21:15:03 UTC
        expires: 2016-05-03 21:14:05 UTC
        expires: 2016-05-03 21:14:01 UTC
        expires: 2016-05-03 21:14:03 UTC
[root@usdev-ops-ipa-01 ~]# date
Tue Apr 26 22:38:10 UTC 2016
[root@usdev-ops-ipa-01 ~]# getcert list | grep CA_UNREACHABLE -A 2 -B 2
Number of certificates and requests being tracked: 12.
Request ID '20160109004901':
        status: CA_UNREACHABLE
        ca-error: Server at https://usdev-ops-ipa-01.internal.com/ipa/xml failed request, will retry: 907 (RPC failed at server.  cannot connect to 'https://usdev-ops-ipa-01.internal.com:443/ca/eeca/ca/profileSubmitSSLClient': (SSL_ERROR_EXPIRED_CERT_ALERT) SSL peer rejected your certificate as expired.).
        stuck: no
--
        auto-renew: yes
Request ID '20160109005609':
        status: CA_UNREACHABLE
        ca-error: Server at https://usdev-ops-ipa-01.internal.com/ipa/xml failed request, will retry: 907 (RPC failed at server.  cannot connect to 'https://usdev-ops-ipa-01.internal.com:443/ca/eeca/ca/profileSubmitSSLClient': (SSL_ERROR_EXPIRED_CERT_ALERT) SSL peer rejected your certificate as expired.).
        stuck: no


Please let me know if you need any other status of this IPA server, or any suggestions?

Comment 23 Petr Vobornik 2016-09-14 10:12:01 UTC
Jan, the guide contains:

   # ldapmodify -x -h localhost -p 7389 -D 'cn=directory manager' -w password
   dn: uid=ipara,ou=people,o=ipaca
   changetype: modify
   add: usercertificate;binary
   usercertificate;binary: MII...PNQ=
   -

I.e. add another certificate to the user entry. In comment 22 I see that the user entry now has 3 certs.
  userCertificate;binary:: TUlJRG
  userCertificate:: MIIDnz
  userCertificate:: MIIDoj

Could it cause harm? IMO it should have only the one which matches the serial number. Is that correct?

Comment 24 Rob Crittenden 2016-09-15 02:23:07 UTC
Not sure what is going on but here are a few observations:

1. serial #7 is the original certificate, not a renewed one.
2. I'd pull these 3 certs and see what the heck they are. Are they all for the IPA RA or something else?
3. Pretty sure that guide is wrong (and it might be my fault). This results in a double-base64-encoded cert (TUlJRG...)

Comment 25 Jan Cholasta 2016-09-20 11:10:15 UTC
Petr, I don't think multiple userCertificate attributes should cause harm. However, the guide has errors, as Rob pointed out:

1) rather than userCertificate;binary, userCertificate should be used, as that is the attribute type used by Dogtag,

2) when adding the certificate using ldapmodify, the value of userCertificate must be specified using double colon ("userCertificate::"), otherwise it won't be properly encoded.

Comment 26 lmgnid 2016-09-22 00:21:56 UTC
Hello, I followd your advice and tried again, but still cannot renew the certificates, any ideas?

Here is the current status of this IPA server:

[root@usdev-ops-ipa-01 tmp]# ipa-getcert list
Number of certificates and requests being tracked: 12.
Request ID '20160109004901':
        status: SUBMITTING
        stuck: no
        key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-INTERNAL-COM',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-INTERNAL-COM/pwdfile.txt'
        certificate: type=NSSDB,location='/etc/dirsrv/slapd-INTERNAL-COM',nickname='Server-Cert',token='NSS Certificate DB'
        CA: IPA
        issuer: CN=Certificate Authority,O=INTERNAL.COM
        subject: CN=usdev-ops-ipa-01.internal.com,OU=pki-ipa,O=IPA
        expires: 2018-01-09 00:40:45 UTC
        key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
        eku: id-kp-serverAuth,id-kp-clientAuth
        pre-save command:
        post-save command: /usr/lib64/ipa/certmonger/restart_dirsrv INTERNAL-COM
        track: yes
        auto-renew: yes
Request ID '20160109005609':
        status: SUBMITTING
        stuck: no
        key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
        certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB'
        CA: IPA
        issuer: CN=Certificate Authority,O=INTERNAL.COM
        subject: CN=usdev-ops-ipa-01.internal.com,OU=pki-ipa,O=IPA
        expires: 2018-01-09 00:40:48 UTC
        key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
        eku: id-kp-serverAuth,id-kp-clientAuth
        pre-save command:
        post-save command: /usr/lib64/ipa/certmonger/restart_httpd
        track: yes
        auto-renew: yes

[root@usdev-ops-ipa-01 tmp]# getcert list
Number of certificates and requests being tracked: 12.
Request ID '20160109004901':
        status: SUBMITTING
        stuck: no
        key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-INTERNAL-COM',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-INTERNAL-COM/pwdfile.txt'
        certificate: type=NSSDB,location='/etc/dirsrv/slapd-INTERNAL-COM',nickname='Server-Cert',token='NSS Certificate DB'
        CA: IPA
        issuer: CN=Certificate Authority,O=INTERNAL.COM
        subject: CN=usdev-ops-ipa-01.internal.com,OU=pki-ipa,O=IPA
        expires: 2018-01-09 00:40:45 UTC
        key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
        eku: id-kp-serverAuth,id-kp-clientAuth
        pre-save command:
        post-save command: /usr/lib64/ipa/certmonger/restart_dirsrv INTERNAL-COM
        track: yes
        auto-renew: yes
Request ID '20160109005301':
        status: CA_UNREACHABLE
        ca-error: Internal error
        stuck: no
        key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
        certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
        CA: dogtag-ipa-ca-renew-agent
        issuer: CN=Certificate Authority,O=INTERNAL.COM
        subject: CN=CA Audit,O=INTERNAL.COM
        expires: 2016-05-03 21:14:05 UTC
        key usage: digitalSignature,nonRepudiation
        pre-save command: /usr/lib64/ipa/certmonger/stop_pkicad
        post-save command: /usr/lib64/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca"
        track: yes
        auto-renew: yes
Request ID '20160109005303':
        status: NEED_TO_SUBMIT
        ca-error: Error 7 connecting to http://usdev-ops-ipa-01.internal.com:8080/ca/ee/ca/profileSubmit: Couldn't connect to server.
        stuck: no
        key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
        certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB'
        CA: dogtag-ipa-ca-renew-agent
        issuer: CN=Certificate Authority,O=INTERNAL.COM
        subject: CN=OCSP Subsystem,O=INTERNAL.COM
        expires: 2016-05-03 21:14:01 UTC
        key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
        eku: id-kp-OCSPSigning
        pre-save command: /usr/lib64/ipa/certmonger/stop_pkicad
        post-save command: /usr/lib64/ipa/certmonger/renew_ca_cert "ocspSigningCert cert-pki-ca"
        track: yes
        auto-renew: yes
Request ID '20160109005305':
        status: CA_UNREACHABLE
        ca-error: Internal error
        stuck: no
        key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set
        certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB'
        CA: dogtag-ipa-ca-renew-agent
        issuer: CN=Certificate Authority,O=INTERNAL.COM
        subject: CN=CA Subsystem,O=INTERNAL.COM
        expires: 2016-05-03 21:14:03 UTC
        key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
        eku: id-kp-serverAuth,id-kp-clientAuth
        pre-save command: /usr/lib64/ipa/certmonger/stop_pkicad
        post-save command: /usr/lib64/ipa/certmonger/renew_ca_cert "subsystemCert cert-pki-ca"
        track: yes
        auto-renew: yes
Request ID '20160109005307':
        status: MONITORING
        stuck: no
        key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
        certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB'
        CA: dogtag-ipa-ca-renew-agent
        issuer: CN=Certificate Authority,O=INTERNAL.COM
        subject: CN=Certificate Authority,O=INTERNAL.COM
        expires: 2034-05-14 21:13:57 UTC
        key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
        pre-save command: /usr/lib64/ipa/certmonger/stop_pkicad
        post-save command: /usr/lib64/ipa/certmonger/renew_ca_cert "caSigningCert cert-pki-ca"
        track: yes
        auto-renew: yes
Request ID '20160109005308':
        status: MONITORING
        stuck: no
        key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB',pin set
        certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB'
        CA: dogtag-ipa-renew-agent
        issuer: CN=Certificate Authority,O=INTERNAL.COM
        subject: CN=usdev-ops-ipa-01.internal.com,O=INTERNAL.COM
        expires: 2017-12-29 00:52:21 UTC
        key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
        eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
        pre-save command: /usr/lib64/ipa/certmonger/stop_pkicad
        post-save command: /usr/lib64/ipa/certmonger/renew_ca_cert "Server-Cert cert-pki-ca"
        track: yes
        auto-renew: yes
Request ID '20160109005609':
        status: SUBMITTING
        stuck: no
        key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
        certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB'
        CA: IPA
        issuer: CN=Certificate Authority,O=INTERNAL.COM
        subject: CN=usdev-ops-ipa-01.internal.com,OU=pki-ipa,O=IPA
        expires: 2018-01-09 00:40:48 UTC
        key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
        eku: id-kp-serverAuth,id-kp-clientAuth
        pre-save command:
        post-save command: /usr/lib64/ipa/certmonger/restart_httpd
        track: yes
        auto-renew: yes
Request ID '20160109005623':
        status: CA_UNREACHABLE
        ca-error: Error 7 connecting to http://usdev-ops-ipa-01.internal.com:8080/ca/ee/ca/profileSubmit: Couldn't connect to server.
        stuck: no
        key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
        certificate: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB'
        CA: dogtag-ipa-renew-agent
        issuer: CN=Certificate Authority,O=INTERNAL.COM
        subject: CN=IPA RA,O=INTERNAL.COM
        expires: 2016-05-03 21:15:03 UTC
        key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
        eku: id-kp-serverAuth,id-kp-clientAuth
        pre-save command: /usr/lib64/ipa/certmonger/renew_ra_cert_pre
        post-save command: /usr/lib64/ipa/certmonger/renew_ra_cert
        track: yes
        auto-renew: yes
Request ID '20160913204943':
        status: NEED_TO_SUBMIT
        ca-error: Error 7 connecting to http://usdev-ops-ipa-01.internal.com:8080/ca/ee/ca/profileSubmit: Couldn't connect to server.
        stuck: no
        key pair storage: type=NSSDB,location='/var/lib/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
        certificate: type=NSSDB,location='/var/lib/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
        CA: dogtag-ipa-renew-agent
        issuer: CN=Certificate Authority,O=INTERNAL.COM
        subject: CN=CA Audit,O=INTERNAL.COM
        expires: 2016-05-03 21:14:05 UTC
        key usage: digitalSignature,nonRepudiation
        pre-save command:
        post-save command:
        track: yes
        auto-renew: yes
Request ID '20160913204944':
        status: NEED_TO_SUBMIT
        ca-error: Error 7 connecting to http://usdev-ops-ipa-01.internal.com:8080/ca/ee/ca/profileSubmit: Couldn't connect to server.
        stuck: no
        key pair storage: type=NSSDB,location='/var/lib/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
        certificate: type=NSSDB,location='/var/lib/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB'
        CA: dogtag-ipa-renew-agent
        issuer: CN=Certificate Authority,O=INTERNAL.COM
        subject: CN=OCSP Subsystem,O=INTERNAL.COM
        expires: 2016-05-03 21:14:01 UTC
        key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
        eku: id-kp-OCSPSigning
        pre-save command:
        post-save command:
        track: yes
        auto-renew: yes
Request ID '20160913204945':
        status: CA_UNREACHABLE
        ca-error: Error 7 connecting to http://usdev-ops-ipa-01.internal.com:8080/ca/ee/ca/profileSubmit: Couldn't connect to server.
        stuck: no
        key pair storage: type=NSSDB,location='/var/lib/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set
        certificate: type=NSSDB,location='/var/lib/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB'
        CA: dogtag-ipa-renew-agent
        issuer: CN=Certificate Authority,O=INTERNAL.COM
        subject: CN=CA Subsystem,O=INTERNAL.COM
        expires: 2016-05-03 21:14:03 UTC
        key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
        eku: id-kp-serverAuth,id-kp-clientAuth
        pre-save command:
        post-save command:
        track: yes
        auto-renew: yes
Request ID '20160913204946':
        status: MONITORING
        stuck: no
        key pair storage: type=NSSDB,location='/var/lib/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB',pin set
        certificate: type=NSSDB,location='/var/lib/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB'
        CA: dogtag-ipa-renew-agent
        issuer: CN=Certificate Authority,O=INTERNAL.COM
        subject: CN=usdev-ops-ipa-01.internal.com,O=INTERNAL.COM
        expires: 2017-12-29 00:52:21 UTC
        key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
        eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
        pre-save command:
        post-save command:
        track: yes
        auto-renew: yes



[root@usdev-ops-ipa-01 tmp]# ldapsearch -x -h localhost -p 389 -D 'cn=directory manager' -W -b uid=ipara,ou=People,o=ipaca
Enter LDAP Password:
# extended LDIF
#
# LDAPv3
# base <uid=ipara,ou=People,o=ipaca> with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#

# ipara, people, ipaca
dn: uid=ipara,ou=people,o=ipaca
description: 2;7;CN=Certificate Authority,O=INTERNAL.COM;CN=IPA RA
 ,O=INTERNAL.COM
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
objectClass: cmsuser
uid: ipara
sn: ipara
cn: ipara
usertype: agentType
userstate: 1
userCertificate:: MIIDnzCCAoegAwIBAgIBBzANBgkqhkiG9w0BAQsFADBDMSEwHwYDVQQKExhJ
...
 8PsnY504OGaodT9IWdsaffmpffA==
userCertificate:: MIIDojCCAoqgAwIBAgIFA4/9AAowDQYJKoZIhvcNAQELBQAwQzEhMB8GA1UE
...
 BY5uQHsTpO5O4haY68jXGSlX+a/TlKg==

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

Comment 27 Petr Vobornik 2016-09-23 15:08:28 UTC
The procedure from comment 25 alone won't renew the certs. 

Usually after each adjustment the "usual renew procedure" needs to be tried to check if it unblock it:

The usual renew procedure:
- set time back
- make sure PKI server and IPA are runnning
- run date
- run getcertlist
- wait a bit if stuff are happening, optional observer PKI debug log what is happening
- if the affected certs are renewed, put time back.

Then when pasting output of commands here, run 'date' with it as well paste the 'date' output so we new if the output is from current time or the "past time"

Comment 28 Petr Vobornik 2016-10-07 12:10:54 UTC
Closing for inactivity.


Note You need to log in before you can comment on or make changes to this bug.